Stupid advice
cs grads: dropout now, there is not a single job in CS anymore. AI will take over, this field wonβt exist in a year.
Christian Szegedy
8,782 posts
Christian Szegedy
@ChrSzegedy
Sunnyvale, CA
Joined June 2015
- IDEAS has been a great partner at pushing AI for sciences and it has developed into an exceptional powerhouse of European innovation. Now, I find it very quite troubling that there was no transparency about the recent leadership selection process and I have mostly hearedDeeply concerned about the future of IDEAS NCBR, a pioneering AI research institute & newly approved ELLIS unit. International advisory board members, incl. Aleksander MΔ dry, have resigned after the sudden and unjustified replacement of its founding director. @donaldtuskEPP
- Tried @grok 4 on a dozen non-trivial math (under/)grad level math problems. So far, it has failed to fail me even once. Congrats to @Yuhu_ai_, @ericzelikman and the whole xAI reasoning team, their progress has exceeded all my expectation!
- A few months ago, I had a long chat with Terry Tao. While he is one of the knowledgeabe and forward-looking mathematician, I still had the impression that he was out of touch withe the reality that was coming up quickly.
- "Not too bad from a bunch of monkeys"For the first time, there is a rocket that can make all life multiplanetary. A fork in the road of human destiny.
- Deep learning is not just memorization, it also compresses. And really good compression amounts to intelligence. That's why we get few/zero-shot capabilities from LLMs, since they "discovered" a lot of the underlying patterns just by being trained to compress.Deep learning takes data points and turns them into a query-able structure that enables retrieval and interpolation between the points. You could think of it as a continuous generalization of database technology.
- New jobs in the 21st century: Model restart specialist Hyperparameter psychic Prompt engineer Model janitor Tensor shape mediator Quantum state observer Model footprint accountantReplying to @awnihannunThe paper mentions 35 (!) manual restarts to train OPT-175B due to hardware failure (and 70+ automatic restarts).
- Replying to @ibabIt was a great privilege working with you, Igor. Wishing best for your new ventures.
- I talked to several non-ML people over the past half year (including my kids) about chat bots and what i feel is a general frustration about their being moralizing and not getting answers even to relatively innocent prompts. So being more unhinged will be a feature not a bug.
- User: "When was Einstein Born?" LLM: .. Let's revisit a compressed form of all existing knowledge on the web ... once for *every* single input token ... and each token generated, including the GDP of Armenia, the name of all kings ever lived and all famous chess games, etc ...
- I still stick to these predictionsReplying to @joshua_saxe and @fcholletOne on math: super-human mathematician AI by June 26. The other: 95% at the end of 2025 on the current test set of of ARC. We can work out the details on the first. I feel much less confident about the second, esp. given the current challanges with evaluation on proprietary
- Here is a thread in paper references demonstrating that current deep learning, especially transformers are amazingly powerful for symbol manipulation already: @GuillaumeLample, @f_chartonarxiv.org/abs/1912.01412 Solving hard integrals using deep transformers π§΅1/n
- The most sincere from of flattery is when something seems so outrageous that most (knowledgable) people don't even believe it.AI Agenda: Why Muskβs AI Rivals Are Alarmed by His New GPU Cluster Elon Musk claims to have finished a 100,000-strong H100 cluster in four months. How likely is that? theinformation.com/articles/why-mβ¦ From @anissagardizy8
- My view on this has not changed in the past eight years: I have given many talks and written position paper in 2019 (link below). Progress is faster than my past expectation. My target date used to be ~2029 back then. Now it is 2026 for a superhuman AI mathematician. While aReplying to @ChrSzegedy @colin_fraser and @Lang__LeonI am less sanguine but would love to hear your thoughts in more detail if you can share. @ChrSzegedy
