Got promoted to a full professor at CMU:) Thanks to my amazing students, collaborators, and my mentors.
CMU is a remarkable place, but what makes it truly unique is its brilliant students! Hard to find a better place to do ML and AI.
Russ Salakhutdinov
1,766 posts
Russ Salakhutdinov
@rsalakhu
CSO @ Sooth Labs, Professor @ CMU, President Elect ICML Board, Ex-VP of Research @ Meta (Multimodal LLMs, AI Agents), ex-Director of AI at @Apple.
- I get asked a lot: why stay in academia, all the excitement in AI is happening in industry with massive compute. And I am seeing some profs leaving academia, but also seeing lots of researchers in industry looking to go back to academia, especially those who don’t work on LLMs.
- 1) My observation when working at Apple and many other industrial partners is that deep learning researchers who are well versed in graphical models, variational inference, CRFs, MCMC, uncertainty quantification are on average much more creative when solving real-world problems.Researchers in speech recognition, computer vision, and natural language processing in the 2000s were obsessed with accurate representations of uncertainty. 1/N
- This semester I am teaching a graduate course in Deep RL with over 500 graduate students enrolled so far, the biggest class I ever taught! One of my colleagues was joking that in the near future, we might be teaching ML and deep learning classes at the CMU football stadium.😀
- Being in academia is amazing. Being in industry is exciting. Being in both is electrifying.
- I hear a lot of folks in our AI community complain about openAI -- they don't publish, don't release models, maximum-for-profit, etc., so they are more like closedAI rather than openAI. This is true, but you have to give it to those guys -- they showed the true potential of LLMs
- Someone told me once: You can do AI or you can just talk about it. #AIDebate
- Excited about joining Apple as a director of AI research in addition to my work at CMU. Apply to work with my team
- Slides from my talk on Integrating Domain Knowledge into Deep Learning at the New York Academy of Sciences @NYASciences. Special shoutout to @ZhitingHu and Bhuwan Dhingra for leading this amazing work and helping me with the slides: cs.cmu.edu/~rsalakhu/NY_2…
- Very excited for the launch of Apple’s Machine Learning blog: machinelearning.apple.com. Check it out!
- To be fair, @openAI had a big impact on AI/ML. Their OpenAI Gym env for RL, their CLIP - a de facto model for any image-text research, their Whisper speech recognition is amazing - all open sourced. In my view, they've had a bigger impact than many labs publishing 100s papers 🧵Data on the intellectual contribution to AI from various research organizations. Some of organizations publish knowledge and open-source code for the entire world to use. Others just consume it.
- Congratulations to Zhilin Yang for successfully defending his PhD thesis at CMU in just 4 years! Zhilin introduced XL-Net, Transformer-XL, Mixture of Softmaxes High-Rank LM, HotpotQA, GLoMo Unsupervised Learning of Relational Graphs, just to name a few: kimiyoung.github.io
- New #ICLR2021 paper on Self-supervised Learning with Relative Predictive Coding: A new contrastive learning objective that maintains a good balance between training stability, minibatch size sensitivity, & downstream task performance. arxiv.org/abs/2103.11275 w/ H. Tsai et al.
- Replying to @rsalakhu3) My advise to deep learning PhD students is to also learn variational inference, belief propagation, MCMC, HMMs, graphical models, generative models, etc. -- no matter how irrelevant you think this is for the current speech, vision, and NLP systems.
