Voozh

Dan Roberts

1,117 posts

Dan Roberts

@danintheory

Scientist @OpenAI. Prev. co-founder @diffeo, acquired by @salesforce // co-authored The Principles of Deep Learning Theory // studied gravity.

San Francisco, CA

Joined June 2009

Pinned
👁 user avatar
Dan Roberts
@danintheory
Jun 4
TL; DW I say things (on reinforcement learning, the process of scientific discovery, and how physicists approach work on AI).
👁 user avatar
Matt Turck
@mattturck
Jun 4
Why AI Can Now Make Discoveries - my conversation with @danintheory, Lead of the Foundations of Reinforcement Learning team at @OpenAI 00:00 Intro: AI's wild week in mathematics 01:21 What OpenAI's Foundations of RL team does 03:08 Dan's journey: from black holes and quantum
👁 Image
00:00
👁 user avatar
Dan Roberts
@danintheory
Jun 18, 2021
I'm delighted to share publicly "The Principles of Deep Learning Theory," co-written with @ShoYaida, and based on work also with @BorisHanin. It will appear on the @arxiv on Sunday and will be published by @CambridgeUP early next year: deeplearningtheory.com 1/
👁 user avatar
Dan Roberts
@danintheory
May 26, 2022
The Principles of Deep Learning Theory, by @ShoYaida, @BorisHanin, and myself, is available in print today from @CambridgeUP! You can find links to purchase (as well as a link to a free arxiv draft) here: deeplearningtheory.com 1/4
👁 Image
👁 Image
👁 user avatar
Dan Roberts
@danintheory
May 16, 2024
Update on this: I joined @OpenAI.
👁 user avatar
Dan Roberts
@danintheory
Apr 27, 2024
Finally tried the new AI assistant: seems like it has a long way to go before it matches even the most basic capabilities of the old AI assistant…
👁 Image
👁 Image
👁 user avatar
Dan Roberts
@danintheory
May 9, 2025
My short talk at @sequoia AI Ascent on how we at @OpenAI are attempting to flip the old "cherry-on-top" meme about "Scaling RL"
👁 Image
00:00
👁 user avatar
Dan Roberts
@danintheory
Mar 27, 2024
Do LLMs really need to be so L? That's a rejected title for a new paper w/ @Andr3yGR, @kushal_tirumala, @Hasan_Shap, @PaoloGlorioso1 on pruning open-weight LLMs: we can remove up to *half* the layers of Llama-2 70B w/ essentially no impact on performance on QA benchmarks. 1/
👁 Image
👁 user avatar
Dan Roberts
@danintheory
Jun 18, 2021
Replying to @danintheory
This formula accurately gives the output of a deep MLP after training. 11/
👁 Image
👁 user avatar
Dan Roberts
@danintheory
Feb 20, 2022
I respectfully disagree w/ this and @ylecun's perspectives on causality: (a) causality is an essential part of microscopic physics and (b) even at the effective level of Newtonian physics, F=ma is *not* symmetric since it's a differential equation, not an algebraic equality. 1/
👁 user avatar
Judea Pearl
@yudapearl
Feb 19, 2022
You don't need to go to CPT to argue that "deep down" there is no such thing as causality. Newtonian mechanics is also governed by algebraic equalities, so it is symmetric, telling us that f=ma is as valid as a=f/m. Where causality comes in is when we venture to model the 1/2
👁 user avatar
Dan Roberts
@danintheory
May 8, 2025
I gave a talk where the only real thing to say is that we are inverting the meme
👁 Image
👁 Image
👁 user avatar
Sonya Huang 🐥
@sonyatweetybird
May 7, 2025
Replying to @sonyatweetybird @sequoia and 7 others
@danintheory of @OpenAI Strawberry 🍓: * Reinforcement learning is shifting from a “cherry on top” of the pre-training cake, to the main ingredient driving breakthroughs going forward * Imagine einstein-v1907-super-high had access to unlimited test-time compute. What new science
👁 user avatar
Dan Roberts
@danintheory
Jan 11, 2023
If you like both physics and AI -- and in particular if you are interested in using physics tools to study AI -- please apply to the "Theoretical Physics for Deep Learning" workshop at the Aspen Center for Physics this summer (May 28 to June 18): aspenphys.org/physicists/sum…
👁 Image
👁 user avatar
Dan Roberts
@danintheory
Nov 1, 2022
New work on the origin of @OpenAI's neural scaling laws w/ Alex Maloney and @jamiesully2: we solve a simplified model of scaling laws to gain insight into how scaling behavior arises and to probe its behavior in regimes where scaling laws break down. arxiv.org/abs/2210.16859 1/
👁 user avatar
Dan Roberts
@danintheory
Aug 6, 2021
This week @ShoYaida & I gave 5 lectures based on our book at the Princeton Deep Learning Theory (PDLT) Summer School (…-learning-summer-school.princeton.edu) masterfully organized by @BorisHanin. For those interested, videos are online (linked below), & next week I'll summarize our talks here.
👁 user avatar
Dan Roberts
@danintheory
Dec 22, 2022
Replying to @djstrouse
I thought when you catch an AI in a logical paradox it’s supposed to self destruct?
👁 user avatar
Dan Roberts
@danintheory
Jan 24, 2025
I’d be very glad to vouch for @shaunmmaguire ‘s work with DARPA and in physics: I affirm he did a tour with DARPA in Afghanistan, his startup Qadium worked with DARPA as part of the Memex program along with my startup @diffeo (cf. forbes.com/sites/thomasbr…), and he did his PhD in
👁 user avatar
Steven Balik
@laurenbalik
Jan 23, 2025
The beautiful thing about Shaun Maguire is nobody can confirm date/time specifics about his DARPA tour in Afghanistan. He was about to flunk out of every school he attended before his PhD from Caltech. Nobody from his background can confirm anything. I've done the work.
👁 Image

URL: https://x.com/danintheory

⇱ Dan Roberts (@danintheory) / X