My paper Linear Algebra with Transformers was published in Transactions of Machine Learning Research (TMLR). This new version includes many new results and experiments.
openreview.net/pdf?id=Hp4g7FA…
The source code should be available in a few days.
AI for mathematics and theoretical physics
Tomorrow's problems on yesterday's machines
Axiom - École nationale des ponts et chaussées
France
Joined December 2019
- Transformers can be trained to solve a 132-years old open problem: discovering global Lyapunov functions. New paper on Arxiv (accepted in NeurIPS 2024), with @albe_alfa and @Amaury_Hayatarxiv.org/abs/2410.08304 1/8
- The source code, datasets and trained models for our paper "Learning Advanced Mathematical Computations from Examples", with @Amaury_Hayat and @GuillaumeLample, are now available at
- Transformers for discrete optimisation problems 1- Train a model on candidate solutions 2- Use the model to generate more candidates 3- Improve the solutions with local search 4- Use the best candidates to fine tune the model 5- IterateNew preprint up! "PatternBoost: Constructions in Mathematics with a Little Help from AI," with F. Charton, A.Z. Wagner, and G. Williamson: arxiv.org/abs/2411.00566
- My talk at Physics ∩ ML last week youtube.com/watch?v=81o-Ui… Recent results on transformers learning mathematical properties (instead of just memorizing and interpolating) at 40:00. And a first attempt at particle physics, and gluons, at 57:00
- The source code for my two papers "Linear Algebra with Transformers" (TMLR) arxiv.org/abs/2112.01898 and "What is my math transformer doing?" (NeurIPS 2022 Math-AI Workshop) arxiv.org/abs/2211.00170) is now available at github.com/facebookresear… (with trained models and test sets)
- The code for our paper: Global Lyapunov functions: a long-standing open problem in mathematics, with symbolic transformers, with @albe_alfa and @Amaury_Hayat is available at github.com/facebookresear… We will be in NeurIPS: come see us at the poster session next Thursday at 5PM
- Transformers can be trained to compute the roots of polynomials f-charton.github.io/polynomial-roo… It is often said that language models "cannot compute", evidence to the contrary is accumulating.
- I am joining Axiom Math, a seed-stage startup on AI for maths. I will lead discovery: AI for advancing math research. 6 years after Deep Learning for Symbolic Maths, our first paper with @GuillaumeLample, I am proud of the field's progress, and excited about what comes next.
- Transformers can be trained to solve problems of linear algebra (matrix transposition, addition, multiplication, inversion and eigenvalues) to very high accuracy. 1/4 Our new paper is on Arxiv: arxiv.org/abs/2112.01898
- Transformers solve an open problem in symbolic mathematics: discovering Lyapunov functions, joint work with Alberto Alfarano and @Amaury_Hayat. My talk in IAIFI today (starts at 5:00)Our first IAIFI Colloquium of the semester is starting now with @f_charton! "Transformers meet Lyapunov: Solving a long-standing open problem in mathematics." Watch live on YouTube: youtube.com/live/yCzV97QNG…
- My talk in Harvard yesterday. Transformers for symbolic regression (11:30), theoretical physics (26:00), and results on explainability in linear algebra (39:00) and arithmetic (50:00)
- My talk at the IHES workshop: Mathematics for and by large language models youtube.com/watch?v=k9xLg-… and the full seminar on carmin.tvcarmin.tv/en/c/1539 featuring talks by @Amaury_Hayat, @KempeLab, Yiannis Vlassopoulos, Andrew Dudzik and @syhw
- Looking for a postdoctorate student to work with me on applying transformers to open problems in mathematics and theoretical physics. This is an 18 month position, based in Paris. DM me if interested. metacareers.com/jobs/771404296…
