ATALA best PhD thesis prize 2025

I’m very happy to share that my PhD thesis, Improving Representations for Language Modeling, was awarded the 2025 ATALA best PhD thesis prize at the TALN conference (Marseille, July 2025).

The full citation from the jury (in French) is available on the ATALA website. A short excerpt, translated:

Nathan Godey’s thesis sets out to go beyond the structural limitations of representation learning in neural architectures. Through detailed analysis using interpretability methods, he proposes a strong conceptual contribution: a method for training language models that does away with computing the distribution over output symbols, by leveraging contrastive learning techniques. This original approach, called headless, aims to reconstruct sequences of input token embeddings without resorting to a decoding head during training. A demanding, systematic, and technically accomplished experimental study demonstrated the effectiveness of this new paradigm, both in terms of memory usage and convergence speed.

The prize was awarded jointly with Urchade Zaratiana (LIPN, Université Sorbonne Paris Nord, CNRS). Congrats to him as well!