Tom Schaul, Ph.D.

I am a senior staff research scientist at DeepMind London.

My research interests include (modular/hierarchical) reinforcement learning, (stochastic/black-box) optimization with minimal hyperparameter tuning, and (deep/recurrent) neural networks. My favorite application domain are games.

I grew up in Luxembourg and studied computer science in Switzerland (with exchanges at Waterloo and Columbia), where I obtained an MSc from the EPFL in 2005. I hold a PhD from TU Munich (2011), which I did under the supervision of Jürgen Schmidhuber at the Swiss AI Lab IDSIA. From 2011 to 2013 I was a postdoc at the Courant Institute of NYU, in the lab of Yann LeCun.

Selected papers

NeurIPS 2022	T. Schaul, A. Barreto, J. Quan and G. Ostrovski. The Phenomenon of Policy Churn. Advances in Neural Information Processing Systems. [arXiv]
Nature Comm. 2020	N. Tomašev, J. Cornebise, F. Hutter et al. AI for Social Good: Unlocking the Opportunity for Positive Impact. Nature Communications 11 (2468). [Link]
Nature 2019	O. Vinyals, I. Babuschkin, W. Czarnecki et al. Grandmaster level in StarCraft II using multi-agent reinforcement learning. Nature 574 (7780). [Link] [Preprint] [Blog] [Video]
RLDM 2019	T. Schaul, D. Borsa, J. Modayil and R. Pascanu. Ray Interference: a Source of Plateaus in Deep Reinforcement Learning. Multidisciplinary Conference on Reinforcement Learning and Decision Making . [arXiv]
ICLR 2016	T. Schaul, J. Quan, I. Antonoglou and D. Silver. Prioritized Experience Replay. International Conference on Learning Representations. [arXiv][BibTeX]
ICML 2015	T. Schaul, D. Horgan, K. Gregor and D. Silver. Universal Value Function Approximators. International Conference on Machine Learning. [Pdf] [BibTeX]

See the full list of publications for 60+ additional ones.

News

2024-25: organizing committee for RLC 2025, keynote chair.
12/2024: invited talk on language games at NeurIPS 2024 workshop on Language Gamification [W23].
08/2024: attended the first-ever Reinforcement Learning Conference in Amherst, MA.
07/2024: position paper on open-endedness[67] at ICML 2024 in Vienna: oral presentation.
06-09/2024: area chair for NeurIPS 2024.
02/2024: attended Dagstuhl workshop on AI for Social Good.
03-06/2024: area chair for RLC 2024.
02-05/2024: area chair for ICML 2024.
08/2023: paper on discovery via pruning proto-goals[66] at IJCAI 2023 in Macao.
05-10/2023: area chair for NeurIPS 2023.
05-07/2023: papers on transformer-based meta-learning of evolution strategies[64], or genetic algorithms[65], presented at ICLR in Kigali and GECCO in Lisbon.
02-05/2023: area chair for ICML 2023.
02/2023: attending Barbados workshop on Lifelong Reinforcement Learning and presenting [W21].
11-12/2022: attended NeurIPS 2022 in New Orleans, presenting "Policy Churn"[63].
09-11/2022: senior area chair for AAAI 2023.
06-09/2022: area chair for NeurIPS 2022.
07/2022: attended ICML 2022 in Baltimore, presenting [62].
06/2022: presented meta-learnt exploration ideas[W20] at RLDM 2022.
04/2022: spotlight talk on "When to Explore"[61] at ICLR 2022.
02-03/2022: co-organizing Dagstuhl workshop on AI for the Social Good [T7].
07/2021: teaching at the International Summer School on Artificial Intelligence and Games.
11/2020: Keynote talk at the BNAIC/Benelearn conference.
07/2020: mentoring at the AI4Good Lab
05/2020: Perspective paper on guidelines around AI for Social Good published in Nature Communications [60].
10/2019: Our work on StarCraft made it into Nature [58]!
12/2019: co-organizing Dagstuhl workshop on Artificial and Computational Intelligence in Games.
07/2019: invited talk on the Ray[W18] and panelist at Lifelong learning workshop at RLDM.
07/2019: attended RLDM 2019 and presented Rays[W18] and Unicorns [W17].
07/2019: launched the Dagstuhl Declaration on the Application of Machine Learning and Artificial Intelligence for the Social Good.
06/2019: mentoring at the AI4Good Lab
02/2019: co-organized Dagstuhl workshop on AI for the Social Good [report].
04/2018: best reviewer award at ICLR 2018.
03/2018: invited talk at the Deep Learning: Theory, Algorithms, and Applications workshop at RIKEN, Tokyo.