2017

NIPS[51] Natural value approximators: learning when to trust past estimates.
Zhongwen Xu, Joseph Modayil, Hado van Hasselt, Andre Barreto, David Silver and Tom Schaul.
Proceedings of the Neural Information Processing Systems (NIPS-2017, Long Beach)
NIPS[50] Successor Features for Transfer in Reinforcement Learning.
André Barreto, Will Dabney, Rémi Munos, Jonathan Hunt, Tom Schaul, David Silver and Hado van Hasselt.
Proceedings of the Neural Information Processing Systems (NIPS-2017, Long Beach) [arXiv]
arXiv[*] Rainbow: Combining Improvements in Deep Reinforcement Learning.
Matteo Hessel, Joseph Modayil, Hado van Hasselt, Tom Schaul, Georg Ostrovski, Will Dabney, Dan Horgan, Bilal Piot, Mohammad Azar, David Silver. [arXiv]
arXiv[*] StarCraft II: A New Challenge for Reinforcement Learning.
Oriol Vinyals, Timo Ewalds, Sergey Bartunov, Petko Georgiev, Alexander Sasha Vezhnevets, Michelle Yeo, Alireza Makhzani, Heinrich Küttler, John Agapiou, Julian Schrittwieser, Stephen Gaffney, Stig Petersen, Karen Simonyan, Tom Schaul, Hado van Hasselt, David Silver, Timothy Lillicrap, Kevin Calderone, Paul Keet, Anthony Brunasso, David Lawrence, Anders Ekermo, Jacob Repp, Rodney Tsing. [arXiv]
arXiv[*] Learning from Demonstrations for Real World Reinforcement Learning.
Todd Hester, Matej Vecerik, Olivier Pietquin, Marc Lanctot, Tom Schaul, Bilal Piot, Andrew Sendonaris, Gabriel Dulac-Arnold, Ian Osband, John Agapiou, Joel Z Leibo, Audrunas Gruslys. [arXiv]
ICML[49] The Predictron: End-To-End Learning and Planning.
David Silver*, Hado van Hasselt*, Matteo Hessel*, Tom Schaul*, Arthur Guez*, Tim Harley, Gabriel Dulac-Arnold, David Reichert, Neil Rabinowitz, Andre Barreto, Thomas Degris.
Proceedings of the International Conference on Machine Learning (ICML-2017, Syndney). [arXiv] [openreview]
ICML[48] FeUdal Networks for Hierarchical Reinforcement Learning.
Alexander Sasha Vezhnevets, Simon Osindero, Tom Schaul, Nicolas Heess, Max Jaderberg, David Silver, Koray Kavukcuoglu.
Proceedings of the International Conference on Machine Learning (ICML-2017, Sydney). [arXiv]
ICLR [47] Reinforcement Learning with Unsupervised Auxiliary Tasks.
Max Jaderberg*, Volodymyr Mnih*, Wojciech Czarnecki*, Tom Schaul, Joel Leibo, David Silver, Koray Kavukcuoglu.
Proceedings of the International Conference on Learning Representations (ICLR-2017, Toulon). [arXiv] [openreview]
US Patent[P3] Training neural networks using a prioritized experience memory.
Tom Schaul, John Quan and David Silver.
United States Patent Application 20170140269. 2017.
Based on [42].

2016

NIPS[46] Unifying Count-Based Exploration and Intrinsic Motivation.
Marc G. Bellemare, Sriram Srinivasan, Georg Ostrovski, Tom Schaul, David Saxton and Rémi Munos.
Proceedings of the Neural Information Processing Systems (NIPS-2016, Barcelona) [arXiv] [Video (100k+ views)]
NIPS [45] Learning to learn by gradient descent by gradient descent.
Marcin Andrychowicz, Misha Denil, Sergio Gomez, Matthew W. Hoffman, David Pfau, Tom Schaul and Nando de Freitas.
Proceedings of the Neural Information Processing Systems (NIPS-2016, Barcelona) [arXiv]
IEEE-CIG[44] Analyzing the Robustness of General Video Game Playing Agents.
Diego Perez-Liebana, Spyridon Samothrakis, Julian Togelius, Tom Schaul and Simon Lucas.
Proceedings of the IEEE Conference on Computational Intelligence in Games (CIG-2016, Greece). [Pdf] [BibTeX]
ICML[43] Dueling Network Architectures for Deep Reinforcement Learning.
Ziyu Wang, Tom Schaul, Matteo Hessel, Hado van Hasselt, Marc Lanctot and Nando de Freitas.
Proceedings of the International Conference on Machine Learning (ICML-2016, New York). [arXiv] [BibTeX]

Best Paper Award

ICLR[42] Prioritized Experience Replay.
Tom Schaul, John Quan, Ioannis Antonoglou and David Silver.
Proceedings of the International Conference on Learning Representations (ICLR-2016, Puerto Rico). [arXiv] [BibTeX]
AAAI-WH[W14] General Video Game AI: Competition, Challenges and Opportunities.
Diego Perez-Liebana, Spyridon Samothrakis, Julian Togelius, Tom Schaul and Simon Lucas.
AAAI What's Hot Track (2016, Phoenix). [Pdf]
US Patent[P2] Selecting reinforcement learning actions using goals and observations.
Tom Schaul, Dan Horgan, Karol Gregor and David Silver.
United States Patent Application 20160292568. 2016.
Based on [41].
US Patent[P1] Method for Creating Predictive Knowledge Structures from Experience in an Artificial Agent.
Mark Ring and Tom Schaul.
United States Patent Application 20160012338. 2016. [Link] [BibTeX]
Based on [32].

2015

ICML[41][W13] Universal Value Function Approximators.
Tom Schaul, Daniel Horgan, Karol Gregor and David Silver.
Proceedings of the International Conference on Machine Learning (ICML-2015, Lille). [Pdf] [BibTeX]

Also presented at the 9th Barbados Workshop on Reinforcement Learning (2015). [Slides]

T-CIAIG[40] The 2014 General Video Game Playing Competition.
Diego Perez, Spyridon Samothrakis, Julian Togelius, Tom Schaul, Simon Lucas, Adrien Couetoux, Jerry Lee, Chong-U Lim and Tommy Thompson.
IEEE Transactions on Computational Intelligence and AI in Games. [Pdf] [BibTeX]

2014

T-CIAIG[39] An Extensible Video Game Description Language.
Tom Schaul.
IEEE Transactions on Computational Intelligence and AI in Games. [Pdf] [BibTeX] [Code]
ICLR[38] Unit Tests for Stochastic Optimization.
Tom Schaul, Ioannis Antonoglou and David Silver.
Proceedings of the International Conference on Learning Representations (ICLR-2014, Banff, Canada). [BibTeX] [arXiv] [Code] [Public reviews]
JMLR[37] Natural Evolution Strategies.
Daan Wierstra, Tom Schaul, Tobias Glasmachers, Yi Sun, Jan Peters and Jürgen Schmidhuber.
Journal of Machine Learning Research. [Pdf] [BibTeX] [arXiv]

2013

Dagstuhl[36] General Video Game Playing.
John Levine, Clare Bates Congdon, Michal Bida, Marc Ebner, Graham Kendall, Simon Lucas, Risto Miikkulainen, Tom Schaul and Tommy Thompson. Dagstuhl Follow-up, volume 6. [BibTeX] [Preprint]
Dagstuhl[35] Towards a Video Game Description Language.
Marc Ebner, John Levine, Simon Lucas, Tom Schaul, Tommy Thompson and Julian Togelius. Dagstuhl Follow-up, volume 6. [BibTeX] [Preprint]
IEEE-CIG[34] A Video Game Description Language for Model-based or Interactive Learning.
Tom Schaul.
Proceedings of the IEEE Conference on Computational Intelligence in Games (CIG-2013, Niagara Falls, Canada). [Pdf] [Code] [BibTeX]

Runner-up to Best Paper Award

ICML[33][W10] No more Pesky Learning Rates.
Tom Schaul, Sixin Zhang and Yann LeCun.
Proceedings of the International Conference on Machine Learning (ICML-2013, Atlanta). [arXiv] [Pdf] [Supplementary material] [Code] [BibTeX]

Also presented at the NIPS Workshop on Optimization for Machine Learning (NIPS-OPT 2012, Lake Tahoe).

IJCAI[32][W12] Better Generalization with Forecasts.
Tom Schaul and Mark Ring.
Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI-2013, Beijing, China). [Pdf] [BibTeX]
Basis for patent [P1].

Also presented at the 8th Barbados Workshop on Reinforcement Learning (2013, Holetown, Barbados). [Slides]

ICLR[31] Adaptive Learning Rates and Parallelization for Stochastic, Sparse, Non-smooth Gradients.
Tom Schaul and Yann LeCun.
Proceedings of the International Conference on Learning Representations (ICLR-2013, Scottsdale AZ). [Pdf] [BibTeX] [Code] [Public reviews]
GECCO[30] A Linear Time Natural Evolution Strategy for Non-Separable Functions.
Yi Sun, Faustino Gomez, Tom Schaul and Jürgen Schmidhuber.
Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2013, Amsterdam). [arXiv] [BibTeX]
Springer[29] Optimization with Surrogate Models.
Tom Schaul.
Chapter 3 in Numerical Methods for Metamaterial Design, edited by Kenneth Diest. [Link] [BibTeX]

2012

ICDL[28][W11] The Organization of Behavior into Temporal and Spatial Neighborhoods.
Mark Ring and Tom Schaul.
Proceedings of the International Conference on Developmental Learning (ICDL-2012, San Diego). [Pdf] [BibTeX]

Also presented at the AAAI Spring Symposium on Lifelong Machine Learning (AAAI-LML-2013, Stanford University).

NYC-ML[W9] Adaptive Learning Rates for Stochastic Gradients.
Tom Schaul, Sixin Zhang and Yann LeCun.
New York City Machine Learning Symposium.
GECCO[27] Natural Evolution Strategies Converge on Sphere Functions.
Tom Schaul.
Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2012, Philadelphia). [Pdf] [BibTeX]
BBOB[W8] Comparing Natural Evolution Strategies to BIPOP-CMA-ES.
Tom Schaul. Black-box Optimization Benchmarking: GECCO Workshop for Real-Parameter Optimization (BBOB-2012). [Pdf] [BibTeX]
BBOB[W7] Investigating the Impact of Adaptation Sampling in Natural Evolution Strategies.
Tom Schaul. Black-box Optimization Benchmarking: GECCO Workshop for Real-Parameter Optimization (BBOB-2012). [Pdf] [BibTeX]
BBOB[W6] Benchmarking Separable Natural Evolution Strategies.
Tom Schaul. Black-box Optimization Benchmarking: GECCO Workshop for Real-Parameter Optimization (BBOB-2012). [Pdf] [BibTeX]
BBOB[W5] Benchmarking Exponential Natural Evolution Strategies.
Tom Schaul. Black-box Optimization Benchmarking: GECCO Workshop for Real-Parameter Optimization (BBOB-2012). [Pdf] [BibTeX]
BBOB[W4] Benchmarking Natural Evolution Strategies with Adaptation Sampling.
Tom Schaul. Black-box Optimization Benchmarking: GECCO Workshop for Real-Parameter Optimization (BBOB-2012). [Pdf] [BibTeX]
Snowbird[W3] Decoupling the Data Geometry from the Parameter Geometry for Stochastic Gradients.
Tom Schaul, Sixin Zhang and Yann LeCun.
Snowbird Learning Workshop (2012). [Pdf] [BibTeX]

2011

TU Munich[26] Studies in Continuous Black-box Optimization. Tom Schaul.
Ph.D. Thesis at Technische Universität München.
[Link] [Pdf] [BibTeX] [Paperback at lulu.com]

Summa cum laude

ICDL[25][W2] The Two-Dimensional Organization of Behavior.
Mark Ring, Tom Schaul and Jürgen Schmidhuber.
Proceedings of the International Conference on Developmental Learning (ICDL-2011, Frankfurt). [Pdf] [BibTeX]

Also presented at the AAAI Workshop on Lifelong Learning from Sensorimotor Experience (AAAI-2011, San Francisco).

AGI[24] Coherence Progress: A Measure of Interestingness Based on Fixed Compressors
Tom Schaul, Leo Pape, Tobias Glasmachers, Vincent Graziano and Jürgen Schmidhuber.
Proceedings of the Fourth Conference on Artificial General Intelligence (AGI-2011, Mountain View). [Pdf] [BibTeX] [Video]
Acta Futura[23] Artificial Curiosity for Autonomous Space Exploration.
Vincent Graziano, Tobias Glasmachers, Tom Schaul, Leo Pape, Giuseppe Cuccu, Jürgen Leitner and Jürgen Schmidhuber.
Acta Futura. [Pdf] [BibTeX] [Link]
IJCAI[22] Q-error as a Selection Mechanism in Modular Reinforcement-Learning Systems.
Mark Ring and Tom Schaul
Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI-2011, Barcelona). [Pdf] [BibTeX] [Video]
GECCO[21] High Dimensions and Heavy Tails for Natural Evolution Strategies.
Tom Schaul, Tobias Glasmachers and Jürgen Schmidhuber.
Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2011, Dublin). [Pdf] [BibTeX]
CEC[20] Curiosity-driven Optimization.
Tom Schaul, Yi Sun, Daan Wierstra, Faustino Gomez and Jürgen Schmidhuber.
Proceedings of IEEE Congress on Evolutionary Computation, (CEC-2011, New Orleans). [Pdf] [BibTeX]

2010

PPSN[19] A Natural Evolution Strategy for Multi-Objective Optimization.
Tobias Glasmachers, Tom Schaul, and Jürgen Schmidhuber.
Proceedings of Parallel Problem Solving from Nature (PPSN-2010, Krakow). [Pdf] [BibTeX]
Scholarpedia[18] Metalearning.
Tom Schaul and Jürgen Schmidhuber.
Scholarpedia. 5(6):4650. [Link] [BibTeX]
GECCO[17] Exponential Natural Evolution Strategies.
Tobias Glasmachers, Tom Schaul, Yi Sun, Daan Wierstra and Jürgen Schmidhuber.
Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2010, Portland). [Pdf] [BibTeX]

Nominated for Best Paper Award

JMLR[16][W1] PyBrain.
Tom Schaul, Justin Bayer, Daan Wierstra, Yi Sun, Martin Felder, Frank Sehnke, Thomas Rückstieß and Jürgen Schmidhuber.
Journal of Machine Learning Research. [Pdf] [BibTeX]

Also presented at the ICML Workshop for Open Source ML Software (ICML-2010, Haifa) [Video]

JBR[15] Exploring Parameter Space in Reinforcement Learning.
Thomas Rückstieß, Frank Sehnke, Tom Schaul, Daan Wierstra, Yi Sun and Jürgen Schmidhuber.
Paladyn Journal of Behavioral Robotics. [Pdf] [BibTeX]
AGI[14] Frontier Search.
Yi Sun, Tobias Glasmachers, Tom Schaul and Jürgen Schmidhuber.
Proceedings of the Third Conference on Artificial General Intelligence (AGI-2010, Lugano). [Pdf] [BibTeX]

Kurzweil Best Paper Prize

AGI[13] Towards Practical Universal Search.
Tom Schaul and Jürgen Schmidhuber.
Proceedings of the Third Conference on Artificial General Intelligence (AGI-2010, Lugano). [Pdf] [BibTeX] [Presentation Video]
ICANN[12] Multi-Dimensional Deep Memory Go-Player for Parameter Exploring Policy Gradients.
Mandy Grüttner, Frank Sehnke, Tom Schaul and Jürgen Schmidhuber.
Proceedings of the International Conference on Artificial Neural Networks (ICANN-2010, Greece).[Pdf] [BibTeX]
NIM-A[11] Assessment of Neural Networks Training Strategies for Histomorphometric Analysis of Synchrotron Radiation Medical Images.
Anderson Alvarenga de Moura Meneses, Christiano Pinheiro, Paola Rancoita, Tom Schaul, Luca Gambardella, Roberto Schirru, Regina Barroso and Luís de Oliveira.
Nuclear Instruments and Methods in Physics Research, Section A. [Link] [BibTeX]

2009

GECCO[10] Efficient Natural Evolution Strategies.
Yi Sun, Daan Wierstra, Tom Schaul and Jürgen Schmidhuber.
Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2009, Montreal). [Pdf] [BibTeX]

Best Paper Award

ICML[9] Stochastic Search using the Natural Gradient.
Yi Sun, Daan Wierstra, Tom Schaul and Jürgen Schmidhuber.
Proceedings of the International Conference on Machine Learning (ICML-2009, Montreal). [Pdf] [BibTeX]
ICANN[8] Scalable Neural Networks for Board Games.
Tom Schaul and Jürgen Schmidhuber.
Proceedings of the International Conference on Artificial Neural Networks (ICANN-2009, Cyprus). [Pdf] [BibTeX]
KI[7] Ontogenetic and Phylogenetic Reinforcement Learning.
Julian Togelius, Tom Schaul, Daan Wierstra, Christian Igel, Faustino Gomez and Jürgen Schmidhuber.
Zeitschrift Künstliche Intelligenz - Special Issue on Reinforcement Learning. [Pdf] [BibTeX]

2008

IEEE-CIG[6] A Scalable Neural Network Architecture for Board Games.
Tom Schaul and Jürgen Schmidhuber.
Proceedings of the IEEE Symposium on Computational Intelligence in Games (CIG-2008, Perth). [Pdf] [BibTeX]
PPSN[5] Fitness Expectation Maximization.
Daan Wierstra, Tom Schaul, Jan Peters and Jürgen Schmidhuber.
Proceedings of Parallel Problem Solving from Nature (PPSN-2008, Dortmund). [Pdf] [BibTeX]
PPSN[4] Countering Poisonous Inputs with Memetic Neuroevolution.
Julian Togelius, Tom Schaul, Jürgen Schmidhuber and Faustino Gomez.
Proceedings of Parallel Problem Solving from Nature (PPSN-2008, Dortmund). [Pdf] [BibTeX]
CEC[3]

Natural Evolution Strategies.
Daan Wierstra, Tom Schaul, Jan Peters and Jürgen Schmidhuber.
Proceedings of IEEE Congress on Evolutionary Computation (CEC-2008, Hongkong). [Pdf] [BibTeX]

ICANN[2] Episodic Reinforcement Learning by Logistic Reward-Weighted Regression.
Daan Wierstra, Tom Schaul, Jan Peters and Jürgen Schmidhuber.
Proceedings of the International Conference on Artificial Neural Networks (ICANN-2008, Prague). [Pdf] [BibTeX]

2005

EPFL[1] Evolving a Compact Concept-based Sokoban Solver. Tom Schaul.
Masters thesis at the École Polytechnique Fédérale à Lausanne. [Pdf] [BibTeX]