Theses

TU Munich[26] Studies in Continuous Black-box Optimization. Tom Schaul.
Ph.D. Thesis at Technische Universität München. 2011. [Link] [Pdf] [BibTeX] [Paperback at lulu.com]

Summa cum laude

EPFL[1] Evolving a Compact Concept-based Sokoban Solver. Tom Schaul.
Masters thesis at the École Polytechnique Fédérale à Lausanne. 2005. [Pdf] [BibTeX]

Peer-reviewed Conferences

ICLR [47] Reinforcement Learning with Unsupervised Auxiliary Tasks.
Max Jaderberg*, Volodymyr Mnih*, Wojciech Czarnecki*, Tom Schaul, Joel Leibo, David Silver, Koray Kavukcuoglu.
Proceedings of the International Conference on Learning Representations (ICLR-2017, Toulon). [arXiv] [openreview]
NIPS[46] Unifying Count-Based Exploration and Intrinsic Motivation.
Marc G. Bellemare, Sriram Srinivasan, Georg Ostrovski, Tom Schaul, David Saxton and Rémi Munos.
Proceedings of the Neural Information Processing Systems (NIPS-2016) [arXiv] [Video (100k+ views)]
NIPS [45] Learning to learn by gradient descent by gradient descent.
Marcin Andrychowicz, Misha Denil, Sergio Gomez, Matthew W. Hoffman, David Pfau, Tom Schaul and Nando de Freitas.
Proceedings of the Neural Information Processing Systems (NIPS-2016) [arXiv]
IEEE-CIG[44] Analyzing the Robustness of General Video Game Playing Agents.
Diego Perez-Liebana, Spyridon Samothrakis, Julian Togelius, Tom Schaul and Simon Lucas.
Proceedings of the IEEE Conference on Computational Intelligence in Games (CIG-2016, Greece). [Pdf] [BibTeX]
ICML[43] Dueling Network Architectures for Deep Reinforcement Learning.
Ziyu Wang, Tom Schaul, Matteo Hessel, Hado van Hasselt, Marc Lanctot, Nando de Freitas.
Proceedings of the International Conference on Machine Learning (ICML-2016, New York). [arXiv] [BibTeX]

Best Paper Award

ICLR[42] Prioritized Experience Replay.
Tom Schaul, John Quan, Ioannis Antonoglou and David Silver.
Proceedings of the International Conference on Learning Representations (ICLR-2016, Puerto Rico). [arXiv] [BibTeX]
ICML[41] Universal Value Function Approximators.
Tom Schaul, Daniel Horgan, Karol Gregor and David Silver.
Proceedings of the International Conference on Machine Learning (ICML-2015, Lille). [Pdf] [BibTeX]
ICLR[38] Unit Tests for Stochastic Optimization.
Tom Schaul, Ioannis Antonoglou and David Silver.
Proceedings of the International Conference on Learning Representations (ICLR-2014, Banff, Canada). [BibTeX] [arXiv] [Code] [Public reviews]
IEEE-CIG[34] A Video Game Description Language for Model-based or Interactive Learning.
Tom Schaul.
Proceedings of the IEEE Conference on Computational Intelligence in Games (CIG-2013, Niagara Falls, Canada). [Pdf] [Code] [BibTeX]

Runner-up to Best Paper Award

ICML[33] No more Pesky Learning Rates.
Tom Schaul, Sixin Zhang and Yann LeCun.
Proceedings of the International Conference on Machine Learning (ICML-2013, Atlanta GA). [arXiv] [Pdf] [Supplementary material] [BibTeX]
IJCAI[32] Better Generalization with Forecasts.
Tom Schaul and Mark Ring.
Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI-2013, Beijing, China). [Pdf] [BibTeX]
Basis for patent [P1].
ICLR[31] Adaptive Learning Rates and Parallelization for Stochastic, Sparse, Non-smooth Gradients.
Tom Schaul and Yann LeCun.
Proceedings of the International Conference on Learning Representations (ICLR-2013, Scottsdale AZ). [Pdf] [BibTeX] [Public reviews]
GECCO[30] A Linear Time Natural Evolution Strategy for Non-Separable Functions.
Yi Sun, Faustino Gomez, Tom Schaul and Jürgen Schmidhuber.
Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2013, Amsterdam). [arXiv] [BibTeX]
ICDL[28] The Organization of Behavior into Temporal and Spatial Neighborhoods.
Mark Ring and Tom Schaul.
Proceedings of the International Conference on Developmental Learning (ICDL-2012, San Diego). [Pdf] [BibTeX]
GECCO[27] Natural Evolution Strategies Converge on Sphere Functions.
Tom Schaul. Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2012, Philadelphia). [Pdf] [BibTeX]
ICDL[25] The Two-Dimensional Organization of Behavior.
Mark Ring, Tom Schaul and Jürgen Schmidhuber.
Proceedings of the International Conference on Developmental Learning (ICDL-2011, Frankfurt). [Pdf] [BibTeX]
AGI[24] Coherence Progress: A Measure of Interestingness Based on Fixed Compressors
Tom Schaul, Leo Pape, Tobias Glasmachers, Vincent Graziano and Jürgen Schmidhuber.
Proceedings of the Fourth Conference on Artificial General Intelligence (AGI-2011, Mountain View). [Pdf] [BibTeX] [Video]
IJCAI[22] Q-error as a Selection Mechanism in Modular Reinforcement-Learning Systems.
Mark Ring and Tom Schaul.
Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI-2011, Barcelona). [Pdf] [BibTeX] [Video]
GECCO[21] High Dimensions and Heavy Tails for Natural Evolution Strategies.
Tom Schaul, Tobias Glasmachers and Jürgen Schmidhuber.
Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2011, Dublin). [Pdf] [BibTeX]
CEC[20] Curiosity-driven Optimization.
Tom Schaul, Yi Sun, Daan Wierstra, Faustino Gomez and Jürgen Schmidhuber.
Proceedings of IEEE Congress on Evolutionary Computation, (CEC-2011, New Orleans). [Pdf] [BibTeX]
PPSN[19] A Natural Evolution Strategy for Multi-Objective Optimization.
Tobias Glasmachers, Tom Schaul, and Jürgen Schmidhuber.
Proceedings of Parallel Problem Solving from Nature (PPSN-2010, Krakow). [Pdf] [BibTeX]
GECCO[17] Exponential Natural Evolution Strategies.
Tobias Glasmachers, Tom Schaul, Yi Sun, Daan Wierstra and Jürgen Schmidhuber.
Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2010, Portland). [Pdf] [BibTeX]

Nominated for Best Paper Award

AGI[14] Frontier Search.
Yi Sun, Tobias Glasmachers, Tom Schaul and Jürgen Schmidhuber.
Proceedings of the Third Conference on Artificial General Intelligence (AGI-2010, Lugano). [Pdf] [BibTeX]

Kurzweil Best Paper Prize

AGI[13] Towards Practical Universal Search.
Tom Schaul and Jürgen Schmidhuber.
Proceedings of the Third Conference on Artificial General Intelligence (AGI-2010, Lugano). [Pdf] [BibTeX] [Presentation Video]
ICANN[12] Multi-Dimensional Deep Memory Go-Player for Parameter Exploring Policy Gradients.
Mandy Grüttner, Frank Sehnke, Tom Schaul and Jürgen Schmidhuber.
Proceedings of the International Conference on Artificial Neural Networks (ICANN-2010, Greece).[Pdf] [BibTeX]
GECCO[10] Efficient Natural Evolution Strategies.
Yi Sun, Daan Wierstra, Tom Schaul and Jürgen Schmidhuber.
Proceedings of the Genetic and Evolutionary Computation Conference (GECCO-2009, Montreal). [Pdf] [BibTeX]

Best Paper Award

ICML[9] Stochastic Search using the Natural Gradient.
Yi Sun, Daan Wierstra, Tom Schaul and Jürgen Schmidhuber.
Proceedings of the International Conference on Machine Learning (ICML-2009, Montreal). [Pdf] [BibTeX]
ICANN[8] Scalable Neural Networks for Board Games.
Tom Schaul and Jürgen Schmidhuber.
Proceedings of the International Conference on Artificial Neural Networks (ICANN-2009, Cyprus). [Pdf] [BibTeX]
IEEE-CIG[6] A Scalable Neural Network Architecture for Board Games.
Tom Schaul and Jürgen Schmidhuber.
Proceedings of the IEEE Symposium on Computational Intelligence in Games (CIG-2008, Perth). [Pdf] [BibTeX]
PPSN[5] Fitness Expectation Maximization.
Daan Wierstra, Tom Schaul, Jan Peters and Jürgen Schmidhuber.
Proceedings of Parallel Problem Solving from Nature (PPSN-2008, Dortmund). [Pdf] [BibTeX]
PPSN[4] Countering Poisonous Inputs with Memetic Neuroevolution.
Julian Togelius, Tom Schaul, Jürgen Schmidhuber and Faustino Gomez.
Proceedings of Parallel Problem Solving from Nature (PPSN-2008, Dortmund). [Pdf] [BibTeX]
CEC[3]

Natural Evolution Strategies.
Daan Wierstra, Tom Schaul, Jan Peters and Jürgen Schmidhuber.
Proceedings of IEEE Congress on Evolutionary Computation (CEC-2008, Hongkong). [Pdf] [BibTeX]

ICANN[2] Episodic Reinforcement Learning by Logistic Reward-Weighted Regression.
Daan Wierstra, Tom Schaul, Jan Peters and Jürgen Schmidhuber.
Proceedings of the International Conference on Artificial Neural Networks (ICANN-2008, Prague). [Pdf] [BibTeX]

Journals

T-CIAIG[40] The 2014 General Video Game Playing Competition.
Diego Perez, Spyridion Samothrakis, Julian Togelius, Tom Schaul, Simon Lucas, Adrien Couetoux, Jerry Lee, Chong-U Lim and Tommy Thompson.
IEEE Transactions on Computational Intelligence and AI in Games. 2015. [Pdf] [BibTeX]
T-CIAIG[39] An Extensible Video Game Description Language.
Tom Schaul.
IEEE Transactions on Computational Intelligence and AI in Games. 2014. [Pdf] [BibTeX] [Code]
JMLR[37] Natural Evolution Strategies.
Daan Wierstra, Tom Schaul, Tobias Glasmachers, Yi Sun, Jan Peters and Jürgen Schmidhuber.
Journal of Machine Learning Research. 2014. [Pdf] [BibTeX] [arXiv]
Dagstuhl[36] General Video Game Playing.
John Levine, Clare Bates Congdon, Michal Bida, Marc Ebner, Graham Kendall, Simon Lucas, Risto Miikkulainen, Tom Schaul and Tommy Thompson. Dagstuhl Follow-up, volume 6. 2013. [BibTeX] [Preprint]
Dagstuhl[35] Towards a Video Game Description Language.
Marc Ebner, John Levine, Simon Lucas, Tom Schaul, Tommy Thompson and Julian Togelius. Dagstuhl Follow-up, volume 6. 2013. [BibTeX] [Preprint]
Acta Futura[23] Artificial Curiosity for Autonomous Space Exploration.
Vincent Graziano, Tobias Glasmachers, Tom Schaul, Leo Pape, Giuseppe Cuccu, Jürgen Leitner and Jürgen Schmidhuber.
Acta Futura. 2011. [Pdf] [BibTeX] [Link]
Scholarpedia[18] Metalearning.
Tom Schaul and Jürgen Schmidhuber.
Scholarpedia. 5(6):4650. 2010. [Link] [BibTeX]
JMLR[16] PyBrain.
Tom Schaul, Justin Bayer, Daan Wierstra, Yi Sun, Martin Felder, Frank Sehnke, Thomas Rückstieß and Jürgen Schmidhuber.
Journal of Machine Learning Research. 2010. [Pdf] [BibTeX]
JBR[15] Exploring Parameter Space in Reinforcement Learning.
Thomas Rückstieß, Frank Sehnke, Tom Schaul, Daan Wierstra, Yi Sun and Jürgen Schmidhuber.
Paladyn Journal of Behavioral Robotics. 2010. [Pdf] [BibTeX]
NIM-A[11] Assessment of Neural Networks Training Strategies for Histomorphometric Analysis of Synchrotron Radiation Medical Images.
Anderson Alvarenga de Moura Meneses, Christiano Pinheiro, Paola Rancoita, Tom Schaul, Luca Gambardella, Roberto Schirru, Regina Barroso and Luís de Oliveira.
Nuclear Instruments and Methods in Physics Research, Section A. 2010. [Link] [BibTeX]
KI[7] Ontogenetic and Phylogenetic Reinforcement Learning.
Julian Togelius, Tom Schaul, Daan Wierstra, Christian Igel, Faustino Gomez and Jürgen Schmidhuber.
Zeitschrift Künstliche Intelligenz - Special Issue on Reinforcement Learning. 2009. [Pdf] [BibTeX]

Patents

US Patent[P2] Selecting reinforcement learning actions using goals and observations.
Tom Schaul, Dan Horgan, Karol Gregor and David Silver.
United States Patent Application 20160292568. 2016.
Based on [41].
US Patent[P1] Method for Creating Predictive Knowledge Structures from Experience in an Artificial Agent.
Mark Ring and Tom Schaul.
United States Patent Application 20160012338. 2016. [Link] [BibTeX]

Book chapters

Springer[29] Optimization with Surrogate Models. Tom Schaul.
Chapter 3 in Numerical Methods for Metamaterial Design, edited by Kenneth Diest. 2013. [Link] [BibTeX]

Workshops and Abstracts

AAAI-WH[W14] General Video Game AI: Competition, Challenges and Opportunities.
Diego Perez-Liebana, Spyridon Samothrakis, Julian Togelius, Tom Schaul and Simon Lucas.
AAAI What's Hot Track (2016, Phoenix). [Pdf]
Barbados[W13] Universal Value Function Approximators.
Tom Schaul, Daniel Horgan, Karol Gregor and David Silver.
9th Barbados Workshop on Reinforcement Learning (2015, Holetown, Barbados). [Slides]
Barbados[W12] Better Generalization with Forecasts.
Tom Schaul and Mark Ring.
8th Barbados Workshop on Reinforcement Learning (2013, Holetown, Barbados). [Slides]
AAAI-LML[W11] Organizing Behavior into Temporal and Spatial Neighborhoods.
Mark Ring and Tom Schaul. AAAI Spring Symposium on Lifelong Machine Learning (AAAI-LML-2013, Stanford University, California). [Pdf]
NIPS-OPT[W10] No More Pesky Learning Rates.
Tom Schaul, Sixin Zhang and Yann LeCun. NIPS Workshop on Optimization for Machine Learning (2012).
NYC-ML[W9] Adaptive Learning Rates for Stochastic Gradients.
Tom Schaul, Sixin Zhang and Yann LeCun. New York City Machine Learning Symposium (2012).
BBOB[W8] Comparing Natural Evolution Strategies to BIPOP-CMA-ES on Noiseless and Noisy Black-box Optimization Testbeds.
Tom Schaul. Black-box Optimization Benchmarking: GECCO Workshop for Real-Parameter Optimization (2012). [Pdf] [BibTeX]
BBOB[W7] Investigating the Impact of Adaptation Sampling in Natural Evolution Strategies on Black-box Optimization Testbeds.
Tom Schaul. Black-box Optimization Benchmarking: GECCO Workshop for Real-Parameter Optimization (2012). [Pdf] [BibTeX]
BBOB[W6] Benchmarking Separable Natural Evolution Strategies on the Noiseless and Noisy Black-box Optimization Testbeds.
Tom Schaul. Black-box Optimization Benchmarking: GECCO Workshop for Real-Parameter Optimization (2012). [Pdf] [BibTeX]
BBOB[W5] Benchmarking Exponential Natural Evolution Strategies on the Noiseless and Noisy Black-box Optimization Testbeds.
Tom Schaul. Black-box Optimization Benchmarking: GECCO Workshop for Real-Parameter Optimization (2012). [Pdf] [BibTeX]
BBOB[W4] Benchmarking Natural Evolution Strategies with Adaptation Sampling on the Noiseless and Noisy Black-box Optimization Testbeds.
Tom Schaul. Black-box Optimization Benchmarking: GECCO Workshop for Real-Parameter Optimization (2012). [Pdf] [BibTeX]
Snowbird[W3] Decoupling the Data Geometry from the Parameter Geometry for Stochastic Gradients.
Tom Schaul, Sixin Zhang and Yann LeCun. Snowbird Learning Workshop (2012). [Pdf] [BibTeX]
AAAI[W2] The Two-Dimensional Organization of Behavior.
Mark Ring, Tom Schaul and Jürgen Schmidhuber. AAAI Workshop on Lifelong Learning from Sensorimotor Experience (AAAI-2011, San Francisco). [Pdf]
ICML[W1] PyBrain.
Tom Schaul, Justin Bayer, Daan Wierstra, Yi Sun, Martin Felder, Frank Sehnke, Thomas Rückstieß and Jürgen Schmidhuber.
ICML Workshop on Machine Learning Open Source (ICML-2010, Haifa). [Pdf] [Video]

Technical reports

(not otherwise published)
arXiv[T1] Measuring Intelligence through Games.
Tom Schaul, Julian Togelius and Jürgen Schmidhuber. [arXiv]