AIMS course: Physics for Machine Learning

Information

• Lecturer: Prof. Hugo Touchette
• Lectures: Mon and Tues afternoons
• Tutorials: Thurs afternoons

References

• Basic references on probability theory:
• S. Ross, A First Course in Probability, Prentice-Hall, 2010.
• G. Grimmett and D. Stirzaker, Probability and Random Processes, OUP, 2001.
• K. Jacobs, Stochastic Processes for Physicists, CUP, 2010.
• Main references on MC:
• J. S. Liu, Monte Carlo Strategies in Scientific Computing, Springer, 2001.
• C. A. L. Bailer-Jones, Practical Bayesian Inference, CUP, 2017.
• A. Guyader, Méthodes Monte Carlo, Lecture notes, Sorbonne Université, 2022.
• Others:
• S. Asmussen, P. W. Glynn, Stochastic Simulation, Springer, 2010.
• D. J. C. MacKay, Information Theory, Inference, and Learning Algorithms, CUP, 2003.
• K. P. Murphy, Machine Learning: A Probabilistic Perspective, MIT, 2012. Chaps 23, 24.

Primary notes

From the SU course: Monte Carlo Methods for ML (ML822)

• Chapter 1: Probability theory and sampling
• Chapter 2: Markov chain Monte Carlo
• Chapter 3: Stochastic approximations

Secondary notes

From the SU course: Applied Markov Processes (AM783)

• Chapter 1: Probability theory and sampling
• Chapter 2: Markov chains
• Chapter 4: Brownian motion
• Chapter 5: Stochastic differential equations

Courseworks

• CW1: Probability theory and sampling: pdf
• CW2: Markov chain Monte Carlo and simulated annealing: pdf
• CW3: Optimization: pdf

Demos

• List of Python commands (cheat sheet)
• Histograms in Python: pdf | ipynb
• Statistical estimation: pdf | ipynb
• Limit theorems: pdf | ipynb
• MCMC for sampling Gaussians: pdf | ipynb
• Solving ODEs and SDEs: pdf | ipynb

Extras

• Paper of Metropolis et al.
• Hastings's 1970 paper
• Stationary distribution for MCMC: See Arizona notes or Rochester notes

Tutorials

• Introduction to Python with Jupyter by Maxime Sangnier, Paris University
• Plotting in Python by Maxime Sangnier, Paris University
• Scientific computing in Python by Maxime Sangnier, Paris University
• Python Numpy tutorial in Jupyter or Colab by Justin Johnson