I am a Phd student interested in optimization applied to (deep) neural networks, under the supervision of Pascal Vincent at MILA. This website is intended at sharing some notes or short articles that I wrote during my exploration of deep learning.
For a list of publications, please refer to scholar.
- Fast Approximate Natural Gradient Descent in a Kronecker-factored Eigenbasis - NIPS 2018 - Poster - Pytorch code
- Derivatives through a batch norm layer (15 Feb 2019)
- What is the empirical Fisher ? (09 Nov 2018)
- How to compute the Fisher of a conditional when applying natural gradient to neural networks? (29 Oct 2018)
- The algebra of second order methods in neural networks (29 Oct 2017)