notes

old and recent short notes

Derivatives through a batch norm layer

7 min read · February 15, 2019

2019 · note
What is the empirical Fisher ?

3 min read · November 09, 2018

2018 · note
How to compute the Fisher of a conditional when applying natural gradient to neural networks?

4 min read · October 29, 2018

2018 · note
The algebra of second order methods in neural networks

12 min read · October 29, 2017

2017 · note
Demystifying Natural Neural Networks

10 min read · May 23, 2017

2017 · note