A Step by Step Backpropagation Example

[shawndumas]

http://mattmazur.com/2015/03/17/a-step-by-step-backpropagation-example/

[discardorama]

Every such example stops at vanilla backprop.

What I'd like to see (and which I think would be a lot more challenging) is: BPTT for a recurrent network. BP for an LSTM or GRU. Can someone do that?

[jmount]

Backprop is fun. You may also enjoy a look at the generalization: reverse accumulation http://www.win-vector.com/blog/2010/07/gradients-via-reverse... .