> So while this looks like a great resource - Who's it for?
Undergraduates, or graduate students who didn't happen to take the right prerequisites. Most STEM degrees require vector calculus, but few require matrix calculus. A physics undergrad might see matrix calculus if they studied general relativity, or math undergrad interested in optimization or differential geometry. A statistics major might have seen it when working with multivariate distributions and regression. But it would be easy to miss.
Nevertheless, matrix calculus, which is not in fact a large subject, but only some new notation and a handful of theorems, is the key to understanding back-propagation. It's not the only way to approach it - you could just keep track of all those subscripts and indices - but it's one of the best. The differential form[1] is particularly good to learn because it maps almost 1-1 onto the error terms in a gradient descent implementation.
> So it’s for people that have been getting into DL but also haven’t bothered or needed to look up the math concepts?
Everyone has to start somewhere. The usual pedagogical technique to teach a subject twice: once at an "undergraduate" level, omitting the technical details of proofs, with the goal of providing a big picture intuitive understanding of the subject and some practical symbol pushing ability; then again at the "graduate" level, with more formal definitions and detailed proofs. Your own education presumably used this structure, no? Even if you've already graduated, this "two pass" approach to learning new material is still a good idea. Few of us are von Neumann, able to dive immediately into the deepest depths of theory in a new field: we can all benefit from taking the time to develop some good intuitions first.
This is where all textbooks come from - a lecturer presents the material the way that seems clearest to them. They prepare notes to keep everything straight in their own head. Sometimes they find that their presentation resonates with students and is superior to what's currently available, so they start to develop their notes into something publishable. Most such projects get abandoned before too long, but many end up in some form on the internet, and a few go on to be developed into standard texts. As long as you can find even one new way to explain things that helps students, the exercise is not in vain.
> Most STEM degrees require vector calculus, but few require matrix calculus
Gradients, Jacobians, etc are typically covered in a multivariable calculus class along with vector calculus (line integrals, Green's theorem, Stoke's theorem, etc).
This is required for engineering and physics degrees.
Undergraduates, or graduate students who didn't happen to take the right prerequisites. Most STEM degrees require vector calculus, but few require matrix calculus. A physics undergrad might see matrix calculus if they studied general relativity, or math undergrad interested in optimization or differential geometry. A statistics major might have seen it when working with multivariate distributions and regression. But it would be easy to miss.
Nevertheless, matrix calculus, which is not in fact a large subject, but only some new notation and a handful of theorems, is the key to understanding back-propagation. It's not the only way to approach it - you could just keep track of all those subscripts and indices - but it's one of the best. The differential form[1] is particularly good to learn because it maps almost 1-1 onto the error terms in a gradient descent implementation.
[1]: https://en.wikipedia.org/wiki/Matrix_calculus#Identities_in_...
> So it’s for people that have been getting into DL but also haven’t bothered or needed to look up the math concepts?
Everyone has to start somewhere. The usual pedagogical technique to teach a subject twice: once at an "undergraduate" level, omitting the technical details of proofs, with the goal of providing a big picture intuitive understanding of the subject and some practical symbol pushing ability; then again at the "graduate" level, with more formal definitions and detailed proofs. Your own education presumably used this structure, no? Even if you've already graduated, this "two pass" approach to learning new material is still a good idea. Few of us are von Neumann, able to dive immediately into the deepest depths of theory in a new field: we can all benefit from taking the time to develop some good intuitions first.
This is where all textbooks come from - a lecturer presents the material the way that seems clearest to them. They prepare notes to keep everything straight in their own head. Sometimes they find that their presentation resonates with students and is superior to what's currently available, so they start to develop their notes into something publishable. Most such projects get abandoned before too long, but many end up in some form on the internet, and a few go on to be developed into standard texts. As long as you can find even one new way to explain things that helps students, the exercise is not in vain.