I get it, and I've actually used the same metaphor just a couple days ago. What I mean though, is that the maths are OK, and although the heavy notation goes over my head (as I'm not academic), I understand the ideas. But I'm lacking some practical examples of how this is put to use. I understand how neural networks work for example, but I do not understand the maths behind how they work. There was (is?) a site called ai-junkie which used (does?) have very practical and layman docs about A.I. before ML became the buzz word du jour.
Honestly, the best thing you can do is try to implement your own shitty neural net with only Python + Numpy, from scratch, with only a basic understanding of the math. It will make most of the math very concrete very fast.
I think this is the only way to really grok backpropagation. The hours of staring at the update formula till your eyes glaze over the subscripts and superscripts and the summations would not give you as good an understanding as implementing a toy neural net with just a single hidden layer. Its actually a whole lot easier than parsing those low-level notation. It can be done better with high level notation but then you would need familiarity with the relevant mathematical abstractions.
I respectfully disagree. Of course you need to really understand backpropagation for any advanced stuff, but it is easier to ignore it at the beginning. Take Keras container, copy some MNIST example from somewhere and tweak it. Then, when you have a general feel of how training NNs works, gradually learn about each concept - BP should of course be one of the first. By the time you get into math stuff it will probably make much more sense because you will understand how it applies to your case.
But I guess the approach depends on how you best learn, so there is no wrong answer. Just jump in!
> This strategy is equally effective for most things in life.
Not if we understand 'effective' to also mean 'cost & time effective'
You'd learn a lot by building a nuclear reactor from first principles, but it's not the most effective way to develop an intuition about how one operates.
I think you want to talk about whether the strategy is efficient, which I agree it is not. However, if you already tried understanding several general descriptions and it didn't work out, implementing something from scratch is an inefficient but effective way of really grokking it.
Some require a lot more understanding than others (for instance I'm not sure I'd be comfortable implementing a kernelized SVM from scratch, even though intuitively I know how it works) but basic neural networks (simple perceptron, simple feedforward network, simple recurrent network) are quite easy to grasp, and backpropagation is very intuitive. You can even use finite difference approximation [1] to bypass the derivatives when you're starting (at the cost of some efficiency) and figure out the rest as you go.
Oh I do understand you. The manual for driving a car has to be different from the manual for designing a car. I believe you are looking for a manual to drive the car where you don't really need a lot of visibility into the inner workings.
A problem is that ML is not quite as mature as a car yet, so the driving manuals will be a bit on the thinner / shallower side.
> I understand how neural networks work
Quickly write that down please, that would do the world a favor. Researchers are still grappling with the question 'why the hell does this freaking thing work as well as it does, when it does'.
Haha, I meant, the practical idea behind their usage. The example of OCR via a NN was very good in drilling that concept into my head (many inputs leading down to output).
The HOW (in capitals) they work bit - I'm not going there. This thread here got me searching and I'm reading through the basic TensorFlow docs. That's, so far, sinking in.
