On: Mistakes Programmers Make when Starting in Machine Learning

[grovulent]

Advice question - how far into the linear algebra should I go? I'm currently working my way through a book - and it's not too bad, but I really don't want to invest more than I really need. Any suggestions?

[agibsonccc → grovulent]

The main thing to understand is the basics. For example element wise multiplication vs full blown matrix operations. Think something like this: http://en.wikipedia.org/wiki/Matrix_multiplication

Matlab/Octave is a great way to practice this due to the native data types. If python is your thing numpy's arrays are also pretty easy to digest.

Subtle little tricks like this: https://www.youtube.com/watch?v=evF-3ykjRU0

And understanding the dynamics of scalar operations vs matrix - vector operations.

The machine learning class has some good fundamentals if you need a refresher on how something works.

There will be more complex things like some optimization algorithms have different uses for eigen values: http://see.stanford.edu/materials/lsocoee364b/11-conj_grad_s...

See: http://en.wikipedia.org/wiki/Eigenvalues_and_eigenvectors

One other thing might be understanding different ways you can manipulate data. In this case, numerical representation here is an example per row when I toss in one matrix for training. This is applicable to many machine learning problems.

[grovulent → agibsonccc]

Very much appreciated. That's really helpful and feels very doable. :)

[fretless → grovulent]

You could look at the appendixes in texts by Bishop, MacKay, Barber and Rasmussen/Williams to see what they expect (and they expect a pretty thorough understanding. The last 3 are freely available content

http://www.inference.phy.cam.ac.uk/itila/book.html

http://web4.cs.ucl.ac.uk/staff/D.Barber/textbook/090310.pdf

http://www.gaussianprocess.org/gpml/chapters/RW.pdf

Also, an excellent list of ML resources: http://www.reddit.com/r/MachineLearning/comments/1jeawf/mach...