The original linked text is probably better for a linear algebra learner if they want to apply it (especially if they want to apply it earlier rather than later). Your linked text looks (I've read the first 5 parts) to be a good introduction but at a higher level of abstraction (category theory and abstract algebra). In fact, I think it may present a good motivating case for understanding that level of higher abstraction (while it's ostensibly about linear algebra, the ideas transfer with some adjustments to other algebras).
It could also use some polish. It's awkward to be starting a new "episode" and find that it's two pages of off-topic discussion (presenting the case for the blog, which would serve better in an introduction than in the middle of the material).
Where is the text? Or is this an advertisement for a book?
Since this might get the right eyeballs here:
1. Does anyone have a good intuitive explanation for the meaning of the transpose operation? It seems to tie into a central symmetry between columns and rows (or equations and variables) that I'm having trouble articulating for myself
2. Why does every linear algebra book have a chapter on norms? It's always so incredibly boring and esoteric and I still haven't hit a situation where it's strictly necessary. The 2-norm seem adequate for most situations and comes out naturally from the least squares problem
PS: Strang is great. Meyer's "Matrix analysis and applied linear algebra" is even better (but more advanced)
- Let <-,-> denote an inner product. Then <Au, v> = <u, A^T v> is a definition of the transpose operation. Proof: (Au)^T v = u^T A^T v = <u, A^T v>
- The transpose maps a vector v to a linear functional (a kind of function) v^T, such that v^T (u) = <v, u>. Conversely, a linear functional of a vector can be transposes to a vector. In fact, it's the latter (linear functionals to vectors) that's more interesting. This fact generalises to Hilbert spaces as the Riesz representation theorem, where the consequences can be pretty amazing.
- An arbitrary linear operator L: H -> K tranposes to a mapping between their dual spaces L^* = K^* -> H^* by L*(f) = f o L.
- The outer product produces rank-one matrices, and all rank-one matrices can be expressed as outer products.
So the transpose is really ties to the whole idea of inner product operators/function? (I'm not sure what the exact terminology is there)
So thinking out loud a bit about the first interpretation... The inner to me represents an operator that maps vectors to a scalar that denotes the degree of "sameness" between two vectors. And it's a whole family of operators that all share certain properties. It just so happens that to write out this operator in matrix notation (for the standard inner product) we need to transpose/rotate the first vector which is yucky and why we prefer the bracket notation
It's a cool insight but it just makes it seem like a mechanical operator that flips a column into a row to make things work for one family of operators. I'm thinking more like in the case of an orthonormal matrix/basis. We know the transpose is it's inverse. It stems from the standard inner products of the columns being equal to one which is I find kinda interesting b/c the inner products suddenly become part of linear systems and not just a separate measurement of a degree of sameness.. but the headache for me is that I can picture an orthonormal basis, but I can't picture it's inverse! :)
PS: The orthonormal basis case and the transpose by extension further makes me feel that only the standard inner product seems to really map to something directly applicable in linear systems
PPS: The 3rd bullet is a bit beyond me. And the 4th is also very interesting but I'm not sure how to expand on that to build an intuition haha. I'll need to digest it further
For one example the notion of a generalized inner product and its induced norm is important for the derivation of conjugate gradient algorithm. My smarter friends have told me that anything involving cost functions and covariance matrices (as might arise in a log likelihood thing for a multivariate normal random variable) will probably end up with weighted norms as well, but I have likely mangled this. So to mangle things further I think that this is very important to many fields eg control or statistics
As for 2, norms (and general inner products) show up in more theoretical parts of math that use linear algebra. Maybe an applications-focused treatment doesn't need to focus so much on it?
Look for the word "Next" to find the button that takes you to the next page. Even though this is arguably the most important thing on the page, it is rendered in a very small font, and styled to not look like a link. Can't say I understand this design decision.
i mean, probably the sexiest application of linear algebra is machine learning and different exotic norms are very important there. sometimes you even have to learn a norm from data (metric learning).
at the very least i feel like it’s important to understand 0, 1, 2, and infinity norm.
Looks like some great information but you need to change the way you style the links to the different chapters. The links look like ordinary text and I had no idea that I could click them until I accidentally hovered over one.
Thank you for doing this. I'm currently working with deep learning but have never had Linear Algebra in school. Resources like this one, and the 3Blue1Brown video series on YouTube are a really nice way to get started.
3B1B is by far the best resource on LA anywhere in the Web. I might be chastice for saying it but i will go as far to say that 3B1B is better than MIT's LA.
This is just one example of why it is usually better to learn stuff from more authoritative sources. (On the other hand, I can’t help but recommend this amazing video: https://m.youtube.com/watch?v=CliW7kSxxWU.)
Certainly you would want a more abstract follow up for many cases. Even engineering or computer science students may have to learn about e.g. inner product spaces and orthogonal polynomials for their numerics class.
Obligatory remark that "reading" a math textbook, and even moreso a basic one, is an active task; you can get the intuitions from youtube videos of the 3Blue1Brown variety, but to really learn you need to follow the text closely, put question marks on passages you're not 120% comfortable with, do the proofs that are left to the reader and the recommended exercises. If you're super comfortable with the material, you should do the more challenging exercises, usually marked with asterisks.
Basically you need a pencil, a lot of scratch paper and a capable waste basket; or a whiteboard all to yourself (but I get tired from being on my feet for hours on end).
----
E: My litmus test is the Gershgorin circle theorem. Every so often I check if I can still prove it, and if I do I try to review the surrounding matrix. I will literally vote against hiring anyone who can't at least handwave why that's true.
>My litmus test is the Gershgorin circle theorem. Every so often I check if I can still prove it, and if I do I try to review the surrounding matrix. I will literally vote against hiring anyone who can't at least handwave why that's true.
Interesting comment with no context—i.e. not hire someone for what? I don't know if I would ever not hire someone for not understanding a math problem (unless the job was heavily math related in the sphere of the problem). I would definitely not hire someone who communicates as ineffectively as you did in this comment.
