Why does linear algebra elicit so much confusion despite being an elementary topic. I think part of the problem is that the connection between matrices and matrix operations and their applications is not explained well enough. Linear algebra is typically one's first foray into abstract mathematics, and if taught poorly sets one up for failure down the road.
I took LinAlg the same semester I took Computer Graphics, back in 2001. The first half of LinAlg was all about solving equations, transformation matrices and that sort of stuff, and the first half CG was all 2D drawing (Bresenham and such). Second half of CG was all about OpenGL, and I was able to apply all the stuff I'd just learned in LinAlg.
The second half of LinAlg was a bunch of stuff about Eigenthis and Eigenthat, with no particular explanation as to what you'd use it for or why you would care. I pass the class with an A and was even recommended by the instructor to be work as a tutor for other students, but 23 years later I couldn't tell you the use of any of that stuff.
T-matrices, though - I've used that stuff my whole career, working in medical imaging and related fields. I still couldn't tell you what an Eigenvalue or Eigenvector is useful for, as I never had any reason to apply that stuff.
It's not particularly abstract if you can explain why and how this stuff is used - and I know it's heavily used in many fields - just not those that I've worked in.
Eigenvectors characterise a linear transformation by answering a simple question: What lines map to themselves after the transformation?
Say for example you rotate something in 3D. The rotation axis remains unchanged. That's an eigenvector. Or say you mirror something in 3D, then all the lines lying in the mirror plane remain unchanged (all eigenvectors with eigenvalue 1), and the line orthogonal to the mirror plane remains unchanged - or rather, flipped, so it's an eigenvector with eigenvalue -1.
Yeah, eigenvectors and eigenvalues are abstract and best understood in terms of the linear mapping between vector spaces and change of basis, but change of basis for a vector space and writing a matrix in a new basis gets really complicated. Many people get lost there.
Because it's taught in the 1st year of most math courses and most people don't take it seriously and think that it's "easy 1st year math" when its actually the base for a whole lot of maths down the line.
I've actually stopped asking questions about Cayley-Hamilton and Jordan form (both covered in year 1 of any decent BS math course in Europe) when I used to do inteviews for trading/quant positions because so many people failed.
You can probably reduce it down to a single definition. As soon as matrix multiplication is introduced without careful motivation, almost everyone is lost. And those who aren’t should be.
It’s mostly because, as Axler explicitly tries to address, linear algebra is a subject that can be viewed through at least two different lenses: the intuitive, (possibly higher-dimensional) geometric lens, or the algorithmic and numerical lens. Of course they are equivalent, but almost every course in linear algebra teaches the second and barely even touches on the first. It’s like learning Euclid’s algorithm before you’ve learnt to count. No wonder everyone’s so confused.
> Why does linear algebra elicit so much confusion despite being an elementary topic
I'm not sure that linear algebra is an elementary topic, if the word "elementary" is as in math of elementary school. That said, I don't think there's much confusion about linear algebra either. It's more likely that linear algebra is substantially more abstract than high-school algebra, so many students have a hard time deeply understanding it, just like many students already bail out on high-school math. When I was a TA in college, I also observed that many students were not prepared for the fact that college-level math is fundamentally different from high-school ones. College-level maths has far less time for students to grasp the concepts through sheer brute-force practice. College-level maths requires students to focus on intuitive understanding of they key concepts so the students won't get bogged down by hundreds and hundreds of concepts or corollaries or theorems. Of course, high-school maths requires intuitive understanding too, but because high-school maths is so simple that many students get the intuition naturally so they are not aware of how important such intuitive understanding is.
This book used to help my students build intuitions: https://www.amazon.com/Algebra-Through-Geometry-Undergraduat.... It starts with 2D geometry to teach linear algebra, and then moves to 3D, and then 4D. The author also chooses to use calculations and constructions to prove most of the theorems instead of more abstract or algebraic approach.
Thinking back to high school and college, the biggest issue with I had math in both is how it's frequently taught in absence of practical examples. Speaking personally (though I believe there are many who feel similarly), there's a need to see real world examples to develop a true grasp on new concepts in any reasonable amount of time, for reasons related to both motivation and intuition.
> how it's frequently taught in absence of practical examples
Unfortunately this has been the debate for thousands of years. One of Euclid's students asked him what practical use geometry had. In response, Euclid instructed a servant to give the student a coin, saying, "He must make gain out of what he learns."
Legend aside, I'm actually surprised to see, many times actually, that people on HN criticized the "absence of practical examples". If we compare the textbooks written in the US and those in China or Europe, there's a sharp contrast. The textbooks from the US are thick, full of discussion of motivations and practical examples across multiple domains (Thomas' Calculus, for instance). In contrast, Chinese and European textbooks are terse and much thinner. They focus on proofs and derivations and have only sparse examples.
Personally, maths itself is practical enough. I'd even venture to say that those who can naturally progress to college-level maths should be able appreciate high-school maths for its own sake.
I agree with this in theory, but I don't necessarily need the "real world"-ness.
A lot of the stuff you learn initially, about systems of equations and how they relate to the matrix inverse, are very interesting because it's clear how this can be applied.
But as you move forward into vector bases and change of coordinates there's a very long dry spell that you sort of have to slog through until much later when you start to see how it is actually useful. I'm not sure how to fix this -- maybe take a step back from the symbolic and do some numeric computations because that's where they start becoming useful again.
IMHO there should be two versions of linear algebra. One for computer science majors and one for mathematicians. I regularly run into stuff at work where I say to myself, "self, this is a linear algebra problem" and I have next to no idea how to transform the problem I have into a matrices or whatever.
But I can write a really stinkin' fast matrix multiplication algorithm. So there's that I guess.
Modern CPUs with big ass SIMD registers are incredibly fast at slogging through a linear algebra problem. Lots of incredibly intelligent people (ie, not me) spend an incredible amount of effort optimizing every last FLOP out of their BLAS of choice. For several years the only question Intel asked itself when designing the next CPU was, "How much faster can we make the SPECfp benchmark?" and it shows. Any time you can convert a problem using whatever ad-hoc algorithm you came up with into a linear algebra problem, you can get absurd speedups. But most programmers don't know how to do that, because most of their linear algebra class was spent proving that the only invertible idempotent nxn matrix is the identity matrix or whatever.
Discrete math has the same problem. When I took discrete math in college, the blurb in the course catalog promised applications to computer science. It turns out the course was just mucking through literally dozens of trivial proofs of trivial statements in basic number and set theory, and then they taught us how to add two binary numbers together. The chapters on graphs, trees, recursion, big-O notation and algorithm analysis, finite automata? Skipped 'em.
Yes I’m currently dealing with text that has a line “you will end up with ~2^32 equations and from there it’s just a trivial linear algebra problem” without further guidance (from the general number field seive).
I get that 2^32 simultaneous equations may be a straightforward linear algebra problem but I am now going deep to understand the exact mechanism to solve this.
one issue that comes to mind is the nature of the determinant. when one considers the determinant defined by the recursive definition, it seems like a highly contrived object that is difficult to work with (as it is from that definition!). avoiding that confusion requires that a lot more scaffolding be built (ala Axler in the "Done Right" book). either way you have some work: either to untangle the meaning of the weird determinant or get to the place where you can understand the determinant as the product of the eigenvalues.
Do you know why introductory textbooks don't define the determinant in terms of the exterior product? This is how some "real" mathematicians I've talked to define it. It also is more intuitive (in my opinion) to define determinants as "signed volumes" than some sum of products multiplied by signs of cycles.
The product of eigenvalues definition is also somewhat intuitive to me ("How much does the matrix scale vectors in each direction? Now multiply those numbers together."), but it's harder to motivate the fact that adding rows together doesn't change the determinant, which is kind of important to actually computing the determinant.
It's hard to make the connection between matrices and a linear mapping between vector spaces. A lot of the time students in a linear algebra classes do not have a great grasp of what a function is, so when talking about vector spaces and linear maps between them they're lost. Then connecting that to a matrix is hopeless.
Linear algebra is definitely not elementary. There are layers and layers of linear algebra, and depending on the audience, you have to select which topics to emphasize.
