It seems like college Linear Algebra expeiences fall into one of two buckets: either you took a "Linear Algebra / Differential Equations" class where you learned no theory and saw a bunch of practical examples and did problem sets that were routinely north of 10 pages, OR you took a highly theoretical course where everything was far too abstract for your first year of college level of comprehension.
I took the first route and retained none of the information, despite doing well in the course. Then I took computer graphics and cried all the way home.
I found Linear Algebra Done Right by Axler to be a fantastic theoretical text that makes everything click together from day 1. It's derived from his paper Down with Determinants.
He starts by explaining that linear algebra is about studying the properties of linear transformations over vector spaces, and then goes on to explain what this means. In many theoretical books it takes ages to get this message, and in some practical ones you get lost in a sea of matrices without even understanding what a matrix actually is. Furthermore all his proofs are very slick.
Seconded. I was very skeptical of LADR when I was forced to use it freshman year. I thought I already knew all the practical stuff from Strang's book, but learning linear algebra from the mathy angle has paid off again and again and again. Choices of units & coordinate systems are impossibly confusing without having a foundation in how to reason independently of such arbitrary choices and add them in later.
I think the turning point came when I realized that "abstract" linear algebra was actually much closer to physical reality than "applied" linear algebra. Superficially, it seems like the exact opposite state of affairs from computer engineering, where grids of numbers take you closer to "bare metal" and "higher level reasoning" takes you further away. In math, it's the grids of numbers that are further removed from "bare metal" and the abstract reasoning that you can count on when all else fails!
I didn't need to be convinced that I would eventually come to regret having an insecure foundation so much as I was mixed up about up vs down and trying to build a foundation in the middle of the sky.
I'd be quite interested to keep in touch with some people studying these topics, even if we don't follow the same pace or curriculum.
Despite having studied a decent dose of math during my undergrad (and currently during my grad), I keep on finding awesome books like Axler, Hubbard, Halmos, Neuveu... that make revisiting math so much more illuminating. Everything clicks much more quickly and I never loose momentum now.
Some of these books I found online, so I'd definitely like to exchange more references, problems, tips & experiences.
I can't recommend this book more. I read through about half of it before I took my first real course on linear algebra. The result? I got literally 100% in that class -- I didn't miss a single point on anything the entire semester. The teacher was flabbergasted, but I had my secret weapon -- Axler. The rest of the class was relying on the course textbook, some awful fat thing that I'm pretty sure nobody could ever really learn from. Axler's book made the concepts so crystal clear to me that it felt as if I'd known them since birth.
I can't guarantee that everyone is going to have the same experience, but yeah, it's a good book.
Wow, really? I mean, I got an A- in linear algebra with some normal old textbook I don't even remember, and certainly learned it all in terms of linear transformations over vector spaces... but then again, I didn't have it as a doubled class with differential equations in the first place. Is Axler that good?
It was partially a fluke I'm sure. I'm notoriously sloppy with arithmetic and no book could save me from that.
It's not necessarily that Axler is that good. It's that it's just solidly good all throughout. It has the right focus. The goal of the book was very clearly to explain linear algebra, not to make money or get contract deals with college bookstores. And the proofs are very clearly written.
I'm sure there are other books out there which are just as good as Axler. But compared to the more popular linear algebra textbooks, at least at the time, it seemed like it was in a class of its own.
I don't have any particularly noteworthy recommendations for ordinary analysis or real analysis, but for complex analysis I did find that Needham's Visual Complex Analysis was just as revolutionary for me as Linear Algebra Done Right (if not more so).
In my experience, Linear Algebra Done Right by Axler is a great second book on linear algebra. But it makes sense to most students only if they've seen and done concrete matrix calculations before. Giving it to a mathematical novice is likely to lead to baffelment. Too abstract.
I don't think it's too abstract. It contains plenty of geometrical intuition. And it makes things very clear and he says them very _early_. E.g., a matrix represents a linear transformation. It took me ages to understand this at high school because I was lost in a sea of matrices.
I got a D in linear algebra, my lowest grade in university and one of my lowest grades ever, but excelled in computer graphics, and even got permission to take a graduate level course in it for elective credit as an undergrad.
My recollection is that the problem sets in the linear algebra class seemed really abstract and vague, and I couldn't reason through it very well because I couldn't visualize what we learned, given the information from the class.
Computer graphics, however, let me visualize what we were covered, and suddenly everything made sense. I could see, literally, what the end result was supposed to be, and it just clicked.
Now that I think about it, it's kind of a shame I had the linear algebra class first. The CG experience would have helped me out quite a bit.
You can't really get around linear algebra. Geometric algebra is really neat, and you can benefit from taking a geometric algebra approach to linear algebra (Macdonald's book is pretty good for that), but it enhances linear algebra, it's not a substitute for it.
Linear algebra is cool, no doubt. This blog post is certainly well written and explains how to think of matrices as functions.
I have written a blog post[1] that describes matrices as functions. This blog post is titled "Correcting common mathematical misconceptions" and walks through N dimensions, linear functions and linear algebra. After explaining what linear functions are, I mentioned nonlinear functions and how they often don't have a closed form solution and why mathematicians find bounds.
>We’re getting organized: inputs in vertical columns, operations in horizontal rows.
OH! Bloody hell, so that's why we write it like that!
>The eigenvector and eigenvalue represent the “axes” of the transformation.
>Consider spinning a globe: every location faces a new direction, except the poles.
>An “eigenvector” is an input that doesn’t change direction when it’s run through the matrix (it points “along the axis”). And although the direction doesn’t change, the size might. The eigenvalue is the amount the eigenvector is scaled up or down when going through the matrix.
Very nice! I'd had an algebraic (ahaha) understanding of eigenvectors/values before, but hadn't a geometric intuition. Thank you. Now I can neatly imagine why the eigenvector is orthogonal/perpendicular to the "direction" of the transformation.
A lot of programming puzzles companies hand out can be solved with linear algebra. One company I interviewed at gave me a pretty complex problem, solved it with linalg. Didn't get the jib since I didn't use the usual data structures, but it was still a fun experience.
Lie algebra is a group whose underlying set is smooth. For example, angles (rotations) on a circle (or units in the complex numbers) form a Lie algebra. You can add and subtract angles on a circle, and if you perturb one angle you perturb the result slightly: (theta1 + delta) + theta2 = theta1 + theta2 + delta.
That's a trivial example, of course.
Contrast with a symmetry group of a hexagon. There's no (non-degenerate) notion of a continuous transition from identity to rotate-by-pi.
A Lie _group_ is a group which is also a manifold, such that the group operations are smooth. (I don't know what it means for a set to be smooth.)
A Lie _algebra_ is a vector space together with a bilinear operation (the Lie bracket, written [a, b]) which also satisfies the Jacobi identity:
[a, [b, c]] + [c, [a, b]] + [b, [c, a]] = 0.
How are they related? If G is a Lie group, the tangent space at the identity is a Lie algebra. What does the Lie algebra represent? Well, unlike a plain manifold, in a Lie group, given a tangent vector a, there is a path starting from a the identity in the Lie group always in the direction a. Given two tangent vectors a, b, you can flow a little in the direction a, and then a little in the direction b, or vice versa. The Lie bracket can be thought of as a measure of how the flow fails to commute.
Very roughly:
It's a group with a an infinite number of elements which is smooth, in the sense that there is a notion that elements can be close or far from each other, and locally around each element the group looks like R^n (for some n). Of course the group operation must play nice with this notion of "close". I.e. if elements A and B are close, after multiplying both of them by X, AX and BX can't be too far: the group operation does not "tear" the structure apart.
So in the example above of rotations on the plane, you can visualize the group as a circle where a point represents the rotation by that angle. Locally it looks like R, but it has a different global structure.
Actually, a Lie Group is a group which is simultaneously a manifold, such that the group multiplication corresponds to diffeomorphisms of the manifold onto itself. The Lie Algebra corresponding to that Lie Group is the tangent space of the identity. The elements of the Lie Algebra generate the given Lie Group via the exponential map.
p-adic numbers are actually quite simple; so simple, I've actually seen children invent it on their own!! They're just like ordinary decimal notation numbers, except instead of allowing these to extend infinitely to the right (as in 3.14159...), you allow them to extend infinitely to the left. So, in addition to familiar numbers like 45 or 67.8, you also have numbers like ...95141.3 or ...999 or what have you.
You add and multiply these according to the same rules you're already familiar with. So, for example, if you add 1 to ...999, the last digit of the output is 0, and you get a carry. Making the second to last digit 0, with another carry. And so on and so on, making the result ...0000 over all. Thus, ...999 acts like -1.
(If we were working in base two, this last example would be just like the "two's complement" you are perhaps familiar with from computer arithmetic!)
In fact, most discussion of p-adic numbers isn't done in base ten. Instead, people typically focus on p-adics in a prime base (hence the p). Why? Because in a prime base, you will find that every nonzero p-adic number has a multiplicative inverse, which is very convenient (while in a composite base, you will find that sometimes nonzero numbers multiply to zero). But there's nothing actually stopping you from making use of the notion for non-prime base, should you be interested in doing so; the notion is still perfectly coherent. (That having been said, another reason mathematicians focus on p-adics in prime bases is that the Chinese Remainder Theorem essentially allows one to reduce the study of all other bases to this case.)
There's a lot of beautiful further theory to explore here, but again, the basic idea is hopefully quite simple. Let me know if you have any questions and I'll be happy to try explaining further or more clearly.
Interesting. Very similar to the way ints are represented in Python (since they have arbitrary length, negative numbers are represented with infinitely many 1's in the most significant bits).
p-adic numbers are measure of necesary- resolution to describe. A pair of p-adics is very close when you can use a very coarse ruler (of tick size p^n, large n) to measure the exact difference (1 vs 10000000001). p-adic numbers are far away when you need a very fine grid (1.0 vs 10.000000000001) to measure the exact difference.
I'd request the same for whatever mathematical concepts Quants use for stock market trading. Or at least a list of concepts useful for such an endeavor.
If you like this, then go read the No bullshit guide to math & physics after this: http://minireference.com.
Love the trend of indie publishing that has been exploding in the past 2 to 3 years. The biggest issue is that the marketing & design is often mediocre. Content wise, completely awesome though. I lost interest in books from larger publishers years ago.
I think so – it covers the fundamentals. Numbers, Algebra, Functions. Not exactly in depth, but if you've done high school math you should quickly remember and be able to get to the fun stuff.
Having group or ring structure isn't too surprising and isn't too intimately related to the idea of metacircularity or the Y-combinator. It's actually more interesting to really see that matrices encode linear transformations, and multiplication is function composition.
Perhaps I don't recall my linear algebra well (we called it engine math), but there's a hell of a lot more to it than just passing inputs through a series of operations. Linear algebra is more about the methods used to design very elegant models and transforms and understanding the limits of those models and transforms.
For context, I don't think I ever knew the thing we were studying was called linear algebra, so it took me another 17 years to realize that is the common name for what we studied in that class. Now I'm wondering if maybe one of the intents of the small engineering-focused school's tightly integrated math-science-engineering curriculum was to avoid an introductory linear algebra class. We had to understand matrices and vector spaces earlier for other classes and didn't do what you call the "applications" of linear algebra until sophomore year.
That's because matrix multiplication is noncommutative, and it's traditional in linear algebra to write the inputs matrix on the right of the operations matrix. Just like in normal algebra, you write 5x+7y instead of x5+y7. So the picture matches up with the notation.
I took the first route and retained none of the information, despite doing well in the course. Then I took computer graphics and cried all the way home.
This link seems like a happy middle ground.