I've used it to prepare my linear algebra exam in an italian university (which usually emphasize theory over practice, even in engineering courses), and I've loved it.
There are few exercises but they're to the point. Complex inner product, self-adjoint operators etc. are treated. I find the notation used in there particularly easy to understand, and proofs are elegant but very easy to follow.
I have skimmed some books about Linear Algebra, both in Italian and in English, and my favourite so far is "Geometria e Algebra Lineare" by Enrico Schlesinger.
The book is very well written but sadly it won't be enough to pass a typical italian Linear Algebra exam. This is because, in my opinion, examples are too vague or too "abstract": when you're first learning such subject, it would be better to have layman terms, that is simple examples with simple numbers. More complex examples might be okay too, but simple examples should come first.
My favourite linear algebra book in italian is "Geometria e Algebra Lineare" by Abbena, Finno and Giannella.
Really great book, with a wealth of exercises and extensive material.
I can also suggest the youtube lectures[0] by Francesco Bottacin, who actually was the one that eventually did my oral examination.
He's also written two books based on the course, the one with exercises[1] is very solid and I think you can pass the written exam by just doing it whole.
They told me he is a wonderful lecturer. Pity that all this material is in italian.
I actually don't like the Strang book and his MIT course.
I watched 6 or 7 of his lessons, and to be honest, I think they're nice if you already know the subject.
I have skimmed both an italian version of his book and an english one, still didn't like it.
The nice thing about the Schlesinger book is that it builds all of the concepts basically from scratch. Euclid’s Elements are quoted too, in the introduction.
That was not my experience -- I was utterly confused by my college linear algebra class which was taught as a bag of techniques. I couldn't figure out why anyone would care or what you would do with any of it. And I definitely didn't get the connections to the geometry that Strang goes over in Ch4 iirc.
Strang taught me linear algebra. His course -- and I read a couple other books afterwards -- is the only reason I understand it. Not only the how of the techniques, but why and what they do.
I don't suppose there's an english translation of the Schlesinger book? Google says no.
Unfortunately that book was too lightweight on theory to be of use to me, but it's certainly a very good book.
I've recently started Strang's more advanced book on computational linear algebra/numerical analysis and it's pretty good though.
I was on the fence about that one and ultimately took it off the list. It seems to be fairly numerical, sort of in defiance to Axler's "Done Right" book. But then as a numerical book how does it compare with the others in that category, or with the old Russian books? I ask quite honestly because I haven't gone through Treil's book. Does it hold its own?
Actually, I think it's very similar to Axler's in spirit. Not numerical at all (there is a description of the min squares problem, but that's it as far as I remember).
Matrices are used more explicitely than in Axler, but only when necessary, because the author claims he's an operator theorist so the focus is on linear operators.
Determinants are introduced as multi-linear functionals and are not avoided as in Axler.
Supporting the fact that the book is quite theoretical, this is what you can find in the last chapters:
Chapter 6. Structure of operators in inner product spaces.
Chapter 7. Bilinear and quadratic forms
Chapter 8. Dual spaces and tensors
Chapter 9. Advanced spectral theory
Unfortunately I'm not familiar with the Russian books.
Given the frequency and popularity of these "math texts for programmer audience" threads on HN, I think there's a solid business model where programmers pay a onetime subscription fee for unlimited email/hangouts access to a math professor.
The problem with learning math all by yourself is, to quote my favorite math professor- "math is a foreign language".
So its a bit like trying to learn chinese sitting in a non-chinese setting, with only a chinese grammar text. You are simply not going to get very far. Far easier to buy a plane ticket to China and surround yourself with Chinese speakers, so you learn through osmosis - which is what the university experience is all about. But not everyone can take time off and go to a university. So then, atleast have a Chinese mentor on call, who can guide you through the difficult parts, tell you why things are the way they are etc.
I'd be glad to pay $1000 for a 1year unlimited email access to a math prof who can help me with some of the harder portions of measure theory and Banach spaces. Sign up enough students like myself and you have a lucrative gig.
As a former math professor, I'm not confident that this would be as scalable as you might be assuming. Given a nominal fee of $1k/year and a reasonable target of 100 such subscriptions, that is 100 people requesting "unlimited" amounts of your time in 1on1's. And decent 1on1's require non-trivial prep. And, given 100 customers, at least one is bound to push the limits of the "unlimited" premise.
The "unlimited" is mostly branding, like gmail's unlimited email storage. In practice, professionals like myself simply don't have the time to bug you at all hours of the day. I have a dayjob and maybe an hour I could devote to math every weekday. So worst case you'd receive 1 question a day...usually far less. Moreover, students tend to ask the same questions, so after the initial phase you can simply redirect most queries to preexisting URL. Also, you as a professor don't have to chime in for everything - have a graduate student preprocess the majority of the questions and you attack the harder 10%.
Its definitely feasible. Chegg does something similar - what's missing is the quality and rankings
Or you could just take a class at a local university or community college.
You could contact the professor and see if they're okay with you auditing and using office hours and the like without paying or you could contact the university and see if you can pay to take a class without enrolling in a degree program.
So has the author read all these books? Is there a review? What makes these the best? Isn’t a significant part of the content in these books overlapping?
What is someone who isn’t a graduate student algebraist supposed to take away from a list of 33 books? At typical textbook prices, that’s like $4000 and 5 years of doing nothing but reading about linear algebra.
(Sorry if this sounds negative; I’m sure there’s some audience who will find this list useful.)
Honestly, I'm not even sure how much a review would help either. I struggled quite a bit with linear algebra in school with Curtis' Abstract Linear Algebra, and eventually I decided to hole up in the math library for the better part of a week and went through every linear algebra book they had (definitely more than 33) to find the best supplementary material. I guess it was a pretty hard class because about half of the other students had the same idea. None of us settled on the same book as our favorite.
My sample size is exceptionally small, but the opinion I took away was that linear algebra was just one of those things you have to plod through lots of books to find the one that works for you.
I can't believe he didn't mention Gilbert Strang. This should be the starting point for everyone, IMO.
The oddball Russian books look very interesting, but for a computer guy to learn, I'd suggest Strang, then Trefethen & Bau (my favorite). If there is anything else worth knowing about numeric linear algebra it is in Gollub, or Demmel's applied numerical linear algebra (another unforgivable omission; he only helped write LAPACK!).
The problem books can also be important; they're all pretty good if you pick the problems right.
Not sure the highly theoretical stuff is actually worth anything as far as getting things done, unless you go on to study Algebra and mathematics in general. Ultimately the "real world" stuff where linear algebra is studied as systems of equations and geometric constructs is going to be closest to what you actually do with linear algebra on a computer.
> I can't believe he didn't mention Gilbert Strang. This should be the starting point for everyone, IMO.
Not everybody loves this book, you know. A lot of people find its sloppy, disorganized, chatty style quite irritating.
> Not sure the highly theoretical stuff is actually worth anything as far as getting things done, unless you go on to study Algebra and mathematics in general.
You've answered your own question. It's "highly theoretical", not "highly practical". The focus is different by definition.
Really appreciate this effort. My mathematical background is somewhat lacking and I'd love to pursue ML/Data science where I'm led to believe this is a prerequisite.
Anyone able to tell me what other areas of math I might need as a baseline before diving in here?
My advice to you would be to start backwards. Find a few interesting-looking papers in the field and get a feel for the math involved and learn it. If you learn your mathematics as a tool to make sense of the papers you're reading, you're likely to be going down the right path for your goals.
That said, linear algebra and statistics (+ prereqs) are an absolute necessity.
If you have already studied linear algebra at an undergraduate level, read Trefethen and Bau’s book. Otherwise, perhaps find an undergraduate course somewhere first (OCW? Some MOOC?). Most of the stuff on this list is very abstract and technical, and won’t necessarily be relevant to you unless you want to become a professional mathematician.
Practically, a main difficulty in ML/Data science, is getting the data. Applying algorithms is largely limited to applying implementations of algorithms.
So, yes, it helps to be familiar with this level of stuff. But don't let it stop you tinkering with what you have access to.
I own that book and have worked through a few chapters. It is one of the first linear algebra books in a "modern" style, being published in 1958. The reason it's not on the list is I just didn't see what it had to offer that isn't covered in later books. Happy to be convinced otherwise though.
I do have Halmos' linear algebra problem book on the list because it's quite good.
bourbaki is fairly early. the date on your site is related to translation.
although the halmos book appears wordy, i think he has a sound pedagogical style appropriate to his target audience, having glanced at the text mentioned here and really enjoyed his book on set theory.
> by choosing one road I am turning my back on a thousand others
ah, no need to paint such a glum picture: there are only a few facts that will appear and reappear in many of the theoretical books, b/c there is only one thing called, "vector space," one thing called, "hilbert space," and so on.
for hacking, of course, numerical methods are what many readers will be interested in...
> Weird Russian
this might be the best section, except for the categorical "weird" label. the iron curtain must have been sort of like a semi-porous membrane. probably just about every book list on math topics considered during the mid-20th-century ought to include some books by russian authors.
> The reason it's not on the list is I
just didn't see what it had to offer that
isn't covered in later books. Happy to be
convinced otherwise though.
Halmos wrote Finite Dimensional Vector
Spaces in 1942 -- shortly after he got
his Ph.D. from J. Doob at University of
Illinois -- at the Institute for Advanced
Study in Princeton as an assistant to von
Neumann.
The book is a mostly finite dimensional
introduction to von Neumann's Hilbert
space theory. Some physics profs tell
their students to read that book to learn
the Hilbert space theory they need for
quantum mechanics. Generally the book
uses Hilbert space, that is infinite
dimensional, notation and arguments when
they also work for the finite dimensional
results -- cute. This approach is easier
and cleaner when it works.
There is emphasis on duality, as is
important in functional analysis including
Hilbert space.
Some of the special emphasis is on
spectral theory, important in
transformations on Hilbert space.
Halmos includes some multi-linear algebra
(can see again in exterior algebra and
more modern approaches to general
relativity) and near the end has an
ergodic theorem!
At one time the book was one of three
basic references for Harvard's somewhat
famous or notorious Math 55 as in
The other two references were Rudin's
Principles and Spivak's Calculus on
Manifolds. So, from Harvard get a good
recommendation for excellence for the
Halmos book.
I very much enjoyed that Halmos book and
later more of his books. He was one of
the best writers in math of the 20th
century.
His proof of the Hamilton-Cayley theorem
is a little less general than the one in,
say, E. Nering's book, and I wrote Halmos
about that. He wrote back a nice letter.
I learned linear algebra from a course in
abstract algebra, Murdoch, Linear Algebra
for Undergraduates, (about the easiest
linear algebra book anyone could read),
one by E. Nering, some work I did in
mathematical physics, the first chapters
of Nickerson, Spencer, and Steenrod's
Advanced Calculus, my ugrad math honors
paper on group representation theory,
Herstein's Topics in Algebra on abstract
algebra, Halmos, FDVS, Forsythe and
Molar on numerical linear algebra,
applications in multivariate statistics,
some work in optimization, carefully wrote
out my own notes, enough for a book, and
more.
The OP list also includes Horn's book.
After all that study, mostly independently
and part of my career in applied math and
computing, eventually I took Horn's
course, intended to be an advanced course.
As part of my application to his grad
school, I did include a copy of the Halmos
letter!
There was next to nothing in the Horn
course new to me, and I didn't try very
hard. Everything in the course was
carefully graded. I did the homework but
didn't study for the tests. On an early
homework assignment, the grader made a
mistake on my paper; I corrected him; and
he made no more mistakes. About 3/4ths
the way through the course the grader
showed me the class scores -- I was first
on everything by wide margins. Did that
again on the final. Apparently did that
again on the corresponding Ph.D.
qualifying exam.
In the course, when Horn got to the polar
decomposition, I got excited and blurted
out in class "That's my favorite theorem"
which it was and is. Horn's remark was,
"Thank you Dr. Halmos".
The polar decomposition? Go through a lot
of stuff in linear transformations and
then discover something really simple:
All a linear transformation can do is just
a rigid motion (rotations, reflections)
followed by stretching/shrinking on
orthogonal axes. A singular
transformation? One or more of the axes
go to zero. In the non-singular case, the
proof is really simple. In the singular
case the difficulty is that there are many
choices for the rigid motion part and just
have to pick one.
Horn did at times have some especially
good proofs.
Horn is very precise with high quality.
It's possible to have a quite fast
introduction to linear algebra, but then
are missing out on a lot if don't work
carefully through a classic text such as
Hoffman and Kunze. IIRC it is available
as PDF on the internet for free.
E. Nering's book is also fine except for
his material in the back on linear
programming which is easy (for a lot of
it, just slightly tweak Gauss elimination)
but he makes difficult. For linear
programming, get a good book on linear
programming, e.g., Chvatal or Bazaraa and
Jarvis. In the back, Nering also does
group representations -- very cute.
I'd recommend Halmos, FDVS as ice cream
and cake desert after something like
Hoffman and Kunze.
For the numerical stuff, should look at
the documentation for Linpack. Maybe also
look at Forsythe and Molar. Some that has
been done since then for numerical
methods, e.g., for models of oil
reservoirs, is impressive -- might at
least glance at that.
Broadly there are two directions in linear
algebra: One direction assumes that the
field is the real and/or complex numbers
and is aimed at analysis and applied
math. The other direction assumes finite
fields and is aimed at high end abstract
algebra and has applications in, say,
algebraic coding theory (high end error
correcting codes).
The direction with real and complex
numbers is, sure, a start on functional
analysis (you are in a vector space, but
each point is a function -- get to learn
how to approximate functions by other
functions, e.g., how to approximate random
variables by other random variables, e.g.,
learn Fourier theory), but for that also
should have measure theory, e.g., from
Royden, Real Analysis and/or the first
half of Rudin, Real and Complex
Analysis. For that, as in Math 55, a
prerequisite is Rudin's, Principles.
Rudin's Functional Analysis is more
advanced, still. I got accepted to grad
school at Brown's Division of Applied Math
(didn't go there), and Rudin's FA was
popular there, but I haven't seen a lot of
popularity for it elsewhere.
A huge fraction of applied math, somewhere
on the way to implementation and code,
boils down to linear algebra.
Oh god, I still have nightmares about Herstein. I took an undergrad algebra class from a very old-school prof who thought it would be good for undergrads to be exposed to some of the problems - they gave all of us quite a bit of trouble.
Herstein is a bit obscure for a first course. Once I was pushed into a course from Herstein. At first it was in group theory below what I'd already done. I asked the prof if he was going to get to group representation theory. He said that that topic was "deep". I told him it was my ugrad honors paper. The course was beneath me, and I saved time and didn't go. For the final exam, he gave me an oral. He asked me something about Galois theory, which is in Herstein and which I learned from there for his exam. His other question was about duality. He wanted me to show that the dual space of the dual space is isomorphic to the original space. That's true and easy to show, mostly just pushing symbols around, and the notation can get a little confusing, but is not true in some cases.
Some point set topology courses consider duals of duals, and when there is equality say that the original space is reflexive. Then they want to define the weak topology and the weak* topology, etc. Can see some of this in, say, G. Simmons, Introduction to Topology and Modern Analysis, super nicely written book although I have never seen any applications for it.
Pushing around duality in linear algebra can get to adjoint transformations. To me it turns out to be a lot of careful notation for something that turns out to be nearly trivial.
Yes, on problems in Herstein, there was one on rings. It took me a while but I got it. I showed it to the prof, and he sent me to another prof who thought that solving that problem was good. He gave me a more difficult related problem, and I made a mess out of it. The problem needed transfinite induction, and I was supposed to understand that but didn't understand it very well. So, I got pissed off: I should have been studying stuff I didn't know, e.g., transfinite induction, instead of wasting time in the Herstein course that had, say, only Galois theory I didn't already know.
At that school same thing happened in a topology course: The book was Kelley, famous. Heck as an ugrad, I'd taken that book and given lectures once a week to a prof and did a stack of the exercises. I did all but the chapter on compactness at which time I dropped everything to finish my honors paper. For me to sit in that topology course was stupid.
I went to the department chair and tried to get into some material I didn't know but wanted to, especially differential geometry for relativity. He told me no.
So, I did my teaching, started violin, met my wife, and left for a good job!
The books in the list are pretty advanced. It would be nice to see some sort of "star" rating for each book to indicate which books the authors found best.
I studied under the author of this book [1] and I can anecdotally say that I finished with a very strong understanding of the subject. Learning linear algebra and being able to whip out some of the less common theorems has been very valuable to me as someone who uses numpy, pandas and tensorflow regularly.
I am a big fan of Hoffman and Kunze. I also think many would benefit from reading a book on ordinary differential equations after they read a linear algebra textbook. Much of the elementary theory of linear ODEs consists of combining relatively simple observations with linear algebra. Coddington and Levinson is the one I used for a course in college, but it is quite detailed. I am sure there are less ponderous options available, but I don't know any off the top of my head.
The Lipschutz (that is quoted) is good. While looking stupid and repetitive it is giving very good basis.
Applying linear algebrae to grok more complex problems can be done with the Mc Grawhill book on Mechanic (solid & punctual). It is the same pattern : simple explanation and a lot of exercises.
Thermodynanmic, quantum mechanics, relativity and electromagnetism are introduction to even more interesting algebrae.
Maybe Feynman physics lessons are the right books :)
My favorite LA books are Linear Algebra by Friedberg/Insel[0] which is a combination of Axler style book with more computation oriented one (Terry Tao has a set of lectures based off this book). Another one I like is Modern Intro To LA by Henry Ricardo[1] which implicitly introduced me to Replacement theorem which is really overlooked in a ton of LA books. Again, this book's a rigorous mixture of both theory and computation done very well. If you've never seen higher level math before, there's Linear Algebra: Gateway to Mathematics by Robert Messer[2]. It has tons of commentary about elementary set theory and proof techniques along the way. Whenever someone mentions Axler's book, someone else brings up Treil's book. But there's a third one in the same league/group which is Linear Algebra: An Introduction to Abstract Mathematics by Robert Valenza[3]. Other favorites are Coding the Matrix by Philip Klein[4] for Python aficionados and
Linear Algebra Through Geometry by Banchoff/Wermer[5] for those who like geometry.
If you are way beyond all this, you can still pick up new things from Advanced Linear Algebra by Steven Roman[6].
Seconding Klein's Coding the Matrix. It's a great alternative introduction and actually implementing linear algebra ideas in Python is both useful and a great aid for learning.
The auto-grader is helpful for making sure you got the right solutions.
I've used it to prepare my linear algebra exam in an italian university (which usually emphasize theory over practice, even in engineering courses), and I've loved it. There are few exercises but they're to the point. Complex inner product, self-adjoint operators etc. are treated. I find the notation used in there particularly easy to understand, and proofs are elegant but very easy to follow.