Upvoted for bringing up The Mathematical Experience in a relevant way.
If I could get everyone to read just one book about mathematics, that book would be The Mathematical Experience. More than any other book, it gives the actual experience of what mathematics is like.
Much of it is accessible to someone who is still in high-school. And yet it has valuable lessons for people who have PhDs in the subject. I know a number of mathematicians who say that it explains why they went into math. And I can say that in its pages I can find explanations of both what caused me to love math, and eventually to leave it.
For anyone interested, a new book by Reuben Hersh is scheduled for this Christmas: "Loving and Hating Mathematics: Challenging the Myths of Mathematical Life".
This discussion is even more interesting in the context of natural sciences, since no scientific law is ever proven. What were the odds, in 1850, that Newton's laws would be "disproved"?
From Jaynes' The Logic of Science
It depends entirely on this: Against which specific alternatives are we testing Newton's theory?
For example, if you ask a scientist, "How well did the Zilch experiment support the Wilson theory?" you may get an answer like this: "Well, if you had asked me last week I would have said that it supports the Wilson theory very handsomely; Zilch's experimental points lie much closer to
Wilson's predictions than to Watson's. But just yesterday I learned that this fellow Wolfson has a new theory based on more plausible assumptions, and his curve goes right through the experimental points. So now I'm afraid I have to say that the Zilch experiment pretty well demolishes the Wilson theory."
I think that the first ~6 chapters of the book should be required reading for anyone in the natural sciences. An early drafts can be found here: http://www-biba.inrialpes.fr/Jaynes/prob.html
The Kelvin problem: what is the optimum way to partition 3D space into identical cells, such that the surface area between them is minimized? Lord Kelvin conjectured that the Kelvin cell gave this optimum partitioning. This conjecture stood for 100 years until Denis Weaire and Robert Phelan found a better cell in 1993.
http://www.amazon.com/Mathematical-Experience-Phillip-J-Davi...