For all the glory Einstein deserves as one of the greatest minds I find more interesting the history of this supposed "failure" in later life, but it's even more admirable his tenacity at trying to tackle the problem at different angles for decades. And boy it must be a hard problem if Einstein himself could not crack it!
There's some argument that Einstein was in the right place at the right time. Mercury was wobbly, and there was about fifty years of non-euclidean geometry research built up, including (eg) Riemann breaking ground on differential geometry.
Maybe matter didn't crack because the right tools weren't available.
Einstein solved three very different open problems in one year: the photoelectric effect, Brownian motion, and special relativity.
Others may have gotten close to one of these solutions, but nobody else came anywhere close on all three.
General Relativity as his own research programme was designed to capture the consequences of constraining the speed of propagation of changes of gravitational influence; it was literally about relativizing gravity. GR was not designed to explain Mercury specifically (or even really motivated by Mercury), the theory just happened to explain why its orbit traces out a daisy-like pattern rather than a perfect ellipse. GR also correctly predicted a deflection of background starlight around the limb of the sun (1919 eclipse and eclipses since), stellar gravitational redshift (Sirius-B initially, many many objects since), and gravitational redshifts induced by Earth (all reliable results were posthumous: Pound-Rebka and similar since 1959, and more recently precision lunar and satellite ranging).
Indeed, Einstein liked to explain that his mental toy in understanding relativistic gravitation was someone jumping off the roof of a house, or the behaviour of things (e.g. flashlights/torches) riding in office-tower elevators/lifts (which date from the 1870s). That's a far cry from precision measurement of Mercury's perihelion!
There were no real astrophysical or terrestrial problems calling out for General Relativity. Even after General Relativity was a published theory, real gravitational problems were solved with low-order correcting terms to Newtonian gravitation and with linearization: an approach which Einstein practically invented, and which he used himself when thinking and writing about early problems in cosmology (the discovery of Cepheid variables opened up a lot of those).
There were quickly alternatives to General Relativity which predicted some but not all of these early classical tests of the theory. Eddington's 1922 book was the first shot in a body of literature analysing different theories of gravitation and how they differ in their predictions of dozens of tests of General Relativity. Will's work in particular is useful: https://en.wikipedia.org/wiki/Parameterized_post-Newtonian_f... -- you can see how it's used in practice in this open access paper https://www.nature.com/articles/s41467-017-02558-1
That said, there are mathematical tools available now that weren't available to Einstein in the early 20th century, and he might have chosen to arrive at a different (but equivalent) formulation of General Relativity. The standard Hamiltonian and a variety of modern Lagrangian formulations are particularly useful, and it could have been nice to have had <https://en.wikipedia.org/wiki/Initial_value_formulation_(gen...> before the 1960s (even though it arguably only shines brighter when you have 21st century supercomputers).
Indeed, one can imagine Einstein starting with Lie theory, such that Special Relativity from the start is just the theory of spacetime with SO(3,1) symmetry at every point. But Cartan, Killing, and Noether came decades later and were motivated by Einstein. And quite a bit of group theory developed as the Standard Model developed after Einstein was already dead (notably Goldstone's theorem, 1960s). So to first formulate General Relativity as a GL(4,R) group theory with spontaneous symmetry breaking to SO(3,1) one would need to rearrange an awful lot of physics history.
Any of these alternative-universe mathematical origins would just have been using different tools to arrive at the central result: we inhabit a Lorentzian spacetime in which there is an exact matching of moving matter and a metric tensor (at each point in spacetime) encoding durations, spatial lengths, and angles, or the equivalent set of orthonormal vector/covector fields.
And worse, the physical content of the theory -- however formulated -- probably would still have been considered interesting but practically useless until the 1970s.
>> Indeed, Einstein liked to explain that his mental toy in understanding relativistic gravitation was someone jumping off the roof of a house, or the behaviour of things (e.g. flashlights/torches) riding in office-tower elevators/lifts (which date from the 1870s). That's a far cry from precision measurement of Mercury's perihelion!
Something I don't understand about GR is whether Einstein had real, empirical evidence to guide him or whether it all just sprang fully-formed from the world of ideas into his waiting mind. I know about his thought experiments, but those are, well, thought experiments. Did he have concrete observations to base his theory of GR on?
Forgive my lazy approaches to citation and linking below. I didn't have much time, and ran into the realization that you and I are probably the only HN users who will ever read this.
> concrete observations
No, certianly not. As far as I remember all of the early tests of post-Newton/pre-GR but "relativity compatible" gravitation were motived by sketched-out systems in Einstein's 1907ish-1912ish writings, with the observations and experiments taking place years later.
> base his theory of GR on
The entire programme of Einstein gravitation started from purely theoretical considerations, starting with thoughts about accelerated bodies in special relativity and relativistic ballistic bodies (which aren't accelerated beyond the initial impulse: c.f. Newton's cannonball (1728) -- ignoring friction a cannon ball's descent back to the ground is free-fall so the motion in free-fall must therefore be inertial, and in fact the whole trajectory up and down, or into orbit, or out to infinity after it is shot out of the cannon must be inertial <https://physics.weber.edu/schroeder/software/NewtonsCannon.h...>).
General Relativity certainly didn't spring fully formed from Einstein's mind -- it took about eight years for him to develop the ideas and mathematics along the way.
(It's also handy to see how interested in diffusion and dissipation -- and thermodynamics more generally -- he was before his famous Brownian motion paper, 1905, Doc 16.)
Relativity, 1905 (Doc 24 ("Does the inertia of a body depend on its energy content?"))
First articulation of equivalence principle (observers in free-fall are in inertial motion), 1907 (Doc 47, "On the relativity principle and the conclusions drawn from it"; this is practically a short textbook and is awesome in total, but for your question the relevant bit is in part V, starting at the bottom of p. 301 (direct link because pagination is often weird on the site) <https://einsteinpapers.press.princeton.edu/vol2-trans/315>).
A choice quote: from the end of part V §19: 'There exist "clocks" that are present at locations of different gravitational potentials and whose rates can be controlled with great precision: these are the producers of spectral lines. it can be concluded from the aforesaid that the wave length of light coming from the sun's surface, which originates from such a producer, is larger by about one part in two million than that of light produced by the same substance on earth.' (Footnote 1 is also interesting for noting that his initial thinking was about a uniform gravitational field, which does not exist in nature, and thus he was forced into assuming that his thinking also applied in a more realistic gravitational field; this was well before he started developing the mathematics of General Relativity in 1911-1915 and of course also before Schwarzschild 1915).
Such a small spectral shift was untestable in 1907; it took until 1959-1960 (Pound-Rebka experiment, "Apparent Weight of Photons") and the 1970s (Harfle-Keatin and Gravity Probe A) to demonstrate and measure the effects of Earth's gravitational time dilation.
and in particular Doc 23, "On the influence of gravitation on the propagation of light"
There is a lot of interesting stuff in that paper, including an early rough estimate of the deflection of light around the sun and Jupiter. However it's important to note that there are no tensors and no (pseudo-)Riemannian geometry in the 1911 paper: the theory with those (and the general covariance and curved spacetime they encode) was still about four years away. His light deflection estimates were illustrative of generic effects in any sort of principle-of-relativity-compatible gravitation (again, he was personally in no position to be thinking about a geometric theory, not yet having started learning differential geometry) rather than serious predictions, let alone existing observations that he needed to capture.
It is really only after 1912 that Einstein begins developing a description of gravitation in terms of curved spacetime. By then there were several observations (motivated by Einstein's pre-GR work!) showing that Newtonian gravitation becomes inaccurate near the sun, with Newton's formulation (and also Poisson's and Gauss's and other equivalents) proving difficult to adapt via correcting terms.
So: SR first; then thoughts about accelerated v freely-falling observers where relativistic effects are expected, including some thought experiments and some sketches that could be weakly checked by astronomy; becoming convinced that non-Euclidean geometry could represent some of these effects; then learning (and developing) relevant mathematics; then GR; then modelling systems and predicting behaviours using GR; then early precision astronomical and eventually (and sadly, posthumously) laboratory tests of those.
It's possible to imagine that astronomical discoveries could come before a fully non-Newtonian theory of gravitation. Let's omit General Relativity and the early work in the 1920s-1930s, the theoretical developments of General Relativity from the 1950s&1960s, and more (thanks to computers) in the 1980s, but keep the dates of astronomical discoveries. We then wonder how what we realized were "manifestly relativistic" astrophysical systems would have been dealt with by theoreticians.
The "Newtonian gravitation needs to go" trigger would probably have been clearly non-Keplerian orbits like pulsar+other binaries (Hulse-Taylor, 1970s; the orbit is contracting/speeding up) precision studies of stellar orbits in the Milky Way's central parsec (1990s; there is an unseen compact mass distorting the innermost stars' orbits), or could have come as late as the discovery of PSR J037-3039 (a two-pulsar binary, 2003) or PSR J30337+1715 (a pulsar a close binary with a white dwarf and with another white dwarf orbiting the central pair, 2007, which has been used to test the strong equivalence principle). However, in our history, predictions of "highlight" aspects of these astronomical oddities date from early work with GR in the 1920s and 1930s.
Another trigger could be compelling evidence for the non-ballistic trajectories of distant galaxies, which started racking up in the 1990s. Einstein and others toyed with an accelerated expansion in the late 1920s-1930s (that other galaxies were distant was freshly discovered then), and rejected the idea for want of astronomical evidence.
In all of these, with some thought, a pattern would emerge: in all these systems there are orbits or trajectories which appear to reach velocities comparable to the speed of light. Corrections in powers of v/c might seem obvious. However, with our real history, Einstein was doing this in about 1916 or so, because it was easier to calculate orbital evolutions that way, and the approach was (allegedly) easy to extract from (or trace back to) GR. I wish I had a handy reference for that. Perhaps some proper historian of relativity will update <https://en.wikipedia.org/wiki/Post-Newtonian_expansion#Uses>.
Likewise linearized gravity captures the speeding-up of orbits implying the loss of non-electromagnetic energy radiated out of the binaries/triples/central parsec as gravitational waves. It is somewhat easier to relate to GR than expansions in v/c. But what form would early theories take when thinking about the decay of the Hulse-Taylor orbit?
Wow, thanks for the amazing reply. Clearly this is a subject of passion to you. To clarify, I'm a computer scientist and my physics background is sadly very poor, so much of what you say and link to goes straight over my head. But I think I get the gist of it, thanks to your summary:
>> So: SR first; then thoughts about accelerated v freely-falling observers where relativistic effects are expected, including some thought experiments and some sketches that could be weakly checked by astronomy; becoming convinced that non-Euclidean geometry could represent some of these effects; then learning (and developing) relevant mathematics; then GR; then modelling systems and predicting behaviours using GR; then early precision astronomical and eventually (and sadly, posthumously) laboratory tests of those.
From this I take that Einstein was mainly a theoretician, which is perhaps a little bit surprising. It is to me, because I have an idea of physics as a discipline with a very strong empirical component.
In the AI and machine learning community there is a bit of a debate about the lack of a theory of intelligence, or artificial intelligence, to guide research. Turing award winner Yann LeCun for example has argued that a theory is not necessary because very often technological advances precede theoretical understanding. He's used the Wright brothers as an example and claimed there was no theory of aerodynamics before their first powered flight, but I think he's wrong about that: George Caley set the foundations of aerodynamics a good 100 years before the Wrights.
I asked my question just because I was curious but as I was reading your answer it occurred to me that it is relevant to that debate, that I'm very interested in (and have my own strong opinions about). Einstein comes across as a powerful example of how far a dedicated, competent theoretician can go without a need for empirical results. A little bit scary to be sure, and certainly a lone theoretician plugging away at a theory on their own will not go very far, there needs to be the right environment, a scientific community that can properly evaluate such work. In any case, despite what LeCun says I am convinced that there can be no true scientific progress without theoretical understanding. Even if empirical results come first, we only really make progress when our understanding of the world improves. But all that is a bit political; apologies if you find that disagreeable.
And now I wonder whether Einstein was really one of a kind, as we're taught since school. There must have been others, other physicists, and possibly astronomers, who made progress only based on mathematics and theoretical understanding, and didn't have direct empirical confirmation of their results for a long time; or possibly never -because those results were confirmed after their death; yeah, sad for uncle Albert :( .
There's no real focused thesis below, more a set of reactions to your comment: Einstein wasn't in practice a lone wolf, he worked in academic environments; he focused on three passions and didn't stray much; he worked "on paper" with very few physical artifacts attributed to him; he was extremely successful where his interests took him, even quite late in life where he failed to meet some of his own hopes; while he is a household name in much of the world, there are other well-known names in his areas of interest (that he admired & used the work of); and after he died the work of thousands of others in those areas of interest carried on.
> Clearly this is a subject of passion to you
Not exactly - the passion here is for physical theories of gravitation, with a side helping of obviously unphysical uses of gravitation (e.g. spacetimes with arbitrary numbers of dimensions), and wrong theories of gravitation. This thread was more history of science, which isn't a passion. It's just that I've read a lot of 20th century scientific papers on gravitation, which are often interesting but rarely of practical relevance.
(It also goes into "metahistory" of science or metaphysics or whatever which is even less of a passion, so I'm afraid I won't engage with your summary of LeCun. I'm not familiar with him (yes I know, it's HN, but...), there's no obvious overlapping expertise. I'm not sure I'd get it right if I were to comment much more on the relativist/astrophysicist "dance" I mention below, and I simply would not know what I'm talking about when it comes to AI, computer vision, etc.)
> Einstein was mainly a theoretician
Yes, very much.
We can tie this to the article at the top: late in his career he pursued theories which didn't bear fruit for one reason or another.
In astrophysics and physical cosmology there has been centuries of "dance" between theoreticians and experimentalists -- sometimes, the former have 'go look for this particularly' ideas which can pan out; sometimes the latter produce a 'huh, we weren't expecting and can't explain this result'. Theory and observation co-evolve.
Again, though that goes into the history and philosophy of science more broadly, and that's too far from my island of comfort. There are many many HN commenters who will share their thoughts on this stuff quite readily.
> whether Einstein was one of a kind, as we're taught since school
He was, but it doesn't really matter -- he died before there were tools for theoreticians like supercomputers, before the cosmic microwave background was discovered, before neutrinos were observed in a lab, before ultra cold atomic gasses could be made and studied in a lab, before the first practical atomic clock, before the discovery of astrophysical masers, before the first lasers, and so on and so forth, before CCDs, before good slew studies of the "variable universe" <https://www.astronomy.ohio-state.edu/asassn/index.shtml>. Some of this stuff would probably have been harder to invent without his photoelectric effect paper (for which he got the Nobel prize in physics) or special relativity. But if all those tools and techniques had been around when he was at his most intellectually productive, it would have been interesting to see how he would have adapted his thinking, and what he would have chosen to work on.
(Not to mention that scientists can collaborate with one another much more easily these days, and don't have to wait for access to experimental results as long as before there was the Internet. Just the way he would do things now day-to-day would be incredibly different. It's really hard to imagine being stuck with the long long lonnnnnnnnnng waits he had to put up with, but on the other hand he wasn't distracted by streaming entertainment, computer games, social media, etc etc, and had more time to fill as a result -- notably while travelling via HORSE (!), train, and ship to academic conferences, instead of flying or being remote.)
> There must have been others
Sure, physics and the mathematics of physics are totally littered with surnames like Gauss, Laplace, Lagrange, Lorentz, Poincaré, Newton, Leibniz, Galileo -- many of them were "polymaths" that studied many areas of science and even many non-scientific things (for example: economics, social sciences).
By comparison,Einstein stuck to his three lanes (thermodynamics, quantum theory, and gravitation), writing down theories that used groundbreaking formulae and theories named for / developed by all of the above and more.
Interestingly, one experiment/invention of Einstein's that I do know (there may be others; I just deal in theories he developed alone and with collaborators) came from his interest in thermodynamics: https://en.wikipedia.org/wiki/Einstein_refrigerator
I would imagine Einstein would have been interested in the cooling of electronic equipment and especially computers, had he lived into the 1980s. He also would likely have been interested in computer display technology (CRTs use and LEDs/OLEDs are related to the photoelectric effect) and digital photography (which has uses in astronomy, obviously).
> A lone theoretician plugging away on their own
is pretty rare, since it is so easy to get feedback from other theoreticians, results from experimentalists, and so on. There are many single-author theory papers still, but they generally aren't written by recluses.
Einstein certainly didn't work in a vacuum, particularly when it came to General Relativity and his later Grand Unified Theory work. He got help from mathematicians and published a number of papers with coauthors.
Finally, because he didn't work in a vacuum it is safe to say that if he were suddenly resurrected today with all his previous memories intact, he would have a lot to catch up on to be able to make sense of most papers in quantum mechanics and gravitation. He'd have to learn how to browse the web safely; he'd have to pick up some LaTeX and probably a bit of computer programming. These are all things a modern physics student will pick up no later than their first year at university. There's quite a lot of graduate-level stuff in regular use that postdates Einstein too, that he would have to become familiar with. I'm pretty sure he would still be productive and even prolific very quickly, but he would need help from others practically right away. (That wouldn't be alien: he had help from Hilbert, Levi-Civita, Noether, and many others, during his career).
No matter how expert one is in one's "silo", there will always be people much more expert than you in other fields, even closely-related other fields. And even in your "silo" you won't always be preeminent (and even then can learn from others).
Sorry for the delay in replying and thanks for another high-effort (the opposite of low-effort) reply.
To clarify, I didn't think that Einstein was "A lone theoretician plugging away on their own", that was a more general stereotype and kind of hypothetical. My understanding is that even in his time Einstein had ready enough access to others' work and there are at least two famous results of his collaboration with Podolsky and Rosen I'm aware of. It's an interesting question how differently he would have worked in today's world. Even the idea of scientific research is very different today than in his time. I don't reckon he had to spend so much time writing grant proposals as modern senior academics, for instance (fortunately I'm just a post-doc at this point, but I observe the effect it has on my PI; basically, their research career is over. Which is where I come in I guess. Bit of a digression this).
>> (It also goes into "metahistory" of science or metaphysics or whatever which is even less of a passion, so I'm afraid I won't engage with your summary of LeCun. I'm not familiar with him (yes I know, it's HN, but...), there's no obvious overlapping expertise. I'm not sure I'd get it right if I were to comment much more on the relativist/astrophysicist "dance" I mention below, and I simply would not know what I'm talking about when it comes to AI, computer vision, etc.)
I appreciate this. I, too, don't try to have an opinion about everything and anything and prefer to stick to my area of expertise. There are enough different disciplines in AI research that I can probably keep learning until I'm dead; which is what I intend to do.
Thank you for the conversation and I appreciate also your kind deviation from the matters of your immediate interest. It has been a pleasure to read.
