What a wonderful spirit he was, never to be forgotten. We had lots of great interactions over the years, and he greatly enriched computing with his ideas and actions.
Years ago I emailed Joe Armstrong about concurrency stuff. I had no idea about anything like multithreaded programming or distributed systems or anything like that, but I had read Erlang was a language well-suited for that stuff.
I sent an email asking some seriously noobey questions about concurrency (about why mutation was bad, what made the Actor model work, etc), and instead of a response like "Go read a book and stop bothering me", he responded back with an incredibly long, well-written email explaining a lot of the minutia of how Erlang avoids a lot of pitfalls and generic concurrency theory. He was really good about explaining things in a way simple-enough for me to understand, without coming off as patronizing or rude.
About a year later I got a job doing Erlang, and I sent him another email telling him this, saying something to the effect of "sorry for bothering you a year ago, but your email was really helpful to me".
His response was basically "You have nothing to apologize for! I've always thought it was important to help people asking questions, especially early in their career".
I can honestly say that his kindness and patience changed my life for the better. If he hadn't held my hand a bit to make distributed systems make sense to me, there's a very high likelihood that I would still be writing Flash applications for a Tae Kwon Do studio.
It occurs to me that this is one fundamental commonality across great people that includes likes of Einstein to Feynman to Knuth to Erdos. They can sense the curiosity of a person regardless of their background or expertise and cannot prevent themselves from infecting you with the fundamental joy of discovering nature's secrets. Passion is something they feel obligated to spread even if you are just a gardner or a cook or a high school student while they happen to be towering figures with Nobel or Field medals. If you have that email, please do share!
I actually did share that email in a sister thread; I also wrote about it specifically in a blog post [1] if you don't want to traverse through the comments.
I completely agree with your thesis. I was a recent college dropout, and Joe's enthusiasm about Erlang, distributed programming, and pretty much everything else about computers was really a good springboard for self-education.
Thanks for sharing that experience. This matches my interactions with him at a couple conferences. Always willing to spend a moment explaining things to anyone. I had the pleasure of drinking beer with him at one of them and he was genuinely excited when I asked to have him sign my erlang book. His Twitter was one of the few I followed. His teachings and musings changed how I view software engineering. He will be missed.
> he responded back with an incredibly long, well-written email explaining a lot of the minutia of how Erlang avoids a lot of pitfalls and generic concurrency theory.
I love that about Joe. He really really love Erlang and the thing he built and share it with everybody. It shows in his blogs and the talks he gave at conferences.
This only makes me want to cry.. I never got the opportunity to meet him but I will miss him so much. I've always known since high school that his work in certain ways just had no equal. The Open Telecom Platform is straightforwardly just a masterpiece of programming - it basically turns any server into this data processing god machine with almost no possibility of a (code) failure.
Basically, he explained that shared-memory systems simply don't make sense for distributed programs.
It was years ago, but I believe he said something to the effect of "imagine that there's a server in Europe, and a server in America. Where does that shared memory exist?? Shared memory violates the laws of physics!"
He then explained that since Erlang processes don't share any memory, and the only way for them to communicate is via message-passing, there's no need for locks or anything like that.
I was able to find his reply. The stuff after the `>` is him quoting me. There's another part to this somewhere, but I'll have to keep looking.
> Hello!
> I am learning Erlang and really liking it.
Great
> I am finding myself really liking the immutability in regards to being able to reason about my code, but that leaves me with a probably easy question: why does immutability make concurrency easier?
Tricky I guess it makes parallel execution easier (no locks) - it certainly makes programming easier
closures really are closures :-)
> Is it because it's difficult to have something in a thread reach-across and modify the state of another thread?
That's not really immutability - actually I hate threads - the difference between a thread and a process is
crucial - threads can modify shared state (horrible) -
In distributed systems there is no real shared state (imagine one machine in the USA another in Sweden)
where is the shared state? In the middle of the Atlantic? - shared state breaks laws of physics.
State changes are propagated at the speed of light - we always know how things were at a remote site
not how they are now. What we know is what they last told us. If you make a software abstraction that
ignores this fact you'll be in trouble.
> Does that mean concurrency wouldn't be so bad if you kept state mutations contained within one thread?
Yes - if you use the word process, not thread. Processes were invented to provide protection
from other computations and to share resources in a clean way - threads share resources in a messy way.
> If you don't have time to answer right now, a link or some guidance into the right resource would be appreciated!
I think what you're interested in is the distinction between message-passing-concurrency and shared-memory-concurrency.
Ah, found the email he responded to a year later. As before, the stuff after the > is him quoting me.
----------------------------------------
> Hello!
>
> Just under a year ago I send you a message asking you about multi-process computing and the like. You gave me a long, detailed email that not only answered my questions, but also explained in pretty-decent detail.
>
> I was so impressed by that email that I actually purchased your "Programming Erlang" book, and enjoyed that just as much, and today I started my first job doing Erlang full-time (well, mostly; there's some JS involved).
Great - thanks a lot for buying the book - these kind of stories
make it feel all worth while
> I just wanted to thank you for the help. I know a compsci-celebrity probably gets bombarded with emails,
Actually not - I guess it's a 'being polite' thing - I rather like twitter it
lowers the entry barrier for communication between people.
I have in the past mailed several of my CS "heros" - and was always
rather reluctant to do so. Then to my great surprise I often got long
and helpful replies (from Niklaus Wirth, Alan Kay, Jim Gray)
This kind of fits in with my view of life - 95% of people are basically helpful.
I've always thought that converts were gained "one at a time" - imagine
a drip drip drip of water falling into a pan - one day it overflows.
We always think it's fantastic to hear the stories of what you guys do with
Erlang. (I say we because the Erlang you're using is the product of
many people fantastic work) - I'm just the fall guy who writes the books
and happened to start it all :-)
I was sat listening to Brian Acton who founded WhatsApp and made 3.5B$
and thought "I invented this crap" - which is a great feeling - and they gave me a tee-shirt.
So the side effect of what I do is I get to meet loads of interesting people
and do what I want.
Anyway thank again, and I hope things go well for you.
> and I think it's really cool that you took a good amount of time responding to some random guy on the internet asking for help.
Well every body is a random guy until you share stories.
My wife was a random woman before we met.
Anyway life would be very boring if it were not for random events.
I never run for busses or trains - it was after I read about a guy who missed
a train and had a random conversation with a guy on the platform
who had also missed the train - this gave them an idea that they
turned in a multi-million dollar business.
I guess the other reason that you get long replies is that I'm an author
and I don't do short mails.
> As usual, keep up the great work.
I'm trying - though I've recently retired - gives me more time for hobby programming - though I have to tend my garden and so on
Please consider sharing this email exchange with lettersofnote.com. I am not affiliated in any way with that site, but I have found it to be a great source of inspiration and a reminder that very successful people are still people, and often very approachable and kind people. Reading this sort of thing can restore one's faith in humanity, and provide impetus to change one's own behavior towards kindness and generosity.
Yeah, I'm glad I was able to dig it up. It has an almost-inspirational sentiment to it.
Also, rereading this line genuinely made me laugh:
> I was sat listening to Brian Acton who founded WhatsApp and made 3.5B$ and thought "I invented this crap" - which is a great feeling - and they gave me a tee-shirt.
Me too; it's an cleverly simple way to say something that seems obvious, but bizarrely isn't. I guess like Erlang itself, Joe was good at expressing the complex tersely.
Back in 2015, I somehow ended up as part of a panel discussion at the end of a conference. The panel was going to include Joe Armstrong, Don Syme (inventor of F#), Tony Hoare (!), and myself (a nobody compared to any of them). During the conference, I wanted to meet each of the other folks on the panel, and ended up in a somewhat lengthy conversation with Joe Armstrong -- mostly about Erlang, but at some point the panel itself came up. I mentioned how nervous I was; me, sitting on a panel with such other esteemed and brilliant folks -- of course referring to all of the other people on the panel, including Joe Armstrong himself. I'll never forget his response: "yeah, me too! Wow, Tony Hoare, seriously?!"
I'm not sure if he deliberately misunderstood me or not, but it definitely put me more at ease, realising that there's always someone bigger, even for the folks you see as bigger than you!
Not at all hard to believe that he was sincere. He seemed to be pretty starstruck when he was interviewing Alan Kay on stage just three years ago: https://www.youtube.com/watch?v=fhOHn9TClXY.
And his twitter @joeerl [0] which I feel unfortunate to have just discovered today is filled with him discovering emojis and dictation, talking up the joys of letters, and discussing FPGAs.
It’s sad to hear of his passing, and rewarding to see he was just as kind and curious as you could ever hope to be even just this month.
Other people have already mentioned Joe Armstrong's 2003 thesis, and I recommend that too.
I very much appreciate Armstrong's sometimes contrarian criticism of the tech industry, and I hope I managed to match the tenor of his frustration when I wrote "Joe Armstrong figured out the right way to do everything, and nobody cared"
"I mentioned how nervous I was; me, sitting on a panel with such other esteemed and brilliant folks"
I feel the same when I get put into lists with folks like Joe Armstrong. I've made no significant contributions to the tech industry, whereas other folks have made historic contributions. I'm flattered by stuff like this:
Joe always seemed like a super approachable and super humble guy. I never interacted with him but he always left a good taste in my mouth whenever I watched him on video--which happened quite often in the period during which I was obsessed with Erlang.
That is so sad, he was in his late 60s and it seemed like he had a lot of life left in him, talking about how software can get better and having a great (and sometimes snarky) outlook on the profession.
Highly recommend his thesis (2003) and a few of his great interviews/presentations for anyone who isn’t familiar with Joe, it captures a lot of what he thought about and pushed for in his professional life.
> That is so sad, he seemed like he had a lot of life left in him talking about how software can get better and having a great (and sometimes snarky) outlook on the profession.
I'm in actual shock, he was tweeting about pretty much that (also brexit and playing with his phone's voice recognition) just 2 weeks ago… He wasn't even 70…
It goes much further than that, it shows how to tackle reliability even in systems that are not distributed. The primary insight is that all software will be buggy so you need to bake reliability in from day one by assuming your work product will contain faults.
Yes, I know. Erlang was not distributed till 1991, roughly 5 years after it was born.
It's also really illuminating how they implemented the first versions of Erlang as a reified Prolog [1]. But that is not explained in the thesis, just in his 1992 paper which he briefly cites.
"And then we've got this sort of dichotomy between efficiency and clarity. You know, to make something clearer, you add a layer of abstraction and to make it more efficient you remove a layer of abstraction. So go for the clarity bit. Wait ten years and it will be a thousand times faster, you want it a million times faster, wait 20 years."
That's... one way to make something clearer. It's also a way to hide complexity behind leaky facades. "Just add a layer of abstraction" is horrible advice.
Thanks a lot, I never watched this, it’s absolutely awesome.
I laughed a lot and
I actually cried in the slide about legacy code when the first line was “programmers who wrote the code are dead” :(
It is a great talk. Based on that I took his course on Erlang at FutureLearn, which was also very good. Sadly, when I sent that link to some colleagues at work they shrugged and have since kept adding more and more dependencies and complexity to the front-end build. It's very hard to stop momentum once it's got going, culturally.
Guess I'm alone in not seeing what's so good about this talk. He only presents problems, and in an extremely disorganized way. The closest thing to a solution is "wait, hardware advancement will eventually make your slow code fast."
It's definitely entertaining, but that's about all I can give it.
There's a time and place for things. Posting multiple critical comments in a commemoration thread is maybe not the best time or place, and better kept for later?
The idea of "no copies – everything exists in one place and has a unique ID" was new to me. I still don't know if it's a good idea, let alone practicable, but it's great food for thought!
> At 17, Armstrong began programming Fortran on his school district's mainframe. This experience helped him during his physics studies at University College London, where he debugged the programs of his fellow students in exchange for beer. While working for the Ericsson Computer Science Lab, he helped develop Erlang in 1986.
This is pretty legendary. He was quite the programmer.
Joe was so full of life, consumed by curiosity. I had the pleasure of escorting him around Chicago during an Erlang conference several years ago, and I still have in my Reminders app to look up a water painter named Zorn that he recommended highly as we tried to find our way to some building or another.
He’s also, I believe, my only LinkedIn contact. I opened a new account a couple of years ago when Basho was winding down, but I strongly dislike the service and didn’t do anything with it...but when Joe found me and added me, I had to approve that, obviously.
Godspeed Joe. I’m sure if the interface between this world and the next is in need of debugging, you’re the man for the job.
“Make it work, then make it beautiful, then if you really, really have to, make it fast. 90% of the time, if you make it beautiful, it will already be fast. So really, just make it beautiful!”
Last year, I was at a conference giving a talk about error handling and Joe Armstrong was in the audience. I went through a bunch of strategies for mitigation and handling of errors and Joe yelled from the back "just let it crash! just let it crash!" I had to reply, "but we're not all using Erlang yet."
It was superb, I think it's unfortunate that he had to rush through it because of time constraints, his anecdotes and delivery were really engaging and funny.
From the day I learned about his work he became my greatest influence in my career, and still is to this day. I'm grateful for all the insights and the perpetually fresh perspective, on distributed systems, on PL research, on databases and on OOP.
I'm grateful that I have someone like him to look up to in my profession: able to solve hard problems while still having fun with novel solutions, while still keeping it all simple.
We'll take at least 20 years just to catch up with him.
I got really into Elixir/Erlang a while ago, and it made me love programming again. One of the first 'useful' things I built was to scratch a personal itch: tracking all the comments on HN that I've read and, with a bit of JS, highlighting any new comments.
Erlang, OTP and Elixir were pretty new to me, and on top of that I built everything in a rush. The code is messy, naive, and the whole thing barely works. Over time I've added some other Applications (in the Elixir/Erlang sense of the word) that scratch various other itches. Most of that code is also pretty shit.
The fact that the whole thing has been chugging along for over a year, with no restarts or crashing the entire system, is a regular reminder to me of Armstrong and the "let it crash" philosophy. I sometimes log in to the server and face a wall of red because the happy path wasn't followed, and instead of having to fix the bug and start the thing up, I can just forget about it. Or I fix the bug, enter recompile() in the interactive console, and get on with my day.
I'm not exaggerating when I say that seeing how my duct-taped-together contraption chugs along and keeps doing its job gives me a small sense of joy every single time I SSH into the server, and it motivates me to get better at all this so I can do the same on "MongoDB level web scale".
And it's not just Armstrong's work. His openness and participation on the Elixir forums, his 'stamp of approval', so to speak, as well as his talks, played a big role in my decision to get into Elixir/OTP.
I met Joe in 2012 when I spoke at the Erlang Conf in SF. he is the reason I discovered functional programming and the whole concept of message passing (actor model) in distributed systems. Quite a legend and contributor to the world of computer science. RIP joe.
I also met him at a conference in '16. He gave a talk but mostly wanted to shoot the breeze in the hallway with fellow hackers. He lamented scheduling gripes with his PM, being a slave to Jira, and some new things he was playing with. Very approachable and humble.
He asked a few of us hanging around what talks we were thinking of attending and we agreed to check out something about concurrency in some modernish platform. The speaker was working on issues with handling failures, message passing, memory management, etc, but then realized Joe was in the back and pointed out it had all been solved many years ago by that guy over there. Everyone laughed and gave Joe the acknowledgement.
I don't feel like we've finished learning from and building on his work. RIP.
This is devastating. I’m in shock. Joe was one of my programming heroes and it was one of my goals to, one day, see one of his talks in person. He was incredibly articulate, good-humored and didactic in his posts on the Erlang list. He will be very much missed.
This is absolutely devastating! His talk "The Mess We're In" was one of those talks which were incredibly funny and informative at the same time. I absolutely lost it, when he told the story of the single comment his coworker put into the Erlang code.[1]
This recent panel (Armstrong, Hewitt and Hoare) discussion on concurrency was the first time I got to know about him and it gave me a lot of food for thought.
RIP Joe. Every interaction I had with you in the past was always a great time. You'll be missed, but never forgotten. Thanks for everything you've done for the Erlang community and distributed systems design.
5 or 6 years ago I saw a presentation of Joe in Infoq. There he said "solution of the dining philosophers problem is to give fork to everyone". It really made me interested in actor based system. Now here I am, building financial system in java where akka is the vital component of the system. Thank you very much Joe. Rest in peace.
Life is short and Joe went much younger than one would expect. When I find myself doing something that I do not enjoy, I'm always trying to meditate to the moment of my future-death in order to psychologically slap myself in the face and re-evaluate what it is that I'm doing. We only get so many years and we should strive to use them to the best of our ability and potential. Joe Armstrong certainly did that. RIP.
Erlang is one of the most Perlis languages for me. I'm always a little bit cynical and think Erlang deserve more attention. Several months ago Joe post this tweet about a game company refused to tell other people they are using Erlang.
That is pretty awesome, and I love the fact he had to come back and say, no seriously guys, I was just kidding, no need to flood Erlang. Man I swear Erlang is highly underrated. I've not had the pleasure of building much with it, only read through parts of the book by Joe and kept up with some of his blogs and posts from time to time. Anybody who loves servers should really pay attention to his work.
I remember his joke .. Erlang is "write once, run forever". Fascinating. The BEAM will run forever since there are also other languages compiled to it.
I was following him on Twitter, and it was always amazing that a genius like him took the time to respond to people on Twitter with insightful comments. Showed what a great human being he was. RIP Joe.
Not just that, but he also struggled in public with things we all struggle with, often with incredible humor. His struggles with Grunt.js come to mind. His humanity and charisma was in full force, and you can tell he worked in full grace and gratitude. His work of course is a testament to this.
The black bar usually indicates the passing of someone I'm not especially familiar with. When I looked down the page and saw Joe's name I was really caught off guard. I never met him but followed him and the Erlang scene long enough to know we've lost someone special.
I only recently came across his Twitter account and started following him (he had some interesting ideas about and experiments with TiddlyWiki). He seemed like a passionate and enthusiastic technology lover, which was a good enough reason to follow him.
And it is only just a few minutes ago that I discovered that Joe is the guy with the moustache from the famous Erlang: The Movie video…. I had not pieced those two things together.
So I knew nothing about his history, just one little thing he was working on/interested in. That may make me seem ignorant, but I share this because it goes to show that the light that shines in people like Joe was a genuine light of curiosity and sharing. Bright enough to attract people like me.
So sad. Humble, open minded genius. I was hugely impressed the first time I learned about Erlang and impressed again when I saw how welcoming and open minded he approached Elixir and it's community. A great man has left the planet :-(
This is a shock. I have been following his twitter and all is looking well. Learned a lot from his erlang paper. Wish I had the courage to write him an email and thank him. I can't believe it.
I'm another person that learned about Erlang through Elixir. Never got to talk to Joe Armstrong in person. But I do want to acknowledge that his work with Erlang, via Elixir, has given me the opportunity and hope of becoming an entrepreneur this year. I'm thankful for what he has given to the world. Thank you, Joe Armstrong, rest in peace. ️
He sounded like a wonderful person in terms of ideas, communication & empathy. I feel sadder than on the day Alan Kay passed.
To those with whom I share the morbid curiosity about how it happened, I'd like to mention a movie ending that came to mind.
Griever: Tell me how he died.
Friend: I will tell you how he lived.
Foot in mouth there! For that, I apologize as profusely as possible online.
P.S: FWIW, I was thinking of John McCarthy there, must have gotten confused by recalling Joe Armstrong's interview with Alan Kay. I am also a big fan of Alan Kay, landing up at Erlang's doorstep after playing a little with Squeak.
I had Joe as a lecturer in parallel Erlang back in January and February this year. Amazing person, seemed like he knew everything about computers. He told us that he was sick with lung fibrosis. His lungs capacity was at 50% something and kept shrinking every day. The last few lectures he was supposed to give got cancelled due to sickness. Got the news that he had passed today. A very sad day I must say.
I keep refreshing looking for the answer. Seems that it would be the top question for somebody who was 30. He was 68 but he had way more energy than many 20 year olds that I see.
I'm deeply sad. Despite being absolutely brilliant he was also extremely positive and full of humility. It's a rare combination.
I'm always curious about these things because if it's something medical that might be preventable or that has some nonprofits raising funds for, etc., then that can increase awareness. But I can understand if people don't always rush to talk about it.
>>> This is Helen wife of @joeerl It’s fantastic to read all these comments. Joe died from an infection which his lungs couldn’t cope with due to a quite recent diagnosis of pulmonary fibrosis. Incurable and rather unusual. Donate to lung research!
This is incredibly saddening. Joe has left an incredible legacy for us to continue on. I'm saddened that I never got to meet him and thank him for all that he has done.
I've known about Erlang for at least 20 years, but I never took the time to learn about it.
After reading about it, and more importantly after reading user "tombert"'s interaction with Joe Armstrong, I think I'm beginning to understand the significance of our loss of him.
Godspeed, Joe. You set a high bar for many of us to aim for. Your patience, kindness and desire to help and spread knowledge is truly inspirational.
No! Without question one of the finest computer scientists I have ever met. Fearlessly independent and willing to try anything and everything.
Edit: the thing I loved most about him was that he was brilliant but also kind. A living breathing rebuttal to the idea that it’s alright to be an asshole in this field if you’re smart enough.
Sad to hear this. I've begun picking up some Erlang, mostly Elixir over the past few years, and have watched (and watched and watched and re-watched) most of his various (brilliant) talks, all of which are eye-opening.
Aside from his technical skill and knowledge, he seemed like a really gret person with an undying sense of humor and humility.
Erlang was one of the first interesting languages I learned after the “basics” (C, C++, Java, JavaScript, etc) and even though I never actually used it for anything real, just learning it opened my eyes to a bunch of concepts that have been useful over the years.
RIP.
Thank you for Erlang, the great programming language, it saves me a lot of time. And thank you for the book: Programming Erlang. It guides me to think the programming world in different, interesting, and simpler way.
What an incredible guy. I’ve interacted with him and Robert briefly on the Elixir forums a few times, and I can’t think of anyone more giving, humble, and insightful. His presence will truly be missed.
Joe was such a kind person and was one of the few role models I had (I'm not a role model type of person, so this is saying a lot). You'll be missed more than you know, Joe. Rest in peace.
Truly heartbreaking. One of my favorite computer scientists of all time. Loved his perspective on all things programming. So much wit, and such a vibrant and wonderful guy. Joe, you will be missed.
Joe had to be the most engaging, famous developer I’ve ever heard of. Very sad day. He was so supportive of the Elixir community and a true blessing to the people who were able to learn from him.
Rest In Peace, Joe. I’m surprised and ashamed that I have not heard about him but thanks to the HN community for posting links to his thesis and talks. Wonderfully approachable and a great guy.
Possibly the most influential person in my programming career. Always curious, always full with ideas, but never lacking a pragmatic approach to the real world.
This is indeed sad news (I feel he deserves the black bar of mourning once confirmed). Although Elixir gets the most attention these days the solid engineering of BEAM is the foundation that Elixir is built on.
Joe was also a part of the Elixir community. He learned the language early on and participated in the mailing list and forums. Actually, he had just very recently become an admin at elixirforum.com. I will really miss him.
That's not what you're doing. You're shitposting about Elixir developers--while Joe was enthusiastic about Elixir to the point of being a mod on elixirforum.com.
"Keep your friends close and your enemies closer."
If you knew Joe, you'd know he was strict about feature-creep in Erlang. He removed Paramaterized Modules, even though it broke some popular Erlang projects -- including the Chicago Boss Web Framework. This was one of the reasons Elixir was created. To make it more "approachable" to people who didn't have the inclination to be FP purists.
Very sad news. Thank you to HN mods for commemorating Joe's passing with the black bar. His contribution to our world is immeasurable. Although I never had the pleasure of meeting him in person, by all accounts he was much loved. Joe you will be missed.
I like the black bar, it shows both mourning and respect.
Occasionally I'll see the black bar and the notice and not even know who the person was - at which point I get to find out about someone amazing I didn't know existed and to appreciate their contribution.
Obviously I'd prefer that there not need to be a black bar though.
