This problem will be faced by many developers soon. The Internet is huge. Very large. The big companies are going to be dealing with huge data. You'll need to understand algorithms and math, and frankly, this stuff is a bit difficult to learn on your own. I thought I knew it all till I went into the algorithms class - that when I realized that not only did I not know it all, I was not as smart as I thought I was, and I would never have had the motivation to go through with this if I had not been forced to. And that goes for many developing.
Programming is a scarce profession now, but the simple stuff will soon be done by too many people. Software will become a real engineering task. In 20 years, the age of the code monkey will be gone.
Oh no, it will be cyclical. Programming is all about attaining higher abstractions, hiding more technology under a simple interface. Every now and then some new set of abstractions will be useful enough that we'll need a bunch of people to explore a field of opportunity. These explorers are called entrepreneurs.
Maybe in 2016, you're going to need deep credentials to be a useful web dev, but none whatsoever to start something useful with 3-D printing.
EDIT: that said, nothing makes you more employable than knowing things at a deep level. A friend of mine, a former Plan 9 kernel contributor, quit the tech industry after the first bubble to become a wildlands firefighter. Returned to the tech industry in 2008 and resumed being a highly-paid infrastructure geek like nothing had happened.
I think it will be the exact opposite. As computers take over more and more jobs from us, we will need less "office workers" who know how to shuffle documents around, and more workers with programming skills.
You want to be a mathematician? You need to know how to program. You want to be a "secretary", you need to be able to dig through your boss's e-mail using regexpes when he needs to find sth, you are a dentist - you will install your own scripts on the website because you know how to do it from high schools.
In 2016 (or 2012) it won't be "oh, we need more skilled programmers", it will be "sure, you know programming, everyone knows, but what really you know?". Programming will have the same place on CV like "MS Office", or "keyboard typing" has right now. No big deal if you know it, but much harder to find your job if you don't.
Of course there will still be place for real computer experts - algorithm designers et al, but the basics will be known to more and more people.
This isn't going to manifest itself as more people knowing how to program, though. Easier interfaces and smarter searching, but not programming as we know it.
In 1980 you'd say that in 20 years the average office worker would be performing calculations on thousands of rows of data, generating charts, typesetting documents, creating full color presentations, doing business with clients in multiple continents and they'd wonder how people would cope with the increase in cognition required to do all that. But it's just button-pressing for most people.
In 2016 they'll say "we have a database with 4 billion data points and we need to infer customer behavior patterns from it". You'll say "Sure.", sit at a desk, click "Segment", click "Demographic: 18-21", click "Intersect", click "Products", click "Make Recommendations", click "Apply" and a discount coupon for "Justin Beiber's Comeback Tour" will be beamed directly into the eye sockets of anyone who bought canned salmon last fall.
I don't see a society where 80% programs, I see a society where 10% builds things for the other 90% and a huge part of the middle class will be automated out of existence. This, to me, is the big issue that will shape this generation and the next.
Excellent thoughts. I think that more people need to know how to program their computers. But, as you have so elegantly pointed out, the inexorable march of progress will not bring this to pass. It hurts a little to think about, but in a large way you seem to be on the money.
These ideas are worthy of more than a two paragraph comment on HN. I third the notion that you should pen a full blog post.
Give this test to the next 5 random non-technical friends and family you talk to:
A = 1
B = 2
C = 3
A = B
What does A equal?
I'm not saying people can't be taught. But think about how big the workforce actually is, think about how widespread MS Office skills are. For every power-user analyst and project manager that's really taking Excel out for a workout, there are 10, 20, 50 people who use Office in every day non-challenging tasks.
I've given that little test to my MBA wife and a GP family member and several other people. Hardly anybody gets it right.
Edit:
The x-factor here, btw, that determines whether or not somebody understands it, is whether they see that assignment is happening, not some sort of "wha? 1 equals 2? what is that?" And those that didn't just get it, even after I explained assignment they were just as puzzled. Just the concept of variable symbols confused and (i presume) disinterested them.
What we do here everyday, this is difficult, challenging stuff, that I don't think most the workforce will ever understand. Instead, people like us will be busy for decades to come, building tools so they don't have to.
There was a time when machines were new concepts versus simple tools. You could say, in the early stages of the industrial revolution, that soon everybody would understand and be able to fix their machines. But machine complexity has out-paced the desire and ability to learn those skills.
I don't think it's unreasonable for people to assume that = means equality, not assignment.
Edit:
In particular, the link posted in the second edit has a rather poor test using a and b, because it uses = in two different ways with no indication that the meaning of the symbol has changed. Maybe the problem with the test isn't just the people, it's the sloppy notation that assumes people with no programming background are able to infer when we mean equality and when we mean assignment.
The population that took that test were self selected computer science undergrads!
And even after three weeks of instruction most of the people who didn't understand it immediately never understood it. I'll quote from the article:
"Either you had a consistent model in your mind immediately upon first exposure to assignment, the first hurdle in programming-- or else you never developed one!"
My wife is a brilliant woman, fantastic at what she does. The GP I mentioned in my post is a very good doctor who had no problems getting into a medical school, passing his boards, or running a successful practice. But that doesn't mean that everybody is meant to be able to understand the abstract concepts you have to master in our line of work.
I wonder how the experiment would change if you change it to "let A = 1, let B = 2, let C = 3, let A = B", or "make", or some other verb that seems more like assignment.
Or if this were explained to be a sequential process and not a just a descriptive list of unrelated declarations. It seems reasonable to think that that list contains a contradiction if you've never been exposed to these concepts before.
So education and the availability technology will eventually result in many people with basic programming skills will result in not only computer literacy in the "I can use MS Office" sense, but in the "sure, I can hammer out a Python script to get this task automated or sift through that data" sense.
Is there some sort of disruption of (basic) programming skills coming the way blogging has disrupted journalism?
I'm of two minds on this one. Part of me agrees completely with what you say. Further tools will exist that automate a lot of what the code-monkey does. The level of work done by a lot of us will be push-button, or "plug these couple of things together".
On the other hand, in 1998 as a nerdy guy getting out of high school, with minimal html, javascript and programming experience, and running linux on a pentium pro I faced a big choice. I went to the local ISP to pick up a "real modem" (vs a winmodem) to connect my awesome unix box to the internet. The guy there asked why I wanted these modems vs going to circuit city for some amazing sale they were having for a faster winmodem. When I told him I was running linux I was offered a $40K/yr job on the spot - just for getting linux installed on a computer and understanding the basics (quote "we can teach you anything else you need to know, you got the spark"). I was 18, and that was a HUGE deal. Anyway this wasn't uncommon, at the time wired was running stories about "HTML factories" where people were making pages and pages by hand all the time. Minimal programming skills got you a job.
Some of this was just normal boom-time labor shortage. There were lots of stories about how after the crash these guys would never work again. Some aspect of this was true, but some of it was bunk. The 2000 version of this story would be "sure you can do html, and you can do CGI, and you understand http headers and can whip up a server, but we need people who understand SQL and how to work with record objects and how to do live updates to a system, stuff you need a real degree for. It's 2005 not 2000"
So basically I am suggesting that while Rails may be a non-skill (like HTML has become) and maybe good REST APIs will be auto generated, and some Backbone.js future version or successor will do most of our tricky js stuff, there will likely be good toolkits that allow people to plug together data-mining and data-management without needing super deep algorithmic understanding, we are already seeing the emergence of such tools.
So the other part of me disagrees, the code-monkey will be needed, just that they will be putting together different bits than they are today.
> So the other part of me disagrees, the code-monkey will be needed, just that they will be putting together different bits than they are today.
It seems to me the Hiring Manager in the story was fishing for a jack-of-all-trades kind of applicant. The interviewee was obviously a web developer, but the position to be filled was a data analytics job.
What difference does it make if Rails/PHP/Node.js/Backbone/etc become commodity jobs? In order to get the raw data that requires complex and high speed algorithms to parse in to useable data, you'll still need a website, built by a rails/backbone code monkey and a Photoshop designer and with the help of a decent DBA, at the very least. The data position, if it ever comes to it, will just be another job type that a well rounded development team will need to fill, not a replacement for everyone else on the team.
"When I told him I was running linux I was offered a $40K/yr job on the spot - just for getting linux installed on a computer and understanding the basics (quote "we can teach you anything else you need to know, you got the spark")."
Yeah, my technical screen for an 8-year career in Schlumberger was exactly this.
Math is overrated. Nobody really needs math; if you find yourself needing some math you're probably reinventing some kind of wheel. Same with algorithms.
How do you deal with huge data: you just do.
There are tools for that, and you apply those and perhaps make new tools yourself, but math and algorithms tend to never enter the equation.
Programming is a scarce profession now, but the simple stuff will soon be done by too many people. Software will become a real engineering task. In 20 years, the age of the code monkey will be gone.