Basically is anyone of the opinion that we will ever be able to create software development methods which are significantly (orders of magnitude) more productive than what we have now? And if so, will we ever reach the point where users/customers/clients can have very capable and sophisticated custom made or customized applications for a price that warrants "wasteful" or "trivial" (for trivial reasons) software development?
Don't forget the one key issue that almost all hackers (including myself) continually overlook.
We do the HOW.
They determine the WHAT.
Doesn't matter how efficient we get at doing the HOW until someone figures out how to get more efficient determining the WHAT.
All of us (users, hackers, managers, entrepreneurs) are so bad at determining the WHAT that we don't even try anymore. We build something "close" and we iterate until it's "close enough".
Until someone comes up with a methodology or technology for tying down the WHAT into a rigorous functional spec, I wouldn't worry about anything becoming too cheap.
So perhaps we've been focusing on the wrong series of problems when it comes to software development productivity. Doing the HOW is not the bottleneck, but rather determining (rigorously enough) the WHAT that is the bottleneck.
It won't be "wasted" it will be performed by end users, in much the same way that Excel allows financially knowledgeable folks to develop applications in a few hours that would have previously taken man months. Wikis and blogs eliminate the need for most of what a "webmaster" used to do (put new content on a website). Google Alert does what a clipping service used to do. I am sure that folks here can think of many other examples.
This is already occurring. I work at a large engineering company. The engineers, even those with no coding experience, are writing one off scripts to solve problems and then throwing them away. Some write 10-15 scripts a day. All of which used to be done by IT. Our IT department has shrunk to glorified data managers. Make sure the data is reliable and the engineers write all the logic themselves.
It's much nearer than you expect.
It may not be agile or maybe it's the extreme of agile.
Our finance guys started using the scripting language just 6 months ago.
The requests are drying up from them as well...
None of the script would pass even basic IT standards, but they work. And they only have to work once. Answer the immediate question. They don't care if it takes 1 min or 8 hours.
It's not pretty, but it is disposable code with no IT involvement.
As performance increases we'll to have worry about it less and we'll get the luxury of choosing more powerful abstractions. This makes us more productive (e.g. garbage collection - static typing too is partly compiler optimization).
Regardless of performance increases, sometimes more powerful abstractions appear anyway. e.g. the relational database, Amazon S3, the git filesystem.
Better abstractions are what makes us more productive, not methodologies.
Will we ever be able to create abstractions or systems of abstractions that make software development significantly (orders of magnitude) more productive than what we have now? And if so, will we ever reach the point where users/customers/clients can have very capable and sophisticated custom made or customized applications for a price that warrants "wasteful" or "trivial" (for trivial reasons) software development?
I think there will be types of software that folks will be able to bark orders into their microphone and have the computer barf out software. Maybe an iTunes app creator. "Make it pink! Ok, now put the play bar on the left. No wait, the right. A little more. Good, right there. Now I want my playlist to be black with yellow text and put pretty stars everywhere."
In engineering land, we continue to get better tools for building certain type of applications. Ruby on Rails has shown us the benefits of constrained thinking. A blog in 10 minutes! But Rails only excels at building one type of application, standard fare web apps, with loads of standard database-driven tables/forms.
It's very very hard to generalize programming. We have a long way to go still. Software gets the job done today, but often, it's not pretty and often fights against the user.
"Ok, now put the play bar on the left. No wait, the right. A little more. Good, right there."
Users would put up with this for about 10 seconds before just grabbing the mouse and dragging things where they want them.
I think that's why voice interfaces have not become popular. PC applications tend to be developed for a direct manipulation (window, icon, menu, pointing device) paradigm, so adding a voice interface doesn't help much. A useful voice interface requires an entirely new UI paradigm.
> ... A useful voice interface requires an entirely new UI paradigm.
I think this would require natural language processing far better than what we have now and great deal of contextual knowledge as well as a great deal of "common sense" built into the software (agent? interpreter?). Mice and keyboards remain easier to work with.
The question is what applications you consider trivial - and herein lies the answer. What is considered hard now will be trivial in the future, and what is considered impossible now will be considered hard in the future.
Take a simple calculator application for instance. If you asked a team of engineers to build one in 1950 they would consider it hard but possible given some months of work programming their punch-tape spewing tube-filled constantly overheating monster of a computer. Now it is considered trivial.
In the future it will probably be as simple to build what we consider a complicated application as it is for us to program a calculator application. But the only thing that has changed is that the bar has been moved up a few notches.
What about whole databases and the various applications that use them? one time complex system models for climatology or finance or spread of diseases? Simulations of alternate systems designs for traffic and infrastructure to optimize particular attributes? Will that ever get "simple"?
Basically is anyone of the opinion that we will ever be able to create software development methods which are significantly (orders of magnitude) more productive than what we have now?
His argument, is that there isn't a single programming methodology that creates order of magnitude productivity gains. It's worth a read.
I don't really think that the answer to your question lies in user of programming methodologies. I do think that you can get great improvements in productivity based on tool selection, however. See PG's essays on beating the averages.
will we ever reach the point where users/customers/clients can have very capable and sophisticated custom made or customized applications for a price that warrants "wasteful" or "trivial" (for trivial reasons) software development?
I think it's already being done.
Non-programmers are already using specialized scripting languages like BBS code, or html tags to customize their web pages, forum posts and blogs. People are using Excel scripting to write their own mini financial applications. Graphics programs like Maya, Houdini and After Effects have built in scripting languages for procedural graphics. And, years ago, the Word Perfect word processor was built on the idea of a template language for document layout.
I'm already working on a language that does this. It's called Maroon, and it's as high-level as they come. For instance you'd type in "make fantasy sports site" and have one working in seconds. Or "clone Google".
I'm going to put the project on elance, since I obviously have no experience writing programming languages. How much do you want to bet some Indian firm says they can program it for $1,200 in 2 weeks?
Jerry Weinberg told a story about how IBM used to send him around to their most expensive projects. He would report back about which ones deserved further investment and which were hopeless and should be killed. Usually they took his advice, but there was one he couldn't get them to kill no matter how hard he tried. This was a project to create an "automatic debugging machine". The executives were too enthralled by the diagram the people on the project had drawn for them: a box with an arrow going into it saying program with bugs and an arrow coming out the other side saying program without bugs. (At this point he was met with incredulous laughter and insisted it was true.) Although they had sunk millions of dollars into this with nothing to show, the executives had been so burnt by programs with bugs that they refused to give up the idea.
At long last, Weinberg said, he finally came up with the argument that got the project killed. It went like this. "Say you have an automatic debugging machine. I could feed it a blank sheet of paper, and out would come the program I wanted. So you'd have to have a mind-reader in there!"
Of the two bids on your elance project, one is from a German company and the other is from a US student. I suppose now you'll stop berating Indian firms? :-)
1) If you make something (i.e., a software development tool) that does everything, it will do it really badly. That's unlikely to change soon, if ever.
2) When you put the invisible hand at work, if developing software of type X becomes too easy, there will be a surplus of X apps, so it will get unprofitable and people with good software development skills will start pushing the envelope and invent Y apps, etc. Because every piece of software depends on another one, the software that interacts with Y apps will also need to advance, so it's an infinite loop, if you will. :)
That is the problem that plagues Lisp and to a lesser degree other powerful languages, because you can develop things so quickly and express yourself easily, a lot of code is single use, throwaway code. No one bothers with developing a library, complete and properly documented, for something that only took you a couple of hours to write.
I couldn't find the original article but the scene in a Star Trek movie where a photon torpedo is reconfigured is a utopia of what programming/integration could be in the future...
Unless we become extinct, there will always be people whose expectations exceed what is trivial to produce with the existing state of the art.
The only question I wonder is how long before software development as we know it becomes no more intrinsically difficult than other forms of expression (e.g essay writing, public speaking, etc).
Several people have pointed out that people lacking the word "programmer" in their job description write code to get their jobs done. But what happens when they have two scripts that look similar to them but one finishes in an hour and the other is still running two weeks later? Then you need someone to break out the big O notation, look at the source code for the high level language and figure out what's really happening.
The need for this will continue to diminish over time in most firms. However, for companies that provide software over the internet this will remain very important (make something at Google 5% faster and you're probably saving them a whole lot of money). So, just one more reason to make sure your career does not hinge on getting paid to write software at a place where software is not the main source of revenue.
depends on what method you mean by "we" Do you mean agile methods are not working for your team or do you mean the old waterfall method is not working for you?
I think it doesn't matter to much what method you use just as long as you have excellent communication between client and the end development team.
Agile works well enough for our teams at work as long as everyone is aiming for the same goal. I used waterfall only once and it ended up a "failure" or a "qualified success" depending on your point of view. What we produced wasn't really what the users and customers needed or truly wanted even though it was to the spec that we all agreed to.
Think beyond agile and other current cutting edge methods. What I'm really asking about is will software development ever become so productive that significant, useful applications could be created for such a low cost that "throwaway" one-time development that occurs only to take advantage of fleeting opportunities or needs is fairly common and is not considered "extravagant" but rather a smart thing to do?
i hope so. digital cameras let me take tons of photographs without worrying about cost. it would be cool if i could turn out 100 Web apps every day, just to see what works and what doesn't.
We do the HOW.
They determine the WHAT.
Doesn't matter how efficient we get at doing the HOW until someone figures out how to get more efficient determining the WHAT.
All of us (users, hackers, managers, entrepreneurs) are so bad at determining the WHAT that we don't even try anymore. We build something "close" and we iterate until it's "close enough".
Until someone comes up with a methodology or technology for tying down the WHAT into a rigorous functional spec, I wouldn't worry about anything becoming too cheap.