I almost spit out my drink. High barrier to entry? The barrier is lower than it ever has been. Any nut-job with a few weeks of training can make a shitty website with PHP or Node.js and have it instantly accessible to the most of the English-speaking world.
Most of the tools I see for software development seem to be organized around the needs of the bazaar (or thatched huts) not the needs of the cathedral. A million toy languages which might solve your problem well, but don't scale to a million users. Websites like GitLab and GitHub so you can share last week's 1 kloc project with collaborators. Libraries that do that one weird thing you need for your project, and nothing else.
By comparison, the cathedral builders (Google, Facebook, Apple, Microsoft, etc.) seem to be building a lot of their own tools. This includes programming languages, frameworks, build systems, version control, operating systems, and so many other things. They build their own stuff because the tools of the bazaar don't work quite well enough for cathedrals.
> Most of the tools I see for software development seem to be organized around the needs of the bazaar (or thatched huts) not the needs of the cathedral. A million toy languages which might solve your problem well, but don't scale to a million users
Not the OP, but when I see comments like this one I do realize that sometime HN has a very strong echo-chamber. The world doesn't need more than 100 (give-or-take, maybe 1,000, maybe 10,000) apps/websites which need to scale to "millions of users", a situation which doesn't keep that many workers occupied (Google&FB and the like employ much less people compared to the industrial giants of the early 20th century).
But the world does need millions of apps for the 10-100-1000 users, if needed built using the "toy languages" you decry. If we make it easy enough for people to build these apps, using "toy languages" if need be, the world would be in a much better place (we'd have higher productivity).
I'll give you my example at the company I used to work for in the early 2000s (when the "Cathedral and the Bazaar was written"). I was doing some office work, along with my other 20 or so colleagues, which involved having to check that two separate folders on our computers had the same files. This took each of us about one-hour each, so, there were 20 man-hours spent each working day on this mundane task. Lucky me I was a (already close-to-dropout) CS student, and I had heard about Python and about how easy it was to do stuff with it, and lo and behold, it really was. Just:
> import os
> l1 = os.listdir('first folder')
> l2 = os.listdir('second folder')
>a_call_to_a_custom_function_which_was_comparing_l1_to_l2() //which was probably quadratic, but it didn't matter
then use py2exe to put it all up in an .exe file which could be also run on my colleagues' computers (along with some inputs and the like) and that was about it.
A task that used to take an hour each day now was only requiring a script/program call. I fail to see how this program would have required a grown-up language which would need to be scaled to "millions of users", even though it proved to be pretty useful. And there are countless examples like my anecdotal one I gave above, you just need to go into any institution or company office, look at how people work on their computers and realize that the world needs millions of small programs like mine that would substantially increase productivity. The problem is, like the OP said, that us, "programmers", like to keep the playing field only for ourselves.
I think there's a huge unmet gap there that's actually being blocked by big businesses for business (not engineering) reasons. The ability of end-user to quickly and simply tune their device to their needs and automate stuff.
The very concept flies in the face of what is today's accepted UX "best practices", i.e. to make software trivial, engaging and masterable in 5 seconds. It naturally happens by removing any kind of thing there is to be mastered.
The task you performed with Python should be easily scriptable at the OS level. It shouldn't require one to know complex programming languages and toolkits. Similarly, I think that a tool like Tasker[0], maybe with a bit better interface, should be available by default in vanilla Android. We're vastly underutilizing the power of computing devices by restricting end user's ability to work with them.
And there are countless examples like my anecdotal one I gave above, you just need to go into any institution or company office, look at how people work on their computers and realize that the world needs millions of small programs like mine that would substantially increase productivity. The problem is, like the OP said, that us, "programmers", like to keep the playing field only for ourselves.
You're going to need a better example than that. This program already exists, it's called diff(1), md5sum(1), or cmp(1). You could wrap its use up in a shell script to make it even easier, or the companies/people could spend some money/time to learn how to use the tools already at their disposal. In a lot of cases, lack of training is the issue that should be addressed. I've said before "Those who don't learn /bin are doomed to reinvent it, poorly"
This isn't to say you're wrong about a lot more little, customized programs that could be written. The focus needs to be on the specific processes of a particular company, because every company's processes are unique (maybe not on all axes, but on more than one). Enterprise software often errs in either of the two extremes: either it's overly customizable and doesn't fit anyone's needs completely; or it's highly specific and tries to force working its way. And this is done out of a desire, from the software vendors, to capture market share. Customized software is highly expensive, and people are expecting something tangible for that purchase. What they should be concentrating on (at least from an efficiency standpoint) is empowering their own people to automate the processes they do: after all, they are the experts in these processes.
In the business software world, the barrier to entry for end-users seems a lot higher than it used to be. Learning PHP, standing up a database and getting it all hosted is a lot more difficult that taking an application suite like MS Access, creating some tables, and then putting forms up on top of them.
Microsoft created a tool a few years ago called Lighswitch that allowed end-users to throw together CRUD apps quickly, and it seems to have been met with deafening silence. I wonder if managers and CIOs in BigCorps would tolerate their end-users throwing together little apps that solved their problems in today's equivalent of VB6, or MS Access (the ultimate agile experience since users are solving their own problems). Experience suggests not, and although those apps could have become unmaintainable, it seems that there is little effort being made by vendors to address that market, and to provide ease of use, with better maintainability and scalability.
In the business world the barrier to entry is often imposed (and for many good and bad reasons) by the IT/OPs/Security teams.
Going to the PHP example, you could pick one of a number of deploy and hosting providers and have your code running and world visible in minutes for less than a Starbucks coffee a week (specific example Laravel Forge + Digital Ocean).
The problem is that even mediocre software developers with a couple years of experience can miss critical things in any language with any framework that can leave them incredibly vulnerable to attack.
For homegrown internal systems, the barrier to entry isn't the code, it's putting it somewhere people can access. In ye olde days you could slap together some VB6 and throw it in an Excel template and have a workable product- but have you ever inherited something like that? I have, multiple times. It's AWFUL- but I also have made a lot of money on not making it awful.
As an engineer, my rapid prototype basically means I eschew some things like a cache layer or performance optimization for just getting the concept out- but at an organization with no real devs, I can see the value in someone who can hack together anything with whatever they have to prove the idea, then calling in the mercenaries like myself to make the concept a real thing. The problem (and expense) usually lies in the fact that they wait until the concept is completely untenable in its current state and everyone is in a panic.
I agree with you largely. I've inherited plenty of unmaintainable code, including a few user-created abominations. But that doesn't necessarily mean that the idea of end-users writing applications is bad per se, rather, that more effort should go into making it harder for them to shoot themselves in the foot.
I think yes and no. For example, for public web application development the barrier is already low enough that you can put your entire company at risk pretty easily. I think there is little reverence for what it actually means to craft a proper web based application, and that it's not even all about the code, and then there's the never ending maintenance and administration of the server(s).
Now, if you're a spreadsheet jockey and you just need to gather and display your data in a non-trivial way, there are quite a number of things already out there. Business Objects (or whatever it's called now) and Tableau have basically formed large companies upon this idea and there's open source options like Jasper Reports.
I think the days of being able to slap some VB together and write a desktop application are just about completely dead in most situations, which means you really do need a vast breadth of knowledge that a weekend warrior developer didn't need to have a number of years ago.
>Microsoft created a tool a few years ago called Lighswitch that allowed end-users to throw together CRUD apps quickly,
Lightswitch relied on Silverlight and VB Studio, which made it useless for almost everyone.
The problem with bazaar culture is its obsession with tools and systems, and its lack of interest in users. When you get a product that inverts that - like Wordpress - it's often incredibly successful, in spite of its many the technical shortcomings.
The hierarchy of value in bazaar-land is:
1. New tool/framework/language/OS (that looks good on my CV)
2. Elegant, powerful product for customers
3. Fully productised, reliable, scalable, and easy to maintain combination of 1 & 2.
2 and 3 are more or less on equal levels. 1 is far, far ahead.
Because the culture is so tool-obsessed, a whole lot of makework and work-around fixing is needed just to get things to build, never mind work well for customers.
Basically there are dumb tools, dumb products, and occasionally elegant commercial products fall out of the combination - but usually only when they're designed by someone who cares about the user experience.
Hacking culture massively undervalues the user experience, and massively overvalues tinkering and tool-making as ends in themselves.
There's a basic disconnect between the talent needed to write code that works, and the talent needed to design a user experience that's powerful but elegant - whether the user is a non-technical user, or another developer.
The cathedral/bazaar metaphor is utterly unhelpful here, because neither really captures the true dynamic.
manyxcxi summed it nicely. I want to highlight part of what (s)he wrote.
I've watched this play out for 25 years with dbase, Paradox, Access, and countless other tools intended to empower end-users. Typically only one person in a User Area (UA) has the gumption to want to develop an application. It's wildly successful at first. As time goes along, the person devlops the app based on new requirements as is true with any app. At some point, the complexity exceeds the user's skill and time. Often, it's when they want the app to become multi-concurrent user.
I saw that one play out around 1995 with an app built on Access 2.0. The department had a copy installed on each of 20 desktops. The manager came to realize it needed to be a shared app. The power user didn't know how. My colleague spent the better part of a year doing it.
Whatever the reason, IT gets called in. Then we have to salvage a good-for-an-amateur app. Usually the app has become critical to that department so the developer resource has to be pulled from other priorities to salvage the situation.
The problem isn't the lack of tools or CIO's protecting their turf. It's IT being left with messes when a power user gets into trouble. Whether it's Oracle Glue, Access, Gupta SqlWindows, Crystal Reports, or Frontpage, the scenario consistently plays out the same way.
I don't see a problem here. Basically, the amateur build the MVP and validated the use case. And when it was shown that the software served actually a need (maybe one people couldn't even articulate before, but when they saw the app they knew "that's fine, I just need this feature too) the app got used more. At some point the app will have to be replaced. Software ages and rots. My software, your software, everyones software.
So, now we are at the point where the app is breaking down under its own weight. What do we have now?
- Clear specification: The users already know what they want from the app, something very rare in our business
- Proven value: The app is not something someone designed by looking at people from the outside and saying "I think that can be done better ..." but something which stems from their own daily needs and pains.
- Experience with likely extension points: From the history of the app and where new features had to be bolted on you can already see where new feature requests will likely come in, so a new design can accomodate that
And last but not least: A working app, so you have less stress to finish something, but instead can iterate on your new version until it really is better than the current version, without anyone bothering "when is it finished? when is it finished? We need that yesterday. When is it finished?!"
...plus a long, long list of new requirements such as, "it has to be blue" and "it must send email, which must be received, but only on Thursday when the stars are right."
And it must work exactly like the existing semi-manual system, including the ability to make random edits on legal records.
I've done these a few times before, and usually pulled it off, but there are solid reasons why they say, "don't rewrite software".
In particular, the "clear specification" usually has to be thrown out immediately and previous extensions are no guide to extensions for a new system.
And no one wants to do a serious job of it until the absolute last possible moment, so "when is it finished?" is the most important question.
If the organization is setup such that empowered super-users develop apps to the extent of their knowledge and then have a scheduled handoff to a developer in IT, what you’re describing can work quite well. I haven’t seen it work that way in any organization. Usually a department decides to let their super-user develop something without informing IT or they inform us in the vein of “We’re doing this one on our own because we’re tired of waiting for project approval.”
The Access example, from my previous comment, was the “we’re tired of waiting” vein. The app was a critical part of their work day: they used it while on the phone with customers. We had to get involved when the app had become unusable. The developer had to be drawn from another project to “throw it on a server” so it could be shared. Unfortunately, Access 2.0 had a primitive locking scheme that prevented it from being shared between 20 or so people. To compound the lunacy, they fought recommendations, like migrating to a relational database, every step. We had a developer unavailable for the better part of a year while she had to make the desktop app into a department-level app. She had to make the changes while the app was in active use. This example is not one of a partnership for a planned MVP handoff to IT. It was, probably unintentionally, a way to jump the queue to have their project done.
I’m all for a partnership like you described. But, it has to be a partnership with the parties involved agreeing on some kind of a schedule so resources can be available without hurting other projects/UA’s.
Maybe this is an argument for the inherent complexity of the solution being (at least) an order of magnitude more than the tools themselves?
Honestly, I'm very unimpressed with tools these days solving actually useful problems BECAUSE they're so dependent on their assumptions of the simplicity of the problem space.
I don't think we're disagreeing, necessarily. Just speculating on how to put a conclusion on the end of your thought.
> High barrier to entry? The barrier is lower than it ever has been.
I agree with you, but everything is relative. It's expensive to produce a custom microprocessor, but it's cheaper than it's ever been.
> Any nut-job with a few weeks of training can make a shitty website with PHP or Node.js and have it instantly accessible to the most of the English-speaking world.
The barrier to entry can be much lower than that. Someone without any programming experience could fork and deploy a Node service in 60 seconds if the tools were designed for that. I think you and I are just putting our parameters for "low" and "high" in different places. You are comparing Google (cathedral) to entry-level programmers (bazaar). I am comparing a random engineer in your company (cathedral) to one of your customer support staff who is requesting a copy change (bazaar).
> It's expensive to produce a custom microprocessor, but it's cheaper than it's ever been.
My impression is that it's more expensive these days, which is why we don't see as many startups like MOS or Acorn, and see instead partnerships between larger companies. It also seems less likely for anyone producing an ASIC to get funded in the first place these days. I couldn't find good data to settle the cost issue, though.
> I am comparing a random engineer in your company (cathedral) to one of your customer support staff who is requesting a copy change (bazaar).
I don't understand this argument. I'm not sure what "copy change" means in context, and I don't know how customer support relates to the discussion.
I guess the main point I was trying to make was that the tooling for bazaar-style development is at your fingertips from the moment you sit down at a computer, but the cathedral is harder to make and the publicly available tools aren't as good.
Customer support are the people who know which words in the software should change to confuse customers less. When I say "copy change" I mean changing some words in the software. The barrier to entry I'm talking about is the one preventing that support person from making that change, instead of having to ask their boss to ask one of the engineering bosses to ask one of the engineers to do it.
Okay, but if you lift that barrier there is still a major fundamental problem: the people in customer support don't know how to code. The few people in customer support I've known who knew how to code changed jobs on fairly short order.
The fact is, even in the bazaar model where the barrier is low, when does customer support make code changes? I'm talking here about instances where customer support for open-source projects exists.
Most of the tools I see for software development seem to be organized around the needs of the bazaar (or thatched huts) not the needs of the cathedral. A million toy languages which might solve your problem well, but don't scale to a million users. Websites like GitLab and GitHub so you can share last week's 1 kloc project with collaborators. Libraries that do that one weird thing you need for your project, and nothing else.
By comparison, the cathedral builders (Google, Facebook, Apple, Microsoft, etc.) seem to be building a lot of their own tools. This includes programming languages, frameworks, build systems, version control, operating systems, and so many other things. They build their own stuff because the tools of the bazaar don't work quite well enough for cathedrals.