Hacker News new | past | comments | ask | show | jobs | submit login
DigitalOcean’s quarterly report on developer trends in the cloud (digitalocean.com)
145 points by sandGorgon on July 9, 2018 | hide | past | favorite | 92 comments



> Serverless computing is in a much earlier stage of adoption, with nearly half of developers failing to clearly understand what it is.

I've seen this called Function as a Service and honestly I think using that term instead of Serverless would go a long way to fixing this issue because 1) it clearly communicates what it is 2) there's not actually such thing as serverless since the functions are still running on servers. It's a made up term that isn't grounded in reality. It's not serverless any more than a Heroku Dyno is Serverless.

The first time I heard "serverless" I assumed it was a totally in-browser application.


Also it fits in nicely with the other naming conventions such as the following:

- Software as a Service (SaaS) - a product such as GitHub or Slack

- Infrastructure as a Service (IaaS) - VM or LB as you find in Azure or AWS or Digital Ocean

- Platform as a Service (PaaS) - sometimes called Container as a Service as you find in Heroku or Zeit Now

- Function as as Service (FaaS) - a single function as you find in Lambda


Technically CaaS could be IaaS, if you're just using containers as lightweight VM substitutes.

Practically CaaS tends to be PaaS, because the hosting service is also providing a platform such as Kubernetes.


What's the practical difference between FaaS and the old style mod_php, mod_perl, mod_python and friends ?


I think the difference is that you don’t know if the server is running Apache or nginx or IIS because it’s abstracted away.

The only interface is the input into the function and the output it returns.


"/cgi-bin/ Rides Again" was left on the cutting-room floor.


I see people getting excited about something new "past REST"... which by that what they mean is RPC.


> The first time I heard "serverless" I assumed it was a totally in-browser application.

So the term "serverless" made you think there was no server involved? What could have possibly made you think that?

"Serverless" is up there with "cloud" for the dumbest most overused words in technology right now.


BitTorrent and Napster is what made me think that


I think he was being sarcastic...


I was. I refuse to use "/s" though.


I feel like I know more after just reading your post.


I'd like to hear from the Erlang camp. It amuses me to think about what they must think of us darn kids taking their ideas and calling them something else.

What's really the difference between FaaS and Erlang style message passing?


> I've seen this called Function as a Service and honestly I think using that term instead of Serverless would go a long way to fixing this issue

But "Functions as a Service" doesn't cover what it is, though.

The following AWS services are "serverless" but would not be "function as a service:"

* S3

* API Gateway

* SQS

* SNS

* Cognito

* DynamoDB

* CloudWatch (logs and metrics)

* Step Functions

In all cases, you are not provisioning or managing servers. Scaling is linear and costs are linear based on how much you use them, and typically based on what's actually being used (bytes stored, requests made, etc.) and not per-node. Because they're all hugely multi-tenant, they also cost almost nothing to use at low scale.

"Functions as a service" covers serverless compute, but it doesn't cover the huge architectural difference (from the developer's perspective) between the services above and rolling your own (for example) S3.


All of those would fall under IaaS/PaaS.

If we're qualifying anything on a server that developers don't have to manage themselves as serverless than anything with an API becomes serverless...despite all of them actually running on servers.


OK. I guess to me categorizing those things in the same bucket as EC2 or Salesforce is missing a pretty fundamental architectural difference and benefit. I'm not claiming "serverless" is an ideal term, but I haven't seen the case made for anything better.


Those are just managed services, as in managed versions of I/P/F/S-aaS.

"Serverless" can be applied to every SaaS product ever made and is just an all-around meaningless and dumb-sounding term.


(aws employee)

The distinction I've heard and make (regardless of serverless being the right term) is that these infrastructure services expose their control plane and data plane without tying you to the concept of a particular machine. Compare this to all publicly available forms of RDS where I still have to think in terms of machines for scaling and management.


None of those are "serverless"


Like the Stack Overflow survey, this one does not distinguish server-side/client-side JavaScript, making it look far more popular than it actually is when considering the data in context of server-side technologies.

If 100% of the servers were running PHP, the results would likely be 50% PHP, 50% JavaScript.

These survey writers need to modify their question, or add another one specifically requesting server-side tech so JS can be accurately compared with server-side only languages.


Do you think a lot of people are running JS in a container for frontend? I wouldn't consider a webserver container a "JS container", even it were mostly serving a React/Vue/Angular app.

By the way it's worded, it sounds like the question is specifically talking about NodeJS.


> Do you think a lot of people are running JS in a container for frontend?

Hold my beer. --Frontend developers


If the rest of a web app is running on containers, what's the downside to having the test/build process for the frontend in a container as well?


I like using containers as a reliable build context. I think it makes tons of sense.

Once the container's done building, throw it out, take what it built, and serve it.

I'd personally do this separately from the rest of the web app if possible, though. I keep frontends in their own repository.


actually! at my old job there was a project where the frontend webpack build was running in a docker container..


Yeah, I guess when you think about it that way it makes sense. I think I might have assumed this was for production and not build environments.


You’re forgetting the C and C++ for the OS and other binaries. Throw in some python to bring this all together and some Go for a bit of spice.

I get your point, but they probably asked “what language do you develop with”.


Interesting to see the disconnect between hiring managers and what people say they care about most when considering a job offer (salary). Hiring managers claim there is a shortage of talent and that they do not care about offering a higher salary. Perhaps that talent shortage would mysteriously disappear if they increased their salary offers.


The number one thing on that list is "limited pool with relevant skills" and not an actual talent shortage.

IMO this is more from asking for specific combinations of skills used at a company than anything else. You need these two programming languages, this database, this frontend framework, these devops tools and these other skills would also be beneficial.

Generally, you're better off hiring as "if you are competent with one of these and willing to learn the others, you should apply". The broader the stack the more learning curve should be assumed with the hiring process.

But it's not in a lot of cases. This also goes hand in hand with the professional development opportunities cited at the beginning of for retention sake.


Of course it would. It's just standard industry propaganda to suppress wages and "encourage" imported labor.


Computers are a tool of Automation. A seasoned engineer can build amazing things that does the work of many humans. Not only that 10X developers don’t work 10X hard, they are able to solve 10x problems with 1X effort because of their domain experience and approach to problem solving.

This kinds of people are hard to find. It’s a terrible business decision not to pay them well because their ROI is multiple times their salaries.

They are also very rarely on the market sending out resumes. Mostly because they get jobs mostly via friends of friends and referrals.

But I do agree a 100%, we should be getting paid more. If you’re not getting paid enough, look for other opportunities. There are plenty.

The tech industry has never been as profitable as it is now. We are in a huge tech boom right now. AppAmaGoogMsft are all poised to be the first trillion dollar corporations.


I realized recently that Serverless is to developers as credit cards are to college students.

Remember when VMs came out, and people were like, great! Now I can constrain the resources of my OS! I can run 10 OSes on one machine! I'll save so much money !

Then we gave individual teams access to create as many VMs as they wanted, and suddenly all the hardware was used up.

Now do that, but in the cloud, with unlimited resources.


In my experience, those VMs were used briefly then left to rot. Consuming a large reserve of resources while doing nothing. Serverless should at least reduce the overhead of orphaned projects out there.


There's a secondary problem: when there are more resources, application development becomes less efficient over time. Even though there will be less infrequently used resources, the applications will use up resources more frequently in an inefficient way.

Example: I now need a minimum of 4GB RAM to browse the web, which is insane. 15 years ago my computer had 128MB RAM and I was browsing the web at approximately the same rate.


> 15 years ago my computer had 128MB RAM and I was browsing the web at approximately the same rate.

You probably weren't; perceptions of speed are often based on expectations set by experience, and your expectations were probably a lot lower then. If you actually could go back and browse the web of 15 years ago and the sites you could then, you would be disappointed, but the speed then was similar scaled to the expectations you had then as the speed now is scaled to your current t expectations. So your perception of the speed is similar.


I know I could have four browser windows open with 20 tabs each, to a random selection of web pages. If I try to do that today on a laptop with 2GB RAM on a moden browser, it dies from running out of memory. I tried using an older browser, but it won't render modern pages and uses legacy crypto. So I know for a fact that modern browsers simply can't work on older hardware, whereas the older browsers used to work, but are no longer compatible with modern websites.


Firefox didn't even have tabbed browsing in 2003 and everyone else was in the IE6 winter. FF was only 1 year old at this point. Were you using another browser?


Firefox didn't exist in 2003, it was released in November 2004. Before then, the name of the browser was Phoenix. It had tabs since version 0.1, released in September 2002. Later it was renamed Firebird and then Firefox.

Pre-tabs I probably had only 40 windows open at a time.


It existed but was still under development and not enabled by default. E.g. there was no graceful handling of tab overflows https://wiki.mozilla.org/Tabbed_Browsing/User_Interface_Desi...


Yes, I definitely enabled all the beta/testing features. Tabs may have been limited "depending on screen resolution", but as a gamer I had probably one of the biggest screen resolutions you could run, with a CRT to match. I'm pretty sure I was using an AMD K6-2 500Mhz with between 128 and 256MB RAM. Sometime after 2002 I built an Athlon-based system with a gig of RAM, so I could be mis-remembering my computing platform when "peak browsing performance" happened, but I know that no browser has ever been as fast as Phoenix was.


I feel the same way about Phoenix. In 2003 I was running 0.1 on a Pentium III 450Mhz with 128MB RAM and it's still the fastest browser I've ever used.

The newer Firefox Quantum releases are coming close, but even then it's on a quad-core Core i7 2.8Ghz and 16GB of memory MacBook Pro.


More generally: when you detach the consequences from the causes, everybody misbehaves.

(This is, btw, one of the cornerstones of CI/CD. Show people the consequences immediately and their behavior will shift. I've seen this firsthand a number of times.)


Today it's possible to buy a laser-guided sleep-o-matic superbed that walks your dog and cures your back pain.

You can also buy a basic bed that's pretty nice and reasonably cheap.

15 years ago, you might have had a high-tech bed that didn't leave you sore in the morning.

1,500 years ago you had a bed of straw with fleas and bedbugs, but at least you had a place to sleep.

150,000 years you slept in furs on dirt by the fire, breathing smoke and aggravating your sciatica. Your bed was basically free, but since your livelihood depended on your full mobility, sleeping there was slowly killing you.

All these beds provided the feature of "sleep." So why would anybody need a sleep-o-matic, a bed worth more in 150,000 BC dollars than the yearly economic output of your continent? Why should we accept the massive waste of resources it represents?


(Because the sleep-o-matic is better. Those resources are used, not wasted.)


Fifteen years ago, AJAX did not exist. Fifteen years before that our computers had VGA screens and megabytes of RAM. Yours is an oft-heard complaint in many quarters, but with respect, I have trouble taking it as indicative of anything other than the bitterness of age. Yes, the new technology is bloated and wasteful. As far as I am aware, this has been a fact of new technology for some time. Efficiency is one of the least important factors in software development.


if you have the resources use them? 15 years ago your browser had a small fraction of the features a modern browser provides. this argument is tiring.


Which is the problem. Everyone knows that I have resources, so everyone wants to use them by packing in every feature without any consideration for how much memory it uses. It's fine if I only wanted to run a browser or only wanted to run an IDE.

It's not that everyone should always strive to have the minimum footprint possible. It's that no one cares how big the maximum footprint can be.


But Moooooooom! It's all virtual! There are no hardware resources being used up!

(until the bill comes)


Here's the interesting learning for me:

While Kubernetes was most popular overall, the smallest companies (1-5 employees) use Docker Swarm more often (41 percent use Swarm vs. 31 percent that use Kubernetes).

Did not know Swarm was this popular. Even overall Swarm is 35% to K8s 42%

However, there is a huge proliferation of hosted K8s services - AWS, GCP, Azure, Digitalocean, tons of others. Nobody seems to be making a hosted Docker Swarm solution though. I wonder why - is it because the market doesnt exist because of the simplicity of Swarm itself ? Would be interesting to find out.


I can't help but wonder if this is a reflection of Digital Ocean's target market, and its late adoption of block storage and other functionality. Kubernetes is a pretty "serious" tool, and you'd expect "serious" teams to have a preference for bigger, more-established players like AWS/GCE. Digital Ocean having been moving up the chain but at least in my mind they're still associated with the "hobby site" mindset. Certainly I run my hobby sites there!

I wonder if a similar survey from the bigger players would have different results.


Swarm is dead simple to get up and running; it's baked into Docker. Maybe k8s is easy too, but I've always run into issues and the value proposition for pushing past them was never clear to me. There's no doubt in my mind that there are very good reasons to use k8s, I just haven't found them (nor looked especially hard).


And 1-5 employees sounds like the kind of business that would just as well stack a few gray boxes in a room and call it a day.


Sadly Docker is replacing Docker Swarm by Kubernetes


i think they are being very opaque about their plans and this is hurting their credibility.

They built the ability the launch k8s clusters through the docker tool. There are almost no tweets or posts about Swarm instead, their blog is full of kubernetes posts (https://blog.docker.com/). Even their coolhacks section is about "kubeflow". Top rated session at Dockercon was kubernetes.

i really hope they open up more and talk about their future plans and engage with the community.


Um, I was a top rated session that was repeated on Friday and was 100% Swarm, and had a full house https://twitter.com/BretFisher/status/1007099831783862272


Bret - this is not me that's making the claims. Take a look here - https://blog.docker.com/2018/06/top-5-rated-sessions-dockerc...

this is what is scaring us. We love Swarm and really respect all the dev efforts... but the corporate K8s bias in unmistakable. These are some of the kubernetes posts over the last 2 days from Docker's twitter account: https://twitter.com/Docker/status/1016340701800431618

https://twitter.com/Docker/status/1015620961712967681

https://twitter.com/Docker/status/1015535938678738950

https://twitter.com/Docker/status/1015475755428597760

Number of Swarm posts in the same time ? Zero. In fact, I would be hard pressed to find a public tweet about Swarm in a long time.

This was the first Docker event in Lahore - https://events.docker.com/events/details/docker-lahore-prese...

Its Kubernetes everywhere.


According to [1], this is not true - it was intended to be an additive option.

[1] https://www.bretfisher.com/is-swarm-dead-answered-by-a-docke... from https://forums.docker.com/t/is-there-a-roadmap-for-docker-sw...


Yep I wrote this, and it needs an update just months later. Lots happening.

Even today a big PR was pulled in to improve the scalability of Swarm https://github.com/moby/moby/pull/37372

More updates here https://github.com/docker/swarmkit/issues/2665#issuecomment-...


hi, what do you think about the approaches of CiliumProject ? (https://opensource.googleblog.com/2016/11/cilium-networking-...). They do have a Docker plugin (but does not work seamlessly with Swarm), but they claim to use BPF for superior performance https://cilium.io/blog/2018/04/17/why-is-the-kernel-communit...

I think the Kubernetes and Swarm devs have generally leveraged IPVS as the next step from iptables. Any thoughts on going straight to BPF ?


I find it interesting (and perhaps predictable) that while only 11% of people had a top consideration when taking a job of "Quality of manager or management", 47% of people cited "Leadership / management was bad" as a reason for leaving a job.


I find 11% a surprisingly high percentage, since I would think "Quality of manager or management" is something very hard to assess during a selection process.


Pretty hard to gauge the quality of management from the outset though.


I'm curious why you think so?

I normally ask questions about various non-tech topics including conflict resolution, team makeup, strategy, product management priorities, how they mentor junior team members, etc. Those discussions may not be able to tell the difference between a good vs. great leader, but they absolutely will raise red flags from a bad leader.


I disagree. Many bad leaders are very expert at sounding like good leaders in a way that is hard to detect in precisely this type of situation. There’s really only two options: seek advice from someone you trust who worked with those leaders, or else you just have to spend a bit of time working for them to find out. You cannot tell from interview questions, unfortunately.


Agreed, bad management usually shows up when there is a problem / stress point. That typically isn't the case in an interview.

Granted one time I did pass after an interview after talking to an inexperienced manager who had no clue....so sometimes, you can tell.


Something that puzzled me:

"the majority of hiring managers say they make no distinction between bootcamp vs. college graduates"

While the bulk of work at a lot of enterprises (banks, telcos, etc) are CRUD apps with some BPM, I did not expect a (hiring) manager to be unable to make such a simple distinction.

Hopefully it could be the case that 'we don't discriminate based on that' is simply the polite/legal thing to say, especially considering that the article goes to say that "48 percent have not filled any positions with a bootcamp graduate"...


> "the majority of hiring managers say they make no distinction between bootcamp vs. college graduates"

Since the industry norm is "You're all equally worthless until proven otherwise" I don't find it all that surprising. They'll take in your resume and toss you into the pipeline / grinder and if make it through the crucible unscathed, well, congrats you'll get a job offer!

Being neither a college nor a bootcamp graduate, I've found very few places give a damn what you've got framed on your wall at home. What matters is if you can hack it in the real world. The only time I've been turned away for lack of credential was Booz Allen Hamilton, and they exist to arbitrage you to the government which still seems to care about proper documents.

> considering that the article goes to say that "48 percent have not filled any positions with a bootcamp graduate"...

Well, that would mean 52% surveyed have hired a bootcamp graduate which is pretty surprising to me, personally, since the camp(s) in my metro seem to produce about two good candidates per cycle.


Yeah. The worst code camp people have been some of the worst programmers I've ever had to work with. But at the high end? The distinction in success rate is fuzzier.

Several of my most promising mentees came out of code camps. It convinces me of something I already 'knew': that good instincts don't come from a college education, they come from life. But a university will weed out a lot of dilettantes...


>Ninety-four percent of our respondents self-identified as men, 5 percent as women, and 1 percent as non-binary/other.

Seems like this survey was highly self selective, so I'd take the entire thing with a grain of salt. There's no way the industry as a whole is anywhere near 96% male.


> There's no way the industry as a whole is anywhere near 96% male.

What do you think is the correct percentage? 96% seems reasonably close to the anecdotal gender ratio (~1/35 students being female) of undergrads from when I was in university a long time ago.


Also, 76% of respondents were under the age of 35 and 66% reported to be working at a company with < 99 employees.


This stood out to me as well. How the survey respondents were selected could have had a huge impact on the demographics.


The pool of respondents are Digital Ocean customers? I'm assuming that would exclude employees of FAANG plus basically all really large "enterprise" businesses. Is that fair to assume?

If that is fair, I wonder how biased these results are. If we included FAANG and/or big corporations where tech is key but not a cultural staple (e.g. Walmart) I wonder what the results would look like.


It'd also exclude anyone who wanted to host their own site, anyone who stuck with traditional hosting companies/services, anyone who used a service like AWS or Azure to host their site/app, etc.

If so, that'd certainly influence the results a lot, and alongside your examples of FAANG employees and large enterprises, make the survey seem a bit too biased to be all that useful.


This is a nice breakdown but the most interesting thing I see is keeping the statistics to themselves by posting them to the site as svgs, not images and not text. You can't highlight them for copypasting even though a screenshot will work but you can't right click for saving to an image either. Thanks for the technique.


I'm not sure that was the intention. It's easier to cater to different screen resolutions with a vector format. You can always take a screenshot if you want or link to the article.


I did make an assumption that shouldn't have been made, my apologies. I still think this is a cool effect that it can be used for.


Has anyone chosen digital ocean over aws/gcp/aZure when building a scalable setup (for pricing reasons). Eg setting up your own load balancers/ private master slave dB machines / bunch of ansible to organize it all.


Fwiw, Digital Ocean keeps adding more and more options on the service front. Several storage options, load balancers w/ Let's Encrypt built in, etc.

Keeps growing. IMO the biggest gap for them is lack of a database service like RDS. There are providers of that service that can be deployed to DO...but the cost of those providers is high enough to disincentivize the price appeal of DO in the first place.

It would be really interesting to see them partner with a known player like Citus.


FWIW, I have found one of the best parts of using Digital Ocean is their fantastic documentation and tutorials. As a relative newbie, I was encouraged to deploy to a PaaS such as Heroku but grew frustrated at the black box aspect of what was going on underneath the hood. The tutorials on DO allowed me to learn and gain an understanding of how servers really work.

On your point about database services, I was able to set up Postgres on my droplet with relative ease and it has been serving me well ever since, again all thanks to clear and well made tutorials. I think if I could do it without any prior experience, most other people can too. So IMO, these providers aren't necessary at all.


Oh, I totally agree. That's actually how I learned this stuff. Years ago there was a company called Slicehost that was very similar to what Digital Ocean is now. It had a treasure trove of great documentation and articles.

Eventually they were acquired by Rackspace and folded into the Cloud Servers line.

Great company though.


Lack of GPU droplets is also a big gap, and something that has been promised for several years now.


Yes. We are using DO over AWS. Our bill is $120/month. We serve 21m web requests per month for that amount.


possible to share analysis of what the equivalent would've cost in infrastructure on AWS/GCP/Azure?


I havent had the time to do it. I will try to in the upcomjng week


yes I've done that - been pretty happy so far.


the majority of hiring managers say they make no distinction between bootcamp vs. college graduates

Wow. Can this really be true? Maybe strictly a web dev thing?


Interesting to compare the frequently used languages of those using containers to the Stackoverflow Dev survey.

- Javascript 57% use vs 71.5% I had always thought JS was high on the Stackoverflow survey because it included people using it on the frontend. Not necessarily the case.

- PHP was third at 36%

- Golang was fourth above Java


Would have been nice to see what scale these people were running.

Also, if they are only polling developers are these actually production loads ?

Are the developers now doing operational tasks ?


> Would have been nice to see what scale these people were running.

26% were in companies of 5 or less people and 50% in companies of 25 or less. So, this is highly skewed towards smaller scale companies.


So it's the result of a survey? That's a bit disappointing. Quarterly report makes me think it'll be extrapolated from DigitalOcean's usage metrics. Surely some of the things, such as containers could be inferred from real usage.

Also, unlike stackoverflow, if the respondents were DigitalOcean users I would imagine some things would be considerably skewed - for instance the popularity of "Serverless". I assume that since DigitalOcean doesn't offer serverless products people who use serverless aren't on DigitalOcean.


Does anyone know if the graphs and visuals in the post are hand made or are they using some kind of app with a custom theme?




Join us for AI Startup School this June 16-17 in San Francisco!

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: