StackOverflow shows just how powerful using a fast language can be. Compare the TechEmpower benchmarks for, say, Java vs Rails on a very simple JSON serialization benchmark (we can assume .Net would achieve comparable performance to the JVM):
The Java servers get around 900K requests/s on beefy hardware while Rails squeezes out 6K. That's a 150x difference! Any real application will be slower than this, and on cloud hardware you can expect to be a order of magnitude slower as well. You just don't have much headroom if you use languages like Ruby before you have to scale out. And once you scale out you have to worry about sys. admin. and all the problems of a distributed system.
It's a one off cost to learn an efficient language, but it pays returns forever.
> StackOverflow shows just how powerful using a fast language can be. Compare the TechEmpower benchmarks [...] (we can assume .Net would achieve comparable performance to the JVM)
Except those TechEmpower benchmarks show .NET is not nearly as fast as Java. I think StackExchange prove that the platform is NOT the most important: it's much more important to make performance a priority in all engineering decisions, to benchmark everything beforehand and develop new technology where there's no good standard solution available. (think Dapper, protobuf-net, StackExchange.Redis)
I'd like to add one caveat with respect to our C# tests: we did not run the tests on Windows in Round 9. With the help of contributors, we've recently revamped our benchmark toolset, and have yet to pull the Windows implementation of the toolset up to date. The C# data in Round 9 is exclusively from Mono on Linux.
To more accurately judge the request routing performance of C# tests on Windows, see Round 8 data. For example, see Round 8 limited to Java, C#, and Ruby tests on i7 hardware [1].
Other important notes on that data:
* The http-listener test is a low-level test implementation.
* The rack-jruby test is running on Torqbox (TorqueBox 4), which is based on Undertow and has very little Ruby code [2]. This test is mostly about Undertow/Torqbox, with a bit of JRuby.
Another interesting test is Fortunes on i7 in Round 8 [3]. Fortunes involves request routing, a database query, ORM translation of multiple rows, dynamic addition of rows, in-memory sorting, server-side templating, XSS countermeasures, and UTF-8 encoding. Here you will see Java frameworks at about 45K rps, ASP.NET on Windows at about 15K rps, and Ruby at about 2.5K rps.
I agree with your general point, but I believe you need to choose components that have a chance of being fast if you ever want the overall system to be fast.
I didn't notice .Net in the benchmarks -- no .Net languages are listed but .Net is there as a framework. It's still x20 faster than Ruby.
> That's a 150x difference! Any real application will be slower than this
How many applications are rate limited by the speed of the front-end language? Not that many; the speed of the backing store will usually be the bottleneck.
Java servers get around 900K requests/s [...] while Rails squeezes out 6K
Yes, but that's for the /json test, which serves a static object that the framework converts to json.
I'm not sure it is really representative of the overall speed you'll get with whatever framework you'll be using: how often do you have a static page that's best served by your framework instead of leaving nginx serve it directly?
I prefer watching the test results that actually involve the database (/db, /queries, /fortunes and /updates), as it shows less raw speed for serving static things and more overall speed for your dynamic pages.
With /queries for example, Java does ~11.3K, while php (on hhvm) does ~10.7K, Python is at ~7.7K and Dart ~12.8K (For the fastest framework of each language. Rails still does bad though).
The reduced variance of the /queries benchmark suggests to me that the database is the bottleneck in this setup. This would be the first thing you'd attack if you were optimising for speed. For instance checkout the section on StackOverflow's caching -- they have 5 levels of cache, including in-memory caches on the web servers. They are doing a lot of work to take the DB out of the equation.
I prefer the /json test because it gives me a performance ceiling. I know that's the max performance I can expect, and it's up to me to design my system to get as close to that as I can.
On the other hand the db tests don't really tell me much. Does my data and data access patterns match theirs? Probably not. So it is difficult to generalise from their results. If the systems you build are always stateless web servers talking to a relational db then I can see it might be more useful.
This is a good point. It's true that for the high-performance frameworks and platforms, the database is the bottleneck. Or, more accurately—considering the trivial query and small payload—the overhead of the database driver and the wire protocol are significant factors. That said, there remains a fairly broad distribution over the Single-query test and Fortunes tests. For the low and medium-performance frameworks, the overhead of their ORM and other factors are more significant than the database. I find it can be illuminating that in many cases, the ORM code necessary to marshal result-sets into usable objects is more costly than the underlying queries.
Meanwhile, the 20-query test is a bit pathological as it runs into a brick wall with the database wire protocol and efficiency of the database driver. Many otherwise high-performance frameworks and platforms become bottle-necked waiting on those 20 queries per request. But you and I agree, a 20-query-per-request scenario should be the exception and not the rule. When developing a smooth-running web-application, it's common to aim for zero queries per page load. (For those who find this to be crazy talk, note that I'm saying we aim for that ideal, and may not necessarily achieve it.)
I too particularly enjoy knowing the high-water mark set by the JSON and Plaintext tests. When we add the next test type (caching-enabled multiple queries), we should see some interesting results.
Conversely, you might be able to iterate faster with a 'web' language and create a product people really use in a shorter time frame and worry about scaling later!
5 years ago it might have been true that RoR was that much more productive it was worth using, but that doesn't match my experience today using RoR and Scala.
Update: To try to avoid turning this into a language war, there are other good reasons to use something like RoR. Ease of hiring is one. Using known tools is another, if you're not building anything you expect to get load.
Java 8 + an IDE like NetBeans (which isn't even the best Java IDE) is actually a very pleasant development experience. I am a total convert from Python in vim to Java in NetBeans+jvi.
Refactoring in Python or Ruby or Javascript or whatever dynamic language is extremely painful compared to doing it in a static one. The "Software is never finished," argument can also work against dynamic languages.
Yeah, I have. It seemed objectively better, but I started using NetBeans first, and it's what we all use at work. The workflows and keyboard shortcuts are ingrained enough at this point that switching doesn't really seem worth it right now.
http://www.techempower.com/benchmarks/
The Java servers get around 900K requests/s on beefy hardware while Rails squeezes out 6K. That's a 150x difference! Any real application will be slower than this, and on cloud hardware you can expect to be a order of magnitude slower as well. You just don't have much headroom if you use languages like Ruby before you have to scale out. And once you scale out you have to worry about sys. admin. and all the problems of a distributed system.
It's a one off cost to learn an efficient language, but it pays returns forever.