Hacker News new | past | comments | ask | show | jobs | submit login

Some random observations...

PHP

Impressed by PHP's performance - it makes an entrance at position 20. If you exclude Javascript, the next interpreted language is Lua at position 81, then Python all the way down at 206. Ruby makes its first appearance at 255.

Julia and Nim

These are both new, modern languages that tout their performance as a benefit. It's a shame they did not take part in the benchmarks.

***

Regardless of what you think of the benchmarks, the rankings do affect people's perceptions of languages and frameworks (both positive and negative). For example, I don't use PHP, but these benchmarks tell me that PHP has leapfrogged over Python and Ruby in the performance stakes for web development. And not just by a small margin, but by a significant difference.




I like the TechEmpower benchmarks, but you have to read the code to really understand the comparisons being made.

In many languages, a bunch of web server code might end up being implemented in C and not the language itself. When you're creating a minimal endpoint for a benchmark, you might be exercising that C code and not the language itself. For example, the PHP implementation uses `htmlspecialchars` to encode the information which is implemented in C. That doesn't tell you much about the performance of PHP, but just that it has an optimized HTML escaper. The `asort` function used to sort the results is implemented in C and has more limited functionality compared to more general sorting functions that might be able to take lambdas in other languages. The PHP implementation even takes advantage of `PDO::FETCH_KEY_PAIR` which will only work if there are only two columns.

Likewise, the PHP implementation doesn't use templates, but manually builds strings. The fastest Python implementation actually renders a Jinja2 template: https://github.com/TechEmpower/FrameworkBenchmarks/blob/mast.... That's much more realistic in terms of what you'd do in the real world, but you're going to be carrying the overhead of a real-world template system like Jinja2. Part of Python's failure here is that no one wanted to implement an optimized Python version that would just build a string instead of rendering a template.

Changing the test constraints a bit would ruin a lot of the advantages that PHP used there. Let's say that you had to retrieve three columns: `id, sort_key, fortune_text` and sort on the `sort_key`. Now you need to read more information back rather than just being able to make it an associative array (hash map). You need to be able to sort based on that sort_key which means probably giving a sort call a lambda.

This isn't limited to PHP. A bunch of Go implementations do things like allocating a pool of structs and then re-using the same structs to avoid the garbage collector. A lot of implementations create result arrays sized so that they won't need to be re-sized (creating additional allocations and additional GC work). The rules say this isn't allowed, but they do it anyway.

So, before comparing tests, I'd look at the implementations to make sure that they're comparable. A Django implementation that actually returns objects and is rendering templates and looks like a canonical Django implementation is very different from an optimized PHP version trying to avoid running any PHP as much as possible. When we start looking at the popular PHP frameworks which will be executing a bit of PHP like CodeIgniter or Laravel, we start seeing performance similar to Python frameworks as the PHP code is doing similar things like rendering templates. It just happens that no one implemented a Python version that didn't use a fully-fledged template renderer.

And this is the weird thing: the benchmarks changed your perception of PHP while comparing things that weren't similar. I think PHP is often faster than Python and Ruby, but probably not to the extent that your perception might be given these benchmarks.

I actually find it fascinating to look at the implementations and see which communities care about realistic implementations vs. leveraging all sorts of tricks to win the benchmark.


This is intential. Eg. not using a templating engine shows the raw power of the http stack. Good contenders have multiple stacks in the list (e.g. aspcore) with various depth into the stack. For asp.net core it is (a) raw http server, (b) with Middleware and (c) with MVC. And when you read this, you can compare apples with apples


There are 3 Nim frameworks in the plaintext benchmark. Httpbeast, Jester and Prologue.


I did a quick check and only found them under the JSON serialization test. So they are missing from most tests.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: