First of all, for your information, the benchmark code and details are already there: https://github.com/c9s/router-benchmark sorry I forgot to put the link. :p
We firstly tested the pure dispatching benchmark by a simple benchmark tool. (without ab), code is here.
The benchmark with apache is just to show the comparison result (which shares the same configurations).
Pux is basically written in C extension, which reduces the overhead to load classes from PHP files. Also it does a different strategy on route dispatching. to compare the routes as fast as possible, pux uses indexed array to store the route pattern, pcre flag. (In PHP internals, zend_hash_index_find is faster than zend_hash_find)
it's not just iterating all routes in a big loop like Symfony.
> Pux tries not to consume computation time to build all routes dynamically (like Symfony/Routing). Instead, Pux compiles your routes to plain PHP array for caching, the compiled routes can be loaded from cache very fast.
symfony/routing compiles routes only once to PHP code ([1]), and then that generated code is used for matching all subsequent requests. (There are multiple route matchers, but the default one works like this.)
It's amazing that Pux manages to be close to 50x times faster.
- Pux uses simpler data structure (indexed array) to store the patterns and flags. (In PHP internals, zend_hash_index_find is faster than zend_hash_find).
- When matching routes, symfony uses a lot of function calls for each route:
- Pux separates static routes and dynamic routes automatically, Pux uses hash table to look up static routes without looping the whole route array.
- Pux\Mux is written in C extension, method calls are faster!
- With C extension, there is no class loading overhead.
- Pux compiles routes to plain PHP array, the compiled routes can be loaded very fast. you don't need to call functions to register your routes before using it.
My personal opinion is that either no symfony and no pux should exist.
There's no need to reinvent frameworks in PHP as PHP is the framework already.
There's no need to reinvent templating in PHP as PHP is the templating already.
If you require different productivity from a language just don't stick on PHP and make the right choice by using the most appropriate language for you. There's plenty out there.
PHP is great, it rocks and is very fast when used in the right way.
If you're building a web application in PHP, then symfony is a huge improvement over naked php files or rolling your own bootstrapped system. Right out of the box it addresses concerns of security, user management, routing, persistence, and myriads of the other basic needs of a web system. Packages for many common components are already built, meaning you can get sophisticated functionality working quickly. It is modular enough that you can include only the components you need.
The built-in templating language, twig, actually provides a number of benefits over bare PHP templating. It addresses security concerns, has a concise and clean syntax, and compiles down to optimised PHP template code, minimising the performance hit.
If you are suggesting that PHP shouldn't be used for web applications, and should only be used for basic templating, I would be interested to know what you consider to be an "appropriate language".
Look at the php.net website source code. Another example could be the Wordpress source code. It's not handsome code, but it just works.
Things like fat (either slim) frameworks should be memory resident (java or similar), not reloaded from scratch at each web request, it just won't never perform on big sites.
Alternatively, if you need to deal with bigger applications, or massive traffic, you can build some kind of backend memory resident application (c, c++, java, go...) and delegate the hard work to it by using a lightweight rpc protocol (see messagepack, thrift, protobuf), and leave PHP alone to do the job it's born for.
Also, the choice to adopt a framework is serious stuff, it firstly should be well designed and maintained, then it should suit exactly your needs and more importantly you should not just use it, but learn conventions and write your code by sticking on them or else all your code will result in a bad mess.
Honest question: while I know that's true (I was using APC, now using Zend Opcode or whatever the replacement in PHP 5.5 is whose name escapes me at the moment), it seems to me that there's still going to be a performance penalty incurred. The PHP may be all compiled to bytecode at that point and the bytecode may be 100% memory resident, but you're running through all the initialization routines for the framework on each and every page hit.
"Pure" PHP is a pretty fast language when benchmarked, but PHP frameworks tend to benchmark poorly, and I've always assumed that the overhead incurred by PHP's execution model is the culprit -- essentially, PHP was written with assumptions about How Dynamic Web Sites Work that made sense in the late '90s but really suck when every request is hitting a front controller and being dispatched through a router. Is this not the case?
Yes you can use APC, opcache or XCache to load php bytecode in runtime. but you are still calling php functions/methods or creating a lot of objects in runtime. that's the overhead.
Too many people are losing sight of the power of 'pure' PHP and jumping on the framework wagon unnecessarily. That doesn't surprise me, as frameworks provide structure and rules to a language that is very messy.
It's nice to see a PHP extension for routing to improve performance, but is it really necessary to compile the routing definition before being able to use it?
The problem with this is you can accomplish essentially the same thing in userland if you know how to write a good "compiled regex" ... Calling the pcre functions in C land gains you very little. An extension is totally unnecessary.
Using pure PHP, you need to load a lot of class files. and function calls, method calls, hash find are pretty slow in pure PHP.
- First of all, while using C extension, you don't need to reload these php class files again and again. it reduces the class loading overhead.
- Seconds, looping and string comparison is pretty fast in C. it's because in pure PHP, it duplicates the string when calling functions/methods in the runtime.
- Third, there are a lot of spaces to optimize the code in C rather than in pure php.
- Last, pure PHP consumes a lot of memory, but in C extension, the memory footprint is pretty small.
Thanks for your contribution! Always love seeing new attempts with PHP. While I 100% appreciate your efforts, there are a few really big questions or problems I have with the way this has been presented. As a PHP developer, I hope you don't mind me addressing them here.
1) You benchmarked a large framework with all sorts of features and functionality against a very specific library with a few files. What about all of the extra pieces from Symfony? What do those files bring to the table or don't they? Did you only test routing features? If so, how? Are all of the features from that Symfony package available in Pux?
2) The title of the graphs showing the hockey stick effect you were undoubtedly looking for is:
ab -n 1000 -c 10
When creating benchmarks, accuracy and transparency is an absolute must. That aside: looking at the graphs, it's clear there isn't much of a difference at the 1000 level. Boosting that to 2000 appears to have had your desired effect. But what's the reason for it and why does it change?
Also, a concurrency of 10? Why 10? What about 1? 50? 100? 1000? Does it never fail? Does it only work well around 10 concurrent users?
Making massive claims requires massive evidence.
3) You're comparing apples to oranges here. To get to the point you're at in your test, requires a user to install an extension and run a file on the server to compile the routes, every time the routes change. Symfony doesn't require this.
"Pux tries not to consume computation time to build all routes dynamically (like Symfony/Routing). Instead, Pux compiles your routes to plain PHP array for caching, the compiled routes can be loaded from cache very fast."
To get the performance you're wanting, no, Pux doesn't do it dynamically. Which, makes sense because it's writing a flat file to disk and calling it "compiling" or "caching" when it really isn't either in the traditional sense. You can tell it to write a file to disk of the rendered routes, but it must be manually told to do so; unless I'm missing something in the code, it does not happen automatically. Without reading in the file to check for differences, it couldn't. The performance would almost certainly degrade.
Moving bottleneck concerns to a different layer of the application and then specifically benchmarking against that is almost certainly going to give you the benchmarks you're looking for. Since you have moved your concerns to your disk, you now run into all sorts of problems that come along with such a philosophy. How do you ensure the same file is split across multiple servers? Who has access to run the script necessary to create the file? What happens if the application grows so big that the file can't reasonably fit into memory? What happens when disk reading and writing becomes too expensive? Y'know, the regular questions that crop up when you start to move performance concerns to disk.
This would be much more accurate, and in my opinion interesting, if it left off the comparison to Symfony completely, stopped using words like "compile" and "cache," and instead described it by saying what it actually does:
Pux is a library with an extension for fast tokenization and rendering of strings for application routing and a set of files to allow the option of writing those renderings to disk instead of having them executed at runtime.
Without getting into the tests, I suspect he checked his code against the Symfony Routing component[0] instead of the entire framework stack. Symfony is build from stand-alone loosely decoupled components[1].
I gave him the benefit of the doubt and am assuming the same. Supposing it is the case, my previous concerns and questions still remain:
1) How was the component isolated and used for the test?
2) Do the features of the Symfony routing component match the features of Pux?
This is exactly why I'm more upset at the presentation of the project and not the project itself.
These questions wouldn't need to be answered if the README simply talked about what it did and how it performed. Instead, it positions itself as a competitor to a specific product and then tests against it. That would be fine, except the tests appear incomplete, very specific, and not a good comparison. In addition, it doesn't highlight the total throughput numbers or show off the maximum viable performance at all. It simply says it's better than something else. When you're selling something as being performant routing, that means you should be flaunting the maximum number of requests you were able to squeeze out of it. There isn't a single statement about how much you can really push through this until it fails. So, how is it interesting? When comparing performance, you have to know maximums. There are none here except for the amount of requests possible when requesting a single page at 10 concurrent users.
The project stands alone as an interesting dive into attempting to speed up routing for specific use-cases.
This sort of goes to the adage of product placement in that you shouldn't sell your product by simply talking about the competitor. Unless you execute it very, very well you stand to have your product lost in the noise.
From the README, it also looks like that he didn't use the optimised (default) versions at all: "Pux tries not to consume computation time to build all routes dynamically (like Symfony/Routing). Instead, Pux compiles your routes to plain PHP array for caching, the compiled routes can be loaded from cache very fast."
w.r.t. your point 3, symfony modules do sometimes require a one-off command to re-generate cached data after a deploy of new code.
an example of this is the assetic asset manager. during development you can tell assetic to compile assets dynamically, but when you do a deploy to live, you generally run:
php app/console assetic:dump --env=prod
if your app is configured properly this will take all css and js and compile/minify for production. it's a bit of extra "work", but well worth it for the performance benefit.
so yes, it does appear that performance measurements being shown are comparing apples to oranges. however, if developers are willing to go through the extra steps to get this working, it wouldn't be out of line with other symfony framework practices.
> If they would only first explain what the heck is "routing in PHP"...
"Routing" is a common and well-understood term in application development: I'm sure you can appreciate the act of writing documentation for an intended audience. In the context of an application, a router interprets a request from a client and forwards it to the appropriate resource (a controller or some other business logic) which then produces a response.
Web application development. I'm sure plenty of programmers would think of the Layer-3 device that chooses paths to forward packets through nodes to a destination.
I don't know if I'd make that distinction. Routing certainly gets a lot of use in web application development, but routing at the application level is not web-specific: examples of the pattern can be found in the Android SDK,[1] userland Android/iOS development,[2] and in a number of EIPs.[3]
> I'm sure plenty of programmers would think of the Layer-3 device that chooses paths to forward packets through nodes to a destination.
I don't discount this at all, but I do think it's pretty obvious from the context and the README which type of routing this library provides.
A news item a short while ago had the similarly phrased "router in PHP" and for a moment I thought someone had actually used PHP to write a layer-3 networking device.
I agree ... many PHP devs seem to think that programming only exists between when the web server gives a request to a PHP process and when that process closes it's STDOUT handle to complete the response.
Not sure the performance difference, but highly recommend Flight (http://flightphp.com/) as a simple router. Flight is well suited especially for building APIs. Flight has great http helper methods as well (header, json response, etc).
As far as performance, I seriously doubt the bottleneck in modern applications is the router.
Please use 0 as the baseline (zero point of y axis).