The above post proposed that Varnish is not needed because Nginx could handle the task well with the best performance. But Nginx is a bit slower than Varnish. It's been shown in benchmarks that Litespeed is a bit faster than Nginx. By inductive reasoning to show that Varnish is faster than Nginx, I posted a link to show that Varnish is faster than Litespeed, which in turn is faster than Nginx. So Varnish is faster than Nginx.
I'm not going to respond to your personal attacks; just the factual content of you post(s).
I understand that you perceived my posting of the benchmark link to be inappropriate and I agree with you to an extent. It remains to say that the link represents my proof within the context of the conversation chain from earlier.
You seem to be a long-term contributor here and I'm sure it's appreciated. I would be interested if you addressed the flaws or issues with the benchmark results. I do make mistakes just like anybody but own up to them at the end.
Let me know if you have any question and I'll be happy to discuss.
The title says 'Apache vs Litespeed', when in reality it is 'Varnish vs Litespeed', Apache has nothing to do with this particular benchmark.
2. Questionable results
Varnish serving 217.05 req/sec for a static page at 100 concurrency level? Something is seriously messed up in the setup; Varnish can easily handle thousands of req/s for something as simple as looking up a cache entry and returning a static file for 100 clients. Oh wait, 11177.99 [Kbytes/sec] - looks like you saturated the pipe, making it a test of how fast your network is, and nothing to do with Varnish.
3. Flawed benchmark testing procedure
You can't expect a simple `ab -c 100 -n 10000` to provide any insight whatsoever towards the general handling of a server. The only thing obvious is that Litespeed has been saturated with 100 concurrent connections. If you're going to do a benchmark, you must post varying concurrency levels at least (along with exact setup configurations). Likewise, cpu and memory usage are just as important to providing valuable insight.
4. Comparing apples to oranges
I'm not entirely sure what Litespeed is doing (eg. does it cache statically?) since you haven't specified it exactly, but considering that almost 70% of the requests failed and almost all requests after the first few had a different content length, I would assume that Litespeed is returning a dynamic request each time, whereas Varnish is returning a static file. I have no doubt in my mind that if you ran the same scenario (ie. Litespeed serving a static file as well) the results would be much different.
5. Irrelevant to the article benchmark
Saying "But Nginx is a bit slower than Varnish", then linking to a terrible irrelevant benchmark and using circular logic (based on another completely crap Litespeed vs Nginx for serving php!? benchmark) to infer that Varnish is faster than nginx hurts my brain.
I ran a really simple benchmark on Varnish 2.1.3 and nginx 0.8.49's proxy_cache directive on a single url; nginx is faster.
However, what if we test a heavy config with a bunch of complicated rules? Varnish's VCL compiles straight to C and might give it the edge in those cases. What if we use test a million different urls and really test how each one handles their cache hit/miss procedures. How do they fare under real world scenarios where thousands of different clients on various connection speeds are connected? I know nginx is damn efficient memory wise when dealing with a load of keepalives.
1. All is disclosed in the page. Varnish is indeed mentioned on the page as being the crucial piece that made is all happen. But mea culpa, the title "Apache vs Litespeed" does need the Varnish bit added. I'll correct it shortly.
2. You're making the assumption that varnish/VCL is caching the first server-side result indefinitely (ttl). That is not the case. So apache is still doing work behind the scenes, albeit less work than it's used to. Whatever hosting conditions we imposed on Varnish/apache are the same imposed on lsws. It's fair game (for better or worse).
3. Actually Litespeed is able to handle more than 100 concurrent connections. What choked it apparently is its inability to manage the 100 PHP requests well. It actually ran out of memory during the experiments. Again, Litespeed was given the same amount of memory and was under the same conditions as Apache/Varnish. While the environment/conditions are not optimal, both varnish/httpd and Litespeed started equal. At first, we started with -c 10 and -n 10000 but both setups performed fairly well. So we upped it to -c 100. Checkout the load average from the Litespeed node below.
4. One of the goals of the experiment is to install stock Litespeed and stock Apache (RPM). No configuration changes were made to either. Litespeed has static object caching (20MB) enabled by default. Varnishd was given a 16MB cache. Stock installs are common in the Web hosting business, which is the scope/interest here.
5. Based on experience with client portals, Varnish as a cache seems to perform better than Nginx as pure cache. It could a perception. We use Nginx and/or Varnish where it makes sense though. They're both free and open so why not!
6. Can't agree more. Just for the record, we didn't select the hardware and environment to favor either web server. We had a vacant server, network, and resources and went from there. The fact that this environment did not produce good results for Litespeed is purely coincidental.
FYI, Litespeed Tech noticed the benchmark and mentioned that they're working on developing a cache module that will ship with Litespeed web server. We'll see.
What do you need a fully-fledged web server on the edge nodes for? Wouldn't it be better for varnish to proxy everything to the back end dynamic server directly, except for requests that need to use the local edge node nginx?
The post describes a setup with two cacheing layers on each edge node, varnish cache in front of ngingx cache, which has got to be sub-optimal.
Nginx is used as a disk cache and varnish is used as a memory cache. So you're saying that why not scrap nginx and let varnish run the show on the edge nodes? That works just fine too.
Keep in mind that the goal of the article is to build an affordable CDN based on VM edge nodes. Memory is still expensive on VMs (especially if you're going to be renting VMs in costly locations). So if you have say a 20GB static store that needs to be served, on a 2GB RAM VPS, Varnish will only be able to cache that much files and be forced to make round trips to the dynamic node on miss.
Varnish does have "persistent" storage capabilities but from what I've read they're still experimental.
Not sure why you're getting negged here; using nginx without Varnish, you're not going to get nearly the speed, and using Vanish on Apache just doesn't make a lot of sense.
Not sure what you mean by "has some performance penalty compared to Varnish" however.
Nginx "has some performance penalty compared to Varnish"
The performance penalty comes the number of system calls needed to complete the request. According to the Varnish arch docs, it's 18 syscalls (http://varnish-cache.org/wiki/ArchitectNotes). Also, Varnish doesn't malloc for each new request/header. It mallocs once per workspace.
Unfortunately there's no such thing as Nginx vs Varnish benchmarks because Varnish is just a cache (apple to orange comparison).
Header manip is very important if one needs to control caching. I'm not sure Nginx has this capability at such granular level. Varnish's VCL excels at this.
As explained above, all info has been disclosed on the page in question. It's up to the reader to agree or disagree with the results. Let's face it, benchmarks are always going to be controversial and imperfect. One should always take the initiative to verify results by running their own tests.
Nginx does have that capability on a much more granular level in fact.
That benchmark was entirely useless for all the points I mentioned in the other post, but predominantly because you saturated your network on one of those tests, which meant the bottleneck was the network and not the webserver.
All your counterpoints were ignorant/naive and just went to show me how much you don't understand what is going on.
This wouldn't be a bad thing in itself, everyone has to start somewhere, but to think you do this for a living...
Let me clear up a few things. Again, the point of the experiment is not to setup the perfect environment for either VM but to subject both Web servers to fair and equal conditions. Whether it is 1GB RAM, 100Mbps switch ports, stock configurations, etc. It's fair game.
Now, I don't know if you have experience in the Web hosting business but perfection is not and never will be the goal. Clients could care less about the back-end or details as long as the response they get at the browser is acceptable. That's what drives our efforts in technology.
You keep on repeating the idea that no one needs both Varnish and Nginx in their design. Again, there's benefit in running such configuration. It's practical, effective, and thankfully keeps the business flowing. Here's an example of a successful business, besides us ;), doing it: http://www.heroku.com. If you don't realize the benefits of running Varnish over Nginx or Nginx over Varnish in certain situations, that would be one stubborn opinion. So please, stop my wasting my time.
> You keep on repeating the idea that no one needs both Varnish and Nginx in their design.
I never said that at any point.
What I did say was your reasoning that Varnish is faster than Nginx is flawed:
Apache+Varnish is 97% faster than Litespeed [1]
Litespeed is faster than Nginx [2]
*therefore I infer* Varnish is faster than Nginx.
Those cited benchmarks are both crap, and if they are relevant to a particular setup (ie yours), then you cannot use them on a public forum as proof that one is faster than the other.
You see, the problem I have is that you're spreading FUD which many will read and believe (as you do).
2) Nginx is no match for Varnish's VCL for header/cache manipulation
Regards