I've seen Duck Duck Go when you posted before and didn't think much of the idea (hey, I can use Wikipedia!)
Just had a look again and I now think it's pretty awesome. Just for a really speedy "what the hell is this?" type of query it looks absolutely amazing. I love the AJAX extra search links, makes it all feel really snappy.
Works great on the iPhone as well, I've added an icon to my home screen. You might want to consider making an app for the app store (you could pretty much just make an app that loads the site as it is).
Have you considered adding an API? I currently have my Emacs set up to do a dictionary lookup (using the dictd protocol) on a word when I hit C-cl. I'd love to add Duck Duck Go results to the buffer this displays!
Thanks! Yeah, we've gotten a lot better since then. On the zero-click info (what is x?) we use lots of sources now beyond Wikipedia (e.g. Crunchbase) as well as have a bunch of new features (http://duckduckgo.com/about.html). We also have both an iPhone app and an API :)
I'm not exactly a search connoisseur or even pay much attention to startups but wow. This is one of the first search innovations I've found that was any good, in fact it's great.
I've just replaced my google FF keyword, 'g', with duckduckgo link. I'll try it out for a week or so and see if I still think it's as awesome as I do now.
Thanks! I use S3 to serve images that are loaded after the initial page load (incidentally, using YUI's ImageLoader: http://developer.yahoo.com/yui/imageloader/). I use nginx to serve static files, e.g. HTML, as well as images that get loaded on page load, e.g. logo. Why not S3 for the latter? Too slow. Amazon's (relatively) new content network may be faster, though I haven't tried it yet.
Wrt nginx being more stable, both. I've crashed Apache as well as pseudo-crashed it (process still running, but not serving right). nginx hasn't crashed yet. It's also more consistent (and faster) under load. However, in Apache's defense, it does a lot more, e.g. mod_perl vs nginx proxy to FastCGI.
Cool - well I'm on Apache now, on a very undersized server - so I guess I'll have to give nginx a shot.
Not sure how you'd extract more speed from your current setup. The natural next step would be doing work in parallel - this something you do/have considered?
If you do try the Amazon CDN, let us know how you go :)
I use multiple worker processes in nginx (2 x # of cores), which work in parallel. Then FCGI::Engine is configured to run multiple processes as well. Finally, I use the external JS and image files (in part) to speed up the client-side via parallel processing of requests. (Note that IE7 and below limit concurrent connections per host by default often to 2: http://msdn.microsoft.com/en-us/library/cc304129(VS.85).aspx).
Question: why are you going with FastCGI over mod_perl? Issues with mod_perl debugging/memory leaks? Desire to use nginx over Apache for higher throughput? The big advantage -- to me -- of mod_perl is the power of having your code live within Apache's process space.
Initially I didn't think anything could beat mod_perl speed, but after benchmarking, this setup is just as fast. It's also more stable, and uses less memory. And yes, also it is easier to debug and I was getting memory leaks on mod_perl.
Don't get me wrong, I love mod_perl. I used it at my last company and on many other projects before this one. And I still use it for other stuff.
Last I looked I think you can run mod_perl with event_mpm but I didn't like this: "The current Apache::DBI should be usable under threaded mpm, though it doesn't share connections across threads. Each Perl interpreter has its own cache, just like in the prefork mpm." (http://perl.apache.org/docs/2.0/user/performance/mpm.html)
complete side note - but your favicon has a lot of aliasing when it's on a dark background (i.e. as it is when you have ddg in an inactive chrome tab). the circle looks nice on white, but you may want to adjust it so it reads a little better against other mattes.
aside from that tiny-minor issue, ddg seems cool & the overview of the architecture is very clear. i'm going to give it a try as my default chrome search engine & figure out just how behaviorally indoctrinated google has me and how your architecture holds up as more people use the service.
Skimming the headlines of articles on the front page right now, this is more Hackerish, in my view, than several others on the list. He discusses what he is building, and how he's doing it.
Just had a look again and I now think it's pretty awesome. Just for a really speedy "what the hell is this?" type of query it looks absolutely amazing. I love the AJAX extra search links, makes it all feel really snappy.
Works great on the iPhone as well, I've added an icon to my home screen. You might want to consider making an app for the app store (you could pretty much just make an app that loads the site as it is).
Have you considered adding an API? I currently have my Emacs set up to do a dictionary lookup (using the dictd protocol) on a word when I hit C-cl. I'd love to add Duck Duck Go results to the buffer this displays!