One of the points this article made that I like, and argued for in the other thread was
"what matters is fast enough"
people love saying X is too slow etc, the only thing that matters is the ability for X to perform up to your requirements, if you need a million concurrent users then yes access will be the wrong tech. but to say X is too slow, you must specify exactly what it is too slow for.
I agree wholeheartedly, although I have to make the pointy-headed objection that fast and scalable are two different things. You can have a system which scales wonderfully and still takes unacceptable time for pages to render -- if so, you fail. You can also have a system which can't push over 100k page views in a day but renders snappily for those lucky folks -- and if your business model can win on 100k page views a day then you're doing pretty well for yourself now aren't you.
Please, God, send me scaling problems. I want my hardest problem in life to be too many people trying to give me money. (Is it just me or does the "My framework scales better than yours!" argument always rage in inverse proportion to the amount of money at stake per page view? I've seriously never heard it "Whoops, we had this great app people were paying real money for but we outgrew our old framework and needed a rewrite." Probably because the solution would be "Hire somebody to do it while sipping iced teas on a beach in Tahiti.")
I also think people need to be beaten around the head with the YSlow presentations until they get the point: 80% of the user perceptible delay happens on the front end. It does not matter whether you are running Rails or a CGI script written in hand-coded assembly -- stupid little things on the front-end like whether you gzipped your HTML or combined and minified your assets matter more in terms of user-perceived speed. But instead of adapting to that measurable, objective truth of our profession (where best practices are literally along the lines of "There is one line in your Apache configuration which will make HTML gzipped. Make sure it is on.") we waste time dickering over "shootouts" to see which backend gets to wait on IO or wait on rendering 100 ms faster.
Some days we're like the surgeons who scorn checklists because checklists are too easy to possibly save half of the lives lost during surgery.
It's just like "street racers" who badmouth each other's cars because it has this, or doesn't have that. We programmers also succumb to primate instincts that subconsciously have us perpetrate verbal aggression on one another.
However, it is worthy to note that real racers spend their time honing their skills or gathering resources in preparation for actual races. It has been noted that some of the best street racers drive like grannies most of the time, because they're saving all their resources for when it really matters.
to generalize:
1. people often over optimize because they mistakenly believe that every feature matters.
2. If your time is worth $50 an hour researching ways to save $40 for an hour isn't worth your time.
3. the harder a time you have deciding between two things the less likely the decision matters. you're having a hard time because the two choices have similar utility.
and of course this is all just a subset of the larger skill of prioritizing in general. I find a useful exercise is when deciding to invest something (time, money) jot down a quick list of other things you could be doing and see if any of them are obviously better.
And in the other thread you were called out (by me and others) for this false statement, because you are wrong.
What really matters is: "scalable enough".
That's a big difference because today's hardware is very forgiving. Today you can do really stupid things, like write a chat-service that dispatches all messages through an on-disk database, and it will work. Ten years ago that thing wouldn't even have gotten off the ground.
The question remains whether your little toy project will scale to worthwhile workloads. You know, to serve a purpose (novel concept).
This is where this CouchDB nonsense falls apart. Neither would it possibly scale to any worthwhile number of users. Nor does it present any particular elegant or interesting programming concepts.
It's not more or less than the implementation of a horrible anti-pattern. I don't mind that (we had a good laugh, after all) but the author should really stop trying to sell this under the slogan of "Simple wins" - as if any of what he did was a good idea.
Simple did not "win" anything here. And to add insult to injury: His solution is not even "simple" by any means. I can write the same thing in under 100 lines of Python or Perl, without involving an esoteric database, and it would scale better.
I resent being called "smelly". And I WISH I was 15 again ;)
>> "But what none of these detractors seem to understand (except for the script-kiddies) is that the important thing about this demo is the web. CouchDB is part of the web, and serves web applications natively."
Yeah ok, CouchDB includes a web server :/ why is that a good thing? Less scaling options when things are bundled together like that.
>> "The second set of detractors say realtime applications shouldn't go to disk, because it's slow and a lot of data. To them I say "disk is free"."
The only reason a real time messaging system should go to disk, is if persistent logs are required. And in that instance, when you scale, it most certainly is not free.
>> "Call me when performance matters."
Here's a little story. I built the predecessor to Mibbit using php and mysql. It took a long time. It didn't scale so well. I knew using a db was a stupid idea, but I wanted to see just how stupid. I started using shared memory, semaphores, message queues etc instead. It scaled a bit better, but still lame.
In the end I decided to scrap it, and rewrite it from scratch in something I knew would scale. It now does about 2 billion messages a month or something.
You should think about scaling from the outset. Having to scrap everything and change language/architecture/etc is kindof a pain. Just because there's some "new" "so hot right now!!!" language/framework/etc out there, it's probably totally the wrong tool for the job.
The traffic going back and forth from the browser to the demo CouchDB chat couldn't have been less efficient if it tried - it was sending all 50 recent lines of chat everysingletime. I don't know if that's easily fixed, probably not. Maybe you'd have to delve into CouchDB code :/
I hate this trend of "Less code.. more ease for the developer.." etc. Focus on the user and what they want, and how you can best give it to them. Not what's least lines of code/easiest for you.
Yeah ok, CouchDB includes a web server :/ why is that a good thing?
It's not just a good thing, it's a central element of the design. CouchDB is just a ReST based service and ReST servers mediate access to web resources.
1. If you're going to write a "demo app", why not write one that is in the niche that you are targeting. I understand messaging is not it.
2. "The second set of detractors say realtime applications shouldn't go to disk, because it's slow and a lot of data. To them I say "disk is free"" I think in many applications it's safe to say that disk space is free, but disk access time certainly isn't, especially if you're syncing transactions to disk on commit.
3. "Call me when performance matters." You can write nifty apps with little code in many languages/frameworks. The question is, what happens if the shit hits the fan?
4. Hacker News load probably isn't that high. It's certainly not the same as the "slashdot effect".
5. "the important thing about this demo is the web ... your applications can be deployed anywhere there's a CouchDB." - I really don't understand this point. The same applies to AMP or Tomcat+Mysql+Java. If it's there you can deploy it. What's P2P about it? I can write Java code that speaks to other Java code on another Tomcat server, and it's not even particularly hard, as there are frameworks for it. I don't like Java, but then again, I don't like JavaScript either.
6. "Actually, there isn't much complexity to hide as CouchDB is simple on the inside as well." As a user, I don't care. The point of systems programming is for engineers to sit down and write hi-performance C/C++ code that will then be used by thousands of developers to run billions of transactions for several years. If the engineers spend an extra month to make it 10% faster, then it's worth it, because it will be used by so-many people for a long time.
CouchDB can replicate for backup, but either way this test app isn't a "service" that people will care about if the hardware fails. (Software failure can provide lessons, though, since this is a demo of said software.)
I don't think so. Most modern webapps or user interfaces strives for simplicity rather than just piling feature upon feature on top of each other. HN is a great example of this.
Efficiency can often be won by choosing the simple data structure or algorithm. There are numerous examples of the simple choice beating the complex one for all imaginable problem instances (although the advanced datastructure is faster in asymptotic behaviour). Another very important modern player is the cache. Simple and small often wins because it has better cache behaviour.
Note that I am not advocating for "outright dumb". "Outright dumb" is when your search of 100.000 elements is done via a linked list. Fortunately, such blunders can often be corrected with a bit of code that does not impair the readability of the whole thing.
"what matters is fast enough"
people love saying X is too slow etc, the only thing that matters is the ability for X to perform up to your requirements, if you need a million concurrent users then yes access will be the wrong tech. but to say X is too slow, you must specify exactly what it is too slow for.