I will live to be a happy old man if I never read another blog post telling me in absolute terms what I must do and why I'm an idiot if I don't. I blame Joel Spolsky for this, his tone and style seems to have kicked off an entire sub-culture of blogging where the posts read like a lunatic standing on a street corner demanding that we repent before the end of the world and the coming of the messiah.
Bloggers, please try a kinder, gentler tone, one that suggests you and I are colleagues. One that suggests you have discovered some neat thing and won't I have a look at it and isn't it interesting to play "what if" games with it and maybe, just maybe I might experiment with it.
Looking back on my own posts, I would say the strongest ones are those where I found a more reasoned voice. An essay can be strongly opinionated without being imperative.
I agree, but TBH there have been a lot of ridiculously overhyped blog posts/articles telling us how totally awesome every scripting language is, and how it's completely a 'game changer' and how you can do completely revolutionary things in them really easily.
I guess this is just a reaction to the froth+hype. Both sides need to step away from the keyboard, and calm down.
If a blog post says "wonderful, amazing, fun, life-affirming," we can always respond with "oh? what about..." My thought is that there's no need for a post saying "You're gonna die if you try to build a social bookmarking site in Lisp" regardless of how much fun other people are having doing it.
What have people got against enthusiasm, any ways? There's some kind of weird puritanical thing going around programming culture that seems to view youthful exuberance as some kind of a sin, like wearing buttons on a frock coat. Yeah, some twenty-somthing year-old has discovered something way-cool really neat amazing thing while hacking on a macbook in the café. Isn't that exactly the attitude you're supposed to have in your twenties? How would anything ever change if people didn't get super-excited beyond all rational limit?
Can you imagine Wozniak and Jobs trying to blog about microcomputers these days? They'd get flamed as hippies fiddling with toys :-)
What have people got against enthusiasm, any ways?
The ruby and rails fervor never read like genuine enthusiasm. The pro-rails bits were always framed with negative comments about how php and java sucked, or how stupid business people are. Even DHH has a lot of this in his blog posts and talks. I thought it read more like an abuse victim or recovering addict who found a religious support group. It was always a bit manic sounding.
I'm a bit cynical but I think a critical mindset is the bane of the engineer, or really any maker. If people are zealously gushing about their framework, something dark is lurking underneath...
Does my writing read like I'm telling kids to get off my lawn? If so, please tell me, I own a funhouse mirror. When I look into it, I see a guy that still loves programming and thinking about for its own sake...
Sweet merciful lord. Is this guy channelling Zed Shaw or what?
Apart from all the hand-waving and generalizations, this guy doesn't sound like he's ever really a webapp under serious load.
I'm not a Ruby guy, so I don't know how things fly with that, but I've never once had a problem with a webapp being slow because of the language it was written in. Bad database queries, unnecessary assets being loaded, definitely. But CPU-bound limitations caused purely because of the cost of interpretation of a scripting language? Not so much...
I've worked on a webapp under serious load, and I think he's right.
You're correct that database queries and resource bottlenecks are the usual problems for web application scaling, but even taking that into account, I've had more than one problem that was caused by the simple fact that Ruby is a dog of a language. Python is better, but only just so. Perl is faster than both, but still nowhere near as fast as a C++ program.
But let's ignore execution speed, since one can make a reasonable argument that even the slowest scripting language can saturate the network I/O on modern hardware. I think I'd still consider using a compiled language, because the type-safety and compile-time guarantees of a statically typed language would go a long way toward eliminating a huge class of silly bugs that we deal with on a daily basis. That's developer time in the bank.
> I think I'd still consider using a compiled language, because the type-safety and compile-time guarantees of a statically typed language would go a long way toward eliminating a huge class of silly bugs that we deal with on a daily basis.
Not all compiled languages are statically typed.
> That's developer time in the bank.
And the time spent in the edit, compile, test, run, cycle? I'd think there's more developer time in the bank if I can edit, test, run.
No, not all compiled languages are statically typed. I wouldn't use those languages.
The time spent compiling is a red herring. I know that I've spent dozens of hours chasing down weird runtime bugs in Ruby that a C++ compiler would have caught for me during static analysis. That time more than makes up for the 15 minutes a day I might spend waiting on a compiler (I waste more than 15 minutes a day on coffee breaks!)
Yeah I would like to second this... I'm no expert on web apps or frameworks, but I've heard enough that they're generally more I/O bound instead of CPU bound.
Now if he were to propose writing JUST a web application in C that has a custom HTTP server designed from sockets up to HMTL... you can streamline the sh*t out of that.
Much luck to him. Success or failure, he'll still probably be learning...
I have worked with a project for the past several years that, by necessity not choice, required a web application framework written in C (it's an embedded platform) and I can tell you it is pure hell. The most mundane and simple tasks, often trivial one-liners in scripting languages, require many lines of error-prone C code even with a fairly rich set of libraries available. Working in this environment, you'll quickly realize why the rest of the world doesn't do it this way. Thankfully, this project is finally moving away from this architecture and I couldn't be happier about it.
Would Lua have worked there? It's a ~200k library designed to provide an embedded scripting language to C projects. (It's very similar to Python, but a tenth the size, and with a bit more Scheme influence.) While it may be too big for some especially tight embedded platforms, if you could fit it in, it would have likely been substantially less painful.
It's also a pretty good scripting language on its own merits, and has replaced Python as my default language. (Unlike Python, its design expects you to know at least a little C, but in exchange, it avoids cluttering the language with things that C already does well.)
Lua is meant for embedding in C programs; that is different from running on embdedded systems. The latter typically features drastically less RAM and CPU cycles, so most interpreted languages (which are not shy about trading away some performance) don't cut it. There are some exceptions; FORTH, for instance, can fit in about a kilobyte and its performance isn't miserable. But putting e.g. Lua in a confined space would not work out well.
To my understanding "embedded systems" spans from something with 1k of memory to something like a cell phone or a router. Working with Lua is quite comfortable on the latter, hence my qualification. It's been ported to Lego Mindstorms (http://www.hempeldesigngroup.com/lego/pbLua/index.html), for example, which is "a 32 bit ARM7 microcontroller with 256K of FLASH and 64K of RAM."
It's not clear from the post which end of the spectrum he or she works with. (I don't work with embedded systems professionally, I've just tinkered with OpenWRT, cell phones, etc. as an enthusiastic amateur.) It would be usable in the same niches that SQLite is, for example.
The most mundane and simple tasks, often trivial one-liners in scripting languages, require many lines of error-prone C code even with a fairly rich set of libraries available.
What would be an example of this? I'm mostly a C and Perl person, and once a library is available I've never noticed cases where something was much simpler to write in one than the other. That library might take more time to write in C than Perl, but once it's written I find it just about (less than a factor of 2?) as easy to write the application logic in C as in Perl. My instinct is that if you're writing 'many lines of error-prone C' to do something simple, there's probably room for improvement somewhere.
I had that basic idea when web app frameworks starting becoming popular. So I implemented a web framework which took a 'description' of the app written in perl, and generated C++ which compiled into a single exe file which served the app with a built in http server. Small bits of business logic were written in C++, but basically it was done by the perl. All templates, internationalisation, everything, compiled down to a few database queries, string concatenations and the odd if statement.
I then build a couple of production sites in this, one on my own, and another with a few other developers.
Conclusions from this:
1) It's really really fast and efficient, and the single exe model for deployment is lovely.
2) It's a pain to implement such a framework, and a pain to use it.
I like Perl ... using modern Perl practices and CPAN modules. It's the best web development environment I worked with ... and I've worked on projects with PHP (my own framework :) + Symfony), Python/Django/web.py, ASP.NET, Java/Struts and Ruby/Rails.
You should give it another try sometimes. Take a look at Catalyst (the web framework), DBIx::Class, Moose and POE.
I think I just got a bit fed up of the crufty syntax for object-orientated code.
Everything computer-related sucks, you just have to choose something which sucks the least where it matters for your project. And familiarity counts for a lot too.
It's a valid point. I wrote the Mibbit backend from scratch in Java for these reasons. I can only start to imagine the horror I would be enduring if I'd chosen a scripting language.
Having said that, choose the right tool for the job.
edit: Downmod me all you like. A scripted language wouldn't handle 2k req/sec on a couple of VPSs.
I think Mibbit, much like twitter, is one of those apps that is a non-traditional webapp. While its a web based frontend the traffic and distribution of chat messages is a much different beast then what most webapps do. Java is very likely a much better choice for these reasons.
Actually after I wrote that post, that's something I've been tossing around. So many of the current crop of web superstars, like facebook, twitter, friendfeed etc. are built around streams of data as opposed to serving more traditional static(ish) pages.
Perhaps a new type of web framework is needed to address these breeds of apps.
Why would anything need to go to db? :/
Going to db should always be a last, final resort.
People often seem to think a db is the only way to write any webapp, and that everything must depend on the db. That's bad architecture.
In the case of Mibbit, db access is only done for writes, lazily, in the background.
The bottlenecks for me, are CPU, network, and ram. And yes, you can throw servers at it, if you're frivolous and don't intend to make any money. I decided to go the other way and be profitable ;)
The vast majority of web applications are straight-up CRUD. Not making that all about a normalized schema is the stupid mistake.
As far as relational DBs being the problem, that's more a matter of the free ones not cutting the mustard. Oracle and DB2 can handle jaw dropping load in the hands of an expert.
Would you not agree that the web is becoming more "real time". People demand webapps to update as they view them - ajax,comet,getNextBuzzword(). Those demands place a higher load on servers, especially when you scale to large number of users.
For sure, but 99% of apps never scale to "large numbers of users". And when they do, they usually have the time to figure out how to do so without downtime. Often, that may involve rewriting part or all of the backend in a lower level language. If that's needed, so be it - but to do it before you have any performance problems is just premature optimisation.
Judging by the number of blog posts, articles and books about scaling this or that Ruby/Python/PHP framework, people seem to be very busy with "premature" optimisation right from the start, exactly _because_ those frameworks and languages are so horribly inefficient.
Why would I want to run and manage and design for a server cluster just because each and every Rails process needs to load the entire language runtime, the framework and all the libraries using 100MB of RAM for no application functionality at all? Not doing all that work is premature optimisation?
"I predict in the future all the good web engineers will be moving to compiled languages, and all the academic idiots programming social networking applications, ajax widget home pages, and projects to better improve our democracy will be staying behind coding in the latest and greatest slow language, going to circle jerk conferences to hear some scumbag just oozing hubris from every pore tell them to get in shape and learn to fly a kite."
I wrote the upload and download handlers for droplink.me in C as FastCGIs, anticipating performance problems. I built it; but they never really came. It was a fun exercise, but if I wasn't interested in having fun with C, I would have written it in a dynamic language.
I think the reason there isn't a "C web development community" is that programmers are more than willing to trade time for performance. Safe, readable code can be written in high level languages in a snap. You can see your ideas happen very quickly. If you run into performance problems (which I think is safe to say is a fairly rare occurrence for the average web developer who produces decent code, in any language), you can a) refactor or streamline what you already have or b) start fresh, lessons learned, in something lower level.
An aside: the site http://halfbakery.com is implemented entirely as an Apache module.
The reason why people choose to go with languages like Ruby is because the trade-off on speed is worth it.
Web front-ends scale very easily. You can just add more web heads and load balance between them. Databases are much harder to scale since you need to keep data consistency and can't just throw more boxes at the problem.
So, you don't need to be computationally efficient. However, you do need to develop fast. You need to be able to create features before your competition. You need to be able to make your system more reliable and bug-free than your competition. And that's where Ruby (and similar languages) helps you.
Sure, you and Linus have a point that selection of language can keep out the riff-raff (Linus Torvalds having a huge rant against C++ simply because of the programming community), but it also means that you're missing out on the good parts of the community. Ruby programmers tend to put a lot of thought into user experience. While you've ranted against testing, I don't think any IT manager would agree with you there. Testing substantially improves one's ability to create new features because you can easily spot anywhere you've broken something. And that leads to better quality applications too.
There's a difference between speed and scaling and if something scales fine, that's ok. Look at it this way, array indexing in a slower way to go through an array in C than incrementing a pointer. And if you had a lot of code you could go through and change all instances. However, it wouldn't be worth your time. Such an improvement wouldn't net you any real benefit while you'd be neglecting improving your product.
I think your point is exacerbated by the fact that it's hard to get web UIs "right". I was critical of languages like python and ruby until I had to do the frontend for some company code. Box looks like it needs to be shifted a few pixels right? Modify code, compile, check. Logo looks better with an offset? Modify code, compile, check. Writing in a scripting language makes it just Modify, check.
You just can't do that in compiled languages, and the compiled language of choice for the web-- Java --isn't known for blazing compilation speed. You'd probably also have to jar it up and copy the damn thing over, whereas psps could be modified in-place. I think this is something that compiled folks like me simply failed to grasp: all the things about scripting languages that make them orders of magnitude faster for developing, especially for the web.
I've done a lot of webapp development in both Java and Rails. I would agree that the iteration cycle is shorter in Rails, but it's not as bad as your description makes it sound. If you have to redeploy a jar file as part of your development process, then that's a problem with your process, not your language.
When I do Java webapp development in Eclipse, it compiles and hot-swaps my code in the background without requiring any restart or redeployments. Works pretty well.
I do have to restart occasionally because the hot-swapping has limitations, so that does slow things down a bit, but when you consider the time saved by having good compiler integration, then I don't think there's a big difference in productivity.
Your comment adds less to an intelligent discussion than does the article you're calling a troll.
First, this isn't a troll. The guy posted a blog post on his own blog. He didn't post off-topic content to a thread and didn't try to provoke an argument in an otherwise civil conversation. The only possible way that you can characterize this as a troll is that he uses some crude language, and says some things that are unpopular on HN. But say what you will about the content, he didn't try to make you read it.
Second, the guy has a point. I've had a rant like this building in my craw for a long time, and this guy nailed nearly every point I've wanted to make. From a software engineering perspective, web development is in the dark ages. What's worse, it's a luddite process -- a totally self-made exile from the benefits of 50 years of progress: Type safety is shunned. Language efficiency is ignored. Data structures beyond the hash table are exotic and rare (and worse, I've met more than one "professional" web developer who couldn't tell me the difference between open and closed hashing!)
I understand very well that programmer time is more valuable than computer hardware, and that the speed/development time trade-off is often biased toward development time. But if you ask me, what was once an intelligent guideline is now becoming a cult. It is possible to do rapid development in a compiled language, and there are some really good reasons -- beyond execution speed -- to revisit the idea.
"Your comment adds less to an intelligent discussion than does the article you're calling a troll."
Sometimes the truth is very simple. It can be considered a troll because it is in the tone "if you don't do as I say you are an idiot, like everyone else".
Second: not all scripting languages are interpreted on the fly. In fact, many of the most popular frameworks load objects once and keep their compiled forms in memory. Even for languages that run under VMs, some of them run under JIT that, in the end, results in machine-language-like times.
Third: I have profiled a lot of web applications and most of the time they are either waiting for the pipe to deliver a request or waiting for the database to deliver the data. Very few web applications I know of do heavy processing. For those, I agree compiled code may help.
and, finally, by the time the guy finishes his C code, an equivalent competing application that uses Rails will be approaching the millionth registered user.
Even though the tone of the article was abrasive, it was far more interesting and insightful than the one-line comment to which I responded. Your asserted arguments for dynamic languages are well-trodden ground:
Assertion: "Some scripting languages are as fast as compiled languages."
No, not really. I challenge you to prove that statement with a non-trivial benchmark.
Assertion: "most of the time, web apps are waiting for something external to respond"
This doesn't survive a few seconds of thoughtful analysis. If a web app is spending most of it's time blocking for I/O, it's poorly designed. There's no reason that it can't be handling other requests while waiting for a DB response or a pipe. And once you've allowed for that possibility, you're back to a situation where you can handle more requests with faster software.
Empirically, this is obviously true: languages like Ruby and Python don't hold a candle to Java when it comes to web application speed. We choose them because of developer convenience, nothing more. Which brings us to...
Assertion: "C programming is terrifically slow for development. <insert snarky comment about Rails applications with a million users here>"
Ignoring the fact that there are other compiled languages than C (that are probably better choices for web development), I can tell you from experience that by the time your Rails app reaches a million registered users, 80% of your week is going to be spent either working around the myriad scalability flaws in Rails and the dog-slow nature of Ruby, or installing hardware to accommodate the load. Scaling Rails is a nightmare.
Now, it may be true that you can develop that Rails app a bit faster than you could a webapp in a compiled language, but it's a wild exaggeration to suggest that the difference is more than a few weeks of skilled developer time. It's a fallacy to imply that all compiled-code projects end up in death-march hell. Truth be told, the real reason that web apps can be written so quickly is because they're easy, and they tolerate a lot of development error. So long as there are good string processing facilities, the choice of language is a fairly minor point.
I find this similar to Woody's fear in the following exchange:
Cliff Clavin: Boy oh boy, talk about your lucky days, huh, I just found twenty smackers back there in the pool room. And to celebrate my good fortune, I'm going to buy a round a drinks for all my friends.
Carla Tortelli: What are you going to do with the other nineteen bucks?
Sam Malone: Woody, that could be your twenty dollar bill, the one you lost.
Woody Boyd: Hey yeah, you know I was in the pool room earlier.
Cliff Clavin: Hey, wait a second. Let's be fair about this. Give me the serial number.
Sam Malone: Oh, come on man. Nobody knows serial numbers.
Woody Boyd: L21886119B
Cliff Clavin: [hands the bill to Woody] That's amazing.
Sam Malone: How did you do that?
Woody Boyd: I memorize the serial numbers on all my currency.
Sam Malone: Why?
Woody Boyd: For just such an occasion. Tell you though, I pray every day I don't get rich.
Not quite a perfect analogy, but my point is worrying about scaling is a very good problem to have, and is likely to be accompanied by the resources to fix it. Twitter is the poster child for Rails scaling problems, but they re-wrote the parts they needed to write, and people kept using it because they liked the service even when they did have scaling problems.
Twitter isn't really the poster child for Rails scaling. A huge chunk of their traffic isn't handled by Rails, and their web interface is fairly perfunctory.
In any case, at this point I think it's fairly safe to say that I have some direct experience scaling Rails, and it's not as easy as the hand-wavers suggest. It's certainly worth a bit of work early on to avoid the nightmares that come with trying to scale something like Rails later on.
Not to be too silly, but if someone wrote a web framework in C++ that was 10% more difficult than Rails, I'd still consider it. The distribution of suck for Rails is exponential over time, so the expected value of suck is much higher than it initially seems.
"A huge chunk of their traffic isn't handled by Rails, and their web interface is fairly perfunctory."
I guess I wasn't clear enough. I meant they rewrote the parts that didn't scale in another language. Scala, I believe. Is that incorrect?
So, my point then is that writing it in Rails to quickly get it in front of users and quickly iterating on features turned out to be worth it, even though they had to rewrite large portions in another language once they had a lot of users. Do you disagree? I'm not a Rails developer, but this is my understanding of how Twitter development played out, and it seems to have worked very well for them.
I'm curious, if you would accept 10% more difficult, would you also accept 20%? 50%? I'm wondering just how exponential the suck can get.
"There's no reason that it can't be handling other requests while waiting for a DB response or a pipe."
Concurrent request processing most likely belongs in the server, not in the application.
"Scaling Rails is a nightmare."
For someone who complains of "well-trodden ground" in others' arguments, you certainly seem to have a deeply-rooted belief in the myth that particular languages and/or frameworks "scale" or "don't scale".
Write me off if you like, but here's a short list (just off the top of my head) that we've encountered while trying to scale Rails:
Rails has terrible support for database replication (we've had to roll our own). Its caching architecture is laughably bad, and the implementation was/is buggy (again, we rolled our own), and there's next to no support for the kind of robust page cache expiration mechanisms that are necessary to run a large, dynamic site (Observers are only a start). Rails is a memory hog, takes too long to start, has poor translation and locale support, and simply passing a request through ActionController adds huge overhead that cannot be escaped.
And even though you don't want to hear it, Ruby is slow -- even after working around all of the above, we still had to implement a fast page caching layer just to work around the general, all-purpose slowness of Ruby.
Seems like you're picking things which are difficult to deal with no matter the language/framework your application code is written with -- DB replication, cache invalidation, etc.
"If a web app is spending most of it's time blocking for I/O, it's poorly designed. There's no reason that it can't be handling other requests while waiting for a DB response or a pipe"
If your request-handling logic is responsible for concurrent requests, your application (and the platform it runs on) is poorly designed. This belongs to a thread/process dispatch mechanism in the platform, not inside your app. When one thread waits for the database, I am sure the processors can be busy with other requests.
"Scaling Rails is a nightmare."
No. It's not. There is little reason for it to be. While I am no Rails expert (I use Django and was a Plone guy not that long ago), the scalability issues most people have happen because of RDBMS concurrency much more than any problem with Rails itself. Like I said before, that time spent waiting for the database to respond is time spent in the database working. It is incredibly rare to have a front-end having heavier usage than the database that feeds it.
If you want your applications to scale, design it around the database limitations. Use C, C++ (I would prefer ObjC), Java or Assembly if you want, but faster request processing won't get you past database scalability issues.
Using C* or Java will only push back your release date.
In my experience once you get over a few hundred thousand users and have tweaked your web server/proxy setup and tuned your database, you will still have major slowness and memory issues with your Rails code. I.e. it's usually not the database's fault.
Also you might want to look at the profile of the guy who said that rails scaling is a nightmare. I wouldn't use the word nightmare as I'd prefer to think my professional life story wasn't written by Kafka. But, I don't know anyone who has scaled a big Rails site and thought it was a pleasant or fun experience. Most guys who say that it's no big deal haven't really done it.
I will concede that, according to my very limited Rails experience, it generates a lot of database chatter when compared to hand-coded queries. Still, this is mostly a database scaling problem.
The beauty of the ORM is that you can replace the underlying layer without disrupting the code running on top or add fancy things like local memory caching or a networked memcached store. I insist that if your application was built from the start with this kind of issue in mind, it will be easy to scale it if and when the time comes. Until then, you are solving a problem you don't know you have.
This gets harder the lower-level your language gets.
The top line in a gprof profile of an evented C web server had better be select() (or equivalent) or read() (or equivalent). So I don't follow you here.
Yes, the average server thread is going to spend far more time blocked on I/O than not. But the server should be able to handle more than one request simultaneously.
(Or you can use an event model; the point is, if your server is spending all of its CPU time blocked while under load, then something is wrong.)
Now you're not following me, timr. An evented server can handle 10,000 connections simultaneously. Run it under load, profile it. Select is at the top of the profile. The program spends all its time in the kernel, waiting on I/O.
That evented server is also faster than the threaded version of same.
What's the compute task you think a program is bottlenecking on in an application that does nothing but shovel data out of a database into a socket based on a btree key?
You're overgeneralizing. Where your server spends its time depends entirely on the profile of your app. If your pages render (on average) in less time than they wait for I/O, then your server will spend most of its time blocked on I/O.
I suspect that you're thinking of something like a lightweight proxy server, that waits for incoming requests, then quickly hands them off to an app server that does most of the work. That's trivial. If you just profile just the webserver process, then yes, you would expect that most of its time would be spent in wait. But while the proxy server is blocked on I/O, the app server is doing a ton of other things for any non-trivial page rendering.
This is worth repeating: only small part of request lifetime is spent in application code so you should worry about performance of the programming language only if you profiled your app and saw, that this is indeed a bottleneck.
I don't think it's a troll, but just someone who is really in to performance, like I do, and I agree with him on some points.
But what I don't agree is using just C. C is too low level and is a horrific unfit task for web development. Instead C++ would be a good fit (and writing in C mixed in). The STL/Boost provide a wonderful abstraction layer that does pretty much anything you throw at it with ease, plus there are lot of other libraries for anything else you can ever think of. I've written several web applications in C++, and it was actually very much like writing in Python/PHP/Ruby and at time prefered writing it in C++ after writing a poc in a scripting language. They were just a little bit less readable but very clean, but the biggest plus the the multiple magnitude performance increase of the application it self with FastCGI. (all these applications were serious number crunchers and had lot of CPU and Memory Bound problems).
tldr;
C for web dev painful, C++ with STL/Boost good/easy/fast/clean.
I've been hacking C++ long enough to agree C++/STL/Boost is good and fast... but I can't remember the last time I heard anyone accuse C++ of being easy...
I find it easy. The bad thing about c++ is its size. It's large and that tends to make it complex. However, I find that I only use about 40 to 60 percent of what c++ can actually do. Focus on the parts that you use and you'll find a lot of the complexity goes away.
How about a web-centric programming environment that compiles web page code to DLLs, and changes pages "dynamically" by unloading the DLL, recompiling, and reloading? Base the language on a modified Ngnix as an execution environment and generate C as intermediate code. Use TCC to compile web pages very quickly. The modified Ngnix would have a big hashtable of dynamic methods, which would be pointers to functions in the DLLs. Make it concurrency-friendly like Erlang by using isolation and green threads.
The old blog software, Typo, is nowhere near as good as Wordpress. I only used it because it was written in Ruby on Rails and I am a Ruby on Rails coder. This is a very poor means of choosing software.
The VisualWorks Smalltalk Server Pages framework already does this.
SSP results in web pages as compiled machine language.
The HTML file is translated into a new Smalltalk method, the tags become Smalltalk message sends, and the HTML between the dynamic tags just become accesses into an Array. Because Array access is optimized and JITed as machine code, the only message sends are due to the dynamic tags. When the webpage/method is JITed, you have a web page compiled into machine code.
I believe the framework discussion is of a compiled C "Rails" type MVC framework, I'm not familiar with Smalltalk Server Pages but from an initial google search it doesn't seem to be on par with what rails provides?
You're right that SSP is more like ASP/JSP, and not like Rails. But the same idea can be used with a more comprehensive web framework and with a language that has a really good JIT, like LuaJIT, or maybe IronPython/IronRuby.
Guys, his point isn't valid. If you want to write things in C, the current implementation of ruby LETS YOU. That's the whole point of C extensions.
You can then ask, why bother writing in Ruby at all, and i think you'll find the answer to be, it's not anywhere near as much of a pain in the ass as writing C is.
The elitism argument is bullshit. We don't need fewer people programming. We need better ways to tell if someone knows what the fuck they're doing.
In my opinion the tools we use and develop should be as accessible as possible. Anyone should be able to pick them up and get what they need to do, done. Good luck doing that with C.
Web site performance problems usually don't have much to do with the raw executing speed of the language they're written in. The problems derive from poor understanding of how to handle concurrent connections. If your app is written in C, but is accessed through a CGI interface - starting a process for each request - then it's going to be slow. Similarly if you write a web server which makes blocking connections to a backend database, your server will be slow. If you use threads to handle concurrent connections, you're going to allocate 2mb stacks for each user - and your website is going to suck. This all has little to do with how fast a language can invert a matrix.
And then there's situations like porting a Rails app to Lift and seeing a huge improvement in performance without touching the database / storage IO: http://lambda-the-ultimate.org/node/2147
Granted, it's not from the most unbiased source, so take it with a grain ;)
That has to do a lot with Ruby's awful thread implementation. You could possibly port the app again to Ruby EventMachine and see even better performance.
The point is - for 99% of people it's not the language, it's the I/O
There are already a few options for a MVC web framework in compiled code. I'm working on ASP.NET MVC now. It's compiled, but development does not feel like that because of JIT compilation. Also, it will run in Mono on Linux if I want.
Actually, the problem isn't compiled vs. non-compiled, the problem is C vs. languages that you can actually safely manipulate strings in without segfaults, buffer overflows, and other "own the box/own the webservice" flaws.
Arguably, all the languages that show up down near C in the Shootout would be fine languages to write webservices in, since nearly all of them are as fast as C without the eXtreme danger.
Some web services are "shovel strings from the DB out to the user", like CMSs, but for those that aren't, this isn't a terrible point.
(It's easy to forget when you're the type of person who hangs out on HN, but for most people, compiled == C, or C++ if you're lucky.)
I notice comments on the blog post are currently closed. I wonder why. ;)
Although I understand the author's position, it is obvious this is a tech person and not a well-rounded business person. The thing about programming is it's not all about tech.
A program typically serves one purpose: to make a company money. This requires more than just pristine tech. First and foremost it requires time to market so that a company can stay competitive. It also requires the ability for more than just a handful of people to be able to write it for expandability and, God forbid, staff changes.
C is fast. But slow to market. And slow to pick up a program. And slow to test. That's why there is so much WYSIWYG.
Don't get me wrong: there needs to be a lot more code behind and a lot less script out there, but in the end coding is a balance. Anytime anyone says it's all this or all that, you know the position is not sound. It's part speed, part usability, part scalability, part development speed, part readability, part expandability, part pretty pictures, etc. Balance in all things.
That is why there are things called "application servers". Java is not perfect but it has a fine feature called JIT (Just In Time compiling) that produces machine code whenever it makes sense. Raw compiled language goes against the current trend for virtualization, whose goal is to save energy too. So in my opinion JIT is a good solution.
I don't think the guy is troll. I think he simply doesn't know better. This is a classic case of micro-optimization. While he should look at the bigger picture. It is all about global architecture.
Funny article, I hope the author was not being serious. C is everything when you need performance, but having something done quick and maintainable - in C - no sorry. If somebody would dare to actually create a rails or django level framework in C it would simply have no community - not because it won't be 'hip' - I'd use it - but because implementing the same app would take exponentially more time. And that's what matters in the webworld.
No matter how many rps your app can serve, if there is no app you get no requests to serve...
Perhaps unsurprisingly, it shows no awareness of SQL injection, buffer overflows, or concurrency. These seem like things a web framework might need to deal with...
I would NOT trust a website with my private data if they wrote their app in C. Buffer overflow -> execute your own code -> write a function that pipes an entire database out to you. It really is a trivial problem given the buffer overflow, and programmers (myself included) generally suck at not putting buffered overflows into code.
Plus there's the whole portability thing, which is entirely pissed away by C compilation (yes, arguably you can write generic C, but there are fundamental differences (file i/o, sockets, etc.) between different OS's, and restricting yourself is dumb)
Pre-compilation of scripting languages isn't necessarily a bad thing though.
I have done this in c++... not write an entire web framework, just use compiled c++ code to do some heavy lifting on a shared web server. It worked great. Before, I was using python/django. c++/cgi did the same task and was about 30 times faster.
I'm not sure I'd throw away the scripting languages entirely, but making it easy to integrate c or c++ into these languages in this manner is probably a good idea. It's not that hard to write very highly optimized c++ and it can have a huge impact on performance... at least that has been my experience.
While the article was inflammatory, I think the guy has a valid point. He just needs better people skills ;)
The first thing that came to into my head was 'What problem is he trying to solve?'
Does he have performance problems? Is it cost effective to solve those performance problems using C? What is the ROI?
Has the performance problem been analyzed to a degree that its been identified that its the web framework? (Most of the time a decent database index will help)
Go after the lower hanging fruit first and make decisions on quantitive data analysis.. not just your gut.
Ranting aside, this seems like a valid complaint. Scaling and performance seem to be reasonably important in web applications, so why not write them in C? If the only complaint is the lack of good tools, why not build those tools?
Disclaimer: I don't actually work on web apps, and therefore have no experience with these things.
If your site starts to take a performance hit in rendering pages, you can simply write a module in C to handle the part of the script that's using an excessive amount of CPU time, and keep the rest in script.
But for most web apps, rendering HTML pages from script is not the bottleneck, it's the interaction with the database, and the solution is intelligent caching to reduce load on the database. The tools to mitigate these problems are already written in C (memcached, etc).
It is highly unlikely that happens.
Once you lower your DB issues, if the application is responsive enough you start working on other features. By the time you're done the application got new users and the DB is a problem again ... it's a cycle :)
Fortunately HTTP servers do scale by caching and load-balancing.
The author of the post probably never worked in real-world conditions. You first have to have users and real needs for scaling, and that doesn't happen unless you deliver a useful application first, and you also have to deliver it pretty fast, otherwise you might lose to a competitor with the same idea.
Reread what you wrote :) Once you lower the the db io issues, rendering does become the bottleneck.
Not really. Sending and receiving data to/from the browser becomes the bottleneck.
If it takes 50ms to receive the query from the browser, 0.1ms to retrieve the data from the cache, 10ms to render it, and another 50ms to send it back to the browser - the bottleneck is now network communications with the browser.
There are, unfortunately, limits on how much that can be improved.
To me, C makes a lot of sense for software that doesn't change all the time. The point in the article about being an engineer is valid.
However, most website projects are business-driven, which means that you are constantly changing and refactoring stuff as misunderstandings get cleared up, design flaws emerge, business models change, minds change, etc etc.
Business software projects are not just engineering, and if you treat them as such you will endure pain.
The analogy with a house is odd - if a house was purely an engineering problem, then they would probably all be built out of precast concrete or something, not "wood and steel". Aesthetics, tastes, budgets, and tradition all have an impact, and nearly every house is unique, so you use the "best tool for the job" - the one that is the easiest to customize on-site.
True, if house building was business-driven (instead of solely architect-driven) then it would make more sense to use materials more suitable for iterative process. Like cardboard boxes...
When requirements change weekly and by the projects end your original project description is only useful as a piece of history - then using concrete would mean lots of demolition work in the process.
The increased productivity of writing in a scripting language typically outweighs the cost of extra servers. Whereas with a desktop application, you only have X resources to allocate to a task, you can always "just throw more servers at" a web application to some degree.
It is done occasionally though; last I heard, large portions of Amazon.com were written in C.
Because most scaling problems in web apps are ultimately architectural. Yes, you may need to buy a second server sooner rather than later if you run Ruby instead of finely tuned C or C++, but for most apps you're going to need to worry about a second database server and replication before that, and long before reaching either you will have enough traffic that you'll need to prepare to handle front-end scaling properly soon anyway.
An example - from messaging middleware, not web frontends since it's somewhere I do have proper data:
I did a messaging middleware system in C a couple of years back. It's easy. I've done it before. After a while needs were changing and I decided to try writing the replacement in Ruby. It was a lot quicker to write (1/10th of the code size for roughly the same feature set).
We were handling 3-4 million events a day, which could easily result in 2-10 messages each.
The Ruby replacement spent 10% of a single 2Ghz Xeon core handling those messages. Of that 90% was spent in the kernel handling system calls that would've been roughly the same in a C/C++ version.
Even if the Ruby part of the equation was 10 times slower for the userland part, the most we'd have saved for our load was 0.9% of a single core. The most we'd save overall, regardless of scale, would be around 9-10%.
Doesn't mean there aren't things for which a language like C is better suited - I did my MSc. thesis on reducing error rates for OCR, and some of the image processing I did was prototyped in Ruby, but then translated to C for the final version because processing literally would've taken days for things that took less than an hour in C.
But scaling is very often constrained by other things than the performance of the laguage you write your app in. Especially in the web space where most interesting apps are database backed.
If your DB gets maxed out, having a hyper-optimizing compiler is not going to help you. If your app isn't designed to be scale across multiple servers (or at least cores - these days you can rent managed hosting on a 24 core server for less than $1k/month) you could very likely be screwed in ways that no compiler will save you from.
The extra developer time spent doing web development in low level languages is unlikely to be worth it unless you're at a massive scale.
If you can fit in 24 cores with a language like Ruby, your best case saving from switching to a compiled language these days is around $10k/year on hosting if you're ok with a single server, and about $20k if you want redundancy.
That doesn't pay for a lot of wasted developer time. Most web apps can fit in that, and for the lucky (or unlucky, depending on reason) that need to scale beyond that, the CPU time spend on the actual web app is likely to be a relatively small fraction of the total CPU time spent (on databases etc.) - architectural decisions is likely to be a far more important cost driver than the language of the web frontend.
As for developer availability - good luck to him finding good C programmers, and even more so finding good C programmers who aren't expensive and knows web development.
I'm not sure that's accurate. c++ is a large language. Finding developers who use the parts of c++ that you need to get the task done is easy (boost, qt, stl, wx, etc). Finding developers that know all of c++ is next to impossible. c++ is large because it can solve very large, very complex problems, but not every c++ application attempts to do this. c++ can be used in very simple ways too and many people use it in that fashion.
The usual performance bottleneck in a website is the database, which is usually written in either C or C++ anyway. Other parts of a web application are less likely to cause performance issues, and can usually be scaled horizontally by adding more servers.
Most sites aren't CPU or memory intensive. If you are lucky enough to have enough users for that to be a concern you can add more servers. Sites normally bottle neck around the database and writing front end code in C is distraction from fixing the real problem which should be scaling the database.
I think you don´t have to stick with C (or any another language ) for everything. Each language are good for something. Right now, for example, i´m working in a financial intelligent software analyzer. I wrote the web crawler in ruby (not rails) and now i´m planing to write the core of the system, the decision maker, at Lisp. I think a programmer should not be sticked with just one language, there´s a lot of cool things out of the C world. You dont have to write an entire C framework to build an web apps at C ... it´s just too troublesome. You have another challenges. But when it comes to process data very fast in your server, so maybe you should consider another language.
In web the model is different: You develop an app, push it live and find out what works and what does not, change and iterate.
This means that doing the iterations with minimum time and effort are important than worrying about scalability and performance.
You can always throw hardware at the perfortmace problem. If you get enough traffic to become worried about performace, that itself is the mark of success. But to become successful, first you must build the app and get it live somehow instead of worrying about performace issues or wasting months coding in some very low level language!
You can always throw hardware at the performance problem.
No, you can't. That's what scalability means. If you don't have scalability, then additional hardware will only take you so far before you begin wasting it.
Many dynamic-languages frameworks (notably PHP) are more scalable than writing a custom webapp in C. This is because C encourages you to store state in memory on the server, because it's really easy when everything is a single long-lived process. Good for performance, terrible for scalability.
PHP (and Rails/Django, if you ignore their ORMs) encourage you to store state on some collection of backend servers and communicate via RPC to retrieve that. This is bad for performance, but good for scalability. The front-end is just a dumb intermediary that converts data into HTML, so if you need to scale, just add more of them. Scaling the backend is more challenging, but at least you can do it without dragging all the front-end code around.
Absolutely! Just for the record -- since people seem to assume otherwise -- I'm not agreeing with the article. I'm just disagreeing with the idea of ignoring "scalability and performance" and then expecting to solve performance problems by "throwing hardware" at them.
Well, you are both kindof taking the extreme view.
If you build your app such that the webservers can act independently, then by using various methods of load balancing you can just throw hardware at the problem.
However, this just usually leaves the database as the single point of contention among all webservers. There are various ways to reduce load on the database server by some constant factor (memcache, query caching), but in the end database load will be roughly linear with site load (requests/time). You can't just add SQL-based DB servers and get linear speedup like (properly thought out) webservers.
If you think your site will get so big so fast you won't have time to hire engineers to figure out the scaling problems before they become unmanageable (for example, facebook got huge, but it took a few years, so they had time to raise money and adapt their stuff, although they were pretty slow for a long time), then use appengine datastore or simpledb or some other db that promises to scale really well (this is a really hard problem, which is why there aren't a lot of open source things that do this. Note: couchedb can't scale writes, so I don't count it).
My personal thinking is that one big properly designed database can get you pretty far, and database clustering can get you far enough. If you find it pleasant, putting your app in appengine would make you pay the price of scalability a little at a time starting right away, instead of letting you push it off (but with a huge performance wall that you won't be able to get over without basically redesigning much of your apps internals).
Well, if I were forced to write in C these days, I would seriously write a C/C++ generator in Lisp to speed the time to market.
But if I could do it where nobody was watching, I would use a different compiled language, Lisp (SBCL) that would get good enough performance and get to market before the manly C/C++ coder.
So on a semi-serious note, why is (apparently) nobody going for the real performance kill, which is to write a framework in hand-coded assembler? Would it be twice as fast as C?
If you must write web apps in C then consider writing it as an Apache module. But how much traffic would you need to justify this? Web app developers have an unhealthy obsession with performance and scalability that distracts them from building what really matters: features. Twitter proved you can be just as successful if you work on features now and scalability later.
I find it ironic that he mocks "the losers who don’t know how to code and instead just pontificate endlessly about the hippest new blog post that totally changed their way of thinking"
Isn't that exactly who is writing to? Doesn't he WANT people to pontificate about his post and change our thinking?
the blog misses a central point - its not the choice of language thats a bottleneck. I'll trade performance to large number of users any day because I know I can solve the performance issue.
Personally I'd go with perl - as thats the language Im most comfortable with.
His point is valid, but the bottom line is that hardware resources are far cheaper than human resources today. Yes, many of us know C, but a lot of us would rather use the abstractions provided by scripting languages.
False. You only have to write ONE application, you have to buy N servers. It doesn't take very large N to overcome the price of a good programmer. Hardware is expensive when you have to buy _lots_ of it.
On the upside, it means web developers have another way to slack off: "But we can't work, the project is compiling!" As if web devs needed another way to burn our time.
Not really. The shopping basket (this was before everyone settled on "cart") was written in C, but the shopping basket wasn't the back end of the rest of the system, just one of several components that ran in parallel.
I don't see how it's possible to make a good framework in C. It lacks the necessary abstractions. It lacks even basic vital things like namespaces. Why on earth would someone attempt this in C rather than C++?
You could come pretty close to simply rewriting a scripting framework in C++. I don't know what you can get done in C. It will be so dependent on programmer conventions it will be hideous.
People actually have made some damn good C++ frameworks. See asio. What are the C ones? Any complex C system devolves into a mess of macros, casts, and naming conventions.
Isn't that's what CGI is for? If you want a full blown web application framework in C is going to be a lot of work. String manipulation, parsing HTML with standard C libraries is not going to be easy. Java, C# and Python have pretty decent frameworks to do this I don't see the point of writing one using C.
Bloggers, please try a kinder, gentler tone, one that suggests you and I are colleagues. One that suggests you have discovered some neat thing and won't I have a look at it and isn't it interesting to play "what if" games with it and maybe, just maybe I might experiment with it.
Looking back on my own posts, I would say the strongest ones are those where I found a more reasoned voice. An essay can be strongly opinionated without being imperative.