Hacker News new | past | comments | ask | show | jobs | submit login

Makes me wonder why smaller website teams need dozens of engineers to keep their infinitesimally smaller app running.



From my limited experience, it's a lack of the kind of systems thinking present in companies like Stack Exchange, and present on the High Scalability blog, that results in a poor architecture.

And this is compounded by people who have little ability to troubleshoot performance issues. It's quite easy to hunt down the cruddy SQL queries in a DB, or realize that you spinning rust is too slow. But when it starts to come down to things like a blocking network fabric, which has some big fat buffer between two servers, that is killing your transaction speed - many will just start to blame the devs.

Of course, the second point is compounded by the first - the less systems thinking that went into the design in the first place, the harder it is to produce accurate hypotheses about the system in order to troubleshoot.

Whilst there is a lot to be said for the "right tool for the job", if you watch an artisan crafting something, you'll realize that despite having a huge number of tools, they actually get by with relatively few. This is a generalization, but they only use the full range when doing something new, solving a particularly tricky problem, etc. "All the gear and no idea" is certainly applicable in many start ups.


I personally think it's due to overly complex applications and systems with many moving (and breaking) parts.

Also StackOverflow often practices "scale-up" instead of "scale-out".


More specifically, I think it's due to long feature lists and multiple problem domains. I'm also thinking about in-house line-of-business software.


Also due to political managers that want to increase their head-count to give the impression they have more power.


There are hundreds of reasons at nearly every level of a company that can contribute to this situation. Its best summed up by Sturgeons Law, "90% of everything is crap."


Rigid adherence to certain technologies could be a culprit. "We're a Ruby shop!" is fine for problems where Ruby is great, but the limitations of the technology are bound to creep up somewhere. Not using a polyglot approach would be bad.


It can also be caused by too much embrace of polyglot tools. "We use the best tools for the job" can easily mean an architecture with Redis, Memcached, Mongo, MySQL, Angular, JQuery, Node, and several dozen third-party libraries, all glued together with Thrift or JSON-RPC. If all you're doing is a messaging app, you might be much better off with a single in-process Java app.

Usually you want your solution to be just right for the problem; solve the essential complexity but don't introduce any accidental complexity.


I wish more people considered this alternative when recommending the usage of queues etc. I always remember of this post: http://williamedwardscoder.tumblr.com/post/18065079081/cogs-...


Hacker News also runs on a single box using only in-language hashtables and lists. ViaWeb did too, and PlentyOfFish.


Guess they don't have Jeff Atwood and Joel Spolsky onboard.


As of about 2 years ago, Stack Exchange doesn't have Jeff onboard either.

http://blog.codinghorror.com/farewell-stack-exchange/


Joel is very much still around and active. Jeff went off to build Discourse [0] although I think he still has a place at the co-founder table as invisible-benevolent-dictator.

[0]: http://www.discourse.org/


Probably because the engineers are a lot worse.




Join us for AI Startup School this June 16-17 in San Francisco!

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: