22 Cans is a tiny startup and they were expecting 10,000 concurrent users tops -- they're at 2 million and counting now, and they have to capitalize on that momentum and not piss off their obviously fanatical players. So they pulled an all-nighter to build and migrate from a single server to a clustered architecture, and hell yeah, if I was in that position I'd do that too.
Seriously, is there another option? If you think the answer is "clock out at 5:01 PM and go home", you shouldn't be in a startup. Obviously pulling 36 hour shifts on a regular basis is not sustainable, but this is precisely the type of situation where it's warranted.
That he "begged" them to go home is suspect. Is he not in charge?
I've sent people home during crunch times before when they demonstrated they were becoming a liability even though they still wanted to keep working.
You don't beg. You tell them that is how it is going to be. Leave or get escorted off the premises.
> Seriously, is there another option?
Yes. Send people home for some sleep and resume. Unless the people who stayed were carrying out purely menial tasks, odds are having them on the job 36 hours straight got far less work done than if they'd taken reasonable breaks. Sure, you can pull more than 8 hours now and again. Sure, you can even pull all nighters now and again.
But over a 36 hour period the vast majority of peoples productivity at complex tasks will nosedive well before half way, and stay flat to even negative in jobs like software development, as you slow down other people and negatively impact communications once you get too tired.
If you want the most amount of work done in 36 hours, you don't make people work the whole 36 hours.
Actually, if you enter it well rested the first 36 hours (with a couple 5-10 minute catnaps) can be quite productive, with only minimal reduction in productivity. Particularly if you get into the "zone."
It's after that 36 hours, that you need to get about 10-12 hours sleep to get caught up, and then have a few sane days.
Before we do complex cut over onto a new software system, we usually make sure all of our software engineers are very well rested leading into a friday night, in case we've got a couple all nighters ahead of us.
Anybody who tries to do back-back 36 hour death-marches with only 5-6 hours of sleep in between is setting themselves up for the seriously reduced productivity you've described. Nothing good ever seems to come of that.
So - in this scenario, I think they did precisely what they should have, but they shouldn't push their luck.
The question is why did they assume 10k concurrent users, or plan for that to be the cap handled under the current architecture? Any competent project manager should have said "let's pretend we're massively successful... how will it work then?" and have some contingency plan in place so that they're not trying "experimental" fixes.
The flaw here isn't the heroic efforts, it's that heroic efforts were needed. In the days of cloud based architectures, it's a lot easier to be ready for massive horizontal scaling than it used to be, so much so that being unready is malpractice.
Assuming massive success is a stupid move. The majority of the time you are not that successful. The ground is littered by projects that spent vast sums of money (and more importantly time) designing for load they never needed to handle before release.
In about a month my company will be releasing an online game. We think we will get about 15-20k concurrent users in the first few days, but we will be testing the realm for as high as 50k concurrent users using load testing bots.
What will happen if we suddenly get 100k or 200k concurrent users? Even throwing more hardware at the problem might not solve it immediately if there is some new bottleneck we haven't found before.
Testing the realm for more than 50k concurrents and then spending engineering effort required to make it work will consume time and effort that we can't afford to spend. All for an outcome that is unlikely to occur.
As long as you've put in enough consideration to avoid painting yourself into a corner with your system architecture so that you know how you'd scale the system should the need arise, you don't need to invest the engineering time up-front to actually build it. However, if you don't understand how your application is going to scale then you might be totally screwed by bad architectural decisions.
For example, you might assume that the data for all concurrent users fits within RAM on a single server. It's one thing to plan for an 8GB server for 50K concurrent load and then have to scale the RAM to 16GB, 32GB, 64GB, 128GB, and so on. But if your RAM usage is O(n^2) for n concurrent users and both your concurrent user count and the average data stored per user increases such that you need hundreds of terabytes of RAM on a single server in order to continue to function, maybe putting in a modicum of up-front work would have paid off.
I'm not in any way saying that you shouldn't plan for the future. We certainly have designed everything wherever possible not to get in the way of future scaling. It's just the problems you can't predict are what really get you.
For example, in theory the first "designed in" limitation that our architecture has should kick in around 1.5 million concurrents. There is a service in the backend that, due to the way it is designed, could never possibly handle more concurrents than that without redesigning some pretty core internals.
In reality though, when we get to 2 orders of magnitude less concurrents then that the realm starts to develop issues that we are currently resolving.
So yes, don't design things that can't scale, but don't expect your designs to actually scale without serious engineering effort.
Assuming massive success is stupid, but testing for it seems prudent. Given how easy it is to spin up EC2 instances generate bot-load, even just to see what'll happen or to find the collapse point, seems so straightforward as to be routine. And as the commenter below observes, the Curiosity interaction model requires high server load for all interaction.
The problem with your theory is that there are lies, damned lies, and load tests.
Any sensible app is going to have caching layers. A simple load test is going to just be a really good exercise of the caching layers. By contrast a complex load test with lots of calls to do really random stuff is going to stress you out in entirely unreasonable ways. And finding a realistic balance of traffic is difficult unless you've simply sampled real traffic - which you can't do until after you've generated some.
I cant agree with this more, unless you approach me with an experiment that would put most science to shame I do not value your use of Jmeter/Tsung/Siege/Rando-Python-Script. If you test plan does not have pauses in it, and you are not engineering a nuclear reactor I probably wont even start the conversation.
....
At $DAYJOB, when we were putting together a new search engine we spent some time upfront putting in a lot of tooling to worry about and test performance every step of the way. Some of this tooling was built around jmeter and focused on generating realistic test plans (which we could derive from the already operational existing search engine). It was a neat infrastructure it tracked everything by commit, captured all manner of metrics and used huge datasets, mined from the logs, built out with markov models. Everything we reported into an install of the software that runs http://speed.pypy.org, we thought we had it conquered.
Yet it was so flawed! Data would get stale, models would break, caches invalidated datasets and much much more.
It was funny at the same time I wrote a tool that we now know as darklaunch, this tool is stupid simple, it captures incoming traffic to the production system with pcap, and then (after a little mussing with it) sends it through to a always on test cluster. Yet its this tool that produces all the load, has all the weird user behavior, never gets out of sync and can run load, soak, endurance and impact tests.
Guess which one we use to test out new software, or new features, or to tune a garbage collector in as close to production as possible ?
I have not started Jmeter now in over a year, its been no great loss; sure we have microbenchmarks where they matter but for everything else you often hear the words "darklaunch it"
Now for sure we have the luxury of having real traffic to use and abuse, but if you don't have real traffic then realise you are using a scrying device anyhow and I will bet you good money your assumptions are wrong.
Test for the big numbers. With server virtualization you don't have to buy the hardware ahead of time so there's no reason not to do so. Even if it doesn't work at more than 50k currents, you may as well find out whre the failure points are ahead of time rather than when your getting squeezed up against them.
Because people make mistakes? Aren't people allowed to make mistakes anymore?
Most software developers love solving problems. When presented with a tough problem they don't mind skipping out on a few dinners at home or not watching the latest sports ball game. It is just being a developer, not some kind of malpractice, or slave driving, or not having read enough mythical books about herding cats.
Shit goes wrong sometimes, it is in some peoples nature to want to fix the problems.
10,000 concurrent users within a day of release is massive success in today's application market. Engineering a system to handle an order of magnitude more traffic than you expect is sensible; engineering it to handle three orders of magnitude more than you expect would, virtually always, be ridiculously wasteful.
Also, Curiosity is not your average horizontally scalable web app, because all the users are continuously interacting with the same data set. Scaling this up to 2 million concurrent users would be a challenge even with time on your hands.
Very off topic, but what are your thoughts on this;
I built something alone for both the client (iOS with Cocos2d) and server (Ruby, MongoDb, Redis) that managed 3k users on release for a full day. The app was only for a day (Event app for an ad agency)
This took three weeks to develop. The client had 3d graphics and a custom map coded and drawn from scratch.
Keeping the servers up under heavy load is part of the startup archetype. You have to cross the Rubicon.
> The servers melted. Systrom and Krieger worked a straight 24 hours to keep the app running—in that period 25,000 iPhone users downloaded the free service.
The point is that these kind of emergencies ('crunch time') always seem to happen in the games industry. It's _assumed_ that this will happen. Surely the game is not very great _unless_ there is a few months of crunch time!
Industry bullshit artist approaching has-been status... super high-concept game design without any actual game... his nephew as producer... five staff total working on a massively parallel realtime cross-platform title with novel mechanics...
To "just work harder", in this case 36 hours, is the easy decision to make when something goes wrong. It smells deliciously of heroics, bravado and top performer mentality. Most importantly, the soul drenching, exhausting late hours, gives the illusion that "we are doing all that is humanly possible to resolve the problem". How tough are we, right?
But as the author points out, heaps of literature clearly show that "crunching", like it's affectionately called in the games industry, is contra-productive. One guy working 8 hours, going home for a good sleep, and coming back to work an additional 8 hours will vastly outperform the developer-turned-zombie that's been at it for 24 hours straight.
At that point, short term memory is gone, stupid mistakes (blunders even) are rampant. Fixing simple issues takes extraordinary amounts of time, because the ability to focus is all but lost. You effectively have tunnel vision and your reactions and logical skills are reduced to that of a drunk.
This is not anecdotal - there are an immense amounts of studies done on the subject. But time and again, the knowledge gets discarded because it's just easier to "work through the night" at a frantic pace, rather than take the necessary step back, let people rest and do the work in an orderly and planned manner. By burning the candle in both ends, at least no one can challenge your commitment to solving the problem.
While it may sound counter-intuitive to go to bed while the house is on fire, it is, at least if 40 years of research is to be believed, actually the only correct thing to do in a software development crisis. Note this doesn't mean "stick 100% to 8 hours and always go home at 17.01", but it does mean if you are still there at 23.00 you are much better off going home, getting sleep and resuming work when well rested.
The games industry seem to be especially vulnerable to this problem. It might be due to the general inexperience level caused by games development being a young mans game, or it might be due to a failure to understand that games, while glorified in the industry, is "just" software, and therefore also subject to all the established rules of software development.
In any case, I agree with the author that Molyneux should know better than to tweet from the rooftops how many hours everybody is working, but I guess he is not thinking entirely straight after 36 hours of work.
It depends on whether or not a fix is perceived to be further away than what you can accomplish in that one night.
Being down for yet another business day when you don't have to be ('house on fire') is usually not an option.
There is also a class of problems (chasing subtle bugs) that you're much better equipped to tackle when you come in fresh in the morning or when you're actually not looking directly for them, the flip side is there are those that it takes forever to get the mental model set up that maps the circumstances of the bug with enough resolution that you have a chance of cracking them that it pays off to just sit there until it's done, even if that means staying up well past your bed time.
The rule seems to be 'go home if you can no longer positively contribute towards solving the problem', not just 'go home'.
That's a reasoning that is prevalent and very often catch project managers and developers alike: It is "not an option" going home, because of whatever crisis is occurring. Because if we aren't working through the night, we are not committed, and we are certainly not doing the most we can to resolve the crisis.
The outcome is almost always stereotypical: Early in the morning, the fire is doused, and all is good. Until suddenly three more fires spring up out of nowhere, previously undetected due to fatigue and tunnel vision on the one particular issue that got it all started.
Slow and careful consideration of the problem, the potential fixes and especially the side consequences of implementing those fixes, gets thrown out the window after 24 (or even 12) hours of coding.
I agree in theory that the best solution would be if you could somehow tell when you are no longer positively contributing, and only then going home. But it is exceedingly hard, if not impossible, for an individual to accurately determine when he is no longer positively contributing towards solving the problem (i.e. Creating more problems that is being fixed). The outcome is you have people valiantly soldiering on, in the false belief that they are actually contributing, but with actual, measurable setbacks being incurred due to fatigue.
Some things just can't wait. If you have never worked in an industry where there are no commitments to outsiders then that's all fine and good but production (banks, networking, airlines, critical infrastructure such as electricity plants, water pumping, hospitals and a very large number of other services that society relies on) problems take precedence over a lot of otherwise completely valid concerns.
If that is a recurring theme then there is definitely a deeper problem but on occasion things go wrong and 'going home' really isn't an option.
Count your blessings if you are not stress resistant and are working in a line of business where these things don't matter.
Production problems have a habit of not announcing their arrival dates, and some of those can be surprisingly hard to track down to their root cause.
Working until the problem is fixed in those situations has nothing to do with 'showing commitment' it has everything to do with being committed.
There is a right time for knowing when to call it quits and there is a right time for pushing on until the lights are back on.
You simply can't categorically state that it is wrong to push on, it depends on what is on the line and the capabilities of the people involved.
In the case you mentioned, with a clear perception of a problem being fixable within a nights work, I of course agree, that's the time to keep on working.
But I think we will probably disagree on the right time to call it quits.
In my book, that is when the situation is more diffuse, like the hard-to-find bug on a real-time enterprise system with many integrations, and your team is fatigued, having spent 20 hours already trying to fix it, with no clear outlook on when it will be done.
If you keep on at this point, you run the very real risk of turning a manageable bonfire in the kitchen into a wildfire that takes down the whole neighborhood. I.e. direct contra-productivity in the form of unintended side consequences and breakages introduced by bleary-eyed programmers. Such issues can often end up putting the system down for even longer than it otherwise would be.
In a standard, non-live development situation, the effect will be less immediate, like slipped schedules and high defect counts, sometimes turning into the classical "deathmarch" project where there is a constant feeling of one step forward, two steps back.
The risk and impact of course varies a lot from situation to situation, so just to be clear I am not categorically stating anything here: Calculated risks can be okay, and everything has exceptions, especially when it is a production / sysops crisis versus a development deadline crisis.
Actually what is established by literature is that crunch /does/ work, but that it has disproportionate payoffs and only works on teams which already work 40 hours or less per week and haven't been recently crunching...
e.g. 60 hours doesn't give you 1.5x productivity over 40, maybe 1.2x, and 80 hours is even worse return for investment...
... however it then impacts future productivity, i.e. you should take a couple of days at least after a crunch, and expect to still not be performing at peak for a couple more weeks of doing 40 hours.
Unless you're battle-hardened and able to endure multiple 36-hour runs in a single week, which few of us can withstand before something completely snaps, this is probably the worst possible idea.
You do a big push like this as a last resort, not as standard practice.
It reminds me of an anecdote about Icelanders and how they do their off-road driving: You keep your vehicle in two wheel drive. If you get stuck you can always use the four wheel drive to get back out. If you're always in four wheel drive, when you're stuck you're really stuck.
But seriously. Interesting point about Icelander off-road driving style. It makes sense for terrain that could stick you while in four wheel drive, but it's never occurred to me to drive like that. Ex: Chicago snow storm about 2 years ago, I just drove super-careful but many people had to abandon their cars on the side of the road.
heh, cute analogy, and I agree with the sentiment.
but having just returned from an Iceland 4x4 trip in places you absolutely need 12 inches of ground clearance and a snorkel, four wheel drive gets turned on the moment you start on F-roads. you really don't want to be slipping into a fjord on icy gravel roads because your front wheels don't have any power.
if you get to a technically hard part, you engage diff lock and your low range transfer box.
What if the employees were founders with equal shares of the company, and they launched a product and it fell over? Would you blame them for pulling an all-nighter to scale it up in the face of unexpected traffic? Should they give up at 10pm? Midnight? 5AM?
If they're not equal partners, should they immediately give up? What if they own 10%? 1%?
What if they don't have much on their resumes and are excited to make this a successful project, instead of a laughingstock that fell over in the face of scaling challenges? What if they care about each other and want to make this a success for their friends and coworkers?
What's the function into which you plug all these variables and decide when to go home?
It's not black-and-white, pun intended, and if you think it is you're being obtuse. I hope they feel happy and like they accomplished a lot in the face of long odds, and you should too.
I'd rather have a colleague come in to work drunk than pull an all-nighter to work on core functionality.
Yes, we really need to get this thing improved as soon as possible, but having effectively brain-damaged people introduce bugs all night and then be utterly useless the next day is not helping. Take your financial lumps and get over it instead of burning out everyone and ruining the codebase.
These kind of masochistic hero stories perpetuate the destructive belief that every hour your face is planted in front of the monitor is equally productive.
If someone I worked with tried pulling a 36 hour shift I'd physically throw them out of the building.
In my mind, odds are 90%+ that'd they'd do more damage (more bugs, slowing down other people) than they'd do good in aggregate over that 36 window, and odds that they'd be more productive in a single 36 hour bout than doing shifts with proper sleep windows is pretty much 0.
More than 14 hours or so in a single stretch is highly unlikely to be beneficial to the company/project, with rest in the middle.
More than just being in line with research, this is in line with what I've seen over and over again in real projects, where best project during crunch time is generally had by ensuring people leave and rest. Sure, do longer hours for a short period. Sure, ask people to cut down on time off or alternate work/sleep shifts.
But the moment your developers hits 18-20 hours, I could talk most of them into making horrifically bad decisions and have them think they were logically sound - your ability to reason goes out the window pretty quickly. I don't want people in that kind of state anywhere near projects I work on, because I've seen what they can, and will, do in the belief they're helping out.
> What if they don't have much on their resumes and are excited to make this a successful project, instead of a laughingstock that fell over in the face of scaling challenges? What if they care about each other and want to make this a success for their friends and coworkers?
Then learning to understand that there's nothing smart about pulling stunts like this unless you're doing menial work that you can still do reasonably reliably with next to no concentration or short term memory is even more important.
> I hope they feel happy and like they accomplished a lot in the face of long odds, and you should too.
No, I shouldn't, because if they feel that way they've learned the wrong lesson. They'd also most likely be wrong in terms of their relative productivity with more rest. Unless they were all on illegal stimulants, or his staff consists of mutants, it is simply not worth spending much time even considering the option that this might somehow have been the best way for them to spend their time.
I did a 36 hour push once. I did it because there was really no other way to achieve the deadline, since there was about three days worth of bug squashing to reach a deadline and I only had two days left.
I am proud that I managed to pull it off, however I tell it as a tale of caution not as a tale of inspiration.
The work I did was objectively atrocious, by the 24 hour mark the simplest operation took me 5-10 minutes. My short term memory was completely gone by then and I had to rely on writing down each step of an operation as simple as refactoring a block of code into its own function, just to be able to pull it of in a "reasonable time frame".
Thus I would call bullshit on anyone claiming that his team made "huge improvements" and that after such a push some piece of software "started behaving like it should".
Unless that means that they introduced a plethora of bugs that they would be busy fixing for months from now.
> I did it because there was really no other way to achieve the deadline
vs.
> by the 24 hour mark the simplest operation took me 5-10 minutes.
Did you see the incongruity when you wrote this? If it started taking you ages to do things, then there was another way: Take a nap. Speed up substantially.
Of course I have seen it. I also felt it. However If I took a "nap" I would probably just sleep for sixteen hours. After which I would have a hard time facing the absurdity of my situation. Also the massive flow and motivation would be completely gone. I would have probably just quit on the spot.
Also when you are this deep down the rabbit hole you are not thinking rationally and normal reasoning doesn't sound the same. The incongruity of the situation is the whole point.
After a certain level of fatigue settles in, you either quit at the right time (say before the 16 hour mark) or you keep going till the bitter end. Taking a nap would just make things worse.
The experience of going for an all-nighter is akin to watching a full season of 24 in one go, only you're the protagonist. It's memorable, and yes, especially if you manage to pull it off.
I tell it as a tale of inspiration: do it, once.
Whether it's good or bad for business/health/morale is a whole other story. Hint: it's not good.
That's what Global Game Jam is for, it's a 48h competition for writing games. I attended two of those, and during those 48h took maybe total of 8h naps here and there. Amazing experience. But yes, really, at some points your brain just stops working.
Completely agree that it'something required sometimes. But it's nothing heroic, it is a (temporary I hope) dysfunctionment, and shouldn't be put in midly positive light.
Planning failed (not assuming you might have 2m users is sensible, but it's still a failure in it's way), and shit hit the fans.
It's the kind of failure you might wish you had, but next time you should think hard about a way to deal with this kind of situation with minimal emegency, and no need for 'heroes'.
We glorify the noodle-eating couch-sleeping code-generating-machine hacker and let the corporations profit by exchanging our health and personal lives for stupid hacker-perks which costs the companies nothing.
Woohoo! come hack for us, we give you a fancy chair and large monitors to stare at and even food! what else do you want out of your life?
We need to see that we are being taken advantage of.
We have raised expectations so high, that you can't land a decent job unless you have dedicated your evenings to patching bugs in open source projects for free.
What do you think that says to an employer? Sorry to break it to you but it roughly goes like this,
"The poor bastard has no life. When he goes home, he does the same crap for fun. He'll be more than happy if I pay him 2 peanuts"
Have you asked yourself why other professionals don't have "open source" like we do?
If programmers are expected to work for fun to develop your email client, web browser, server softwares etc... it is only fair if lawyers were also expected to contribute to cases for free.
I'm too damn tired of this crap. We have put ourselves into an infinite loop of competition that only hurts ourselves and brings profit to the corporatoins. The guy next to you has to sacrifice his life because he needs to compete with you to stay relevant and you do for the same reason and so on...
We are smarter than this. We deserver better than this. We need to bring this to a stop.
In other professional fields (like with lawyers), doing work for free is called "pro bono". I would think that if you're being interviewed for lawyer position they would ask you how much pro bono work you've done.
So I disagree that doing this stuff for free is inherently bad, but I fully agree with the sentiments of the rest of your post. Particularly this: "come hack for us, we give you a fancy chair and large monitors to stare at and even food! what else do you want out of your life?"
An old manager of mine was once on the phone to a recruitment agency trying to hire some developers complaining that they hadn't made progress, at one point he screamed, "it's software development, it's not fucking rocket science!" He did that in a room full of developers.
For some reason many non-technical people get to become managers of technical people and have that kind of opinion. This then leads to the, "well if it doesn't work you can stay here all night until it does" mentality.
I'm not all that familiar with the field of law but I'm almost sure the "pro bono" work does not compare with the amount of effort that developers put into open source projects.
And even if law is an exception, just consider a lot of other professions, accountants, bankers, managers etc...
How much open-source-equivalent work do they do? How much do they have to do?
What have been the achievements of any of this work for other people (if any)?
Compare that to the things that open source community have achieved, entire billion dollar companies are relying on these technologies that started off as hobbies or what not and are maintained and developed further by the community.
Given the achievements, I think we deserve a lot better than what we are getting.
One difference software engineering has with aerospace engineering is that the people building the rocket get all the time they need. Since it's rocket science it's pretty much a given that if it's rushed or a PM or CEO puts the pressure on (with people working triple shifts for instance) then things just literally blow up.
In software the coders rarely have the option or will power to push back against unreasonable deadlines and fantasy launch dates. This is especially common in corporate cultures where a chain of command makes those calls and large scale plans are built around dates pulled out of the air. At best the coders are given lose-lose choices: miss the date or ship a broken product.
Sometimes I wish it was rocket science, since rocket science is at least a science. Corporate CRUD application usually have so complex and contradicting and vague requirements that sometimes software developer has to be very at mind-reading just to figure out what he has to do. And that works until the next change request when it all starts again.
I'm sure it's a lot harder than the manager to which I refer could do. In fact - it's a lot harder than many could manage, and whilst not literally as hard as rocket science, it's still a lot harder than some other fields. For example, "management".
The danger of incurring so much sleep debt so quickly is that you will start paying compound interest on it immediately - in terms of ability to avoid accidents, struggling with mental coherence, handling relations with loved ones, out-of-it attitude, and dealing with unexpected situations gracefully.
I like Mr Molyneux's imagination, but I have to say that I enjoy his deliverables significantly less. Whether intentionally hyped or not he has become, in my mind, quite the bullshitter. Seeing this I'm not surprised by the disconnect between his vision and products.
I'd advise anyone against purchasing something from his group without having a very generous return policy.
Molyneux has stopped delivering on his promises long, long ago. I think it was from Fable that I noticed that the most. He proposed an open world with all your choices having an impact on all characters and nature around, but it ended up simply a series of binary choices between good and evil. Hardly revolutionary.
Well I enjoyed Populous, Dungeon Keeper and Theme Park. After it all went down the drain.
No direct insight into it, but that is my understanding from reading various things over the years. That said, I seldom play games and don't keep up with much anymore just pick a random weekend once or twice a year to pick up something and binge a bit so I may be completely wrong.
I read here (see below link) they are soliciting user donations in order to speed up the bug fixing of the server code... so presumably to help pay the overtime of the coders. First time I've heard that a game user has been asked to pay to fix bugs in their game. I congratulate Molyneux on his chutzpah.
My advice to _anyone_ in our field is to read this book. Even if you yourself are not manager, and never have managed any team, it will at least make it easier to spot some common pitfalls and be aware of the consequences before it's too late.
I'm a big proponent of working (i.e. laboring) as little as you possibly can in order to leave as much room as possible for convivial activities and personal growth. But I think it is disingenuous in this case to paint Molyneux as a slave driver.
They were ill-prepared for the launch and stuck together to scoop the water out of the boat. It's probably a bad idea to do it for 36-hours straight, but it's understandable. I would certainly stick my teeth in there till the storm's passed. Especially in a company this small, where the line between collegiality and friendship are probably quite blurred.
I'm much more interested in what the RSI is a result of. That shit needs to be taken care of.
Does anyone know to what kind of books or reference is the author talking about when talking about the "15 years of free literature on how to run large-scale online projects"
I'm a gamer and this is very subjective of course but
Peter Molyneux hasn't actually done any good games. Overrated junk ~ and it's not his works, it is the engineers, designers, illustrators, sound engineers and actors who've done the game. He just makes enough noise to get attention and praise.
His early games were interesting, fun, and original, pretty much all the Bullfrog catalog. I'm also not sure you can get away with saying it's not his work when he's the lead designer and executive producer on those titles. Yes, he didn't do it alone, but those roles stamp a considerable amount of personality onto the product.
Anyone else find Curiosity intriguing not for the game (which is arguably about vanity), but for the technical brain-trust that operating such a service creates? Here's hoping they all write books...
Judging from the reviews for Curiosity in the Google Play store, it was not time well-spent. (I would provide my own comments on the game, but it insists on crashing before I can do anything with it.)
If it is a joke...Mr. Molyneux would have better spent the time on fixing his abysmally bug-ridden game.
In fact @pmolyneux BEGGED the team to go home but they wouldn't have any of it :) https://twitter.com/22Cans/status/268387590368030721
22 Cans is a tiny startup and they were expecting 10,000 concurrent users tops -- they're at 2 million and counting now, and they have to capitalize on that momentum and not piss off their obviously fanatical players. So they pulled an all-nighter to build and migrate from a single server to a clustered architecture, and hell yeah, if I was in that position I'd do that too.
Seriously, is there another option? If you think the answer is "clock out at 5:01 PM and go home", you shouldn't be in a startup. Obviously pulling 36 hour shifts on a regular basis is not sustainable, but this is precisely the type of situation where it's warranted.