Hacker News new | past | comments | ask | show | jobs | submit login

Let's go with $15/hr instead, since that's probably what the testers make. That's about 2666 hours of testing.

Now, how many different XBox 360 models are there? Count each one twice, once for the hard drive and once for a USB Stick that people will use to store the game and saves. http://beta.ivc.no/wiki/index.php/Xbox_360_Revisions 8 x 2 = 16.

Now we're down to 166 hours of testing with each revision of the 360.

Let's put 4 people on each one and let them test it for only a week.

And just like that, all that money is used up.

I'm sure some of that money goes into bandwidth, administration, etc etc. So they don't even get 4 people to test it for a week for that money.




Is this close to what's actually happening here? I ask because this contrasts significantly from what happens with Windows Phone app updates, even though you have similar testing issues (different hardware). What's so special about the Xbox?


It's the Cathedral and the Bazaar again. The reason-for-being of the consoles is to establish their brands as delivering a /reliably/ high-quality experience for the users. Users don't worry about variations in hardware, drivers, install requirements, background processes, etc... They pop the game in and it works. Even though it's a huge, complicated, AAA game, it's not going to crash. Can your PC run Crysis? There are a huge number of cross-cutting issues that can screw that up. Can your Xbox run Crysis? Yes. Yes it can.

When you submit a game for approval as an Xbox retail game, it goes through a long, expensive testing process. Any failure means you aren't shipping. Changing one bit on the disk restarts the testing process. So, you are highly motivated to test for yourself very, very thoroughly before submitting.

Contrast this with a very open platform like Android where anyone can do anything with minimal, if any, quality assurance. It's wonderfully liberating. But, you have to accept the fact that with so many changes from so many people being mixed together with minimal, if any, comprehensive test coverage, a lot of stuff is going to be busted a lot of the time.

Microsoft, Sony and Nintendo spend a huge amount of time, money and effort on quality assurance for their consoles because if they didn't, you might as well play games on the PC. They are doing all of this work to establish confidence in their audience that the $60 they spend isn't going to be buying a game that crashes on load on 5% of devices.

Meanwhile on mobile, users demand free apps because they never know what they are going to get. Investing non-zero money to install a mobile app is too much risk for a user. Lots of apps crash frequently for no obvious reason. Instead of up-front assurance, the mobile space relies on freemium, reviews, refunds and other after-the-fact techniques to eventually, statistically, push most problematic apps into areas of less visibility.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: