ArenaNet did this for GuildWars when they encountered particularly inexplicable bugs:
[Mike O’Brien] wrote a module (“OsStress”) which would allocate a block of memory, perform calculations in that memory block, and then compare the results of the calculation to a table of known answers. He encoded this stress-test into the main game loop so that the computer would perform this verification step about 30-50 times per second.
On a properly functioning computer this stress test should never fail, but surprisingly we discovered that on about 1% of the computers being used to play Guild Wars it did fail! One percent might not sound like a big deal, but when one million gamers play the game on any given day that means 10,000 would have at least one crash bug. Our programming team could spend weeks researching the bugs for just one day at that rate!
Thanks for sharing that -- on its own it was a great read.
At first, I was not surprised that they saw 1% of GuildWars players had these issues. My reasoning was given the time of the article (2012) and pointing to a more distant hazy point somewhere between StarCraft and Diablo[0] their user-base probably included a lot of "custom builds", a not-small portion of which would be "built intended for overclocking", in an era when the firmware/motherboard/processors would let you apply settings that would become extremely unstable under load[1]
... and I got about that far before I thought, only 1% of users managed to make their "let's see what happens if we alter the voltages/frequencies such that we run the DDR2-333 memory at DDR2-2048 (it's been a while), change the various "latency" settings throwing caution to the wind. Meanwhile all of this hardware is sitting in a case with a fan configuration that slightly lowers the air pressure in the case[2] ...
Back in the day (less common, today) you could often put any value from a range of "Way Too Low" to "Way Too High" and steps in-between. If it could reach the boot sector,... "Weeeeeeeeee!!" I recall a case of a customer who completely trashed an Intel Pentium (2? 3?) chip via firmware settings "my. kid. was. F---in'. with![3]"
Maybe I'm wrong and it's just a matter that the game involved attracts people who rarely OC/build and the 1% is "just those people", but anecdotally "in my life", everyone I knew back then who was a PC gamer either started with a build of their own or went that route the instant they had a job which allowed them buy their own computer (rather than sharing Dad's or getting his hand-me-downs if you were really lucky/have fewer years on you than I do).
My experience, especially back in the 08-09 era, was that things were ridiculously forgiving. Depending on the use, overclocking would either "do nothing at all" or maybe let you get away with slightly better graphics settings while maintaining a playable game. Generally, I purchased the cheapest memory rated slightly better than what I intended to configure them for (DDR2-xxx slightly higher than where I will clock it) with whatever I decided were the right latency values[4], placebo-overclocked day-to-day, but would push it from time to time. There's been times the system simply doesn't boot but one "tiny notch south" and it'll pass a 72-hour memory stress-test.
I've had crashes, locally, that I can't explain beyond "memory corruption" but I can't say if it's an OS failure, hardware failure or "something that's not memory corruption but malfunctioning other hardware" ... it passes a stress test, I usually find the "other hardware" a little while later (surprisingly often the PSU, for me).
[0] Unclear if editions implied are "I" in both cases, and it's not important enough to matter since reasonably earlier than 2012 puts you in the category of "the firmware/motherboards will let you do incredibly stupid things".
[1] Sometimes subtle "some character's armor texture is transparent" (making your enemies see-through, not naked, much to my teenage chagrin). Sometimes crazy like "Everything is shades of green with partial geometries".
[2] Say, if you had to have the most sleek case where every available air-flow "vent" can take a fan facing inward or outward (where it can sometimes be a bad idea to not leave one empty) has all of the fans blowing out... I've seen "The Vacuum Computer".
[3] So, this really happened to me (CompUSA, filling in at the Parts Department), and that's exactly the way I hear it, but you need to understand that it wasn't some form of "Jersey Shore" accent -- this guy was absolutely Korean with an accent so thickand not a day younger than...60? Aside from the shock of a person very much struggling with the English language dragging out the "F" word, you just don't see a plain-looking-harmless-old-man speak with such vigor and gusto!
[4] This being "unimportant" information for me day-to-day, requires hours of "refreshing via Google" and learning every miserable thing that's changed, coming to a complete understanding, finding that the memory I want either doesn't exist, doesn't work with my mother board, or costs a fortune and settling somewhere pretty close to what is "typical" if not ... exactly typical.
were they testing system RAM exclusively? If it's VRAM, I'd agree with this. Back in the day we'd even overclock VRAM until we got "snow" in the rendered graphics then backoff a bit.
[Mike O’Brien] wrote a module (“OsStress”) which would allocate a block of memory, perform calculations in that memory block, and then compare the results of the calculation to a table of known answers. He encoded this stress-test into the main game loop so that the computer would perform this verification step about 30-50 times per second.
On a properly functioning computer this stress test should never fail, but surprisingly we discovered that on about 1% of the computers being used to play Guild Wars it did fail! One percent might not sound like a big deal, but when one million gamers play the game on any given day that means 10,000 would have at least one crash bug. Our programming team could spend weeks researching the bugs for just one day at that rate!
https://www.codeofhonor.com/blog/whose-bug-is-this-anyway