It's such a pity that he doesn't dig into why this is. He makes this funny statement:
Internet Explorer 8 (I also tested 6, for good measure,
with the same results), Opera, Safari, and WebKit Nightly
all bin their results. There is no 'normal distribution'
whatsoever. Effectively these browsers are only updating
their internal getTime representations every 15
milliseconds.
He can't really think that all those browsers have implemented a binning algorithm. Underlying this is the GetTickCount (http://msdn.microsoft.com/en-us/library/ms724408.aspx) Windows API function: "The resolution of the GetTickCount function is limited to the resolution of the system timer, which is typically in the range of 10 milliseconds to 16 milliseconds." That goes all the way down into the bowels of the PC where there's an interrupt occurring that is used to update the tick count. Depending on the machine/operating system that you are using the system timer updates at varying frequencies (the original IBM PC had an 18.2Hz timer giving updates about every 54ms).
Related to this is how Windows' GetSystemTime is implemented and how often it updates.
He goes on to say:
It's pretty obvious that the Wine layer is tapping into some
higher-quality timer mechanism and is providing it to IE - giving
us a result that is even more accurate than what the browser normally
provides
tl;dr getTime in windows only updates every 15 ms, so if your performance benchmark cares about that kind of period then it isn't going to give very good results.
Every time I hear about performance testing in JS it seems to be that it doesn't really handle IE very well for whatever reason it is today. It looks like this bug would have made IE look like it performs better than it actually does though.
I thought it was pretty well-known that standard system timers (except the High Performance ones) could/would be inaccurate. I'm actually surprised by how reliable the OS X timers are.
Normally, if you do any kind of benchmarking, you would run the test N times and average the times out.
On Firefox, the console.time and console.profile functions provided by Firebug do this. On Chrome, you can use the chrome.interval object for the same purpose.
Related to this is how Windows' GetSystemTime is implemented and how often it updates.
He goes on to say:
Use the source, John! Wine is calling gettimeofday(). GetTickCount is defined here: http://source.winehq.org/source/dlls/kernel32/kernel_main.c#..., it calls GetTickCount64 which is here http://source.winehq.org/source/dlls/kernel32/kernel_main.c#... which calls NtQuerySystemTime which is here http://source.winehq.org/source/dlls/ntdll/time.c#L447 which calls gettimeofday.Wine's GetSystemTime implementation is going the same thing: http://source.winehq.org/source/dlls/kernel32/time.c#L933
In Chrome this is quite well documented, see comments here for Windows: http://www.google.com/codesearch/p?hl=en#W9JxUuHYyMg/trunk/s...