And what would be examples of benchmarking a subjective experience? I mean, a shitty JavaScript engineer can make any UX shitty regardless of speed of JavaScript tests, whereas a good JavaScript engineer can make a UX good with a crap interpreter.
Right, so it's entirely appropriate to benchmark arbitrary computational metrics, but benchmarking UI metrics is inconceivable?
I don't understand your argument - true shitty engineers can ruin a UI, as well as they can ruin a JS interpreter. The point of testing and benchmarking is to evaluate the relative "performance" in each of these domains.
Measuring page loads on an array of popular, representative websites, or measuring the latency between user interactions and some visible response are some examples.
Also, having a group of real people give subjective ratings I think carries more weight than people give it credit for.
it's not that hard to measure latency. point a video camera at the screen and measure how long it takes from click to completion of new screen load.
then pick ten web pages and collect the data.
then pick ten other common UI actions (not in the web browser) and collect that data too. there's some fudge here because the phones have different UIs so you won't necessarily be able to find identical actions but you could get a good idea.
it's entirely possible that the dev teams have instrumented builds that give them this info automatically. i would be surprised if they didn't, in fact.
