I’m guessing they know the terms but ‘consistency’ and ‘accuracy’ are just more common easier to understand terms without sacrificing meaning. I’ve talked to some TripleByte engineers and been very impressed by the sophistication of the statistics and experimental methodology they use.
Using an experimental methodology isn't something you should be impressed by in itself. Experimenting means you don't know what you should be doing. That's great if no one knows what you should be doing and you're trying to figure it out; it's less great if everyone else knows what you should be doing, but you don't.
One specific conversation I had was how they read the literature on the best adaptive testing systems and then developed improvements tailored to their specific data and the advantages they had as a real time online test.
Another was on specifically the psychometric literature, and the big meta-analyses of the predictiveness of different testing factors on job performance, and how that influenced the experiments they did early on and how they honed in on what they do today. As well as downsides they discovered of various methods people commonly suggest they're ignoring.
I came away from those conversations extremely impressed with TripleByte's employees and competence as an organization. They definitely think about this stuff.
