Controlling for academic experience probably raises the average accuracy of labelling, but by how much? Clearly having a degree will not make you omniscient in your major, let alone other subjects.
Most jobs I've known factor in both. I would assume they have processes in place that incentivize quality. Some it is as simple as you have a manager that will fire you if you produce crap.
With billions in funding, and bad results causing bad press etc, you think that OpenAI would not have given this a bit of consideration?
> I would assume they have processes in place that incentivize quality.
> you think that OpenAI would not have given this a bit of consideration?
Those are just assumptions though. The issue is not “this was labelled as a shoe, but it’s a car”, the issue is about depth vs superficially, which is harder to verify. See also https://www.theverge.com/features/23764584/ai-artificial-int... for a well-sourced article on the subject.
Source: https://huyenchip.com/2023/05/02/rlhf.html#demonstration_dat...