I didn't say it has the same competencies and biases as any human - I said we can judge it on it's individual merits, the same way we judge any human on theirs.
No, you can't judge it the same way you judge a human, any more than you could judge a bus engine the same way you'd judge human. It's not an individual, and not remotely like one.
I get this might seem like tedious nitpicking to you, but the number one error people are making with LLM output is anthropomorphizing it. Which I get, because it's built to seem that way. But it's an enormously dangerous misconception.
As one example among zillions, look at the term "hallucination". All LLM output is equally "hallucinated". Some of it, when interpreted by a human may be taken as meaningful. Some of it, the "hallucinated" part, is taken as meaningful but contrary to something else they understand. But it's the human creating all the meaning here. Even calling this class of mismatches "hallucination" is anthropomorphizing LLMs.
Imagine I take the proverbial million monkeys to generate random words. Then I create a statistical filter so that we extract only the plausible sentences. Is this machine a source? Can I "judge just like any human source" here?
I'd say the answer is a clear no. And if you think the answer is yes, then the same has to apply to things like horoscopes, the I Ching, or the intestines of a sacrificial goat.
I can judge the utility of a black box oracle on it's own merits, without needing to know whether it's human, and without being accused of anthromorphizing it.
If I've observed the answers of the oracle to be biased, or useful/correct in some circumstances and not in others, then this is something to take into account when deciding whether or not this is a useful source to pay attention to.
From the POV of whether the output of the black box is useful, it doesn't make any difference whether what's in the box is a human, an LLM, or a rabid monkey. It is what it is, and I'll judge it on it's merits.
Now, YOU may care what's in the box, for some reason, but that's on you.
Well, it's still a known source, who's competence, biases, etc one can judge just like any human source.