> This model was trained by asking people to rate internet comments on a scale from "Very toxic" to "Very healthy" contribution. Toxic is defined as... "a rude, disrespectful, or unreasonable comment that is likely to make you leave a discussion."
> asking people
Gotta wonder: which people?
The examples are good though, I just hope the general results are consistent with that quality level.
That would be the concern. My impression from poking at the API is that it doesn't seem to have any topical biases. The accuracy is nonetheless hard to judge in the 1-40% range.
For example, the API rates this comment as 21% likely to be perceived as "toxic". The use of quotes around the word "toxic" increases the likelihood.
> asking people
Gotta wonder: which people?
The examples are good though, I just hope the general results are consistent with that quality level.