On: Teaching GPT-3 to Identify Nonsense

[derefr]

I wonder if it would identify a question that breaks an implicit rule of English as nonsense. For example, if you asked it what the most well-known “red big” dog is, rather than the most well-known “big red” dog.

[Don_Patrick → derefr]

I expect so. As a next-word prediction algorithm, it knows the probability of "red big" occurring in that order is very low.