The usual deflection is that because spam and child porn still get banned, 4chan actually is moderated. Which is like saying that the FCC censors radio because they don't allow radio jammers.
There's a lot more moderation on 4chan beyond that. Most boards trim excessively off-topic discussion, trolling, or flamebaiting. Half the boards also don't allow NSFW images. These rules are regularly, even if not strictly, enforced.
The low-rules boards are often seen as cesspools by other users.
Your larger point is that 4chan isn't de facto moderated, but it is. Just because you agree with the level of moderation doesn't mean it's not moderated.
The even broader point is that nobody wants to participate in a truly unmoderated discussion, which may not be technically true, but if you allow "nobody" to mean "nearly nobody" (the author's intent), it's accurate.
Besides, one of the key aspects of 4chan is anonymity which could itself be considered a level of moderation; the default and indeed strong preference of commenters is to not be identifiable, which is a form of (self) moderation.
You can try to weasel around with what "moderation" means here, but the effect is largely the same no matter how you slice it; some stuff on 4chan isn't posted by tradition, not by technical capability.