I agree that navigating the ambiguity between different categories requires thinking meta. Where I disagree is the need to drop semantics. Semantics are the foundation of how we organize our society. Perhaps on a small-scale or a personal level, dropping categories can be helpful, even freeing. For that to be done universally would be a disaster. I can see that being possible in the future, after large-scale social change, but not currently. The bulk of modern political discourse is effectively arguments over definitions of various categories and the membership of their constituencies.
Even something as minor as the category of a tomato, would matter very much to a tomato farmer when, hypothetically, all vegetable farms qualify for a higher level of government subsidies than fruit farms. Or to a retailer when, again hypothetically, fruits and vegetables are charged different sales tax rates.
Your example only reinforces GP's point. Arguing semantics and confusing map for territory only wastes everyone's time and effort. It's a trap, and we won't progress as society to handle tough political problems until we learn to stop doing it.
What the farmer in your example cares about is the legal classification of tomatoes in context of subsidies and taxation. That's an entirely different thing that biological, or culinary classification, and needs to be explicitly separated. If it isn't it leads to idiocy.
Real life examples of such inane reasoning can be seen with some very popular complaints about "stupid EU bureaucracy" - for example, that EU "decided that snails are fish" (they aren't, they just get bucketed with fish for some trade deal). People really do use it as a serious argument.
You're assuming that we can distinguish between map and territory and science is definitively proving we cannot. See Dr. Robert Sapolsky on behavior as an example.[1] And philosophers like Kant speculate it's map all the way down, so to speak. That perspective is very compatible with everything we know about the world and ourselves. You'll need extraordinary evidence to prove you have a handle on what territory is. Because at this juncture pointing to anything as territory is an extraordinary claim.
And sure, instead of legal classifications, one can discuss ethics related examples. Such as what makes for the good life? Something as simple as saying, the good life is based on some common sense notion of good or the good life is different for everyone and dependent on unique personal circumstances, already assigns numerous categories implicitly. That doesn't mean that both of those notions of the good life cannot be right. It does mean that we need categories to discuss and reconcile their differences, if we wish to have a pluralistic and liberal [in the 19th century sense of the word] society.
Even something as minor as the category of a tomato, would matter very much to a tomato farmer when, hypothetically, all vegetable farms qualify for a higher level of government subsidies than fruit farms. Or to a retailer when, again hypothetically, fruits and vegetables are charged different sales tax rates.