I hear people say this in various forms. No vote is still a vote kind of concept.
If anything it was the 1.6% of votes that went to other candidates that could be seen as a spoiler. You'd have to say that all of that 1.6% would have gone to Harris for it to have been a spoiler though, and I doubt that's the case.