I'm honestly not sure if I would have had similar issues verbally. I think they maybe sound very slightly different when most people pronounce them, but there's a good chance that's just my brain adjusting based on context. But I don't think reading is the same, and I definitely think some misspellings, even of homophones, can throw me off and cause me to re-read.
In any case, I went out of my way to not be snippy! I just thought it might be helpful to you to know that the mis-use slightly tripped up at least one person. It's hard to know who will find it helpful to have things like that pointed out (like some non-native speakers trying to learn) and who will just get annoyed. Mis-judged this one!