I have not had any such problems living here for 7 years… elevators always have signs all over the station telling you exactly where to go to get to them, with standardized coloring and symbols, and text saying what it’s about in Japanese and English. They tell you even while you’re on the train which car the elevator will be in front of.
I would chalk up your experience to being generally overwhelmed and not used to it, mixed in with being illiterate in the local language (despite there being English and symbols to assist further).
I do know how to read. I was thinking of the case of finding it from the outside, especially at a large station like Shibuya where they're always closing random entrances. Jorudan/Google/Apple Maps also don't incorporate them into search.
It's fine once you've learned where they are of course.
Most stations play sounds to indicate stairs and ticket gates. Typically a slow ding...dong sound, and birds chirping. Though I'm not sure if they indicate elevators with those sounds, I need to check next time I'm in a station.
I would chalk up your experience to being generally overwhelmed and not used to it, mixed in with being illiterate in the local language (despite there being English and symbols to assist further).