iPods became big around 2003. Back then, 128 MB of flash would cost you $40[1]. This was why HDD-backed mp3 players were so revolutionary for the time--flash wasn't viable, yet. By the time flash got cheap enough, portable mp3 players were so established that the time for physical media for audio had mostly passed.
Yep, the reason why the OG iPod was shaped/sized the way it was, is due to the form factor of commodity laptop hard drives. I forget, but I think the OG iPod had a firewire connector (sort of the equivalent of today's Thunderbolt 3/4), which allowed the owner to use the iPod as a portable hard drive. Most macs had at least one firewire port.
1: https://web.archive.org/web/20030520151144/http://www.newegg...