This isn't as difficult as you think. When you know an area well, you know what species of fish are likely to be around - down to specific habitats like rocky vs sandy coastline, and even particular arrangements of rocks or sand formation. After a while you notice their behaviours and specifically behaviours distinct from other species (e.g. bottom feeding, surface hunting, swimming style etc). So it's more of a combination of inputs that gives you an educated guess as to the fish you can't see, based on what you know about the area as well as fish behaviour. Anecdotally, I have an avid fisher of an uncle (more than one actually...) who is able to tell what species of fish he's hooked before he can even see it.
