The actual data source will provide a country, state or sometimes even city and zipcode. Then whatever tool you're using to map drops a pin in the middle of that region. If you zoom in, you get whatever happens to be at the geographic center of whatever the mapping tool (probably Google Maps) thinks is the center. eg if it says "United States" and no other data, you get some random ass place in the middle of Kansas. Sometimes there can be more specific data, but just because you can keep zooming in doesn't mean that that's actually where it is
You're describing GeoIP derived from "public" information sources such as the physical address of the assigned entity or the location information provided to the registrar by the block owner.
However, there is a different kind of GeoIP that has the potential to be much more specific as to the location, based on a join between Internet traffic and transactions that target a specific location. e.g. when you purchase a physical item from an online vendor, with your house as the delivery address, they now have both your IP and location. Obviously for this to work it depends on a) the IP address remaining the same for some period of time and b) sharing of the necessary information to allow the join. afaik both are often true.