Yes, sorry, an image is a bad example. The main issue is with HTML documents. You might open one at the top level, by clicking on it and navigating to it, or you might open one as a child of the current page, by putting it in an iframe. Since they can be opened in both contexts, prefetch doesn't know what to do.
But is it right that the issue is fetching resources from a different domain than the current one? As a user, just because I've connected to domain A, it doesn't mean I necessarily want my computer to connect to any domain B that A links to. Also, I'd rather developers focus on making small pages that are easier to fetch on demand, and am worried that they'll use prefetch to justify bloated pages. If a page is large enough to need prefetch, then I might not want to spend the data pre-fetching it especially if the click probability is not very high. Between all of these, I'm not convinced of the need for cross-domain prefetch.
Apologies that I'm not a front-end person so this may be naive, but it would be great to hear your thoughts!
Yes, this is only an issue for cross domain prefetch.
With HTML resources, the goal of prefetch is typically not to get a head start on loading enormous amounts of data, but instead to knock a link off of the critical path. The HTML typically references many different resources (JS, CSS, images, etc) and, if the HTML was successfully prefetched, when the browser starts trying to load the page for real it then can kick off the requests for those resources immediately.
