1. You can (and must) host AMP content on servers of your own choosing.
2. Google's crawlers ingest your AMP content, validate it, and re-host it in Google's AMP Cache. Bing and other folks run similar crawlers and caches.
3. Google search results will only link to AMP content on Google's own cache.
So yes, you can host your content on your own servers. And you can give people direct URLs to that content. But Google users will only ever see links to the copy of your content that Google hosts; your servers never see that traffic.
There is no way to opt-out and still be considered a valid AMP page.