Spoofing user-agents hasn't worked in a long time for anything but small operati...

Spivak on March 26, 2021 | parent | context | favorite | on: Only Google is really allowed to crawl the web

Spoofing user-agents hasn't worked in a long time for anything but small operations because search engines publish specific IP ranges their scrapers use.