Hacker News new | past | comments | ask | show | jobs | submit login

Facebook is one, Slack (mentioned by someone else) is another, Yahoo Mail, Microsoft...

Does anyone know of a listing of these bots that attempt to preview links? From what I've seen, these bots tend to ignore robots.txt since they are not crawlers, so seemingly need to be handled one by one.




Consider applying for YC's Spring batch! Applications are open till Feb 11.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: