Scraping seems rude, although I might try a gentle scrape if PG doesn't complain... | Hacker News

Hacker News new | past | comments | ask | show | jobs | submit

login

andreyf on April 1, 2008 | parent | context | favorite | on: Post your ideas to increase the quality of YC News

Scraping seems rude, although I might try a gentle scrape if PG doesn't complain... so far, afaik, he's been silent on the topic in re to news.YC...

Tichy on April 1, 2008 [–]

I don't think there is a robots.txt, and I remember comments that crawling HN is OK, at a reasonable rate.

Join us for AI Startup School this June 16-17 in San Francisco!
Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact