Hacker News new | past | comments | ask | show | jobs | submit login

Not only will google index all those pages you are blocking them from but will also penalise you when they have indexed x number of them.

Fix the problem with X-robots-tag, request removal from index (*while such option is available) and then wait for months for the penalty to be lifted.




Consider applying for YC's Spring batch! Applications are open till Feb 11.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: