clicking through is what killed it for me too. I just never went through the trouble of setting up crawlers and stuff. Looks like open source is to the recue nowadays. I am pretty sure there was nothing as easy as that back in the day.
Thanks for sharing this!
