Hacker News new | past | comments | ask | show | jobs | submit login

Very interesting, how tools like these can be so much helpful for journalists and generally transparency in government functions.

Probably world changing, when considering that even semi-technical folks can cook up tools to dig into things like this.

I know this tool was by a developer, but scrapinghub has web UI to make scrapers.




Full disclosure, I work for Scrapinghub and the web UI you speak of is Portia - our open source visual web scraper. It's for those who range from non-technical to technical but want a quick way to scrape data. I think it's extremely important to develop tools to democratize the acquisition of data regardless of technical background and skill. Glad you find the article and tool interesting!


Yes, totally agree with you on the great potential of tools for easy data acquisition.

I have personally used Scrapy in the past, I find it to be a great tool.

Congratulations on your work!


Thank you. Glad you enjoy Scrapy, we're pretty fond of it ourselves!


A similar thing happened in Costa Rica -

    “You can’t visit 160,000 people,” she notes. “But 
    you can easily interrogate 160,000 records.”
http://foreignpolicy.com/2015/05/27/the-data-sleuths-of-san-...




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: