Hacker News new | past | comments | ask | show | jobs | submit login
Mapping CraigsList Housing Ads (trailbehind.com)
15 points by andrewljohnson on Feb 27, 2009 | hide | past | favorite | 21 comments



I hate to be a wet blanket, but didn't http://www.housingmaps.com/ do that like years ago?


Read my blog post above. I acknowledge them and talk about the improvements we made.

For example, HousingMaps.com maps a couple dozen cities, and we map all 326 US cities.


I thought I read somewhere that housingmaps had a special agreement with craigslist... But I hope you stick around because the lack of cities on hm is really annoying.

Craigslist really needs an API... and global search. PLEASE.


First, cool. Next, can you share your crawler code?

Lastly, this violates craiglist's usage policy right? Did you research the implications and what did you find?


This does violate craigslist's usage policy.

craigslist permits you to display on your website, or create a hyperlink on your website to, individual postings on the Service so long as such use is for noncommercial and/or news reporting purposes only (e.g., for use in personal web blogs or personal online media). If the total number of such postings displayed or linked to on your website exceeds one hundred (100) postings, your use will be presumed to be in violation of the TOU, absent express permission granted by craigslist to do so.


First, I don't make a habit of reading ToS, so I don't know if I'm in violation or not :)

Second, I posted the code here. It's kind of a mess... http://www.trailbehind.com/trips/view_report/116


For those interested in the project, I also wrote up a short blog post about it at: http://www.trailbehind.com/trips/view_report/115

Any feedback or bug reports you have are much appreciated.


Good job, I'll be looking for a place soon! I have a few question/comments:

- Open links in a new window. - Have a date picker or a dropdown with 1 day ago, 2 days ago, etc. - Where are the nearby photos near? - Show photos button, why is it different than the checkbox for dogs/cats? Its kind of confusing. - Change it so when you click from list of adds, the map moves to the ad's location on the map. - When you click on an ad on the map you place a pin on the map. What can you do with this? I'd rather see pics/summary information and then I would decide whether I want to save this ad.


All good suggestions

1. I implemented open in new window 5 minutes ago

2. I'll add my calendar widget to replace the data selector

3. I'll overlay the polygon of the photo search area.

4. Good idea for images for cats/dogs.

5. I had it set up to pan to the point originally, but now I just open a new window, so I'm not sure that's useful anymore.


This is good stuff.I like how the list of ads changes as I pan around the map. What else does Trailbehind do?


Suggestion: Open ads up in a new window, when I click back after viewing an ad I have to re-select city, etc.


Thanks for the suggestion! I implemented it :)


It seems like your site is not showing the majority of the adds. If I look at my neighborhood (El Cerrito, CA) there are hundreds of listings in the last week on craigslist but setting the cutoff date to 2/20 on your site only displays 3 or 4 results in the same area.


Sorry, in the future we'll have ads going back further, but we didn't start crawling CraigsList until the 20th.

Each day the bot runs is another day we have stored.


I think you're missing what I'm saying. If you look at San Francisco/East Bay/Albany:El Cerrito on craigslist there are 30 listings for 2/27. If I look in that area on your map I see only 2 listings.


Same here for my neighborhood. For me, this warrants not using the product at all... why use something that has an incomplete dataset? I could be missing out on that perfect listing!

Another thing is that some postings don't contain sufficient information for a machine to know its location... such as "near the Xyz subway station" or other atypical types of addresses that the project's parser might miss.

It's a cool mash-up, but I'll pass.


Ok, I see your point.

For some ads, there is no geo-data. To fix that for the next release, we'll list ALL ads, not just the mappable ones, in the Ad Widget. And we'll indicate in the table if the ad is on the map.

One good side effect though is the ads on the map tend to be less spammy than the non-mapped ads. I assume this is because the spam ads often embed their spam in HTML and don't use CraigsList's structure well. A good landlord is going to make sure their ad links to a Google Map on CraigsList, which also means we will index them.


Not allowing me to enter my location, and instead having me zoom/wait-to-load several times makes me wish that closing the tab somehow inflicted pain on the software.


how come half the kansas and half of nevada is missing?


Nobody lives there? There probably isnt a craigslist to cover those areas.


Are you sure about that? I think we have all the cities that CraigsList maps in the US.

Does Nevada have more than Reno and Las Vegas? We have both of those, in opposite corners of the state.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: