I could be wrong, but I believe Shodan actually portscans the entire internet, whereas Google only crawls known URLs. They also index HTTP headers, which Google doesn't do.
No! I do NOT try to authenticate with username/ password! The only exception to that is for FTP, where I try to do an anonymous/anonymous connection (identical to what Firefox etc. do). I put a lot of effort into making the crawling as benign and unobtrusive as possible, so I definitely do NOT try to brute force devices.
Is that legal? I've seen all kinds of analogies like "if your neighbor leaves the front door unlocked..." or "but if you go down the street testing each lock..." but never anyone who really knew what actual criminal law says.
It is a grey area, at least in the US. The main federal law for computer crimes is the ancient Computer Fraud and Abuse Act. The provisions of the act state all work off the concept of "exceeding authorized access" - but the law never defines what authorized access actually is. Logging in with a default username and password has never been tested in court, as far as I know, and I think there are arguments to be made for both sides about whether that counts as authorized access.
It's run by our very own achillean: https://news.ycombinator.com/threads?id=achillean