This is a bad article. It tries to draw some kind of causality link between the new maps UI and a couple of bad search results that the author received. (And then tries to blame it on some completely random academic paper written by Googlers, which as far as I can tell doesn't even have a hint of being from a geo-context, and doesn't look particularly applicable to geocoding or local searches). Of course blaming the new UI is a real crowd-pleaser, as can be seen in the HN comments, which must be good for getting some page hits on a Saturday afternoon.
The reality is that Google Maps has always had a (frequently changing) set of searches producing bad results. Some of those failures would have been much more visible, widely publicized, and serious than a blogger not finding the closest burger joint. This article presents the past as searches always just working, which just wasn't true. As such there's no compelling evidence there of the search result quality having actually degraded over time.
I tried the new maps when they first released it. Clearly the search was different in some ways because simple directions searches simple wouldn't work. Since the, obviously, they have ironed out those bugs. But in the intervening years, every time I've tried the "new" maps I've run screaming back to the "old" maps because of the terrible UX. Are the search results "worse"? Maybe not. But the same results presented in a nearly-unusable manner probably "feels" like worse results.
No, of course not. What kind of a comment is that anyway? You appear to be doing exactly the same kind of conflating of the UI and the backend that I was complaining about. There's a bunch of people whose job is to improve the search quality. They're largely not going to be the same people who'll work on the UI, so it's not some kind of either-or choice.
But it's a very difficult problem where it's rare that a change is uniformly beneficial, that's insanely dependent on frequently changing input data, and that users are very unforgiving about. My theory on the last point is that a map search is much more concrete than a web search. People have an intuition both about what the right answer should be, as well as an expectation that there is a right answer in the first place. But of course that's not the case, and the results are never going to be optimal. (I.e. your "sub optimal searching" bit is a bit of a truism).
I used to work on the Google maps geocoding team a long time ago. At that time amazing amounts of CPU and engineer time would be spent on verifying the quality of all algorithm and data changes, both during development and during launch. Changes that were unevaluated or were a net negative on quality would only be launched under very exceptional circumstances. Now, the evaluation would of course not be a fully deterministic process, it'd always need to be based on some kind of sampling. But on average that should still mean quality ratcheting up slowly. Maybe things have changed since then and stuff is just randomly launched with no regard to quality, but I have no particular reason to believe so. To me it seems much more likely that the anecdotes from the article don't represent any kind of trend.
>At that time amazing amounts of CPU and engineer time would be spent on verifying the quality of all algorithm and data changes, both during development and during launch. Changes that were unevaluated or were a net negative on quality would only be launched under very exceptional circumstances.
Maybe they should actually try using the product instead of relying on automated algorithms to generate metrics when they evaluate changes. For example, it's quite obvious that something is going wrong with the prioritisation of name place text for the UK at the moment: http://imgur.com/kL0GHfe (for the non-UK readers, no there is not a large important city called "Town Centre", Edinburgh is far larger than Kirkcaldy, Birmingham is the second largest city in the country and not labeled at all).
I'm quite curious about how much real human testing they actually do. I've always had the impression that testing by actual humans is the antithesis of Google culture (automate everything and reduce everything to comparable numerical metrics).
It's a good thing that I didn't say anything about "automated algorithms used to generate metrics", then...
This was machine-aided human evaluation.
Search quality evaluation can't really be done without humans in the loop. If you had an algorithm that could distinguish between a good result and a bad result, you wouldn't use it to evaluate results. You'd use it to generate the results.
The search function was far better prior to the new Google Maps, IMNSHO.
Also, I have an IP based out of Dallas, TX, and for some reason, Google Maps insists that I live in San Francisco. I have even set my default location (in Chrome, no less) and I still get San Francisco by default, forcing me to put a location in.
What the author said about locations is true. I used to be able to search for something unique to my area even if the map wasn't centered over my home town and it would find it, but now, it fails because I'm in San Francisco.
The reality is that Google Maps has always had a (frequently changing) set of searches producing bad results. Some of those failures would have been much more visible, widely publicized, and serious than a blogger not finding the closest burger joint. This article presents the past as searches always just working, which just wasn't true. As such there's no compelling evidence there of the search result quality having actually degraded over time.