Our now-dead app got absolutely slaughtered in the ratings (IMHO rightly) simply over our use of Facebook auth. Took forever for our ratings to rebound to this[0]
Makes sense, I would probably rate an app 1-star also if I had to use Facebook auth to use it - as I don't even have a Facebook account. I've noticed apps that require a Google login seem to do fine on Android though, probably because you need to have a Google account to use Android (or close to).
Did you re-launch the app under a new name to get new ratings after removing Facebook auth? Seems like the sensible thing to do...
I put up a small visual traceroute app up on Google Play recently - https://play.google.com/store/apps/details?id=com.rc.vtrace2 - and it was selling decently (7 sales within the first few hours), but then someone put a 2 star rating and, of course, nobody has touched it since. It's pretty interesting how much impact the first few reviews actually have on an app.
[0] http://monosnap.com/image/ULCknIobEcO6KarrfxwmHdHJC