Related to most of the above: a lot of people like to know their neighbors so that they can be comfortable talking to them if there's a problem. Full time rentals mess that up. Now you might well find yourself going to complain to a stranger who couldn't care less what you think, and it's a different stranger every weekend.
* Increased traffic
* More noise
* Greater odds of criminal activity
* Resultant reduction in nearby property values
These are not theoretical and are part of the reason why zoning laws exist.