Manhattan is pretty much a unique urban environment in the US. (Chicago probably comes closest.) There are a lot of things to like about it but it really is a city on steroids. Crowds, noise, etc. all take getting used to and, for a lot of people (including myself) more than a week or so of it really gets to be too much.
