True, and with enough training we can, that's what meditation is basically - just like we can learn to control our instincts like hunger or sex drive, but still it's not the normal state, it requires a focused mental effort. On the basic level we're social animals and we're probably hard-wired to need the presence of others, and also to seek places that stimulate our senses, as that is where the food is. Being strongly driven to sit in some deserted cave with nothing to eat or do in it wouldn't be a good evolutionary adaptation to any organism - so we surely have an instinct to avoid it - just think of horror movies, how they imagine really scary places? Dark, empty, deserted - low stimulus and it scares the hell of us.
