A Default VirtualBox configuration does not allow direct host-> guest access, it's on some hidden VB-only NAT. You need to change the Network Interface type to Bridged or Host-only or something else to have access to the VM.
I haven't done it for some time, but in my memory it was neither a problem on OSX nor on Linux with VBox. Maybe try Vagrant? I don't remember, sorry. But ask around and you should find a super easy way to make this happen.
It sounds like you are talking about NAT mode on virtualbox. In this case you can set up port forwarding in the network settings. Also, you can actually reach the host from the guest. In my case my guest has an IP of 10.0.2.15 by default, and I can reach the host on 10.0.2.2