When I lived in NYC, I felt this was often true, but my hypothesis was that it was a reflection of the real estate footprint of grocery stores. A restaurant can receive daily shipments from vendors based in the outer boroughs (or further). If I really wanted to, I could have taken the subway to Queens for grocery runs, but that wasn't actually going to happen. But the Whole Foods on Houston takes up a large footprint of valuable real estate, and of course that drives up the price of everything. The choice between cooking and eating takeout weighs the labor of the kitchen staff (ideally low in a high throughput kitchen) vs the overhead of a supply chain which uses more high-value space.
No need to go all the way to Queens for the groceries - just a few blocks from the Whole Foods on Houston you have the grocery stores of Chinatown, and even the produce from the street vendors is decent - and cheap.