Can't say I've experienced that. It always seems to decline into the size of a celebrity's arse. Even one of my friends, an ex-physicist and current private jet pilot has nothing to say past the size of an arse.
Celebrities are an important part of generic conversation... they give us someone to gossip about that neither of us actually know. Public shaming is an important (if offensive) social ritual.
That said, just about anyone can talk about something other than Kim Kardashian. No one actually cares about her. Talking about something like a celebrity's arse is a way of avoiding talking about stuff we actually care about - as someone else noted in this thread, the fear of being seen as uninteresting, or weird, or an outsider, if you talk about stuff that matters to you.
The way to get people to talk about what they care about is to ask them. Your friend the physicist/pilot? Ask him about physics. Ask him about airplanes. Ask him about his clients as a pilot. Pretty soon, he'll be talking about much more interesting things - interesting to him, and probably interesting to you.
And all you have to do is listen, and he'll think you're awesome.
With some people that's really hard. They don't see your interest or disinterest in a topic at all, even if you make it really really obvious (not looking at you, one word answers and even descending into rudeness vs making eye contact and participating in the conversation). Some people just want to complain about trivial things, even if they could tell you about antarctic expeditions instead.
> It always seems to decline into the size of a celebrity's arse
That's the type of issue I typically encounter. You can only talk one subject for so long, especially if the conversation has more than two participants (as diving too deep into a specific subject risks boring everyone else).
I just go and read a book now.