All the fans I've met are Joe Rogan fans in the same way as other people are BTS fans or Ariana Grande fans. They don't really worry about precisely how smart or insightful he is - they just see him as a cool guy producing content that's fun to listen to.