It’s not about whether the audience is large or small.
Performance rights are about whether it’s public or private and the venue is paid or free. For private use where those present are by invitation only and haven’t paid to be there, I don’t think any further license is necessary, beyond simply owning a copy of the work to be performed.
If you are a group of people reading a book and discussing it in an organized manner you are supposed to pay royalties for it. (Source, I've gotten such money from Sweden)
Sounds like a good reason to stick to classics in such groups.
Seriously, if I knew about such a stipulation, I'd be incentivized not to buy copyrighted books to read for a book club in the first place.
I'm really glad there is a corpus of public domain works; so much good is possible (or positively incentivized) because of it. Too bad it isn't growing much (e.g. Steamboat Willie is still, still under copyright restrictions nearly 100 years later).