While 3 is a bit on the low end, user studies are not required to have large sample sizes. You're not trying to estimate what percentage of users are running into a particular issue, but only which issues many users run into. Whether it affects 10% or 30% of users does not really influence whether it should be fixed.
And this is why the field of design gets shit on so much. Jakob Nielsen might have a PhD, but he’s much better at PR than he is at conducting actual research. It’s mind boggling to me that people will still quote his opinion pieces 20 years later as if there was any actual science behind them.
There’s a few other fields that happen to conduct a lot of studies around human behavior. It turns out, there’s actually an established process for calculating the confidence level of your findings:
Is 3 or 5 better than nothing? Sure! But don’t trick yourself into thinking your findings actually mean anything because a Danish dude drew a graph 20 years ago.
So again, a large margin of error and a low confidence level are fine, because it doesn't matter if in reality, 80% of your users run into something rather than the 60% that came out of user testing. In fact, we don't care about the specific numbers at all. All we want to know is what things are unclear to many users. Often, you test a design on five users, and all five of them miss the same thing - that gives you plenty of confidence that you'll want to try something else there.
User testing is not a statistical test, and it's not a way to learn about human behaviour (so indeed, in that sense it doesn't "mean" anything); it's to give insight into your designs. I'd highly recommend everyone to take part in conducting user research sometime, as it will quickly become obvious just how easy it is to miss "obvious" things in something you're closely involved with, and how you need to have just few people interact with it to find out which.
More on this: https://www.nngroup.com/articles/why-you-only-need-to-test-w...