No, definitely not. The intersection of people who would think to do that, have the skill for it, and the inclination to view me personally as their target is probably zero.
Not to mention that I don't think anyone but a few niche enthusiasts even have the hardware to do it even if they had the skills and inclination. Who's going to walk around with a 15 year old laptop brute-forcing wireless USB encryption, or find some obscure hub and do it on a modern laptop?
I'm not going to say the likelihood is "zero", but I am going to say it's so close to zero that it's really not worth even considering.
I inadvertently pranked myself like this. For a while my laptop got super haunted, occasionally the cursor would jiggle slightly even when I wasn’t touching the mouse. Eventually I realised that the Bluetooth mouse in my bag which I’d totally forgotten about was getting bumped and turning on.
You're not concerned that anyone within a 100 ft radius of you can inject arbitrary keystrokes/mouse movements to your PC?