At least for the USB-A versions, the flexing is almost by design. Since it doesn't have the full housing around the USB-A connector, it tends to have a bit of motion in the port. Its not even that the device is flexing, its pushing on the pins on the USB port more or less as it moves. I can't speak for the USB-C version, as I don't have one and I don't suppose it has that same thing going on.
My USB-A yubikey has proven durable so far, but it only requires a very light touch to activate.