I wouldn't even call the SE compact. It's just normal. It's the size that Steve Jobs said fits comfortably in your hand, and since hand sizes haven't changed, it's still true today.
It’s actually a bit bigger, the original iPhone was 3.5” screen and the iPhone 5 got taller.
I have to give up a bit of grip to get my thumb to the top corners of the screen, but it’s a lot better than anything else and I’d say worth the trade off for the extra space. Bigger than this I’m not convinced.