Some people, like me, can only wear headphones or earplugs for a short period of time - like 2 hours - before they develop ear infections.
Furthermore, administrative controls (headphones) for what is actually a safety and health problem (excess noise) are the solution of last resort from a health and safety perspective. A good health and safety person would advise structural mitigation, not forcing everyone to put on personal protective equipment (headphones.)
When open plan offices can regularly burst up to 95 dB (yes, I took in a meter and measured the last one I was in) and are often 65 dB when they are "quiet", noise is absolutely a health and safety issue. Constant noise, even if not immediately damaging to hearing, can cause increased stress and other physiological issues.
Silicon ear plugs are amazing, I sleep with them in (insomnia), they kill sound better than foam, mould to fit so stay in and warm up to body temperature so you don't know they are even there.
Careful with this. I have tinnitus (arguably from 8 hours/day of headphone use), and when I doubled up as you described, my tinnitus became unbearable because it was the only thing I could hear and my brain "latched" on to it. I'm still trying to train my brain to stop listening...
The better 'earmuffs' out there should be enough w/o plugs. The passive model that I have (3M Peltor Optime 105, $21) cut sound by 30 dB. Nearly dead silence.