If your phone is iPhone-ish in size, you may try what I did. After getting a stiff-rubberized phone case (you stretch it around the phone), tape the cards to the case. If you cut a notch in an old card, you can have that be the card against the tape -- the rest of the cards can slide in and out against that card or the phone, and are held in by the force of the tape.
I call it the PhoneWallet. I made one for my droid Incredible.
I ALWAYS have my phone though. Quite frankly, I look forward to the day when my phone replaces my access badge/key-card at work.
Of course there is the arguement "but someone could take your phone and access the MDF!" -- well they can always steal my access badge as well.
I have forgotten my badge countless times in my career though I have forgotten my phone all of once.