The protocol handler support is restricted to a whitelist to prevent an addon from taking over, say, "http://" (or other protocols which could be used to insinuate unsafe content).
It still seems safe though to set up rules where domain ownership could automatically grant ownership for a protocol. E.g. Facebook.com:// or just Facebook:// if .coms were allowed to be dropped.
When you navigate to a custom protocol URL there is a popup asking how to handle it, it will show all the add-ons in a list (and maybe native apps too). You can select you're chosen one and tell it "not to ask again".