Actually you can opt out of the tax without leaving your church (though they might consider this grounds to exclude you from services -- in practice however the only punishment is heartfelt disapproval).
Also it's important to point out that if you are a faithful member of one of the religions that enacts a church tax, that money goes to that church rather than to the state (who just collect it for the church, free of charge). So presumably you actually want them to get that money and are fine with the tax (although in practice most "believers" in Germany either opt out of the tax or complain about it because they're unhappy with the institutions of their religion).
AFAIK it's only the Catholic church and the Evangelical church who've done the necessary paper work and formed public bodies on the national level to be eligible to receive these taxes from self-declared members of their church. There's a small Muslim group that has managed to do this in two states but they're a minority sect (in Germany anyway). I guess for most religious groups it's just too much of a hassle compared to collecting money directly from church-goers (or mosque-goers for that matter).
