Who's gonna take them? Remember that couple in Houston that put the hurt on a swat team. It doesn't take many events like that to really dampen the enthusiasm for kicking down doors (especially when you're doing it to simply confiscate property). The cops are subject to the same risk calculations as any other home invaders.
Due to a sad trilogy of events at the end of last century (specifically Episode 2: Texas BBQ) the ATF of this century mostly conducts stings and information based enforcement (i.e. nabbing people for buying Glock switches online). Were they to go back to kicking down doors of people who are generally considered normal patriotic Americans they would A) be more likely to get shot back at than any other agency (much easier to justify and mentally prepare yourself to shoot back at career snake steppers than your local PD) and B) have some really, really bad optics to contend with.
If you're worried about someone taking your guns be worried about the state police.