That's what they say every time - "it's just to catch the REALLY bad guys, pinky promise". Then five years later we find out that it's routinely used for other things. And ten years later, when the law needs to be renewed or whatever, the same politicians who originally argued that it was a very narrowly tailored measure claim that it became such an important tool in law enforcement arsenal that sky would literally fall if we don't allow them to keep it.
Law enforcement gonna exploit outrage. Evidently so will encryption fans. Both are politicians, in the sense that matters of state powers are of interest to both; and thus both wield hyperbole: "down with terrorists" vs "down with Big Brother".
Maybe let's not fall to 10-foot tall syndrome in either direction. Two things can be true: law enforcement is overstating terrorism and criminal dangers that it's facing, and encryption advocates are overstating totalitarianist danger.
Maybe the problem is that in many Western countries there isn't a huge smoking gun where surveillance powers are "routinely used" for visibly nefarious things. Even things as bad as Snowden exposed (and they were shocking) don't seem to resonate with a lot of folks, maybe because it doesn't seem like the govt was actually able to do anything with all the data that they grabbed anyways. In the US, I haven't really heard of cops blackmailing people over intercepted data, for example. Or jailing of political opponents based on their online habits.
I'm not sure if encryption fans are either not looking to be in office, or not popular enough to be elected. But if it's an issue people care about a lot, there should be no problems getting elected on that issue. Don't be on HN taking to me, go talk to some random people and see how they want to vote on that issue or if they even care. But maybe, just maybe, you'll find that there are other things on their mind right now.
I think most "encryption fans" understand quite well that the current sociopolitical zeitgeist is broadly authoritarian, and legal strong encryption will fall victim to that eventually. As will many other things - e.g. a Chinese-style social credit system with the "right" targets is also likely to get considerable popular support in the long run, and some Western countries have already experimented with similar ideas (e.g. asbo in UK).
So as far as I'm concerned, the political angle is mostly about dragging our feet to buy us more time to develop the anti-surveillance tech like encryption as far as possible in the open before it is heavily regulated.