Development infrastructure like Jenkins has no business being on the internet, so my preferred "second factor" is a VPN, secured with machine specific certificates that offer only VPN connectivity, but not SSH or anything else. This means even if a developer's Git or SSH key is floating around your infrastructure no one without VPN access can get at it, and if a developer loses a laptop there's a good chance you'll hear about that in time to revoke that machine's VPN certificate before anyone has a chance to use it.
That approach is a good defense in depth technique.
Network security by itself doesn't work but entirely neglecting it as some misguided articles I read recently makes me wonder if security is actually moving backwards when it comes to fundamentals and principles.
Granted those supporters are frequently cloud vendors, that dont offer mature enterprise integration. But when smaller shops follow those as best practices I'm getting worried.
Exactly. This is what I employ as one of the safeguards as well. All important resources accessible only to via a jumpbox, and jumbox itself is only accessible in a VPN.
My fear is that if the VPN connection is severed, you're locked out of your environment.
I typically have a public bastion and just shut it down. In AWS you aren't paying for it if it's turned off and you have an "escape hatch" if you need it.
If someone manages to launch another instance or start the stopped one then they would have access, but that's another can of worms.
With AWS, in an emergency you can always use the console to provision access of some sort: SSH bastion with a public IP, second VPN, etc. This definitely implies that explicit 2FA is needed for console access, but I’m pretty comfortable with using smartphones for that.