Storage will never work. Quote me on this.
Nuclear or a mix of nuclear and renewables will be the only way to seriously get away from fossil fuels.
Also, even if storage works some day, hoping we manage to discover how, scale and implement it across human civilization in time is a crazy bet to make. Even if we seriously go all in on nuclear and renewables tomorrow - it might already be too late, so betting on some miracle tech to be found, scaled and implemented in time is not only unwise, it would require several miracles to have any hope.
The storage plan for the EU is a mix of battery and hydrogen storage. It has a timeline that will take a while, but so does nuclear. There is a roadmap for converting europe’s gas transport and storage infrastructure over to hydrogen as part of the EU hydrogen strategy. I know there are people who don’t believe in the feasibility of the hydrogen strategy, but the people actually in the field working as experts seem to believe in it, so I remain unconvinced by the critics.
It already does work though, and it growing exponentially without any tedious political debates since just about anyone can do it at any time, at any cost and any scale. Companies store their own energy, counties store their own energy, private individuals store their own energy.
I see no way of stopping it unless corruption somehow manages to make private power production and storage illegal. There certainly are no technical or economical showstoppers.
This is more of an argument for how pure economic thinking and the current constraints/processes have poor correlation to actual impact and desired outcomes.
It's similar to how Enron would make the most money when California had rolling blackouts - by operating right at the edge of the network crashing, they would make the most money because reserves were low, so they intentionally shut power stations down and caused small grid crashes.
If you really believe in renewables, if anything, we need to go all in on nuclear for the base load, but no one seems to be headed in that direction other than China and India, because the don't have the same market failures we do.
It's much worse - if the data isn't just a ton of tiny files, and you're able to spin up a bunch of workers for parallelism, you can get up to 120 Gbps per storage account (without going to the extreme of requiring a special quota increase).
That means in a little bit over 5 minutes, the data could have been downloaded by someone. Even most well run security teams won't be able to respond quickly enough for that type of event.
At a former team, we went from spending quite a bit of time on code style comments and disagreements to spending no time at all on it, with the simple act of making the code linter a breaking step in our CI build, and deciding no review will start until the build is green.
We had to adjust our linter settings here and there - but it was still super efficient for everyone's time compared to what we had before...
The difference between theory and practice, is that in theory there is no difference, but in practice - there is.
So far, every "provably secure design" I've seen ended up being insecure in practice due to the things people abstract away.
I'm not saying it's impossible, but I have not seen it done perfectly thus far.
We've seen more success by having many many iterations and widespread usage of common designs and patterns. These are not perfectly secure by any means, but they are secure enough against common threats to make it functionally equivalent until we figure it out.
I just feel that our proven insecure system, with default authority, is a really bad foundation to have settled upon. We couldn't have picked a worse default.
Okay, name the "provably secure designs" that were actually proven and validated by a competent security standard such as the Orange Book Level A or Common Criteria EAL 6/7 that turned out to be insecure in practice.
Most people who say that point to designs that were never proven and never validated against anything meaningful, but I am open to seeing a actual example.
For enterprises, it's hard to have a ton of different tools.
I worked at a very large software company, and our security tech stack was so big and convoluted, that just maintaining a compliant CI/CD pipeline was a 5 person job, because there are ~20 different tools to integrate and debug, and each of those changes every year or two, so you're constantly re-learning, re-integrating, debugging,etc.
Having a single (or just a couple) vendor(s) sounds like a dream!
Those aren't cheap, but rolling your own usually isn't any cheaper. Even huge enterprises usually buy instead of build because it's cheaper in both the short and long run.
Think about most managed cloud services - you could deploy your own SQL servers on EC2, configure replication, fail-over, backups, security patching, log collection, observability, etc. - but you'll end up paying a lot for engineers to build, maintain and monitor that solution compared to just spinning up one of the ready made offerings by AWS.
It might be cheaper to do if you have a ton of RDS, but it really has to be a huge huge volume, and even then, AWS will probably find a way to discount your bills to make it still better...
Perhaps I was too cavalier in my original comment, but when I said building tools built on open source software, I meant leveraging things like Matano (matano.dev). So you’re not writing everything from scratch but you are responsible for wiring everything up to fit your environment.
And you’re right, it’s not going to be a universal truth - there will probably be some tool you end up buying. But I’d like to have a security engineering team that is forging something that will fit my organization like a glove instead of us trying to bend over backwards to make some big off the shelf tool fit with all of its features.
I work at one vendor currently and have worked at a few prior. The difference is astounding - my previous gigs, including one of the biggest vendors ever was exactly as you said. My current gig is exactly the opposite - strong focus on real security insights and value, none of the box-ticking bs, and a great roadmap. It is rare, but when everyone at the org, and especially the product side really know how attacks play out - you can make a real impact on the world.
Okay, but how much would it cost to hire a hacker or red team to breach your systems? Is it more or less than $10M? If I had one competent hacker and a year do you think you could stop me? How about three people and a year?
You're right that a generator must be taken care of, but a truck with an inverter is very different from a generator if you plan to power something significant. The truck+inverter will probably keep your fridge and lights running, but can't power your whole house unless you get an oversized aftermarket alternator, which will cost more than the generator probably will (aftermarket car parts aren't cheap).
