Hacker News new | past | comments | ask | show | jobs | submit login
Hacking a Virtual Power Plant (rya.nc)
189 points by bo0tzz 6 months ago | hide | past | favorite | 46 comments



> A major selling point for me was that they have a local network API which can be used to monitor and control everything without relying on their cloud services.

That would be a MAJOR selling point for me, too! Especially because most of the companies that do this for residential are getting a huge portion of the profits using YOUR batteries, rather than getting that profit yourself. Unfortunately, where I live, that is mostly because energy utilities are hostile to anyone except large middlemen installers and their VPPs. It would be simple for them to also allow individuals to sign up for dispatch if they were half-competent. I can't find any companies providing battery systems plus solar that are able to be fully local in my area of the USA, either, even if I wanted to try and go "off grid" using massively overbuilt batteries linked to solar.


Almost bought batteries through a large solar company. Just went bankrupt


None of this would have happened, if the token were a traditional strong random number backed by a database on their API server. I've always disliked JWT for that reason: it keeps too much state in the untrusted client, depending solely on the strength of the cryptography to protect it, and any slip-up (like this one, or the classic alg=none, or failing to correctly using a random number with ECDSA) means the client has full control. With a strong random token, all the state is in the database in the server, and the worst the client can do is leaking the token.


> Expecting developers to know that 512 bit RSA is insecure clearly doesn’t work. They’re not cryptographers. This is not their job. The failure wasn’t that someone used 512 bit RSA. It was that a library they were relying on let them.

I disagree. Anyone using a library should have enough knowledge to use it properly, else they shouldn't be writing serious software. The flip side is that the library should be documented clearly enough to prevent a competent professional developer from making that mistake by accident.

I don't have an issue with a library supporting outdated standards since that can be useful for writing (hopefully temporary) code to interface with legacy systems. Again, as long as it's documented properly, and the devs understand the potential ramifications.


This is a great opportunity for me to cite my new favorite thing: the OHSA Hierarchy of Controls. In the physical workplace, documenting a hazard would be seen as an inadequate control if it can simply be removed.

https://makesafetools.com/osha-hierarchy-of-controls/

If software as an industry wants to improve then its practices need to improve. We can do so much better than “be less dumb”.


Old crypto profiles often can't be removed because there's still ancient hardware that needs to be interacted with. Exhibit A: the SSH server in my modem only speaks diffie-hellman-group1-sha1 and 3des-cbc.

So putting it in separate legacy modules, removing them from automatic negotiation and putting warnings on them is usually the best option.

> We can do so much better than “be less dumb”.

Yes. But in the end "we only had junior engineers working on this piece of infrastructure" would not fly for Boeing either.


We have been doing this software and hardware thing for a decent amount of time now. If the SSH server in your modem uses bad encryption because of a hardware issue, the company should replace it and not make their own infrastructure less security because of that. It's the issue with IoT in general, there's no maintenance plan. But some countries are enacting regulation to ensure these devices aren't left rot (and eventually become a national security issue).

I'm not saying companies should do this for free, btw.


I'm not aware of anything that needs 512 bit RSA for compatibility. At minimum generating the keys should go away.


That's true in theory, but unrealistic in practice. How can everyone know everything about everything?


Requiring people to know everything is a strawman. Raising some basic awareness about risk factors in software systems such as "crypto, authentication, security are hard to get right", is not the same as expecting everyone to be able to implement constant time elliptic curve cryptography in assembly.

Understanding the boundaries of your own knowledge, where to delegate¹ to someone else or do some research to deepen your knowledge is also not the same as knowing everything.

¹ using libraries is not quite the same as delegation if you can't even assess if you're using them correctly or if they do the right thing. You still have to meet it where it is.


You don't have to know everything about everything. You just have to RTFM enough to understand what you're doing and what the ramifications are of doing it wrong. If you're not sure, ask a more senior dev for guidance.


> the legitimate ones I’d initially generated still worked

This spooks me. I take this to mean either:

- They are still using the compromised key for validation, meaning if you have access to any old token, you can still mutate that, maybe needing to play around with the issuing times

- They built an allowlist of all permitted tokens, and check that list first. In which case, might as well use random session ids instead of JWTs, and at the same point where the allowlist is being checked, mutate the request to inject a JWT that the backend can use.

Also, kind of curious why the switch to RSA4096 instead of elliptic curves, since they are generally faster / smaller.


I think very few customers had ever generated API keys, and as best I can tell they made an allowlist for them.

One of my suggestions to them was to switch to elliptic curve, but I imagine RSA 4096 "just worked".

I suspect they'll rework it later now that it's not "on fire".


Ah that makes sense. For sufficiently small values of N, a hardcoded allowlist isn't a problem.

You're probably right that RSA 4096 "just worked", and some library in their stack doesn't have elliptic curve support. And again, if N is small, the verification performance doesn't matter that much.

Nice find and writeup!


My guess is they are still accepting keys signed with the old 512 key but are currently generating new tokens with a 4096 key.


No because Ryan says he is not able to create his own tokens anymore. And he has the same private key as the supplier.


FYI, Ryan uses they/them pronouns (https://rya.nc/about.html).


Ah sorry and stupid for assuming, I would edit but cannot anymore.


Now imaging the amplitude of a deliberate attack not to personal domestic p.v. productions but en-masse against the grid: most hybrid systems also can inject, p.v. and eventual batteries can inject all together, eventually ignoring grid frequency burst. So far probably those systems are not so common do generate widespread damage, but once they'll be spread it would be easy to generate a large scale blackouts.

Just to say that no proprietary devices should be ever allowed to have remote connections, FLOSS must be mandatory from the early stage of a project to allow third party inspect the codebase not starting with a gazillion SLoC, being de facto unable to understand them, instead of starting from the first SLoC following at the project evolutionary speed.

Nowadays it start to be again damn easy to plan any kind of "unexpected logic" in gazillion of devices, potentially at State intelligence level, from cars to energy systems, TLCs active devices, ... not a new thing at all, but easy enough to be dangerous enough to MANDATE FLOSS if those who care of a nation security have at least an idea of the current role of IT in a society and it's enormous mean attack surface.


Here’s a bit of a counterpoint to this fear; Ukraine and Russia have been in a high intensity war for two and a half years now and Russia is a leading cyber-offense power, yet Russia is resorting to flinging long range missiles at Ukrainian power infrastructure rather than cyberattacks, that suggests to me that cyber warfare is overhyped at least as technology currently stands.


Most Ukraine and Russian infra came from Soviet time, I doubt they have much digital attack surface... Most war episodes reported are similar to WWI and WWII combats, so again not much attack surface.

But do not think about an actual war on the ground, just try to image the damage you can give to another country let's say creating a cascade blackouts, than a e-payments system one and so on. Cyber warfare happen generally aside/before a "classic" war, not much during it on the frontline.


Not much about the potential severity of hacking something like this: You could probably blow circuit breakers in MV substations with a vulnerability like this if there are enough batteries under control in the same area and you get them to all discharge at max current at the same time. Serious, expensive damage. They could be substations connected to essential services like hospitals (though they’ll have backup gensets).

The energy transition is going to be a fun time for infosec!


> You could probably blow circuit breakers in MV substations with a vulnerability like this if there are enough batteries under control in the same area and you get them to all discharge at max current at the same time. Serious, expensive damage.

AFAIK, all that would happen in this case would be that discharging the batteries would raise the grid voltage, and if it were raised enough, the over-voltage detection on the substation circuit breakers would trip and open these breakers normally. It wouldn't damage the breakers (or anything else at the substation), they're designed to deal with over-voltage events (for instance, when a large block of power consumers suddenly trips offline, the voltage raises until the power plants either throttle down or also trip offline).


Shouldn't the circuit breakers inside homes trip first?


Stability of the whole grid relies on there being no big synchronized events.

Every home might have a 100 amp circuit breaker, but the grid cannot handle every home taking or giving 100 amps at the same time. 2 amps maybe...


I don’t think so, each battery would be discharging at its rated current? The problem would be what happens “upstream” in the grid


My parents have one of these systems installed - the guy that did it took the password off their WiFi since it "doesn't support it". Even with a password on the home WiFi it defaults to broadcasting it's own open SSID that will happily display the wireless key to anybody that connects to it. Apparently these don't support Ethernet either..


Mine definitely supports Ethernet and WiFi with password, both what it connects to and the SSID it broadcast, though I had to change some settings on my AP (running OpenWRT) to get it to work.


I logged into the SSID and tried to set a password on the interface itself but then apparently it lost connection to their system and needed to be reset. Not sure as to why an electrician can’t just install an Ethernet cable at the same time as the massive battery


They probably can if you ask, but most people don't use wired networks at home.


The Ars Technica article may also be of interest - it has a statement from the company.

https://arstechnica.com/security/2024/08/home-energy-system-...


> not only was I no longer able to mint my own API tokens, the legitimate ones I’d initially generated still worked

can someone explain what's meant by this? i presume the private key identified by the author still works, but will stop working after 7 days?

(based on the sample JWT in the first section, the expiry time of the token seems to be 7 days)


"They’d switch to a 4096 bit RSA key, but not only was I no longer able to mint my own API tokens, the legitimate ones I’d initially generated still worked."

I wonder how the old JWTs signed with the 512-bit key still work safely, isn't that 512-bit key cracked??


I do wonder how you got in contact with them? No security.txt or other obvious ways to report a security issue.


In the footnotes:

> Someone once asked me, as commentary on my ability to figure out email addresses, “Are you a Hacker or in sales?”

Just a bit of OSINT and educated guessing.


It's pretty easy to get in contact with someone most of the time if you search for relevant roles on google with site:linkedin.com, or if you want to reach execs, on their business pages. I usually include several combinations of first and last names @company.com so like I will email to JSmith@company.com and JohnSmith@company.com and others. This is useful for escalating customer service complaints if you're just hitting a brick wall with the low level customer service staff.


AFAICT there's no longer any actual humans reading mail sent to any domain's postmaster@insert-domain.here, even tho it is required by RFPs.


Yeah, this is pretty much what I did, except the demo account had an employee's email address in the details so I knew the format.


They do have a vulnerability policy page, but that was harder to find than figuring out an email address. I did suggest they implement security.txt, as it would have made things easier.


Great blog post. The company response was really great, but there was no mention of a bug bounty. This would be a perfect example if he got paid for his findings.


I'm not male and I use gender neutral pronouns.

The bugs I find are usually just ones I stumble across, I don't go looking for them.

Bug bounty programs generally aren't worth the trouble for me anyway - I have to file taxes in both the US and the UK so the paperwork gets really annoying.


I wanted to see your talk on Hacking Gender, but the link on your about page is broken, it seems to have been taken down :(


The link had been to an unofficial upload. I guess the uploader took it down after EMF's official version was posted.


I've fixed the link.


Adding to Ryan's concerns, submitting to a bug bounty program often means accepting terms and conditions that constrain your ability to publicly disclose the issue if the vendor decides to be a dick about it. Depending on how you think your career potentially benefits from freedom to discuss an issue, the long-term financial benefit is potentially greater from not going down the bug bounty path.


Oh, yeah, I'd forgotten about the NDAs, those are a big NOPE for me.




Consider applying for YC's Spring batch! Applications are open till Feb 11.

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: