Came here to ask just about that but I see it's on your roadmap already, that's good and godspeed with that since encryption is hard.
During my PhD, I worked a little while with threshold encryption schemes (sometimes called horcrux encryption schemes, i.e. make n keys, you need at least m of them to perform some operations) (ref. my noob-ish question here ha https://crypto.stackexchange.com/questions/74763/is-there-an...).
I created a small system that allowed one to reconstruct parametric data from 3D shapes, but if and only if you had at least n pieces of the whole model, because reasons.
Reading this announcement, I recalled several things that came up during my research, which apply to this context as well.
* How can one make sure that losing one key is not the end of the db (i.e. backup keys)?
* How to share db access but not individual keys? (i.e. one key per user BUT all of them can read)
* What if encryption is shared by n users but one of them loses their key?
* How many keys for encrypting (or just one?), how many of them for decrypting?
* Represent all of this in some sort of stateful model, after all it's a db (in my case it was files), it's meant to be cold storage, everything should to be able to be reconstructed/recovered from there.
The list goes on and on ...
Anyway, just wanted to say that this is a very interesting and promising feature to be found in a database. Great work, I'm eagerly waiting to get my hands on this.
You've hit the nail right on the head with this question on how hard group encryption is, and we don't have all the answers yet as we are still working the use cases around it. We are hoping to reach a level of security that you mention in your SE question using something similar to the excellent accepted answer, distributed private key sharing among trusted participants.
The basis we are exploring is using an algorithm called Signcryption (https://github.com/jedisct1/libsodium-signcryption) that is already included in pgsodium. This doesn't solve any of the shared private key issues you mention above, but it is a useful foundation for distributing encrypted messages that separate out sender and receiver identifiers from their keys, a sort of lower level foundation on top of which distributed key sharing can occur.
I also think signcryption is a great foundation for a better token format than JWT or PASETO, as it covers all of their use cases without algorithm confusion attacks (despite PASETO's insistence on "Algorithm Lucidity") and supports more features such as third party verification and streaming shared key generation from any token without having to exchange the key, we hope to use these tokens so that end-to-end peers can exchange tokens, derive streaming shared keys, and then do direct point-to-point message exchange using libsodium crypto_secretstream API which supports key ratcheting for forward secrecy.
Would love to discuss more about your research with you and include it with attribution into our future work, send me an intro at michel@supabase.io when any other ideas or resources you'd like us to see!
Vault and pgsodium dev here at supabase, we're pretty excited about the Vault, and this is just the beginning of some of the advanced encryption functions that libsodium provides that we want to bring into the Postgres ecosystem.
Happy to answer any questions here about how the Vault works now, and always excited to see use cases and suggestions for features from the community. libsodium is a big API, and pgsodium exposes most of it (about 110 functions so far, a few functions don't make sense in SQL) so there is a lot of possibilities for new ideas and projects straight into SQL without having to learn the low level C details of using the sodium library directly.
One of the things i was going to work on this weekend was an enhancement to a side project and it involves storing some sensitive information in a database. My usual way of doing this is strong encryption in the application code with a key from an environment variable then base64 encode the result and put it in a text column.
Is Vault something that can handle this without getting into my app code? Basically, if i gave a someone root access to my supabase instance is that encrypted data safe?
PS the more i read about supabase the more magical it becomes. It's incredible work so congratulations, i love it.
> Is Vault something that can handle this without getting into my app code? Basically, if i gave a someone root access to my supabase instance is that encrypted data safe?
The answer is slightly offset from your question, so let me start by pointing out that the Vault is about Encrypted Data At Rest. This is mentioned in the docs and in the blog and video, but it's something that I like to always mention first in discussions. The main purpose of the Vault is to store your data encrypted, so that it's encrypted on disk, and in backups. In SQL the decrypted secrets are available to you, because that's where you are using them and encrypted data must be decrypted to be useful.
If someone roots access to your database, then yes they can access the decrypted secrets through the view. This is by design, the secrets must be decrypted to be useful in query code. This risk is similar to someone rooting your application code, they will see decrypted secrets via your environment key, so no it won't protect you against anyone rooting processes in your stack that need useful access to secrets and it's not meant to. Like all security you must take a layered approach, the Vault is just one storage level layer strategy.
One big difference from the env var approach though is that the key Supabase uses to encrypt your secrets with the Vault is stored outside the database, it is inaccessible to SQL, which is an enhancement over sticking the raw key into an environment variable or a table that is accessible to your application. Instead of revealing the raw key, pgsodium has a feature called [Server Key Management](https://github.com/michelp/pgsodium#server-key-management) where you do not have access to the raw key, but instead reference keys by an key identifier. It is safe to store this identifier alongside the data it encrypts. The raw key itself is never stored. I'm very intentionally overusing the word "store" here, because that's specifically the layer of security that the Vault provides.
I'm building the "in-use" part of this right now...what if you could encrypt your data with an encryption key (at-rest), but also to a set of code that is allowed to decrypt it (in-use). If that code is identified cryptographically, its identity can't be spoofed or stolen.
One benefit of the approach we are taking is that you can use regular Postgres security rules and policies (GRANTs, RLS, etc) to constrain access to decrypted secrets. It's not quite going all the way to encrypted VMs, but it's better than all or nothing.
Secure enclaves is something that is on our list, one issue we have is that so many of our components we ship are extensions and open source projects, attestation becomes very hard. We've considered a very, very minimal postgres build with almost no extensions enabled except pgsodium, which would run on an encrypted VM purely for the purposes of secret storage, but then "in-use" becomes rather restricted. Definitely open to hearing any ideas you may have on the subject!
Vault is a Postgres extension that wraps pgsodium/libsodium. It enables 2 key features:
1. Secrets management - you can store things like API Keys
2. Transparent Column Encryption (TCE)[0]. This allows you to encrypt a column in any of your tables, with a View for "selecting" out the decrypted data. It enables "row level encryption" too when you create a key for each row.
The blog post details how it works with AEAD[1]. This is a secure way of encrypting "associated data". An easy way to explain this:
Imagine you associated a `user_id` with a `credit_card_number` while you encrypt it. A bad-actor updates the `user_id` to their own ID. When they attempt to decrypt the `credit_card_number` it will fail because the data that is associated is different. (note: please don't store credit cards in supabase)
We're rolling it progressively to the platform over the next month. Michel, the mastermind behind this one will be here to answer the questions that are above my head.
this is the last big launch of the week. You can see everything we launched this week here[2]. Some highlights from today: pg_graphql v1.0[3] (re-written in rust), PostgREST 11[4], and PGroonga release for multilingual search.
I think it’s great too. I wish they would shore up some of their existing releases though. Probably most notably, the ability to query aggregates via the officially supported route is missing: https://github.com/supabase/postgrest-js/issues/206
The workarounds suggested are not ergonomic for most use cases and it feels pretty out of place for such basic functionality to be missing in what otherwise feels like a pretty full featured product.
Their Realtime product is another example of something that languishes while new features get launched.
Very nice. I'm building a new startup MVP with supabase it's been lovely so far. Now if there could be an EC2 competitor I may be able to avoid (re-learning) AWS entirely.
Try fly.io. For our use case, even though I've used AWS/EC2 for 10+ years, Fly.io and similar (render.com probably but I haven't used it) are refreshing because I know a lot of the best practices are just baked in.
I really don't see the point of running a large class of applications on AWS at this point, apart from "X customer won't bother us if we respond with the word AWS".
If only fly/render/all these people come up with an easy way to translate a docker compose config to their architecture, that would be madly powerful (first thing I'd try is to self-host Supabase!)
Fly.io plus Cloudflare have been a great combination for us, and we've added point solutions for other things as needed like email, auth, etc. Feels much more approachable than AWS when you're at a small scale.
> I'm using Postgresql for a service and I'd love to use this feature. Maybe I should just use supabase as my backend...
We'll try to get the other big clouds to adopt some of these extensions we're developing (including pg_graphql). Vault is still in beta, but once it's stable I think it's a no-brainer for them. it can work with secure-enclaves, so it ties in nicely with their other offerings (read: more economically interesting for them). That said, we'd love for you to try supabase.
> We'll try to get the other big clouds to adopt some of these extensions we're developing
This is critical if you're aiming for mass adoption. I need to be able to use this on Google Cloud PostgreSQL and Azure PostgreSQL before I can consider using it.
Came here to ask just about that but I see it's on your roadmap already, that's good and godspeed with that since encryption is hard.
During my PhD, I worked a little while with threshold encryption schemes (sometimes called horcrux encryption schemes, i.e. make n keys, you need at least m of them to perform some operations) (ref. my noob-ish question here ha https://crypto.stackexchange.com/questions/74763/is-there-an...).
I created a small system that allowed one to reconstruct parametric data from 3D shapes, but if and only if you had at least n pieces of the whole model, because reasons.
Reading this announcement, I recalled several things that came up during my research, which apply to this context as well.
* How can one make sure that losing one key is not the end of the db (i.e. backup keys)?
* How to share db access but not individual keys? (i.e. one key per user BUT all of them can read)
* What if encryption is shared by n users but one of them loses their key?
* How many keys for encrypting (or just one?), how many of them for decrypting?
* Represent all of this in some sort of stateful model, after all it's a db (in my case it was files), it's meant to be cold storage, everything should to be able to be reconstructed/recovered from there.
The list goes on and on ...
Anyway, just wanted to say that this is a very interesting and promising feature to be found in a database. Great work, I'm eagerly waiting to get my hands on this.