> Credentials are stored in lib/credentials.json. This is checked into the repo, so USE A PRIVATE REPO.
It's called a "private" repo, not a "secure" repo.
The point of a private repo is to give you control over your collaborators. It is not to provide information security. To my knowledge, Github makes no special promises about the security of private repos. The data is not encrypted on disk at Github, and Github staff can access the content of your repo.
I WOULD NOT UPLOAD SECRETS INTO A GITHUB PRIVATE REPO.
If I'd decide to keep the secrets in the repo, I would have them encrypted with e.g. GnuPG and put a key in the CI's environment variables.
# Something in lines of this...
# TODO: Make sure the keyrings are on tmpfs, too.
# Something like export GNUPGHOME="/run/gnupg" && mkdir -p "$GNUPGHOME"
gpg2 -v --import <(echo "$GPG_PRIVATE_KEY")
gpg2 --decrypt --output /run/secrets/credentials.json lib/credentials.json.gpg
ln -s /run/secrets/credentials.json lib/credentials.json
For this I have to trust the CI, of course... And given that CI jobs run in isolated environments but on a shared hosts, there could be a way for malicious party to access the data. IIRC, there were some exploits that break out of Docker.
If you host in the cloud that offers a KMS (AWS, GCP, Azure), a slightly advanced alternative is SOPS: https://github.com/mozilla/sops (you can use that with GnuPG as well)
I don’t get what this accomplishes. You can either put the secrets in the environment variables, or put all the information required to decrypt the environment variables in the environment variables.
Idea is, if someone gets access to your CI environment but not the repo - they don't have the data to decrypt. E.g. if your CircleCI password is leaked, but GitHub is safe. In my experience, most CI websites don't allow to browse the source code.
That's not ideal - IIRC, CircleCI in particular has a feature to run the build with SSH, so a sufficiently aware attacker still has a way to get the decrypted data. Still, an extra obstacle. Just that security's not binary.
Also, this way you can manage the secrets in your repo. No need to log in to CI and use their web UI to edit the passwords. I think that's convenient.
Sops is a better idea, because you can't extract keys from the KMS.
This problem is one reason why it's important to separate CI from CD. If you do that, then circleci would only have what's needed to test and package it, and the CD can handle the rest, although ideally you'll have a third piece for secrets.
Or at least unencrypted secrets. If you've properly encrypted the data, then it really shouldn't matter where it ends up as long as you keep the key appropriately private.
As long as the encryption isn't broken for the lifetime of the secret. This is enough of a sufficiently hard condition to guarantee that it should not be a general recommended practice.
You should be rotating your secrets regularly. As long as you do that and use an industry standard encryption implementation (never roll your own!) you should be gtg. Everyone differs on how often to rotate secrets. some people say every 10 minutes (projects like vault make this easier) some say 6 months, some say a year.
I've heard this before from people but I can help but think, "companies are paying GitHub to store code which in itself is often very valuable IP that needs to be protected". Are we sure GitHub doesn't have any promises to the protection of private repos? Obviously anything could happen and it's best to encrypt secrets and such... but if it's true that GitHub is not very serious about protecting the contents of private repos seems off to me. Any links to anything regarding their stance on this?
There's a big difference between proprietary code and secrets. If you get my proprietary code you will be able to understand and copy how my application works, but (if I've coded it well) you won't be able to compromise a running instance. This is why open-source software is not necessarily less secure than closed-source software.
But if you get my secrets, you can most likely own that instance.
And I think it's worth pointing out that companies who are serious about the trade secrecy of their code tend to host their own repos. I don't think you'll find critical source code from Microsoft, Google, Facebook, Apple, etc. sitting in private repos on Github.com.
I've been using Github private repos at work since 2013. As I said, I've not seen anything from Github that says that private repos are secure enough to upload secrets. For example, this document does not distinguish between public and private repos:
I use private repos to interact with contractors. Just because I grant a contractor access to collaborate on a private repo, that doesn't mean I want them to have my api keys, passwords, etc.
I use private repos because I want a cheap and easy way to host git repos, but I don't want to bother with the public. I don't want to create the false impression that I'm open to pull requests or forking my code.
The features of github means the data can't just be left encrypted all the time. No matter what, github employees are going to have access to your code. If you absolutely require security github enterprise is the on-prem way to remove the risk of a rogue github employee (to just the risk of your own rogue employees)
> I will not take responsibility for any potential leak of sensitive information.
That's not how it works. You know at least one person will put their key in a public repo by accident. Wouldn't this give anyone complete access to their bank account? Seems like a pretty scary thing to check creds into the repo.
It's worth noting here (though many people have in different ways already) that the reason you should not put credentials in any git repo (private or public) is that inevitably when you share repos with third party services (A CI for instance) you are also giving them credentials as a result and exposing one more vulnerability in the case that they get hacked or otherwise don't store things in an encrypted manner.
Because Github makes it so easy to give developers read access to repos this happens more often that you would realize, and having your most sensitive credentials (banking username and password) exposed in a repo to several different vendors is NEVER a good practice.
It's fine if you're going to use this but at least properly encrypt the secrets, or add your creds in a different file and gitignore it at the least if you're too lazy to use something like vault, credstash etc.
The reality is you're only as strong as your weakest link and there is just no use in having your weakest link be a relative unknown
Source: I built a Plaid competitor once upon a time (that obviously did not work out as well for me as it did for them) and I am currently building a bank, so I know a few things about security and vulnerabilities
haha boy is it a long story ... but yes both banking software and buying part of an actual bank so we can build a full-stack bank for the next 100 years
Interesting. Have you done any write-ups or have any more information out there? As a developer with a degree in finance, this kind of stuff interests me.
We don't have a ton out there because we're still mostly pre-investment, but we have about 9k people signed up for the landing page we built - https://launch.trycrux.com and we should be able to start opening real accounts in the next few weeks
Can I just ask: what you're doing sounds essentially like stuff that's better for the customer but costing you more money (or earning less income) than traditional banks. What's the plan to offset that loss?
Robinhood of course comes to mind, and then deposit insurance ofc pops up as well. Whats your plan on that?
Are you doing debit and credit cards only for the launch? Do you already have open credit lines for interbank loans, or will you need to limit credit issued based on deposits made or other assets? When you're pre-investment, how do you even have enough assets to meet liquidity requirements? What are the toughest requirements and the biggest bottlenecks before you guys can start opening accounts?
In general, you guys should have an FAQ page full of info. And you should probably do an AMA somewhere, I'm sure many people would find the experience of trying to start a bank pretty fascinating.
tbh I just did not think people would find these kinds of things as interesting as I do. I would love to start an AMA for this and go through it if people want me to do that I'm happy to try to make it happen, not sure if there's usually a formal process for setting them up on HN?
I'll do my best to address those brief questions here though. You're right that at the outset it looks like we're doing things that are better for the customer but the overall goal is to actually properly align ourselves with the customer, for instance one of the things we plan to do is pay a higher interest rate if you save a higher percentage of your income. Thus incentivizing users to have better financial habits and as a result become the exact type of borrower we would like to lend to.
The no credit apps thing is actually very simple, it just means we're using a soft pull rather than a hard pull and we still know your credit history so we're not doing anything blind.
The assets and pre-investment question is another good one. We've actually entered into an agreement to become a minority shareholder in a bank that has been in existence since 1921, as they grow (as a result of the things we're building) we intend to keep buying more shares of the underlying Bank Holding Company and to eventually become a BHC ourselves. This means the bank has the capital requirements in the short term and we are just a minority shareholder that happens to build software for the bank!
It's a structure we arrived at after looking at a ton of options and deciding this was the best possible way to test buying a bank without outlaying all the capital on the frontend.
as for the insurance etc. That is relatively explainable as a result of the above. This is an established bank that has been in existence for a very long time, we're just building software we believe will help it grow while being able to share in the upside as an actual shareholder rather than the mis-aligned incentives you see with most of the challenger "banks" in the US since they only make money from interchange.
We are in no way incentivized for our customers to spend money since our number 1 task is to grow overall assets at the bank level.
We're basically ready to start opening accounts, we should have the core ready to go for friends and family around Jan 15 and then from there we will start onboarding the ~9k signups we have as well.
The main hold up has been that Mastercard has a moratorium on new products over the holiday season so we have not been able to issue cards just yet, but we've also been building the app, getting the core provider setup etc. (We actually got a new routing number issued for the bank and a brand new core provider)
Again, if there's a sanctioned way to do an AMA or something similar I'm happy to set it up with @dang or whoever is necessary and walk through this in more detail if its interesting to folks?
EDIT: One other important piece ... we think we may actually have some paid features eventually, things like Metal cards, Free international ATMs etc. that would mean people using us more like a subscription service in some ways. For instance I would gladly pay a monthly fee to be able to have my bank become my financial conduit meaning that they would handle finding the best prices on loans, cars, houses etc. without me having to fill out an application (because they already know the answers) and I think others might be willing to pay for that as well
YES ... I completely plan on having a read only/API account login and more explicit permissions where you see exactly what you're giving access to but that's again a story for another day, can talk about this for hours!
Man I would LOVE to have this. I’m terrified/outraged by the abysmal state of data access at American banks. Just horrified that state of the art is giving your primary credentials to a third party.
Some companies are coming away from the dark side here. I have like 10 accounts on Mint and I think 3 of them are read-only. Coinbase is one for sure, and maybe Capital One?
> Credentials are stored in lib/credentials.json. This is checked into the repo, so USE A PRIVATE REPO. Do not use a public fork and accidentally push your tokens to it! I will not take responsibility for any potential leak of sensitive information.
It seems like using your CI provider to manage the credentials would be significantly more safe than checking them into even a private repository. For instance, there are several services that will scan or otherwise access your private repos for some purpose or another, and you wouldn't want to accidentally connect them while this is also in the mix.
A pre build step that templates the credentials into a file would be pretty easy to set up in most CI systems I've used, but I have no experience in CircleCI. (maybe 10 minutes of tinkering but that's it)
Finally I should note, I am not advising anyone to store their financial credentials with any third party. But if you've already made up your mind to do so, at least consider one that's meant to handle secrets in a secure manner.
It looks like the credentials in credentials.json are your Plaid access key/secret/etc, not your bank account username/password. If an attacker gets them, they'd have read-only access to your bank account until you rotate your Plaid creds, but they wouldn't be able to do anything as simple as logging into your bank account and transferring all your money out.
EDIT: It's actually worse than that, see comment from erichurkman below.
Careful, because Plaid also gives you access to your account's routing and account numbers [0]. I'm not sure if the way this library works gives access to those, but with a routing & account number a thief _can_ write checks, debit your account, etc.
Are routing and account numbers considered “secret”? Potentially any one of the handful of people I still write checks to could be unscrupulous and write checks, debit my account, etc., right?
I must be missing something but making fake checks, with fake banking information is a felony. What am I missing? The links don't explain how this criminal.
Yes! That’s why I recently had to go through the annoyance of changing checking accounts when someone stole the mail of one of the few vendors I still send checks to. Actually, I would have taken the risk, but my bank would not.
Yeah, good point - I figured Auth wouldn't be enabled on a free developer account, but it looks like it actually is these days: https://plaid.com/pricing.
Author here - this was just a weekend hack and didn't really expect it to show up here, but thanks for the security related feedback. The setup now uses environment variables.
Have you considered merging your functionality with a package like gnucash which works as an open source account/budget program for families/small businesses?
It lets you record purchases in your spreadsheet via SMS. I should be launching it in the next month or so. It's geared towards people who want just a little more realtime ability for their current budget without switching what they use. If all goes well, I'll be expanding to other integrations (private firefly III instances, YNAB, alternative input methods, etc).
Disclaimer: I'm the author.
Second disclaimer: Idea shamelessly stolen from a comment on here months ago about what one person does for themself.
Also, I'd be interested in finding a couple of beta testers.
Edit: Aslo, I'm sorry that it looks atrocious on mobile. Should load pretty quick though ...
this is neat! I was just thinking about something like this.
question for you... my credit card (Chase) lets me receive SMS/email immediately from them upon any purchase using the card, including the amount and the vendor. It's a very trivial and consistent text format, it would be super easy to parse.
Could your service learn this format and automatically populate the sheets from these messages? Maybe each user can have a dedicated secret email they can target (username+some_super_long_key@byobudget.com) and they can configure their personal email to auto-forward those emails to your service?
Then, unlike with mint, which syncs occasionally, you would have always-up-to-date information, right from the source of truth.
This is Awesome!! Is the first word after the price the category, the latter description? Can you quote multi-word categories or something, if you had those? Eg "Store Credit"?
I have to fiddle with it, but I'm planning on multi word categories. I'm not sure how ios handles intentional newlines. I might just change it to "category amount description" instead of "amount category description" to remove any questions of how to parse things.
Edit: Yeah, category then description. Quotes are definitely another option. I realized I got caught up and just ignored part of your post while replying...
Not sure if it's the right place/occasion to ask...
I'd love a web application that gives me a one page financial dashboard with:
- bank account balance
- credit card balance. Due date
- list of unusual transactions I should review
- all my bills in one place, with amounts due and possibility to click through to see details
- list of upcoming schedule payment (ie mortgage, gym membership, phone service) with alert/notification of one will bounce (ie: funds insufficient so I can transfer money to the right account)
Current assets/valuation as well. Kinda hack this together in Google Sheets currently with Zillow, 401K, assets, liabilities, investment accounts, etc.
If it’s self hosted do you see it as being free software or you’d pay an upfront cost for a license? Would you use a hosted web application if none of your data was uploaded?
I'd rather pay a monthly fee for something I get value from daily. I would consider using a hosted service depending who the company is. I still don't fully trust Intuit with Mint.
This would be a perfect application built on top of sandstorm.io (which has the model of individual user run apps where there is no possibility of leakage between users).
If you want more data that what you can probably get from Google Sheets easily, an alternative could be modifying this script to make it output a Beancount[0]-formatted ledger text file.
The format is pretty readable plain-text (and so you could store it on Git to track changes easily) and you can generate nice reports out of it with the Beancount CLIs and a web interface with Fava[1]. Also adds another additional level of self-hosted :)
Also Beancount has some nice support for implementing importer scripts inside it if need be, although I haven't played with those yet.
I highly recommend Beancount and Ledger-likes[0] too. Mint-style net-worth trackers have limited usefulness. Once you have visibility into your assets, you'll likely want to take more active management of them. Modelling that with spreadsheets is possible but can become complicated quickly and you'll likely end up re-inventing a lot of established accounting practices. A disclaimer: Ledger-like systems are described as whole "all bran" of personal finance - there's steep learning curve but once you get through that you'll have a powerful, life-long tool.
Are you able to find book keepers and CPA's who can deal with this easily? How do you sync data with them and control access? I have a contractor that helps me with invoicing, a book keeper, a CPA, and a partner. Not sure how this would work.
Yep, fava is a very nice base for viewing finances. Currently trying to see how to extend it further for custom reports, though just using its custom links feature can go a decent way.
Also, have heard of some folks using services like tillerhq.com then exporting to a plaintextaccounting tool to presumably help with dealing with messy data that comes from financial institutions. I just download and parse the csvs, using a mix of browser bookmarks and katalon scripts to aid in the process.
Personally think the goal isn’t to have a mint-like experience of constant updates, but to have a habit where you sit down once a month, update things and go through a checklist of actions / write down improvements.
This is a bit unfair, isn't it? Not in the sense that these practices arent awful and inexcusable from a security standpoint, but in the sense that plaid isn't doing anything that isn't standard practice in the finance industry. Every 3rd party tool that does something with your finances (mint, personal capital, ynab, and even online banks that pull info from other banks) require you to submit login credentials in this manner. The industry needs changing - and really some rather punitive regulation IMHO - but this isn't a Plaid specific problem per se.
The finance industry mainly uses OFX as the standard for online access to statements/transaction downloads (as does quicken, QuickBooks and gnucash...etc). Some banks do not support it and others go through aggregators to provide access.
You are doing people a huge disservice by identifying the companies you listed as "the finance industry". They are scraping data (in most cases against TOS) and exposing it as a paid API service.
It's a terrible state of affairs that easy access to ownership of this critical data is so hard to come by and has essentially forced people to give up their most valuable credentials to in a lot of cases unknown or untrustworthy parties.
At least mint has partnered with some banks that actually provide supported APIs for transaction data (Chase), but that is so far from the norm right now it makes me queasy.
Quicken asks me for my login credentials to my bank and in their policies, says they store my credentials in their infrastructure. How does QFX actually help here?
Again, it's dependent on your bank's support. OFX helps by not requiring remote storage of your online banking credentials.
Direct Connect - Your customer ID and password are not stored on Quicken servers. If you choose to use the Quicken Password Vault, we encrypt them on your computer's hard drive.
Express/Quicken Connect - Your login credentials are stored on Quicken-hosted servers. This makes updates faster for you.
I work for a US based aggregator and it's terribly hard to get direct API connectivity with Financial Institutions. Only big banks like Chase, Wells Fargo provide it and even then you need good business relationship to get them on board.
https://gnucash.org/ is a mature project that allows you to manage expenses & assets, perform analytics, and doesn't require supplying personal information to third parties (plaid in this case)
Download an export and import it into gnucash. It remembers previously processed items. It's a manual process, but much easier to do a monthly export than record items one by one.
If you want to do this without giving your data to any third party (plaid will store your data) or you have 2FA enabled on your FI accounts, try this: https://github.com/skhavari/cheddardog.
It supports a few institutions. If you're a developer, its pretty easy to add more. It generates spending and balance reports, and pretty easy to slap an app (E2E encrypted of course) or local only web UI on top of it.
I'd be enormously hesitant in relying on a B2B product for such an important part of my life. Forget the security concerns, my concern is the company cutting off access for spurious reasons.
If they could sell me access at a price less than $500+ a month, I'd be more inclined to trust them.
This is a nice kickstart for something that has been on my todo list for a while.
Cashflow forecasting is important for my family budget and my small business. My cashflow fluctuates quite a bit. Some months I'll bring in 90k, some months nothing. I have really high expenses for such a small business because I travel and pay contractors.
So far I've been doing this by hand, and trying to use features in Xero, plus a few expensive tools online that integrate with Xero (which do not deserve a free advertisement so I won't name them). I finally figured out if I want cash flow forecasting done right for me I'm going to have to do it myself.
This is great -- I'll be tying this in to my budget calendar spreadsheet that I've been using for the last few months to have a solid grasp of how purchases I make will impact me in the future; I have a template with some example usage of it available at https://docs.google.com/spreadsheets/d/12mkQzq22eM8HHdJVJ_Jm... if anyone is interested.
> You will first need to sign up for Plaid and apply for the development plan.
I guess I'm not the target audience. The only reason I would even consider building something like this is so that I'm in full control. I have no idea what Plaid is. All I know for sure is that I don't want to sign up for it and also introduce a major dependency. I don't understand what the advantage is over Mint. I get to do more work and not be in control. Usually you can only get one of those.
Plaid is an API for banks, basically: https://plaid.com. It's pretty fantastic (especially compared to the way you used to have to go about it.)
As for advantages over Mint, I'm sure Mint has updated recently, but as of ~2 years ago, if you had a bank that didn't expose an API, Mint would take your credentials and more or less scrape the bank websites to get your information into their systems to do their voodoo magic to show you your stats. Plaid has done most of that legwork for you (interfacing with banks) and exposed the data via a nicer API, so you don't really have to worry about anything being wrong because of scraped data -- it's coming directly from the source.
That's my understanding, anyway.
As for how is this better than Mint? No idea. It's just a demo that shows that you could, if you wanted to. And Plaid is more or less the new standard for interacting with bank data, at least in the US.
I'm doing something similar to power an e-ink based display in my house. I see about 24 hours of delay on the transaction API using the free tier which is the only shortcoming of this approach. The Plaid API is very nice.
Does anyone know if the production (non-free) level reduces the delay?
In production, the delay I'd expect is about six hours, but for certain institutions it'll be much higher. Pending transaction support doesn't exist for all institutions, so in the worst case you could swipe your card on a Friday afternoon and not see it in Plaid until the transaction actually clears early Monday morning.
connect to your financial institutions to generate access tokens
Are these generally offered across EU? I know I haven't found a way to get access for a couple of my local (EU-based) banks I checked with, but unsure about the rest of the continent.
In Europe PSD II will come in effect in Q4 this year. By then all banks need to offer account information and payment initiation functionality to psd registered parties. This has spurred banks to implement more than just the required apis and most of them use some form of oauth/accesstoken for authentication. It is only a matter of time before there will be an aggregator site where you will be able to see data from all banks. There are even banks that already show accounts from other banks as well.
If you are looking for a more "packaged" integration with Google Sheets, check out BudgetSheet: https://www.budgetsheet.net
It is a Google Sheets Add-on found in the G Suite Marketplace.
Disclaimer: I am the author. It is currently under review by Google and awaiting publication. There is an email sign at the bottom of the page if you want to be notified at launch. :)
Me. That's the point: I just write the scrapers for the sites only I need, which isn't a lot. And if they change, I simply fix it. I'm willing to bet Mint and others have done exactly this (or are doing it now) as well.
Pretty sure they do, because AFAIK there's not much choice in the US. I think it's therefore forced to use screen scraping (or other APIs not intended for public consumption).
In line with the rest of the US's financial technology (EMV adoption, 3-D Secure) which is pretty immature, I don't think there's a concept of Open Banking enforced by regulators. Hence, no OAuth based flow to use and no standardised transactions API.
Open Banking isn't ideal and has its own issues. It's not anywhere near as revolutionary as some 'thought leaders' make it out to be. But it's better than nothing.
does google really need all your financial info too? could you not do this in https://ethercalc.net/ or even just a csv that you could import/export to a spreadsheet app of your choice?
the appeal of a personal mint is to not share all my financial info with third-party companies, not to keep from paying a few bucks for a useful service.
It's called a "private" repo, not a "secure" repo.
The point of a private repo is to give you control over your collaborators. It is not to provide information security. To my knowledge, Github makes no special promises about the security of private repos. The data is not encrypted on disk at Github, and Github staff can access the content of your repo.
I WOULD NOT UPLOAD SECRETS INTO A GITHUB PRIVATE REPO.