Lack of security? AWS offers very granular, per index, authorizations that are tied into IAM in the same way you would configure S3 or DynamoDB. If user's are failing to implement good policies, AWS is not to blame.
Haven't had time to productise it yet. I think doing this makes you quite a bit safer, because it means you don't end up giving up and allowing more than you need. However, you still need to understand which actions shouldn't be allowed, so it's not the whole solution.
Netflix open sourced a similar tool that watches API calls for a Role and then suggests minimum privilege changes to the attached policy document: https://github.com/Netflix/repokid
That's interesting. That can only work if there's some way of introspecting permissions - which I didn't realise existed. Mine works by experiment. I wonder how fine grained their way is.
If you don't have prior experience in networking or permissions, it will take some time for you to understand the concepts to properly secure a standalone server. The same concept applies to AWS. You are paying for the hardware, not someone to hold your hand through the process.
And if you can't figure security out by yourself, pay someone to hold your hand.
>And if you can't figure security out by yourself, pay someone to hold your hand.
This. Security is as much a tradition as it is a set of technologies. Its better to learn from a master than from a costly mistake, and its better to learn how to do it rather than to pay to have it done for you.
I feel like this attitude (which is very common) holds us back from developing more reliable systems. When something fails, we don't ask what we can do to improve the system.. instead we point the blame at users. It's the easy way out.. instead of designing better systems, we just tell the user to 'do better next time'.
The more difficult a system is to use properly, the more we should demand an alternative. If your users keep making the same mistake over and over again, then at some point you have to start asking yourself what you need to improve.
On the other hand, I always go back to something my grandpa (who himself was an industrial engineer) used to say: "It is impossible to make things foolproof, because fools are always so ingenious."
Unlike every other AWS managed service I had used up to that point, when I was using Amazon ES a ~year ago there was no integration with any sort of VPC offering, and there was no clear published guide on how to establish such a connection. I ended up doing so with a hacky bastion-based architecture, but most other teams I saw using ES at the time just didn't bother.
I don't think you're wrong, its a complete offering- and yet:
- If you want to ingest data with Kinesis Firehose, you can't deploy the cluster in a VPC.
- You can enable API access to an IP whitelist, IAM role or an entire account. You can attach the policy to the resource, or to an identity, or call from an AWS service with a service-linked role. That's all good, perhaps a little complex but as you said, nothing too different than S3 or DyanmoDb, except for the addition of IP policy. Why not security groups? Is DENY worth the added complexity?
- However, you can't authenticate to Kibana with IAM as a web-based service. Recently they added support for Cognito for Kibana, otherwise one would have to setup a proxy service and whitelist Kibana to that proxy's IP, then manage implementing signed IAM requests if you want index level control. Cognito user pools can be provisioned to link to a specific role, but you can't grant multiple roles to a user pool, so you have to create a role and user pool for every permutation of index access you want to grant. You also have delegate ES cluster access to Cognito, and deploy them in the same region.
All told, even a relatively simple but proper implementation of ES+Kibana with access control to a few indexes using CloudFormation or Terraform would require at least a dozen resources, and at least a day of a competent developer's time researching, configuring, and testing the deployment. Probably more to get it right.
Ultimately there is nothing wrong with the controls AWS provides, but plenty that can go wrong with them.
Check out the 2nd to last line of that post. They make the same statement in doc. Lots of services are getting VPC endpoints so traffic never has to hit the public web, but firehouse isn’t one of them(yet).
