At this point AWS has gotten so complex that one has to jump through a dozen hoops to get the simplest of things done.
To scale up our reporting I thought it’s a good idea to offload report generation to a lambda function. Read from MySQL, dump into S3 and mail a link to that S3 file, make the link expire after couple of hours. Reasonably simple enough. But by God was it painful!! IAM is a rabbit hole, now later VPC restrictions on top and couple that with terribly organized document it makes no fun at all.
I get the need for all those layers of security but without well organized document everyone has to go through the ordeal. What we need now is plethora of how-tos for these common architectural patterns.
Sometimes I wonder if the old world where you had to buy computers and drive to the datacenter at 3 in the morning to replace dead hard drives was really so bad.
I guess it was really so bad. Everything you have to build for yourself in the cloud is something you probably needed in your self-hosted stack -- a directory of users, permissions (and escalation), firewall rules, SSH key material, etc. I feel like no matter what production stack you choose, you're always in that "6 of one, half-dozen of another" state. Lots of stuff to interfere with getting your code out there and keeping it running.
(My real question is why the cloud providers don't offer these things for you -- they give you the tools to build solutions, not solutions. But I just want to work on my app. I wonder why things like App Engine never really caught on. I guess being opinionated drives more people away than it brings in with the promise of more simplicity, though to some extent App Engine was too simple, which is always bad. Meanwhile, what cloud providers offer certainly lets you bring your own opinion and build it out yourself. I wish there was a comfortable middle ground.)
Haven’t used it in years, but Heroku was really great in this respect. It was very easy to just focus on making a good app instead of trying to configure everything myself.
I guess there’s probably not much demand for this kind of thing at large scales.
Heroku can be a hard sell to mgmt because of the cost, but if it's framed in terms of "this means you don't need to pay for two AWS cloud architects to be on call" then it's a much easier pitch.
>they give you the tools to build solutions, not solutions.
Hmmm, well the first generation of a cloud providers offerings has to be the bare necessities (EC2, VPC, IAM RDS etc). Then you can start providing a second gen of services that use the first gen, like AWS's ECS, and then a third gen like Fargate.
LOL. That is not a sketch. I swear it is real life in the back end world. The good news is that now we have Micro front ends for the enjoyment of the UI engineers.
The lack of originality hurts I guess. What was more biting was the faux productivity, tech addiction, and social isolation and alienation of such a mundane daily routine.
This mirrors my experience working with Lambda's - I went the "deploy docker images as lambda's" route and that was mostly OK (I mean it's just node/sharp etc) but the actual setup up - good god, I swear some of that documentation if read aloud would summon Cthulu.
If it's of any help, I can promise you that creating a Terraform configuration to set this up will improve your life, happiness and general well being in the future.
Yes, Terraform has a learning curve, but it's also a black magic effery for the cloud.
It can be if you haven't tried any other of the big cloud vendors. In my experience, it's one of the best documented services I have ever used, especially considering it's so vast.
I always tell people "stay off AWS unless you or someone you're paying very well on your team holds an Associate-level certification at the very minimum". It's easy to get in over your head over there.
> you or someone you're paying very well on your team holds an Associate-level certification at the very minimum". It's easy to get in over your head over there.
I will never forget the story about the expert Amazon engineer that lost 80K USD.
A good starting point is elastic beanstalk, at least, but yes it is a big tangled web and one wonders why one is trying to copy netflix for a simple web app most of the time
if yoyr workload isnt simple though, then its time to pay the price until a better abstraction comes along ^.^
There really ought to be sandbox "default" settings or templates that Firebase has for noobs, instead of configuring everything change by change. I still dread configuring IAM and now just resort to using one big EC2 instead of multiple services.
I totally agree with you. A part of my duties is setting up logging from AWS to Splunk and there's so many gotchas and stumbling blocks that's it's infuriating to even attempt and follow best practices.
A great example is Cloudtrail. The best practice is to send your OrgTrail (an org-wide Cloudtrail) to a separate account for security reasons. Cool. Okay, sounds easy enough. The Splunk docs are useless for AWS, so consulting with YouTube, Reddit, etc is the go-to for this.
It's so much easier to just leave the OrgTrail logging into the management account it's not even funny.
>IAM is a rabbit hole, now later VPC restrictions on top
My idea is to build simpler system on top of cloud networking and IAM/permissions.
Why would I want to have networking between components that are permissioned to interact with each other, and the other way, why some app/lambda's service account have permission to interact with particular database, but be blacklisted on firewall? Have GUI on top to drag and drop "routes" between apps.
Of course, make it optional etc, but I think it would save tons of devops time for smaller organisations without audit requirements unlike fintechs, medical stuff etc.
Totally get what you mean. I'll pitch for either terraform or AWS CDK as good ways to make some of this plumbing less painful (they both have you describe the end state you want and then generate resources to get there).
Terraform is—as a colleague described it—a combination very-sharp-knife and fully-loaded foot-seeking gun. Very powerful, but also requires a sufficient amount of respect and care.
e.g. You’ve defined a nice, fully-functional end-state, which was generated n time ago, but now you need to make a change to that state, so you run your
terraform plan
…and it wants to destroy your entire infrastructure and recreate it all from scratch just to e.g. add a new hostname to the certs generated by certmanager.
That may be okay, but you should probably measure at least 3 times before you cut.
Hi, author here. I'm so happy to see this appear on HN front page! I almost didn't post this on HN as I wasn't sure if there would be interest towards a site like this. I spent so many hours tinkering with the UI and crafting questions that I'm happy to see that people are finding value in it :)
I’m curious, did you build the flash card UI yourself, or use a library? I’m not in the market for studying AWS skills, but I really like the flash card interface!
This is exactly how I studied for hard (theory-heavy) exams at the university! If this would cover >90% of what can come up on the certification exam, I would be willing to pay! Please take my money!
AWS Professional Services here -- we all study for our cert exams using the site acloud.guru. They're not always perfect, but generally pretty good and handy for spot study. May be a good choice if you want to go deeper than these flashcards. AWS also offers in-person training sessions that are quality, but I'm not sure what the status of those are given the ongoing pandemic.
Thanks for posting this. AWS is so complex, there are so many products on it that it's often overwhelming to know where to start to learn it, so this is very helpful.
Well done! Thanks for those who needs to pass AWS Certs.
But on another side after checking few flashcards it seems you will turn into an AWS Marketing Parrot, more than really learning skills (Eg: question about AWS Shield)
Very nice. My team is prepping for AWS certs at the moment, and this is helpful.
Passing the exam does mean acting like an AWS Marketing Parrot. That is an unavoidable aspect of it. The exam is AWS certification, not "general IT skills". So I would not call this a flaw of your card content. Your cards show an awareness of this. Your card about SOAP APIs points out that the correct exam answer is technically wrong.
I would strongly suggest that you NOT take hliyan's suggestion of making a "how-to" version of the slides. "How-to" is very different than "pass exam", it would be very difficult to successfully address both needs.
I respect Hhliyan's need (and share it myself often enough); I respect it enough to suggest that it gets addressed in a way which is likely to satisfy it. "Pass the exam" flashcards have another purpose.
In conversations recently around technical hiring I've found a heavy emphasis on "AWS" knowledge rather than infrastructure know-how or proven experience building on top of any stack.
Sometimes, people don't even know what AWS is except that it is needed by their product. They need someone "strong in AWS." They don't know what AWS products they use or what they do.
Separately, I have had a very senior developer convey a sense of respect for candidates that know AWS things that this developer does not know. I was surprised that the developer would give a pass on any subject so easily.
There seems to be some sort of an implied belief in ability simply because the person can recite Amazon's branded solution names and have configured these things before. In part, because the interviewer doesn't know the buzzwords, they can't probe for the actual depth of experience.
It feels almost like a "no one ever got fired for buying IBM" situation. And on the candidate side a "can't beat em, join em" by being able to recite AWS jargon, regardless of how well you can build and choose solutions to fit business needs.
I don't want to hear people tell me only how to solve problems with AWS's legos. Sure, we use them - and sure, knowing them will help you, but those tools are a crutch that is going to prevent you from solving tough problems if it's the only thing you know. You don't want to be stuck on a problem just because it's not in Amazon's toolbox yet.
But on another side after checking few flashcards it seems you will turn into an AWS Marketing Parrot, more than really learning skills
I have a feeling that's partly the intent of the whole certifications program. I'm someone who has no use for AWS (nor interest in any cloud stuff, really) and has never even looked at any of their services in detail, but was once challenged to try one of their practice exams, and recall getting about 75% of the questions right simply by educated guessing with a perspective of "which of these answers seems to make sense and would make the most $$$ for Amazon?"
I have to agree. While the cards themselves are a useful format, I wish this had more "How do you..." rather than "What is..." questions. E.g. Q: "How do you provide temporary credentials to IAM users to access some AWS resource?" A: "STS"
Thanks for the feedback! I'll take it into consideration when I create more cards (AWS deck is done, but I will create Azure deck next). I tried to strike a balance between different question types (e.g. "what", "how", "why", "compare", "which"). I thought that this variation in question format is nice when you're plowing through a lot of questions. Maybe I have too many "what" questions. Those are the easiest to think up and write down.
That's great. That way it can cater to both those who are looking to get a certification and those who just want to apply the skills (like myself). Another thought: would you be open to converting the content from HTML-in-JSON to markdown (one file per card, perhaps?). I wanted to fork your repo and create cards for various different skills, but the format kind of put me off.
Try to fork the repo and run it locally. There's a button for creating cards (button appears only when running it locally). It's still HTML-in-JSON, but you get instant preview while editing, and I think it's a nice way to format cards. HTML offers more flexibility in formatting than Markdown does. That said, I totally understand you might prefer a different format, and if you want to work on it, I'm sure you will be able to replace the HTML-in-JSON format with Markdown.
Thanks for the feedback! You're definitely right that some of the questions are simply parroting AWS marketing points. I tried to optimize the question set for passing Associate level AWS cert exams, and unfortunately those exams require you to memorize certain AWS marketing crap.
The lower level certs are easier to pass, and you might end up sounding like an AWS Marketing Parrot.
The higher level certs are the real deal. You will work your ass off to get them, and I don’t think flash cards will help much. But there are sites out there that help you train for the test, and they can also give you sample tests to work from. If you can pass a number of those sample tests with good scores, then you’ll probably do okay on the real tests.
Acloud.guru is used internally with Amazon (and is free for anyone who has an @amazon.com e-mail address), but it’s not the only option out there. John Bonzo has some of the highest regarded AWS training videos in the AWS Training community, and Amazon SDEs have recommended to me the freecodecamp.org videos as well.
AWS is increasingly a standard that people need to learn to get employment, and the risk of a huge bill because you have misconfigured something somewhere will put off a lot of people, particularly students, from learning it in their spare time in hobby and school projects.
It sounds like the machines plan to abstract human involvement over the long term.... first you change the way they think, then it’s your space of thought to commandeer and command.
You hear horror stories of students/hobbyists playing around with AWS Lambda or some other service and ending up with huge bills. Maybe they get resolved but I certainly don't want to deal with the panic of misconfiguring some setting and losing a chunk of my savings. I'll use this stuff at work if needed but it certainly cools me on playing around with it in my spare time, which is how I best learn new tech.
This happened to me once and AWS entirely reversed the charges no questions asked. And its really something you don't need to worry about. AWS makes more than enough money off of legitimate traffic they don't need to deny someone asking for a refund for a genuine mistake.
And playing around with Serverless is awesome. Things can be created rapidly and very little maintenance. These days I cannot see myself building an application without at least some element of Serverless infrastructure if not the entire solution.
OK, but I'd rather not have to make a panicked call to AWS customer service and throw myself at their mercy just so I can do some coding in my free time, thanks.
And think about how that applies to some student at college or boot camp on a tight budget with rent/bills to pay.
Actually I see flashcards differently. I cannot agree with the "everything Flashcards" idea, it does not work for me, and I can't imagine learning from cards.
With flashcards, I can create an interconnected map in my mind, it supercharges learning, provides structure, makes digesting new content much easier. That ready cache of information has proved immensely useful.
Using just the tools of logic and reasoning is incredibly slow, error prone and exhausting. Especially in a time constrained and pressure setting e.g. a production issue, a discussion with people having differing opinions, a test or an interview.
Flashcards take out a lot of friction from knowledge retention.
You don't need flashcards to build stuff. You can learn it as you go.
You need flashcards to pass exams and interviews, when you need to learn stuff a lot quicker than you could by learning as you go. And sometimes speed is important in life.
Flashcards are orthogonal to whether something is simple and logical.
Hello David! When I posted it had been a while, but I just replied to you on that-forum-not-to-be-named about an Ankidroid issue. The internet sure is making the world smaller.
Content of the cards would probably be easy to convert to other formats, but the cards are formatted with (a little bit of) HTML, so it might take some work to get the cards to look nice in Anki.
I might be too late to not get lost in the comments, but I did notice this flashcard:
> How can you establish a secure connection from your office or on-premises datacenter to AWS?
> Connections to AWS resources are typically encrypted by default (HTTPS, SSH, etc.)
> You can establish a Site-to-Site VPN between your VPC and your on-premises network.
> If you do not want traffic to cross the internet, you may be able to establish a Direct Connect connection, which is a dedicated network connection between your on-premises network and an AWS edge location (from where traffic is routed to your AWS resources using AWS' internal networks).
I want to note that Direct Connect traffic is NOT encrypted by default and this is common misconception, both in real life and in exams. You must combine DX and S2S VPN.
> I want to note that Direct Connect traffic is NOT encrypted by default and this is common misconception, both in real life and in exams. You must combine DX and S2S VPN.
This is true. However, I'm not sure how to improve the card. The examples of "typically encrypted" connections on the card were HTTPS and SSH. If you SSH to an EC2 instance over a Direct Connect connection, your connection is encrypted (because you are using SSH, not because you are using Direct Connect).
Thanks for sharing this and congrats on the product.
BUT: I don't think any amount of flashcards can help me get my head around setting IAMRole's and such, I stay away of amazon's cloud as much as I am able to, when/if I have a choice in the matter.
You would be absolutely amazed how effective flash cards and spaced repetition are at cramming arbitrary information into your head. Once you have definitions and jargon in place, it's significantly easier to learn the concepts and how things go together.
I have been building my own SRS flashcard web app, targeted specifically for language learning. Your implementation, UX-wise, is similar to mine (e.g. I have nearly identical "flip" transition"). Your use of icons is nicer, and the color scheme you chose is excellent.
I like the trade that you made of not having persistent accounts. I wrestled with that idea for a portion of my site that features "pre canned" decks. Without looking at the implementation I suspect it could then be a completely client-side solution served as static files.
Why not just use Anki? It's mature, and plenty of decks exist for language learning. I've been using it for ~4 years or so now and nothing else has really come close in terms of features or stability.
Thanks! By the way, you can use client-side user accounts for custom decks as well. Just save user cards to local storage and allow the user to import/export their cards to a file by clicking a button. (In my opinion, local storage alone provides sufficient persistence for most user data, like which cards should be on which piles, but if the user goes through the trouble of creating a custom deck, we want to offer import/export as a backup feature, and also to be able to move decks across devices.)
I found the flashcards wordy. However, I feel they need to be even more wordy. Acronyms should be resolved, at least once, per card (preferably on the explanation, to aid teaching the acronym)
I know what you mean, I've been going back and forth with this. In some cases I've spelled out the entire acronym, like rarer acronyms, but with some common acronyms it's not worth it. For example, "S3" is "Simple Storage Service", and anybody who studies for AWS cert exams will learn that really fast, so it's not worth it to repeat it on every card. I think a nice solution would be hover tooltips to spell out acronyms, but it would take some work to get that done without breaking all the other functionality that's related to hovers.
Some part of me wishes that programming and stuff ought to be of enough complexity so that only serious programmers lead the tech stuff.
Too much of programming education / online stuff is muddled with millions of media items that cater to absolute beginners.
I rarely come across deep, insightful articles that discuss at-least one level beyond the standard stuff. Any search for any programming related question leads to articles that start from installing Node / Python.
As someone who's produced a large volume of educational content in my career, this idea ends up producing content that even most serious practitioners cannot consume.
The problem is that even amongst "serious" programmers people often have a patchwork of knowledge with small, unpredictable gaps in understanding; many of which will surprise you. You will find these gaps even if you've taken the time to handpick vetted experts. Once you account for this patchwork of knowledge gaps in verified experts you'll often notice that you're really close to a tutorial for relative beginners, which most people are, and just go the full distance.
The problem generalizes to a variety of subjects but is particularly pernicious in programming where small details are often of much greater significance.
There are far more beginners than people with enough knowledge to care about anything beyond that.
And that problem only becomes worse the further you get into these topics.
When you add to that the fact that anything specialized will become outdated very quickly, incentives are not aligned for there to be a lot of deep, insightful articles about anything.
There just aren't enough people who would be interested in it to make it worth the time someone would take to create it.
edit: And of course, StackOverflow is a great place to find these types of things because it serves as a watering hole for the types of questions and discussions it sounds like you would be interested in.
I'm a developer with some experience of AWS, about to start studying for AWS Certs, and looking for good materials. These look amazing - thanks so much!
Flashcards are my favorite way to learn - specifically to pass a test or gain some certification. I've standardized on a simple format of a google sheet with column A being the front side of the card, and column B being the back - and found a few decent iOS flash card apps that allow me to link to a google doc. So, do you have the whole list of questions in a consumable format I could get into a sheet?
Oh, nice! I'm happy to see people importing the content to other apps.
(By the way, the README explains that the MIT license is for the code, not the content. Your excel gives the impression that the content is MIT licensed, and it is not. To clarify: I'm happy to see people copying the content to different places, I just don't want MIT license attached to the content, because I don't want people to make money by selling my content. Distributing the content for free is fine by me.)
Anki seems to be the gold standard. It has desktop, web and native mobile versions and it synchronizes across devices. The interface isn't anything fancy... but it doesn't really need to be for the app to be effective.
Ive been using AWS for a couple years at work now.. never ran into difficulties where flashcards/spaced repetition woulda helped. For example, I don't think you need to memorize IAM privileges or stuff like that. Maybe I'm just working on not so complex things?
This flashcard set is mainly intended to pass certificate exams. Spaced repetition with flashcards is an efficient method to memorize and retain information. When you're actually working on stuff, you don't need to memorize information, because you can google stuff and learn as you go. When you're doing an exam, you can't google, so you need to memorize information.
This would be fantastic to add to a DevOps wiki/forum that I've been wanting to make for ages. I just never get started because I want it to be perfect and can't decide even what to name the damn thing, much less how it should be organized
This is an excellent idea and I look forward to browsing it when you start! Also as you’ve already conceptualised as a wiki - don’t worry about organising it perfectly, you’ll never figure it out yourself. Let the community (which you will bootstrap by contributing your knowledge) do that for you. Oh and just “TheDevopsWiki” is a great name.
I really like this app. I only have one minor piece of feedback, and that's that the card wobbling while you're reading the question is very, very distracting.
Thanks for the feedback! The wobbling effect is supposed to resemble the feeling of holding a physical card in your hand. It's supposed to wobble only when you move your mouse from left-side-of-card to right-side-of-card, or when you move your mouse from outside-the-card to over-the-card. Let me know if it's wobbling in unexpected ways on your device.
In any case, I'm willing to remove the wobbling effect entirely if this feedback is echoed by other people.
It's acting "correctly", it just turns out that I tend to move my mouse around the center of the card while I'm reading it, which makes it super obnoxious.
Actually, I just played with it some more, and you know what it is? My brain knows the mouse is controlling it, so while reading, I'm unconsciously trying to find a middle spot to put the mouse in a vain attempt to make it sit flat, which has the effect of causing nonstop wobbling.
It's like it's designed to irritate me. :D
Just don’t use AWS. There are other options out there that do not lock you in like this. Adding environment specific complexity that is tangled to AWS is to avoid by all means.
To scale up our reporting I thought it’s a good idea to offload report generation to a lambda function. Read from MySQL, dump into S3 and mail a link to that S3 file, make the link expire after couple of hours. Reasonably simple enough. But by God was it painful!! IAM is a rabbit hole, now later VPC restrictions on top and couple that with terribly organized document it makes no fun at all.
I get the need for all those layers of security but without well organized document everyone has to go through the ordeal. What we need now is plethora of how-tos for these common architectural patterns.