For anyone wondering what Ansible is, or what is allows you to do, here's a basic overview screencast I put together [1]. Cannot recommend it enough, as the barrier to entry is simply so small, that you can get up and running in no time. The 250+ built in modules are also a killer feature, no hunting around for added logic to make things "just work" off the starting line. I kind of think Ansible hit a sweet spot, coming to market after Puppet/Chef, in that they learned from the ecosystem, and adapted their tool for the pain points. There is also no reason you could not use Puppet with Ansible (rolling upgrades, quick patches, etc).
Happy to answer any questions too.
PS. Bit of a disclaimer: The screencast is a four part series, the first episode is free, and the remain three parts require a subscription. Did not want to bait and switch you. The first episode will give you enough to know what Ansible is all about though.
Is there a better way to do this? It would be nice to be able to "call" a role/task with parameters, to be able to reuse common elements like templates and task logic.
I was somewhat aware of dependencies, but they don't really do what I want.
I want to be able to execute a role (or task within a role), much like calling a function, that will be executed at the point it is called (dependencies are all executed before the role executes.
I may also want to call the "function" multiple times within a role, with different parameters, to, for example, deploy a template several times to different named files. And yes, dependencies can execute roles multiple times, but this is business logic that belongs in the main role file, not in a meta file. And from what I can see, dependencies can only call a role as a whole, not a part (or task) therein.
Basically what I have with "include" does exactly what I want, but it would be nice if it was a bit more explicit that I was calling a role component, something like:
Where callrole::conf corresponds to roles/common/tasks/deployconf.yml. This might allow for more powerful constructs than is available by simply using include.
IIRC, they specifically removed this functionality a few versions ago. Their stated reasoning was never clear to me. Something about a lack of clarity regarding which variables passed through how.
It might be possible to write a plugin of some sort that adds the functionality you want.
I am not a Ansible dev, so take what I say with a grain of salt.. but BadassFractal has a good suggestion re: pass parameters to roles! Also, there is nothing wrong with what you are doing, especially if these values don't change very often. My rule of thumb is to, go with whatever makes it easier to read / understand, and removes the magic. Are you going to figure out how vars x,y are set at 3am for a troubleshooting session.
Is there a reason why Ansible fans feel it's important to hijack unrelated DevOps tooling articles on HN and chime in about how awesome they think Ansible is? It just seems rude, and reinforces the image that the Ansible community is elitist and intolerant.
I'm guessing there's a glut of such comments because most people comment with their first gut instinct to a topic, and a lot of Ansible users' first gut instinct when they hear "Puppet" is "Ooof, glad I switched to Ansible."
This particular link is sort of hard to comment on, because it doesn't really say anything about what's different in Puppet 4.0. So there's not really a way to comment on the substance.
Same here. I've used Puppet for years and I've even written a book on it. I dropped it completely within days of using Ansible, to me it betters Puppet in just about every way I can think of.
Yep, in my experience, Ansible is less fragile, more flexible, easier to get started with, easier to scale, and easier to learn. It has its frustrations, but they feel minor compared to what Puppet and Chef put us through.
I'm contemplating switching to Ansible as well after 3 years of using Puppet for my personal and company infrastructure, I feel that Puppet has become too complex and that I use no more than 10% of its features.
I've kept an eye on this space---automation is not my day job--and I have to say that I still cannot put my finger on Puppet or Chef or Salt or Ansible.
Automation is my day job and I'm still not convinced any of them is a significant improvement over a well maintained set of shell scripts and Makefiles.
Same here. I recently found out about Babashka [0] and Declarative Bash [1], but I've written something similar already. It's much easier to find talent that knows Bash and can learn a readable library of Bash scripts within a day vs finding talent that knows the flavor of the day (every year people are migrating from the old favor to the new - Bash => CFEngine => Pupper => Chef => Salt => Ansible => ... => Bash).
First I thought this was some dialect of Rexx[1], but then I saw: "Easy to learn, it's just plain Perl". A well. No thank you, I'll stick with ansible. Not that I think Rexx would be a good idea either, it just reminded me of ARexx for AmigaOS...
[ed: forgot to ask: you find it easy to find talent that can not only learn some bash script library, but write new code without introducing some gaping hole somewhere?]
As I mentioned in another comment, I'm not suggesting using m4 with/in-place of ansible -- rather as a compliment to the shell. I suppose one could use jinja2 stand-alone with the shell -- but then you get the python dependency etc... I meant that m4 might be a better fit to go along with bash/ksh.
If we're discussing templating languages, I definitely prefer Mako over Jinja2 and think it's much more appropriate for devops templating. But Jinja2 is the default in a lot of projects, including Saltstack and Ansible, and it's hard to fight the currents all the time.
If you're going to use shell and not ansible, then why not use shell with m4. I'm not suggesting shell+m4 is better than ansible, or that ansible+m4 (wtf?! right?) would be a good idea.
Just why not use a readily available template/macro language along with shell scripts, if you've decided to use shell scripts.
I suppose you could use jinja and bash -- but why pull in two full interpreters as a dependency?
Ansible is a good simple tool with a small learning curve so it's not very intimidating for developers. But it has its strengths and weaknesses just like everything else. Puppet, Chef and Saltstack are all fantastic tools. It all depends on how you use them.
So why ansible? I use puppet. A quick glance at ansible suggests that all of that (certs, servers, services) is replaced in ansible with SSH, right? But you still need to manage keys. What am I missing?
You would need to manage keys whatever you did. Puppet just means that you have to manage keys AND certs, so you have double the headache.
Puppet also means you have to babysit the client service and the puppet master, and if they go down, you have to get them back up again. I deal with this problem a lot and I wish I didn't have to.
Automation is not my day job, and I am very happy that there are frameworks (I work with Chef) with communities that offer standards, guidelines and libraries for a well maintained set of shell scripts and Makefiles. Instead of having to build and maintain them all myself.
I agree that having standards and best practices is a boon, especially if they can be packaged up in an easy to use way. Like anything, Chef brings its own set of idiosyncrasies to the table. However, I don't like how it brings all of Ruby's idiosyncrasies along for the ride too.
If you're not immediately comfortable and familiar with both ruby and the quirks of its ecosystem, you end up investing a lot of time into getting a development environment up and running (and having it reproducible by other members of your team).
Most infrastructure tools, if they're any use, are rapidly evolving. Conway's law applies here, and the maintainability of Chef mirrors that of its ruby ecosystem roots - people tend to freeze dependencies rather than upgrade. That's an antipattern we don't need in infrastructure.
Honestly I just think we can do better. For me, I'm not sure (read: I really don't know, I'm not being facetious) the costs outweigh the benefits, but for someone totally new, perhaps it helps them get stuff done faster.
I've sadly learned that given enough time, you'll likely end up mostly using your own set of private cookbooks, even for configuring things community cookbooks already exist for. The quality of the community cookbooks varies wildly from "pretty good" to "toxic waste," and the likelihood of getting fixes upstream fast enough to reflect back into one's own environment is generally low, even for good maintainers.
And Chef's strict adherence to X.Y.Z semver for cookbooks (even though cookbooks don't fit semver well) and inability to declare "my cookbook foo_prime can be used whenever cookbook foo is called" makes it near impossible to maintain your own branch while you wait for your fixes to make it
upstream.
This makes it difficult to try to get things done when you encounter a cookbook that just doesn't work quite right. The cookbook ecosystem is not yet mature enough at this point in time to rely on.
I think it illustrates the difference well. I agree that you probably could do much of the same with discipline, source control, shell scripts and Makefiles. But there's a reason why we use other languages than shell scripts. Shell scripts are finky, and it takes years for people to stop making "rookie" mistakes. If they ever do. "[" vs "[[", ${var} vs $var, ksh vs bash... the shell gives you way too much rope (and I like shell scripts :-).
I should probably couch this in the caveat that I am using Packer and Terraform to control the infrastructure itself :)
So machines are effectively configured "offline", tested, and replaced. It's not enough though (usual things like databases and development tweaks like config changes are problematic) so I'm back into the world of config management to see how it can help.
Matt makes some good points, and we broadly agree on controlling infrastructure, I just think that most of the CM systems are inadequate abstractions - they don't go far enough, and it leaves them largely as convenience wrappers around shell, but ironically enough, usually without the debuggability of shell.
I've worked with Chef for years, and recently have started seeing what I can do with Ansible. For me, the major thing in Chef's favour was the combination of attribute driven configuration and integration with things like Test Kitchen and ServerSpec.
Chef requries buy in to the ruby ecosystem though, and that's just not something I want to expose non-ruby developers to (it's just too brittle), which is why I'm exploring Ansible. What I'm finding, though, is that for any reasonably interesting tasks, I'm still forced to execute "command" strings, which is not an improvement over shell, and gives people the false impression that it's idempotent (or even just safer).
Also most of these systems are trying to upsell me a centralized management system and UI, handing over the keys to my infrastructure, which is just not something I'm willing to do.
Edit: I'd also add that cross-OS portability of CM (if that's what you're referring to), in my experience, a bit of a red herring.
I think the biggest issue with current CM systems is that they aren't made by people with any experience actually administering large scale infrastructures.
The focus to me seems all wrong. Ansible has a blue million modules, but so many of them are incomplete or downright useless. The cron module can handle cron.d files, but it still tries to manage individual lines in the file and tracks state with comments. It can easily get confused and add multiple entries for the same cron job. Puppet had the same problem. The solution is obvious: just make it barf a pre-made file into /etc/cron.d. But why does no CM actually do this?
Over the last year I have spent a lot of time using Ansible, and I've eventually given up on the cron, sysctl, service and get_url modules and resorted to running commands or copying files. This isn't very clean and it can be brittle, but the alternative of fixing the modules is off the table. The debugging or logging support in most modules is literally non-existent.
There also seems to be a pretty serious scaling issue with the way Ansible processes its inventory, even when using the (absolutely essential) _meta optimization that provides the entire inventory in one command. This is at only a few hundred hosts and tens of thousands of variables.
Contrast this with Puppet: while there are a great many things I hate about it, the dependency system and ability to define your own types inside of the DSL was pretty powerful. Too bad that you eventually had to get down to writing Ruby modules to do certain things. Also too bad that they built it as a client-server system, allow you to export resources between hosts, and took forever to make the execution order deterministic.
The final nail in the coffin for CMs and me is that Ansible (along with Chef, Puppet, Salt) try to do so much but have no concept of absolute state enforcement. The host should always either be in state A or B; the CM system should be able to move in either direction between A and B; and it should never be left in a state inbetween A and B. With Ansible it's quite possible to get permanently locked into a middle state, since it may die before handlers are run (which are then never triggered in subsequent runs).
Building these tools is admittedly harder than it looks, but the fundamental building blocks of a CM system aren't that tough. So many recipes boil down to literally a three step process: install package, copy config file (templating is super nice), start service. Stop trying to play nice with local modifications, the whole point is that I want you to enforce state absolutely -- local modifications are the enemy!
I wish there was a CM that focused on getting this stuff absolutely right and then made it easy to build on top of that.
Automation is also my day job. I've got a ton of shell scripts and a ton of Ansible playbooks. No reason they can't work together.
For some tasks, shell scripts are better/clearer/easier, and you can call those scripts from Ansible. For other tasks, Ansible provides better tools, and hey, you can call Ansible from your shell scripts!
I've added a few ansible-friendly options to some of my shell scripts like adding an --ansible-mode flag that then outputs just a one-line "OK" or "Changed" message that then I have Ansible parse to determine if something changed during the run. Or if you're more ambitious you can just turn your shell scripts into Ansible modules by having them take arguments in a slightly different way and output JSON.
A free templating system (Jinja2) and cross-platform capabilities are nothing to shake a stick at either. Not to mention, Ansible does a good job of helping organize your hosts and parameters in one place, and forces you to modularize your "scripts" (in the form of playbooks and roles) in a beneficial way.
It's definitely worth a look. You can start small, and use it only for one thing, and grow from there. That's one of its biggest benefits over Puppet and Chef, which don't really have a way to go small-scale.
I do like the "mass SSH" functionality that Ansible provides, it's something accomplished years ago with Capistrano, but I certainly prefer having it in Python (now that I've wrangled together a dynamic inventory interface to Terraform's tfstate files).
Shell is not what you want to go to for orchestration, but, essential as it is, I think it's a separate concern from managing the configuration of the machine.
You're absolutely right about being able to start small as a major point in its favour, though. Not enough infrastructure tooling accomplishes this.
I'm wondering about a possible execution mode for Ansible where instead of connecting via SSH and uploading/running each task, you dump each task in a shared volume and run them in a container with docker exec. Then you can commit the final result to an image.
I was under the impression that Packer was used to create machine images for EC2 and DigitalOcean, but apparently it supports building Docker images now. That's perfect!
I have too. After I moved to containers, the hosts themselves are so simple now. In fact, I just have a VM image I use for hosts, and then the devs make the container images running on them.
Immutable infrastructure can be implemented using sole Dockerfile and Docker Compose.
And Docker Machine can create/manage whole environments on various types of cloud providers.
It doesn't take a lot to install and configure the Docker daemon, and if you're running a cluster you'd want to put that in your cloud-init. Of course, you can always use ansible-pull to bootstrap your hosts.
Edit: Actually I have looked back, in horror and disbelief that I put up with Puppet for so long.