For a more full fledged use case FRRouting may be the way to go. We are using Quagga here mainly to maintain internal routes with iBGP and route reflectors. This is a fairly simple use case.
Quagga is available in the default package managers of most distros so its a good place to start.
We have been working on a simplified container build system which does away with layers altogether. [1]
The use of layers at the build stage adds a lot of needless complexity with very little benefits and users
really need to step back and question the value they are getting from the use of layers. [2]
Words like 'immutability', 'declarative' and 'reproduciblity' are often used in ways that can lead to user misunderstanding and can be accomplished with simpler workflows. For instance immutability, reuse, composition do not require layers. There needs to be a lot more technical scrutiny to avoid confusion.
We need to spin up a lot of VMs of various Linux distributions for testing Flockport and we found the existing tooling too heavy and involved.
In the end we rolled our own solution that is much more lightweight, uses QEMU directly and provides better visibility, ease of use and even some potentially useful networking features. Here are a few screencasts [1].
This is mainly designed as a module for Flockport and we will preview this soon.
What is wrong with Docker? This does not address any technical or other shortcoming and only seeks to replace one set of over engineered tools with another with the exact same problems. [1]
The is yet more of the ecosystem continuing to push over engineered tooling and 'winners' breathlessly without basic technical scrutiny that leaves end users dealing with needless complexity and debt.
Containers can be useful as a lightweight, efficient alternative to VMs and those who want containers untouched by questionable ideas should try the LXC project on which all this was based.
Any additional layer on top of this be it non standard OS environment or layers should meet technical scrutiny for end user benefits and most users will be surprised by the results.
This is where containers can help. Ubuntu ships with LXC and its relatively simple to download an Alpine Linux container and then install Nginx and you are set. Alternatively Flockport [self plug] provides an open source app store [1] for LXC and you can download a prebuilt Nginx114 container all set to go.
But containers are some work with both upsides and downsides and upsides compared to having it out of the box in the distribution.
Docker was based on LXC and there are some important differences that should be more well known.
The main difference is LXC boots an init in the namespace as the first process so you get a standard OS environment with multi-process support. Docker runs the application directly so you get a single process environment, unless you decide to run a process manager. IMHO this creates far more problems than it solves as anything nonstandard would, as you now need workarounds for basics like logging, networking, daemons and anything that expects a standard OS environment
Docker also uses layers to build containers and ephemeral data. None of these 3 critical decisions have seen much technical scrutiny resulting in persistent confusion in the ecosystem and with scrutiny many users will find they can add far more management overhead and complexity to containers. We have written about some of these issues here. [1]
For instance OP is referring to unprivileged containers and user namespaces that let you run containers as a user process. LXC has supported these since 2013. Docker added user namespaces support in recent versions. However taking just one issue - because of the use of layers to build containers you can't use unprivileged containers for app builds as only root users can mount filesystems. So a lot of these decisions have all kinds of tradeoffs that should be better known and discussed. [2]
Disclaimer: Founder at Flockport and trying to build a simpler alternative with LXC with support for app builds, provisioning, orchestration, service discovery and an app store. [3]
User namespace support for Docker has been in for years now, not just "recent versions".
Not sure what you mean by "app builds" here. The container process is in a user namespace, not docker itself and "docker build" will work just fine. Maybe you are referring to something else? Let me know, would love to take a look.
Layers are largely an implementation detail in Docker. Yeah they are there, you can even check the sha of them, but not something users interact with. I don't think I've personally heard any of tbis confusion except for very old versions of Docker before content-addressabity and the concept of layers did leak through (each layer was an "image")... maybe there's still some lasting confusion here? I could see this I suppose with things like older books/blogs/etc referring to them as a first class citizen. They are definitely not and certainly should not be. One of the early design mistakes which are long since corrected (docker 1.10 introduced content addressable images, and the concept of the image as a whole rather than an image being a tree of other images).
Also, a side note, "docker run --init" gets you the init you are looking for, can set it as default in the daemon as well.
Note, I've been a maintainer/contributor to Docker for 5 years now.
> Layers are largely an implementation detail in Docker.
You say this, but they are so deeply baked into many tools that there have been many hundreds of talks about how to "reduce the size of your Docker image with this one simple trick!" which reduces down to understanding how layers in Docker images work. Layers definitely leak through a lot of the tooling.
I'm hoping that my OCIv2 proposal will remove the practical need for layers entirely -- and also remove a lot of the cargo-culting which has popped up around "how to get smaller images". Maybe Docker might adopt this in the future (after you guys get OCIv1 support :P).
> Also, a side note, "docker run --init" gets you the init you are looking for, can set it as default in the daemon as well.
This is different to LXC. With LXC, the "init" is actually systemd, not just a zombie reaper -- and when you do an "lxc attach" you are actually put into a new console (getty and all).
Yes, you can run (after some headaches) systemd inside Docker, but LXC definitely handles this much better because that's the usecase it was designed around. It's not really a defect in Docker, it's just not what Docker was designed around.
> I've seen (and largely agree with) your post (somewhere? Can't remember where) here, are you working on a formal proposal?
I'm currently working on a blog post to describe and justify the design, as well as a PoC. The formal proposal for OCI will come after the PoC (and after sufficient benchmarking and discussion within the community). But I plan to write a spec document along the PoC.
Layers are integral in Docker, layers were pushed as enabling reuse, composibility, 'immutability', every 'run' statement is a layer. They are not an implementation detail. Managing all these layers adds complexity and impact users from choosing storage drivers, to using layers to building images, and if you are going to use layers you need to be a privileged user to mount all these layers. [1]
Here is an example. I build an Nginx container with no layers from a base alpine image. Users can download and run it in a layer so the original Nginx container remains untouched or run a copy of it. If there is a security alert or update I rerun the build. No layers used. You build the same Nginx container will multiple layers, and have all the overhead of tracking and managing these layers. And users then download the Nginx container made up of all these layers and run it. If there is a security update in any lower layers it needs to rebuild. You have used a workflow and designed an entire system based on stacking layers to compose and build your image. That is a lot of overhead and complexity in basic image management. What is the benefit?
Right, they are integral but the user doesn't interact with them and they can be swapped out for a different implementation and the user would never know outside of noticing the impact that the change has (or trade-offs such a change would make).
Caching for `docker build` really doesn't have anything to do with layers except as a means of storage, it doesn't need to be layers at all.
To be honest, layers aren't even all that efficient (except in perfectly ideal scenarios), just better than no sharing at all and no one has taken the time to come up with something better yet...because frankly there are many other things that need improvements before bothering with it.
Again, just figured I'd mention again... since you are trying to explain Docker to me... I'm a Docker maintainer and have been for the last ~5 years. Happy to chat about this.
Layers may or may not benefit any particular org. If you build a lot of your own containers from the same base layer image (microserver1-container-a .. microserver2-container-N), then you could save a lot of bandwidth/storage pulling the base layer once. Naturally, if you build everything from alpine:latest, then a large majority of your containers are going to have difference base layers and you're going to lose those savings.
Similar to what linuxftw notes, when Docker hit the scene it was basically competing with the likes of Heroku where you have a few common runtime containers on a shared runtime system. When trying to implement one of those you want to maximize some sorts of shared common components and page/file sharing to create some efficiencies in memory sharing/oversubscription.
@tobbyb Thanks for all the work on flockport your tutorials on LXC helped me to have initial production systems ready. I applied many of those concepts specially for networking using flockport tutorial.
Although today my system with LXD is much easier and getting better with every subsequent release of LXD, I owe you gratitude for helping me to get off the ground.
Layers are interesting but the benefits of layers are vastly overstated, you can achieve the same without layers and the overhead and complexity in image management. Piling layers upon layers to build a container is a really bad idea and adds a lot of complexity in basic image management. [1]
The big problem is technical decisions like the use of layers, single process containers and ephemeral storage that add a lot of complexity and overhead have not met technical scrutiny and thus not well understood. But they should be as they can add a lot of management overhead and debt. [2]
Many users who use LXC will realize a lot of benefits and flexibility actually flow from containers and not these 'modifications'.
Disclaimer: Founder at Flockport and trying to build a simpler alternative with LXC containers.
Thanks for the feedback. This is far from negative. There is definitely value in comparative docs that lets users get a quick overview, and we will put one together.
However we also want to start new conversations in the community about containers and orchestration and reach users who find existing platforms too complex. And we have put together a lot of content to kindle these discussions.
More technical scrutiny could lead to simplification and lower barriers for both users and developers.
Glad to hear you are aware of Flockport. Flockport was initially conceived as an app store for LXC containers. This was still in the early days of containers when the awareness was just building.
We always wanted to do more and our early vision was for a container management platform that is more accessible and easier to use and now of course we have built out a whole platform that delivers a lot more functionality and capabilities, and hopefully delivers on some of that early vision.
We are pleased to preview Flockport with the HN community. Flockport is a new container management platform designed to deliver a lot of functionality and ease of use. We designed this to be simple, it was the core driving force and we hope we have succeeded.
Flockport lets you quickly provision servers and deploy and manage containers across servers, supports overlay networks, distributed storage, service discovery, load balancers and high availability.
Flockport also provides a container build system and an app store. Apps can be deployed cluster wide. We also built in support for rolling out things like databases across clusters. Users can add and remove both database instances and also individual databases cluster wide. There is also similar support for web servers and load balancers builtin.
We are eager to have the community give it a go and get their early feedback and support. For those who do not want to install it there are preinstalled VMs on the downloads page and also screencasts on the resources page.
Quagga is available in the default package managers of most distros so its a good place to start.