Google takes this a step further and creates single binary containers with the minimal OS bits needed [1, 2]. Personally, I think this is where we need to be headed vs running a full blown ubuntu/debian/centos OS inside the container. Three benefits, 1) no OS to manage eg. no apt-get update or configuration management, 2) container has less of an attack surface (think shellshock -- the container does not have bash, wget, curl, etc), 3) they are lightweight. The issue is that, how you do we (container creators) know the dependency tree for the app? Sure this might be easier for Go binaries, but what about complex apps like rails and mysql? It is a major pain to figure this out, so we just use an OS, and it takes all the thinking out of it.
Kelsey Hightower actually published something on this topic called "Building Docker Images for Static Go Binaries" [3].
> how you do we (container creators) know the dependency tree for the app
Nix package manager [1] offers a potential means to know the complete dependency tree. If you're not familiar, a nix expression to build a package takes a set of inputs (specific binary packages of, e.g., make, gcc, bash, libc, libxml2) and produces a binary output (depending only on the inputs). The run-time dependencies can be a smaller set than the build-time dependencies and are deduced by observing shared library linking for example.
I've been using it (outside Docker) for various Ruby apps, and I can't say it's been easy, but a large part of the pain has been Rubygems' inability to encode dependencies on C-libraries (e.g. libxml-ruby depends on libxml2).
There have been attempts at provisioning Docker containers with Nix [2]
Of course, if you are using Nix, some part of Docker's isolation becomes redundant (Nix isolates multiple version of things on the filesystem using plain-old-directories, so it's trivial to run ten different versions of Ruby side-by-side, for example).
Perhaps I'm missing something, but I don't see how [1] can be used to create "single binary containers with the minimal OS bits needed"? It is from https://registry.hub.docker.com/u/google/nodejs/dockerfile/ and uses the full Debian stack that you discuss including apt-get etc.
I've heard whisperings on the wind of research being done with respect to monitoring what files a Docker container uses, and then removing everything that the container doesn't need to run the app. I agree that this is the future- I shouldn't have apt-get, curl, etc. taking up space in my final image if I don't need them - but how do you tell a "good" file from a "bad" one? (Just thinking out loud here - what if my app depends on imagemagick, libffmpeg etc.?) Nix looks pretty cool I suppose.
I wrote https://github.com/jwilder/docker-squash to remove things that I know I don't need in the final image such as curl, wget, temp files, various packages, etc..
I've managed to get most images to basically the size of the base image + my app.
This process is sort of the reverse of building a single binary and adding it to a minimal image. I like that approach but it's not always straightforward w/ some applications.
Ah yes, I've played with docker-squash and like it, I wish there was a built-in docker solution for squashing layers (perhaps any contiguous string of instructions starting with ~ would be squashed into one layer?).
Mostly the problem I've run into is figuring out what to remove without b0rking the containerized app.
Sorry, here's the one I was thinking of: https://registry.hub.docker.com/u/google/cadvisor/ However, I just broke apart that image, and it is using busybox. I swear this was standalone though. I'm going to dig through the layers to see if/when this changed.
If you are willing to do abit of hacking you can do it.
For instance here is a gist I whacked together in a few mins that will build you a runnable Ruby intepreter with NOTHING else installed but it's required shared libraries. (Note this would not be fun to get say Nokogiri working in without knowing what you are doing)
Interesting this approach of building single binary containers.
I think that would be like packr [1] for Java, already discussed here [2].
I wonder if there is something like this for other languages/platforms like python/ruby/node.
Nix is a package management system which knows the full (yes completely full) transitive dependency tree of every package it installs, so you can have an absolutely minimal set of software in your container if you use nix-docker.
Because it's already doing version isolation in the package manager, you can also mount the software from the host into a shim container, which is more the "nix way" of doing things.
There's another gotcha in super-small images - things like "docker exec" will not work because there's nothing to exec. SSH-to-container becomes impossible.
I think there's got to be a middle-ground - small (maybe O(tens of MB) max) but full-featured enough to have a simple shell and the ability to get debug tools.
How small of a debian or fedora image could we get if we REALLY tried? 50 MB? 30 MB?
I thought there was a way to enter a namespace (googled this as I write the comment). Basically you have your shell and debug bits outside the container and make it appear as if it was inside it via nsenter [1]. I have not tested this, but will do that in a second. This might correct the situation you are thinking of.
+--------------------+
| docker container |
| w/ static bin |
| <----- nsenter + bash/debug bits (on host machine)
+--------------------+
He is actually talking about the case where bash binary (or gdb or whatever you're going to exec) does not exist in container. So the hassle of loading bash from the parent, then moving the process into container's namespace with nsentry.
Kelsey Hightower actually published something on this topic called "Building Docker Images for Static Go Binaries" [3].
[1] https://registry.hub.docker.com/u/google/nodejs-hello/
[2] https://github.com/thockin/serve_hostname
[3] https://medium.com/@kelseyhightower/optimizing-docker-images...