If you run Docker in Swarm mode instead (docker stack deploy instead of docker compose up), then you can make use of Swarm secrets. You don't have to make full use of Swarm; it can be a single node with a single instance of the service(s). As I manage the host with Ansible, I'm able to use Ansible's Docker module with a play to communicate the current secrets to Swarm, so that they are available to the services. On the Ansible side, the secrets are encrypted at rest using ansible-vault.
I’ve written a small Python app to deploy to Swarm mode from CI pipelines, and configure secrets and configs from CI environment variables, taking care of rotation and recreating Swarm secrets if the variable content changes. This delegates the whole secret management to the CI tool (BitBucket in my case).
The default implementation in Swarm has the problem that you cannot update secrets, so you’ll need to reconfigure and redeploy the service with a secret with a new name if that changes. That was quite a pain!
I encountered the secret update problem too. I have a secret rotation playbook that stops the Docker services stack, removes the secrets, recreates under the same name, and restarts the Docker services stack. The community.docker Ansible module does all the lifting there.
My CI runs as a container in that stack too, so in Jenkins I have an init.d Groovy script to establish Jenkins Credentials from the current Swarm secrets.
