"poll every N seconds" for readiness checks is an anti-pattern. The problem is that latency cascades. A dependency chain of M services will have an end-to-end latency of N*M in the worst case and (N*M/2) in the average case, if they all rely on polling to check that the next service is ready.
You should instead rely on backpressure to signal readiness. A service accepts connections immediately, and only blocks the client when the service has to block on something else to procure a response. Systemd takes this approach with UNIX sockets - it creates all the UNIX sockets for each service, and then starts all the services in parallel. The socket buffers queue the messages so that clients can send messages to services that haven't yet started, and the OS scheduler will saturate the available CPU/IO so that the startup of all the services are maximally parallelized.
Obviously, this is not always feasible, especially in cases where you don't control the service(s) you depend on. But to the extent that you do, relying on backpressure instead of polling can markedly improve end-to-end latency.
Zero dependencies seems like a weird selling point, especially since it depends on Clap, Tokio, a Rust compiler...
A neat project, though. I think that this is best solved in the application itself (e.g. your server starts but returns HTTP errors while the database is unavailable), but being able to retrofit this behaviour into any existing application seems useful. Feels like something very similar should be built into tools like docker-compose.
>Feels like something very similar should be built into tools like docker-compose.
In docker compose you can use `depends_on` [0] to define dependencies between containers and by default it will wait until a dependent container is "ready".
But you can also use a more verbose syntax to wait until a container is "healthy", as defined by its own healthcheck.
Regarding zero dependencies, I meant that it is a static binary and does not need any external tools to be installed. For example, some alternatives require netcat to be installed.
This is fine, and what I understood from "zero dependencies", that I can drop it in a directory and it'll just work. As far as I'm concerned, this complaint is moot.
`process-compose` is a "scheduler and orchestrator". `is_ready` only checks for services to be ready so it seems great when you want to start the services by other means and wait for them to be avialabe (for example, services started in another node of a cluster).
> Zero dependencies seems like a weird selling point, especially since it depends on Clap, Tokio, a Rust compiler...
Came here to make the same comment. 0 dependencies would mean/infer (at least, to me) a lack of any dependencies but that doesn't seem to be the case[1]. There are 30 matches for the word `dependencies`.
When you see someone advertise a tool to put in your containers to do something, why are you immediately thinking "no compile-time dependencies", rather than "no runtime dependencies"?
> When you see someone advertise a tool to put in your containers to do something...
The title claims "0 dependencies", not "0 runtime dependencies" nor "0 compile-time dependencies". In other words, the word dependencies - by itself - is encompassing both runtime and compile-time dependencies.
Think of reading the title at face-value - but as if someone else had written it, with no exposure to your project, its use-case, behaviour, etc. - before it was read.
That makes a lot of sense for application servers when one controls the stack and can implement something like exponential backoff, and services are expected to already be running.
However it is not realistic to make sure all library dependencies used behave properly for more lambda-like jobs where a library/app like "wait for" is very useful since it reduces the backoff parameters that ops needs to know about, and reduces unnecessary errors in logs.
Take GCP Cloud SQL Auth Proxy [1] for example which simplifies the details of securely connecting to a database without exposing long-term credentials to app code. Typically for the audience that uses the proxy, it is run as a sidecar container in Kubernetes Pod and has a non-deterministic startup time in the range of a few seconds. There is no "depends_on" facility like in Docker (at least when I had to use it). Your batch job could potentially always fail if the proxy cannot establish a connection to the database before your minimal database migration container starts up. Building the backoff inside the migration app makes it more opaque for ops teams. Having this "wait for" run in a startup script before the main app allow ops to tune its parameters, such as through environment variables, so that the job behaves appropriately under health probes [2] for higher level backoffs. All of which can be configured without recompiling the main app code, or embedding the functionality in the first place.
Can you suggest to me one way for an API service to wait for a database to accept connections to run the database migrations without the API depending on the liveness of the database to start up?
I think the pragmatic answer is you should wait for your database to be live before starting up. One just wants to lean on the side of depending on a few critical services rather than all potentially-utlizied application services.
Like other say easy to fix this with bash alone. But maybe if you add things like also waiting on other things it can increase your value u a little like also waiting on files, sql queries it can become a pocket knife of waiting for things? Albeit probably people will start saying it will do too much :)
Sometimes waiting for the address is not enough. For example, you might want to wait for a specific API enpoint to return 200 OK, or you might want to wait for database migrations to get applied. Is `is_ready` aiming to cover those cases in the future?
Interesting idea for a feature request. I would love to read a GitHub issue with your specific case so I can start investigating and think about how to solve a problem like this.
I considered starting to write myself a Rust "wait for it" app myself this past week with more features than the standard one. Not sure how I didn't come across yours during my initial search.
There could be some useful functionality that could be found in startup and health probe topics [1] in Kubernetes for example. But at least in that world, it may be easier, and more transparent, to plug in a startup bash script to run before the main app. For example the Postgres Docker image [2] will run any scripts that may be mounted in a docker-entrypoint-initdb.d directory before starting the server. So putting a bash script in a ConfigMap that gets used as a disk volume, could be easier that downloading/auditing a binary "wait for it" container image.
I built it for many reasons:
- Most docker images do not contain netcat so you would have to download one of them in any case.
- In the case of is_ready, you won't have to write this script yourself.
- Repositories like this had a lot of traffic so I supposed that engineers need a similar tool but this repository requires wget and netcat as dependencies. For this reason, I built my own without any dependencies.
https://github.com/eficode/wait-for
You should instead rely on backpressure to signal readiness. A service accepts connections immediately, and only blocks the client when the service has to block on something else to procure a response. Systemd takes this approach with UNIX sockets - it creates all the UNIX sockets for each service, and then starts all the services in parallel. The socket buffers queue the messages so that clients can send messages to services that haven't yet started, and the OS scheduler will saturate the available CPU/IO so that the startup of all the services are maximally parallelized.
Obviously, this is not always feasible, especially in cases where you don't control the service(s) you depend on. But to the extent that you do, relying on backpressure instead of polling can markedly improve end-to-end latency.
reply