Make sure your docker build is being cached properly, and break infrequently running stuff into their own steps, then move them to the top of the docker file.
Crucially: Make sure that the large layers say they are "cached" when you rebuild the container. Docker goes out of its way to make this difficult in CI environments. The fact that it works on your laptop doesn't mean that it will be able to cache the big layers in CI.
Once you've done that, make sure that the CI machines are actually pulling the big layers from their local docker cache.
30-90 seconds to pull docker images for each run of a golang project's CI environment is too high. You might look into using "go mod vendor" to download the dependencies early in the docker build, then using a symlink and "--mod=vendor" to tell the tests to use an out-of-tree vendor directory. (I haven't tested this; presumably go will follow symlinks...)
Crucially: Make sure that the large layers say they are "cached" when you rebuild the container. Docker goes out of its way to make this difficult in CI environments. The fact that it works on your laptop doesn't mean that it will be able to cache the big layers in CI.
Once you've done that, make sure that the CI machines are actually pulling the big layers from their local docker cache.
30-90 seconds to pull docker images for each run of a golang project's CI environment is too high. You might look into using "go mod vendor" to download the dependencies early in the docker build, then using a symlink and "--mod=vendor" to tell the tests to use an out-of-tree vendor directory. (I haven't tested this; presumably go will follow symlinks...)