Lorixen.
EN

Slow Docker builds in CI? Make the layer cache survive

Short answer: Docker reuses unchanged layers when it builds, which is why builds on your laptop are quick. CI runners usually start empty each time, so that cache is gone and every build installs every dependency again. Export the build cache to somewhere that persists, such as your container registry or your CI provider's cache, with --cache-to and --cache-from, and order your Dockerfile so that rarely changing steps come first. Add cache mounts for package managers and a small build context, and most builds only redo the last few layers.

Why CI throws the cache away

Docker's documentation puts it plainly: CI build environments usually have little or no persistence between runs. The layer cache that makes local rebuilds fast lives on the machine that built them. A fresh runner has nothing, so npm install, pip install or apt-get steps run in full every time, often for several minutes.

Step 1: order your Dockerfile for caching

Docker's guide to optimising cache usage starts with layer order. Each instruction creates a layer; when an instruction's inputs change, that layer and every layer after it rebuild. So:

  1. Base image and system packages first.
  2. Copy only dependency manifests, such as package.json and the lock file, then install dependencies.
  3. Copy the rest of the source code last.
COPY package.json package-lock.json ./
RUN npm ci
COPY . .
RUN npm run build

With this order, editing a source file only rebuilds the last two steps; dependencies come from cache. The common mistake, COPY . . before installing, means any code change reinstalls everything.

Step 2: keep the build context small

The guide also recommends keeping the build context small: fewer files sent to the builder and less chance of an irrelevant change invalidating the cache. Use a .dockerignore to exclude node_modules, build output, .git, logs and local environment files.

Step 3: persist the cache between CI runs

Docker's build system, BuildKit, can export its cache to a storage backend and import it next time. Docker's documentation lists backends including:

  • inline: cache metadata embedded in the image you push
  • registry: cache pushed as a separate image in your registry
  • local: a directory on disk, useful if your CI can persist folders
  • gha: the GitHub Actions cache, marked as beta

The pattern is the same for all of them: --cache-to exports, --cache-from imports. The docs give an example with a registry:

docker buildx build --push -t registry.example.com/app:latest   --cache-to type=registry,ref=registry.example.com/app:buildcache,mode=max   --cache-from type=registry,ref=registry.example.com/app:buildcache .

Note from the docs: the default Docker driver only supports these backends if the containerd image store is enabled; otherwise you'll need a different builder driver, which most CI setup steps for Buildx create for you.

min or max?

Docker's docs describe two cache export modes. In min mode, the default, only layers in the final image are cached; in max mode, all layers are cached, including intermediate stages. Min is smaller and faster to upload; max gets more cache hits, which matters for multi-stage builds where dependency installation happens in a build stage. The docs suggest experimenting with both. For multi-stage builds, start with max.

Step 4: cache mounts for package managers

The optimisation guide describes cache mounts: a persistent package cache available during a build step. Even when a dependency layer must rebuild because one package changed, the package manager only downloads new or changed packages:

RUN --mount=type=cache,target=/root/.npm npm ci

The same works for pip, apt and other package managers with their cache directories.

Don't put secrets in layers

Docker's documentation warns against passing secrets with COPY or build arguments, which can leak into the image or cache. Use the --secret option for tokens needed during the build, such as private package registry credentials. This matters more once your cache is stored and shared.

Check it's working

In the build log, cached steps show as CACHED. After setting up, run the pipeline twice without changes: the second run should skip almost everything. Then change one source file and confirm only the final steps rebuild. If dependency steps keep rebuilding, check the Dockerfile order and what's in the build context.

Pull requests and branches

Cache from the main branch is the most useful starting point for a new branch, because most of its layers will match. A common setup is to import from both the branch's own cache and the main branch's cache, and export only to the branch's own reference. Pull requests from forks may not have permission to write to your registry or cache; let them read and build, and accept slower builds for those.

Housekeeping

Cache storage isn't free or unlimited. CI caches typically have size limits and evict old entries; registry caches take storage. Use one cache reference per branch or per image, and clean up old cache tags occasionally.

A worked example

An illustrative Christchurch booking API's CI pipeline spent most of each run reinstalling dependencies inside Docker. The Dockerfile copied the whole project before installing packages, and nothing persisted between runs. The developer reordered the Dockerfile, added a .dockerignore, used a cache mount for the package manager and exported a max-mode cache to their registry. Unchanged-dependency builds now reuse the cached layers, and pull requests get feedback much sooner.

FAQ

Is this the same as caching dependencies in CI?

Similar goal, different layer: dependency caching speeds up steps outside Docker; this speeds up the Docker build itself.

Why do builds still rebuild after a base image update?

A new base image changes the first layer, so everything after it rebuilds. That's expected.

Is inline cache enough?

For simple single-stage builds, often. Docker's docs note that it doesn't support the cache modes, so intermediate stages of multi-stage builds aren't cached.

Can the cache make builds wrong?

Rarely, if steps depend on the network, such as fetching "latest" without versions. Pin versions.

If your container builds are slowing your team down, send us your Dockerfile and CI setup and we'll help you speed them up.

Sources

WhatsAppBook a call