Whyfire

Blog · 13 October 2026 · 7 minute read

Docker is filling my disk: logs, images and the build cache

Tested with Docker Engine 29.4 and buildx 0.33.

Quick answer

Run docker system df to see whether images, containers, volumes or the build cache are the big one. Container logs grow without limit unless you cap them in /etc/docker/daemon.json, and docker image prune never touches the build cache: that needs docker builder prune.

Docker keeps almost everything until you tell it not to: every log line a container writes, every image version you pulled, every layer it built. On a busy host that adds up quietly until the disk alert fires.

1. See what's using the space

df -h /var/lib/docker
docker system df
sudo du -xsh /var/lib/docker/* | sort -h

docker system df splits it four ways. Here's a real development machine of ours:

TYPE            TOTAL     ACTIVE    SIZE      RECLAIMABLE
Images          40        9         11.82GB   6.945GB (58%)
Containers      12        12        163.6MB   0B (0%)
Local Volumes   11        7         161.5MB   139.5MB (86%)
Build Cache     123       0         3.325GB   1.356GB

RECLAIMABLE is what pruning could free. Container logs don't appear in this table at all, which is why they're so often missed: they live in /var/lib/docker/containers, so check that with du.

2. Container logs: the usual culprit

Docker's default log driver, json-file, keeps every line a container has ever written, with no size limit. Find the biggest:

sudo sh -c 'du -h /var/lib/docker/containers/*/*-json.log' | sort -h | tail -5
docker inspect --format '{{.Name}} {{.LogPath}}' $(docker ps -aq)

To free the space right now, empty the file. Docker carries on writing to it, and the old log is gone for good:

sudo truncate -s 0 "$(docker inspect --format '{{.LogPath}}' my-container)"

To stop it happening again, cap every container's logs in /etc/docker/daemon.json, here at three files of 50 MB:

{
  "log-driver": "json-file",
  "log-opts": {
    "max-size": "50m",
    "max-file": "3"
  }
}

Two catches. Restarting Docker to load this restarts every container, unless "live-restore": true is also set. And the limits only apply to containers created afterwards, so recreate them (docker compose up -d --force-recreate). You can also set it per service in Compose:

services:
  app:
    image: nginx:1.28
    logging:
      driver: json-file
      options:
        max-size: "50m"
        max-file: "3"

3. Images

docker image prune removes only dangling images: untagged layers left behind when a tag moved to a newer build. docker image prune -a removes every image no container is using, including the previous version you might want to roll back to. A middle way removes unused images older than a week:

docker image prune -a --filter until=168h

4. The build cache that image prune misses

If you build images on the host, BuildKit keeps every layer it built so the next build is faster. docker image prune doesn't touch any of it. We've seen a single rebuild leave nearly 10 GB of build cache behind on a server where image pruning had "cleaned up". Prune it separately, keeping a few gigabytes so builds stay quick:

docker builder prune -f --reserved-space 5GB

Older guides use --keep-storage; it still works on current Docker but prints a warning, and --reserved-space replaces it.

Docker does trim the cache by itself, but late. docker buildx inspect shows the policy. On a 1 TB disk with default settings, ours keeps up to 94 GiB of cache no matter what and only starts trimming when free space falls below 188 GiB. On a small server the numbers shrink with the disk, but the shape is the same: the cache can grow until the disk is about 80% full. If you build on a server, prune on a schedule. As /etc/cron.weekly/docker-builder-prune, made executable:

#!/bin/sh
docker builder prune -f --reserved-space 5GB

5. Volumes: careful

docker volume prune removes unused anonymous volumes. With -a it also removes unused named volumes, and named volumes are usually where databases keep their data. A volume counts as unused as soon as no container refers to it, so a database whose stopped container was removed (which docker system prune does) loses its protection. Look before you prune:

docker volume ls -f dangling=true

What docker system prune actually removes

CommandRemoves
docker system pruneStopped containers, unused networks, dangling images, unused build cache
… -aAlso every image not used by a container
… --volumesAlso unused anonymous volumes

Stopped containers include ones you stopped on purpose and meant to start again.

Alert before it fills

A percentage alert on /var/lib/docker fires too late for a runaway log and too early on a big disk. Alert on the trend instead, so a container suddenly logging ten times its normal rate is caught hours before the disk is full:

- alert: DiskWillFillIn24h
  expr: |
    predict_linear(node_filesystem_avail_bytes{fstype!~"tmpfs|overlay|squashfs"}[6h], 24 * 3600) < 0
    and node_filesystem_avail_bytes / node_filesystem_size_bytes < 0.4
    and node_filesystem_readonly == 0
  for: 1h
  labels:
    severity: critical
  annotations:
    summary: "{{ $labels.instance }} {{ $labels.mountpoint }} is on course to fill within a day"

More rules like this in Alert rules that won't burn you out.

Find what started writing

A disk alert says where the space went, not why. The cause is usually a change somewhere else: a service switched to debug logging, a queue backing up, a job retrying forever. Whyfire compares each server's log volume with the hour before, so when one starts writing ten times its usual amount, the incident card names that server and when it started. See the live demo.