|
| 1 | +# Using proxy in Docker builds |
| 2 | + |
| 3 | +Docker builds can share proxy's package cache even when their builders and local |
| 4 | +package caches are new. Configure the package manager inside the build to use |
| 5 | +proxy. Cached archives still need to be downloaded from proxy and installed; |
| 6 | +native extensions still need to be compiled. |
| 7 | + |
| 8 | +## Run a persistent proxy |
| 9 | + |
| 10 | +Run proxy outside the image build and keep its database and artifact storage |
| 11 | +between builds. For a proxy running on your development machine: |
| 12 | + |
| 13 | +```sh |
| 14 | +PROXY_CACHE_METADATA=true proxy serve \ |
| 15 | + -listen :8080 \ |
| 16 | + -base-url http://host.docker.internal:8080 \ |
| 17 | + -storage-url "file://$PWD/proxy-cache/artifacts" \ |
| 18 | + -database-path ./proxy-cache/proxy.db |
| 19 | +``` |
| 20 | + |
| 21 | +Docker Desktop provides `host.docker.internal` for reaching the host. On Linux |
| 22 | +Docker Engine, add the host mapping when building: |
| 23 | + |
| 24 | +```sh |
| 25 | +docker build \ |
| 26 | + --add-host host.docker.internal=host-gateway \ |
| 27 | + --build-arg PROXY_URL=http://host.docker.internal:8080 \ |
| 28 | + -t app . |
| 29 | +``` |
| 30 | + |
| 31 | +`localhost` inside a build refers to the build container. Both the build's proxy |
| 32 | +URL and proxy's `base_url` must be reachable from that container, since some |
| 33 | +registry responses contain download URLs derived from `base_url`. The proxy |
| 34 | +must listen on an interface reachable from the builder, with access restricted |
| 35 | +to the machines that should use it. |
| 36 | + |
| 37 | +For CI or a remote builder, use a persistent service reachable from the builder, |
| 38 | +such as `https://proxy.example.com`, and set proxy's `base_url` to that address. |
| 39 | +Use the same service across jobs and pass its URL with `--build-arg PROXY_URL=...`. |
| 40 | +Creating an empty proxy for every job loses the shared cache unless its database |
| 41 | +and storage persist. If proxy runs in a container, mount persistent storage for |
| 42 | +both. See [configuration](configuration.md) for storage and deployment options. |
| 43 | + |
| 44 | +Metadata caching is optional, but avoids repeated metadata fetches within |
| 45 | +`metadata_ttl`. Artifact caching is enabled by default. Proxy's configured |
| 46 | +cooldown, denylist and scanning policies also apply to Docker builds. |
| 47 | + |
| 48 | +## npm |
| 49 | + |
| 50 | +In your Node build stage, copy the manifests before application source so an |
| 51 | +unchanged dependency set can reuse Docker's installation layer: |
| 52 | + |
| 53 | +```dockerfile |
| 54 | +ARG PROXY_URL |
| 55 | +COPY package.json package-lock.json ./ |
| 56 | +RUN npm ci --no-audit --no-fund --registry="${PROXY_URL:?PROXY_URL build arg required}/npm/" |
| 57 | +COPY . . |
| 58 | +``` |
| 59 | + |
| 60 | +Pass `PROXY_URL` without a trailing slash. With Docker Desktop: |
| 61 | + |
| 62 | +```sh |
| 63 | +docker build \ |
| 64 | + --build-arg PROXY_URL=http://host.docker.internal:8080 \ |
| 65 | + -t app . |
| 66 | +``` |
| 67 | + |
| 68 | +Keep `node_modules` out of the build context so `COPY . .` does not overwrite |
| 69 | +the installed dependencies. Lockfiles with custom registry or download URLs |
| 70 | +may still direct requests to those hosts; check proxy's access log when using |
| 71 | +private registries or non-registry dependencies. `--no-audit` skips the audit |
| 72 | +request, which proxy would otherwise forward upstream. |
| 73 | + |
| 74 | +## Ruby / Bundler |
| 75 | + |
| 76 | +Keep `https://rubygems.org` as the source in your Gemfile and configure its mirror |
| 77 | +inside the Ruby build stage. Install any system packages needed to compile |
| 78 | +native extensions before this step: |
| 79 | + |
| 80 | +```dockerfile |
| 81 | +ARG PROXY_URL |
| 82 | +COPY Gemfile Gemfile.lock ./ |
| 83 | +RUN bundle config set --local mirror.https://rubygems.org "${PROXY_URL:?PROXY_URL build arg required}/gem" \ |
| 84 | + && bundle install \ |
| 85 | + && bundle config unset --local mirror.https://rubygems.org |
| 86 | +COPY . . |
| 87 | +``` |
| 88 | + |
| 89 | +Use the same build command as the npm example. The mirror setting is removed |
| 90 | +after installation so it is not retained in the image, but the URL still |
| 91 | +appears in `docker history` because Docker records the build args used by `RUN` |
| 92 | +lines. Keep credentials out of `PROXY_URL`. Exclude host Bundler |
| 93 | +configuration and installed gems from the build context. Copy any local path |
| 94 | +dependencies required by the Gemfile before `bundle install`. |
| 95 | + |
| 96 | +The RubyGems mirror covers gems fetched from that source. Git dependencies and |
| 97 | +other gem sources need their own configuration. Without |
| 98 | +`mirror.https://rubygems.org.fallback_timeout`, `bundle install` fails if proxy |
| 99 | +is unreachable. Setting it lets Bundler fall back to rubygems.org, so this setup |
| 100 | +does not enforce network isolation either way. |
| 101 | + |
| 102 | +## Layer caches and package caches |
| 103 | + |
| 104 | +Keep existing Docker caching optimizations. If Docker reuses the installation |
| 105 | +layer, the package manager does not run and proxy receives no requests. A |
| 106 | +BuildKit cache mount can also preserve downloaded packages when that layer must |
| 107 | +be rebuilt. For example, the npm step can use: |
| 108 | + |
| 109 | +```dockerfile |
| 110 | +RUN --mount=type=cache,target=/root/.npm \ |
| 111 | + npm ci --no-audit --no-fund --registry="${PROXY_URL:?PROXY_URL build arg required}/npm/" |
| 112 | +``` |
| 113 | + |
| 114 | +Cache mounts are stored by the builder and are excluded from ordinary |
| 115 | +`--cache-to` exports. Proxy can supply packages when a fresh builder has neither |
| 116 | +the installation layer nor the package-manager cache. See Docker's |
| 117 | +[cache guidance](https://docs.docker.com/build/cache/optimize/) and |
| 118 | +[cache-mount reference](https://docs.docker.com/reference/dockerfile/#run---mounttypecache). |
| 119 | + |
| 120 | +To check the configuration, enable proxy's JSONL log with `-access-log` and |
| 121 | +rebuild with `docker build --no-cache`. Look for requests to `/npm/` or `/gem/`, |
| 122 | +then repeat and check `proxy_cache_hits_total` on `/metrics`. Cache mounts still |
| 123 | +retain their contents with `--no-cache`; omit them when testing proxy cache |
| 124 | +hits. Upstream metadata revalidation can still occur. |
| 125 | + |
| 126 | +## Base-image pulls |
| 127 | + |
| 128 | +Package-manager configuration affects commands such as `npm ci` and |
| 129 | +`bundle install`. To cache images referenced by `FROM`, configure a registry |
| 130 | +mirror in the builder as well. For a proxy served at |
| 131 | +`https://proxy.example.com`, put this in `buildkitd.toml`: |
| 132 | + |
| 133 | +```toml |
| 134 | +[registry."docker.io"] |
| 135 | + mirrors = ["proxy.example.com"] |
| 136 | +``` |
| 137 | + |
| 138 | +Create a Buildx builder with that configuration and select it explicitly: |
| 139 | + |
| 140 | +```sh |
| 141 | +docker buildx create --name proxy-builder \ |
| 142 | + --driver docker-container \ |
| 143 | + --buildkitd-config ./buildkitd.toml |
| 144 | + |
| 145 | +docker buildx build --builder proxy-builder \ |
| 146 | + --build-arg PROXY_URL=https://proxy.example.com \ |
| 147 | + --load -t app . |
| 148 | +``` |
| 149 | + |
| 150 | +For local Docker Desktop testing over HTTP, the configuration is: |
| 151 | + |
| 152 | +```toml |
| 153 | +[registry."docker.io"] |
| 154 | + mirrors = ["host.docker.internal:8080"] |
| 155 | + |
| 156 | +[registry."host.docker.internal:8080"] |
| 157 | + http = true |
| 158 | +``` |
| 159 | + |
| 160 | +The registry mirror address must be reachable from the BuildKit daemon itself. |
| 161 | +The build's `--add-host` flag only adds a mapping inside build containers. A |
| 162 | +separate Buildx builder has its own registry configuration; setting Docker |
| 163 | +Engine's `registry-mirrors` does not configure that builder. See Docker's |
| 164 | +[BuildKit configuration guide](https://docs.docker.com/build/buildkit/configure/). |
| 165 | + |
| 166 | +Proxy's OCI endpoint is pull-only. BuildKit registry-cache exports with |
| 167 | +`--cache-to type=registry` need a separate writable registry. Other cache export |
| 168 | +backends can be used alongside proxy. |
0 commit comments