Skip to content

Commit 0b1ca01

Browse files
authored
Merge pull request #627 from git-pkgs/docs/docker-builds
Document using proxy in Docker builds
2 parents cc1b8ac + a06b20f commit 0b1ca01

2 files changed

Lines changed: 170 additions & 1 deletion

File tree

‎README.md‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -138,7 +138,8 @@ go build -o proxy ./cmd/proxy
138138
./proxy -listen :3000 -base-url https://proxy.example.com
139139
```
140140

141-
The proxy is now running. Configure your package managers to use it.
141+
The proxy is now running. Configure your package managers to use it. For image
142+
builds, see [Using proxy in Docker builds](docs/docker-builds.md).
142143

143144
## OpenAPI (Swagger)
144145

‎docs/docker-builds.md‎

Lines changed: 168 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,168 @@
1+
# Using proxy in Docker builds
2+
3+
Docker builds can share proxy's package cache even when their builders and local
4+
package caches are new. Configure the package manager inside the build to use
5+
proxy. Cached archives still need to be downloaded from proxy and installed;
6+
native extensions still need to be compiled.
7+
8+
## Run a persistent proxy
9+
10+
Run proxy outside the image build and keep its database and artifact storage
11+
between builds. For a proxy running on your development machine:
12+
13+
```sh
14+
PROXY_CACHE_METADATA=true proxy serve \
15+
-listen :8080 \
16+
-base-url http://host.docker.internal:8080 \
17+
-storage-url "file://$PWD/proxy-cache/artifacts" \
18+
-database-path ./proxy-cache/proxy.db
19+
```
20+
21+
Docker Desktop provides `host.docker.internal` for reaching the host. On Linux
22+
Docker Engine, add the host mapping when building:
23+
24+
```sh
25+
docker build \
26+
--add-host host.docker.internal=host-gateway \
27+
--build-arg PROXY_URL=http://host.docker.internal:8080 \
28+
-t app .
29+
```
30+
31+
`localhost` inside a build refers to the build container. Both the build's proxy
32+
URL and proxy's `base_url` must be reachable from that container, since some
33+
registry responses contain download URLs derived from `base_url`. The proxy
34+
must listen on an interface reachable from the builder, with access restricted
35+
to the machines that should use it.
36+
37+
For CI or a remote builder, use a persistent service reachable from the builder,
38+
such as `https://proxy.example.com`, and set proxy's `base_url` to that address.
39+
Use the same service across jobs and pass its URL with `--build-arg PROXY_URL=...`.
40+
Creating an empty proxy for every job loses the shared cache unless its database
41+
and storage persist. If proxy runs in a container, mount persistent storage for
42+
both. See [configuration](configuration.md) for storage and deployment options.
43+
44+
Metadata caching is optional, but avoids repeated metadata fetches within
45+
`metadata_ttl`. Artifact caching is enabled by default. Proxy's configured
46+
cooldown, denylist and scanning policies also apply to Docker builds.
47+
48+
## npm
49+
50+
In your Node build stage, copy the manifests before application source so an
51+
unchanged dependency set can reuse Docker's installation layer:
52+
53+
```dockerfile
54+
ARG PROXY_URL
55+
COPY package.json package-lock.json ./
56+
RUN npm ci --no-audit --no-fund --registry="${PROXY_URL:?PROXY_URL build arg required}/npm/"
57+
COPY . .
58+
```
59+
60+
Pass `PROXY_URL` without a trailing slash. With Docker Desktop:
61+
62+
```sh
63+
docker build \
64+
--build-arg PROXY_URL=http://host.docker.internal:8080 \
65+
-t app .
66+
```
67+
68+
Keep `node_modules` out of the build context so `COPY . .` does not overwrite
69+
the installed dependencies. Lockfiles with custom registry or download URLs
70+
may still direct requests to those hosts; check proxy's access log when using
71+
private registries or non-registry dependencies. `--no-audit` skips the audit
72+
request, which proxy would otherwise forward upstream.
73+
74+
## Ruby / Bundler
75+
76+
Keep `https://rubygems.org` as the source in your Gemfile and configure its mirror
77+
inside the Ruby build stage. Install any system packages needed to compile
78+
native extensions before this step:
79+
80+
```dockerfile
81+
ARG PROXY_URL
82+
COPY Gemfile Gemfile.lock ./
83+
RUN bundle config set --local mirror.https://rubygems.org "${PROXY_URL:?PROXY_URL build arg required}/gem" \
84+
&& bundle install \
85+
&& bundle config unset --local mirror.https://rubygems.org
86+
COPY . .
87+
```
88+
89+
Use the same build command as the npm example. The mirror setting is removed
90+
after installation so it is not retained in the image, but the URL still
91+
appears in `docker history` because Docker records the build args used by `RUN`
92+
lines. Keep credentials out of `PROXY_URL`. Exclude host Bundler
93+
configuration and installed gems from the build context. Copy any local path
94+
dependencies required by the Gemfile before `bundle install`.
95+
96+
The RubyGems mirror covers gems fetched from that source. Git dependencies and
97+
other gem sources need their own configuration. Without
98+
`mirror.https://rubygems.org.fallback_timeout`, `bundle install` fails if proxy
99+
is unreachable. Setting it lets Bundler fall back to rubygems.org, so this setup
100+
does not enforce network isolation either way.
101+
102+
## Layer caches and package caches
103+
104+
Keep existing Docker caching optimizations. If Docker reuses the installation
105+
layer, the package manager does not run and proxy receives no requests. A
106+
BuildKit cache mount can also preserve downloaded packages when that layer must
107+
be rebuilt. For example, the npm step can use:
108+
109+
```dockerfile
110+
RUN --mount=type=cache,target=/root/.npm \
111+
npm ci --no-audit --no-fund --registry="${PROXY_URL:?PROXY_URL build arg required}/npm/"
112+
```
113+
114+
Cache mounts are stored by the builder and are excluded from ordinary
115+
`--cache-to` exports. Proxy can supply packages when a fresh builder has neither
116+
the installation layer nor the package-manager cache. See Docker's
117+
[cache guidance](https://docs.docker.com/build/cache/optimize/) and
118+
[cache-mount reference](https://docs.docker.com/reference/dockerfile/#run---mounttypecache).
119+
120+
To check the configuration, enable proxy's JSONL log with `-access-log` and
121+
rebuild with `docker build --no-cache`. Look for requests to `/npm/` or `/gem/`,
122+
then repeat and check `proxy_cache_hits_total` on `/metrics`. Cache mounts still
123+
retain their contents with `--no-cache`; omit them when testing proxy cache
124+
hits. Upstream metadata revalidation can still occur.
125+
126+
## Base-image pulls
127+
128+
Package-manager configuration affects commands such as `npm ci` and
129+
`bundle install`. To cache images referenced by `FROM`, configure a registry
130+
mirror in the builder as well. For a proxy served at
131+
`https://proxy.example.com`, put this in `buildkitd.toml`:
132+
133+
```toml
134+
[registry."docker.io"]
135+
mirrors = ["proxy.example.com"]
136+
```
137+
138+
Create a Buildx builder with that configuration and select it explicitly:
139+
140+
```sh
141+
docker buildx create --name proxy-builder \
142+
--driver docker-container \
143+
--buildkitd-config ./buildkitd.toml
144+
145+
docker buildx build --builder proxy-builder \
146+
--build-arg PROXY_URL=https://proxy.example.com \
147+
--load -t app .
148+
```
149+
150+
For local Docker Desktop testing over HTTP, the configuration is:
151+
152+
```toml
153+
[registry."docker.io"]
154+
mirrors = ["host.docker.internal:8080"]
155+
156+
[registry."host.docker.internal:8080"]
157+
http = true
158+
```
159+
160+
The registry mirror address must be reachable from the BuildKit daemon itself.
161+
The build's `--add-host` flag only adds a mapping inside build containers. A
162+
separate Buildx builder has its own registry configuration; setting Docker
163+
Engine's `registry-mirrors` does not configure that builder. See Docker's
164+
[BuildKit configuration guide](https://docs.docker.com/build/buildkit/configure/).
165+
166+
Proxy's OCI endpoint is pull-only. BuildKit registry-cache exports with
167+
`--cache-to type=registry` need a separate writable registry. Other cache export
168+
backends can be used alongside proxy.

0 commit comments

Comments
 (0)