@@ -93,6 +93,28 @@ These auto-cloned repos are tracked and **removed automatically on exit**
9393(via an EXIT trap), so ` src/ ` directories stay clean in the repo (only a
9494` .gitkeep ` is committed).
9595
96+ Git source caching is optional and is separate from the package caches
97+ described below. Enable it for local source operations with:
98+
99+ ``` bash
100+ SOURCE_CACHE=true STREAM=master tox -ebuild -- nova
101+ ```
102+
103+ When enabled, repository objects are retained as bare repositories under
104+ ` .tmp/source_cache/<host>/<organization>/<repository>.git ` . Branch and tag refs
105+ are refreshed once per repository in each ` build.sh ` invocation, then a fast
106+ local clone is materialized at the exact pinned commit. The disposable clone is
107+ removed on exit while the bare cache remains for later invocations. Set
108+ ` SOURCE_CACHE_DIR ` to override the cache location. Source caching defaults to
109+ ` false ` ; direct temporary clones retain the previous behavior when disabled.
110+
111+ To discard all cached Git objects and make the next cached source operation
112+ fetch them again, remove the cache directory:
113+
114+ ``` bash
115+ rm -rf .tmp/source_cache
116+ ```
117+
96118If a checkout already exists in ` src/ ` (e.g., a local development clone),
97119` build.sh ` uses it as-is and does ** not** remove it on exit. This lets you
98120work on a local branch without ` build.sh ` overwriting it.
@@ -288,6 +310,67 @@ STREAM=stable tox -ecustom -- build watcher/watcher-base
288310tox -ecustom -- list
289311```
290312
313+ ### Optional local package caches
314+
315+ A developer-only Compose stack under ` tools/local-package-cache/ ` provides
316+ [ Proxpi] ( https://github.com/EpicWink/proxpi ) for Python downloads and
317+ Squid for plain-HTTP RPM repository traffic. Package caching is disabled
318+ by default and affects only local ` build ` and ` build-parallel ` actions when
319+ ` LOCAL_PACKAGE_CACHE=true ` . Normal builds and Konflux builds remain unchanged.
320+ The cache-service lifecycle is intentionally separate from ` build.sh ` .
321+
322+ ` podman compose ` requires a Compose provider. On CentOS Stream, install and
323+ verify ` podman-compose ` with:
324+
325+ ``` bash
326+ sudo dnf install podman-compose
327+ podman compose version
328+ ```
329+
330+ Start the services, verify them, and opt a build into package caching:
331+
332+ ``` bash
333+ LOCAL_CACHE_BIND_ADDRESS=0.0.0.0 \
334+ podman compose -f tools/local-package-cache/compose.yaml up -d
335+ podman compose -f tools/local-package-cache/compose.yaml ps
336+ LOCAL_PACKAGE_CACHE=true STREAM=master tox -ebuild -- glance
337+ ```
338+
339+ Compose creates and manages the named Proxpi and Squid volumes declared
340+ in ` compose.yaml ` ; do not create those volumes manually. They persist downloaded
341+ content across container replacement and ordinary ` down ` /` up ` cycles.
342+
343+ Build containers reach the published ports through
344+ ` host.containers.internal ` . Rootless containers generally cannot reach services
345+ bound only to host loopback, so the example publishes on all host interfaces.
346+ Binding to ` 0.0.0.0 ` can expose unauthenticated cache services to the local
347+ network; use host firewall rules or set ` LOCAL_CACHE_BIND_ADDRESS ` to a more
348+ specific reachable host address when possible. The Compose file defaults to
349+ ` 127.0.0.1 ` for safety when no address is specified.
350+
351+ The build fails before invoking Buildah when ` LOCAL_PACKAGE_CACHE=true ` and
352+ either service is unavailable. Endpoint overrides are available through
353+ ` LOCAL_PYPI_INDEX_URL ` , ` LOCAL_PYPI_TRUSTED_HOST ` , ` LOCAL_RPM_PROXY ` ,
354+ ` LOCAL_PYPI_HEALTH_URL ` , and ` LOCAL_RPM_HEALTH_URL ` .
355+
356+ Stop the services while retaining downloaded packages:
357+
358+ ``` bash
359+ podman compose -f tools/local-package-cache/compose.yaml down
360+ ```
361+
362+ Delete the services and their package-cache volumes for a cold package fetch:
363+
364+ ``` bash
365+ podman compose -f tools/local-package-cache/compose.yaml down -v
366+ ```
367+
368+ This proof of concept caches external downloads only. It does not cache wheels
369+ built from the OpenStack source checkouts. HTTPS repositories normally pass
370+ through without TLS interception. Squid fetches the HTTPS origin behind
371+ ` mirror.stream.centos.org ` itself so the existing HTTP repository URLs remain
372+ cacheable.
373+
291374### Initial setup
292375
2933761 . Create ` containers/<project>/sources.txt ` with entries for each stream. Add
@@ -437,6 +520,37 @@ validation, builds, publication, result generation, and cleanup target that
437520host explicitly. The Zuul executor controls Ansible but does not perform those
438521mutations.
439522
523+ The provider copies the RDO base job's ` /etc/pip.conf ` into a job-local file
524+ and mounts it into Buildah so Python downloads use the provider mirror. It also
525+ can run an ephemeral Squid container directly with Podman. Squid caches direct
526+ upstream HTTP traffic or can sit in front of the provider's CentOS mirror. The
527+ cache is published on the disposable builder and Buildah RUN steps reach it through
528+ ` host.containers.internal ` . Proxy values are passed as build arguments so
529+ host-side base-image pulls are not routed through the package cache. The cache
530+ intentionally listens without authentication on all interfaces of the
531+ disposable CI builder. Plain-HTTP CentOS Stream requests can be remapped to the
532+ provider's CentOS mirror. Otherwise, Squid rewrites plain-HTTP CentOS requests
533+ to the upstream HTTPS origin before fetching them, without intercepting client
534+ TLS. The job does not consume host RPM repository files or require
535+ ` podman-compose ` . Post-run cleanup
536+ removes the cache container, its volumes, and generated configuration.
537+ Persistent source caching is explicitly disabled
538+ because Zuul already stages speculative source trees and the builder is
539+ disposable.
540+
541+ ` s2i_ci_use_provider_pip_mirror ` controls whether the parent-provided
542+ ` pip.conf ` is mounted, ` s2i_ci_enable_rpm_cache ` controls the RPM proxy, and
543+ ` s2i_ci_rpm_cache_use_provider_mirror ` controls its optional CentOS Stream
544+ backend remap. Disabling Squid leaves RPM repository access unchanged;
545+ the provider mirror still applies to Python downloads when its pip option is
546+ enabled. The content-provider currently disables the RPM backend remap because
547+ the Vexxhost mirror does not carry the CentOS Stream 10 SIG repositories used
548+ by these images. Squid therefore caches their original repository URLs.
549+ Collapsed forwarding combines concurrent requests for the same object, and
550+ ` repomd.xml ` remains fresh for the two-hour job despite DNF cache-bypass
551+ headers. Squid access and error logs are staged under
552+ ` zuul-output/logs/squid/ ` for troubleshooting.
553+
440554In this repository the provider defaults to ` all ` , so every maintained image
441555is built from its exact ` sources.txt ` pins. A child job can set ` s2i_ci_images `
442556to an explicit list of image targets; the provider resolves the selection
0 commit comments