feat(cache): include Pigsty source in offline bundle

Author: Vonng

cache.yml

@@ -4,7 +4,7 @@
 # File      :   cache.yml
 # Desc      :   Create offline package cache from infra nodes
 # Ctime     :   2024-08-22
-# Mtime     :   2024-12-31
+# Mtime     :   2026-08-14
 # Path      :   cache.yml
 # Docs      :   https://pigsty.io/docs/setup/offline
 # License   :   Apache-2.0 @ https://pigsty.io/docs/about/license/
@@ -48,9 +48,10 @@
 #--------------------------------------------------------------#
 # This play runs on the target infra node to:
 #   1. Detect node identity (OS, version, architecture)
-#   2. Clean dirty packages and recreate repo metadata
-#   3. Create compressed tarball at /tmp/pkg.tgz
-#   4. Fetch tarball back to control node
+#   2. Include the versioned Pigsty source release artifact when available
+#   3. Clean dirty packages and recreate repo metadata
+#   4. Create compressed tarball at /tmp/pkg.tgz
+#   5. Fetch tarball back to control node
 #--------------------------------------------------------------#
 - name: MAKE OFFLINE PACKAGE
   hosts: all
@@ -61,6 +62,7 @@
     #cache_repo: pigsty                 # target repo(s) to be cached, use `,` to separate multiple repos
     #cache_pkg_dir: 'dist/${version}'   # where to store the cached package? dist/${version} by default
     #cache_pkg_name: 'pigsty-pkg-${version}.${os}.${arch}.tgz'  # cache offline package filename pattern
+    #cache_src: auto                    # source policy: auto, local, remote
   roles:
     - { role: node_id, tags: id    }   # detect node identity
     - { role: cache,   tags: cache }   # create and fetch cache tarball
@@ -109,6 +111,9 @@
 #   # Cache multiple repos
 #   ./cache.yml -l infra -e cache_repo=pigsty,minio
 #
+#   # Air-gapped build: use pre-staged dist/${version}/pigsty-${version}.tgz
+#   ./cache.yml -l infra -e cache_src=local
+#
 #==============================================================#
 # Tag Reference
 #==============================================================#
@@ -118,6 +123,7 @@
 #   cache        : full cache role execution
 #   cache_id     : calculate cache tarball filename
 #   cache_check  : verify repo directories exist and not empty
+#   cache_src    : include versioned Pigsty source release artifact
 #   cache_create : clean dirty packages, recreate repo metadata
 #   cache_tgz    : create /tmp/pkg.tgz tarball on target
 #   cache_fetch  : fetch tarball from target to control node

roles/cache/README.md

@@ -13,9 +13,10 @@
 The `cache` role creates an **offline package tarball** from an existing local repository:
 
 - Verify repository directories exist and are populated
+- Include the versioned Pigsty source release artifact when available
 - Clean up dirty/unwanted packages to reduce size
 - Recreate YUM/APT repository metadata
-- Create compressed tarball of all packages
+- Create a compressed tarball containing packages and optional source
 - Fetch tarball to control node for distribution
 
 This enables:
@@ -34,6 +35,7 @@ Before running the cache role:
 3. **`rsync` must be installed** on both control and target nodes (for `synchronize` module)
 4. **SOW 0.3.0 must be available** from the Pigsty INFRA repository
 5. The repo should contain all required packages (run `./infra.yml -t repo` first)
+6. To include source in an air-gapped build, stage `dist/<version>/pigsty-<version>.tgz` first
 
 
 ## Playbooks
@@ -79,6 +81,8 @@ cache                          # Full role execution
 │
 ├── cache_check                # Verify repo directories exist
 │
+├── cache_src                  # Resolve and include Pigsty source artifact
+│
 ├── cache_create               # Clean packages & recreate metadata
 │
 ├── cache_tgz                  # Create compressed tarball
@@ -97,6 +101,12 @@ cache                          # Full role execution
 | `cache_pkg_dir`  | `dist/${version}`                         | Output directory on control node       |
 | `cache_repo`     | `pigsty`                                  | Repo names to cache (CSV for multiple) |
 
+### Source Artifact Settings
+
+| Variable    | Default | Description                          |
+|-------------|---------|--------------------------------------|
+| `cache_src` | `auto`  | Policy: `auto`, `local`, or `remote` |
+
 ### Referenced Variables
 
 | Variable    | Default        | Description                              |
@@ -142,14 +152,61 @@ dist/<version>/pigsty-pkg-<version>.<os>.<arch>.tgz
 ```text
 pigsty/                        # repo_name directory
 ├── *.rpm or *.deb             # Package files
+├── pigsty-v4.5.0.tgz          # Pigsty source release artifact, when available
 ├── repodata/                  # YUM metadata (RPM only)
 │   ├── repomd.xml
 │   └── *-primary.xml.gz       # Plus filelists/other metadata
 ├── Packages                   # APT metadata (DEB only)
 ├── Packages.gz                # APT metadata (DEB only)
-└── repo_complete              # SHA-256 checksums marker
+└── repo_complete              # Package checksum marker
+```
+
+
+## Pigsty Source Artifact
+
+The cache role never packages the current checkout implicitly. It only looks for
+the versioned release artifact or downloads the same versioned filename.
+
+The `cache_src` policy determines provenance:
+
+| Policy   | Behavior |
+|----------|----------|
+| `auto`   | Try local `dist/<version>/pigsty-<version>.tgz`, then download from the versioned URL |
+| `local`  | Copy the local `dist` artifact when it exists; never access the network |
+| `remote` | Download `https://repo.pigsty.io/src/pigsty-<version>.tgz` directly |
+
+Source inclusion is best-effort. Copy or download errors are reported and
+ignored, so the package cache tarball is still produced without the source file.
+The source step runs only when `repo_name` is included in `cache_repo`, ensuring
+the source artifact is part of the resulting cache tarball. SOW scans RPM/DEB
+packages only and preserves the TGZ as a plain repository asset.
+
+For a disconnected build, prepare the artifact explicitly and force local-only
+resolution:
+
+```bash
+# Create this only when the current checkout is intentionally the release input
+bin/release v4.5.0
+
+# Never use network; a missing file is reported and ignored
+./cache.yml -l infra -e cache_src=local
+```
+
+For a normal connected release build, `auto` requires no extra arguments:
+
+```bash
+./cache.yml -l infra
+```
+
+With the default `repo_home` and `repo_name`, after extracting the outer cache
+archive the source contract is:
+
+```text
+/www/pigsty/pigsty-<version>.tgz
 ```
 
+This stable path can be consumed by a later bootstrap enhancement without
+guessing from the build checkout or requiring network access.
 
 
 ## Dirty Package Cleanup
@@ -232,8 +289,11 @@ ls -la /www/pigsty/
 # Only recreate repo metadata (skip fetch)
 ./cache.yml -l infra -t cache_create
 
-# Only create tarball and fetch (skip metadata)
-./cache.yml -l infra -t cache_tgz,cache_fetch
+# Only include source, create tarball and fetch (skip metadata)
+./cache.yml -l infra -t cache_src,cache_tgz,cache_fetch
+
+# Use a pre-staged local source artifact (never use network)
+./cache.yml -l infra -e cache_src=local
 
 # Show cache info only
 ./cache.yml -l infra -t cache_info

roles/cache/defaults/main.yml

@@ -6,6 +6,11 @@ cache_pkg_name: 'pigsty-pkg-${version}.${os}.${arch}.tgz' # cache offline packag
 cache_pkg_dir: 'dist/${version}'  # where to store the cached package? dist/${version} by default
 cache_repo: pigsty                # target repo(s) to be cached, use `,` to separate multiple repos
 
+#-----------------------------------------------------------------
+# SOURCE
+#-----------------------------------------------------------------
+cache_src: auto                   # source policy: auto, local, remote
+
 #-----------------------------------------------------------------
 # REPO (Reference)
 #-----------------------------------------------------------------

roles/cache/tasks/main.yml

@@ -13,6 +13,9 @@
     cache_filename: "{{ cache_pkg_name|replace('${version}', version)|replace('${os}', node_os_code)|replace('${arch}', os_arch) }}"
     cache_dirname: "{{ cache_pkg_dir | replace('${version}', version) }}"
     cache_repo_list: "{{ cache_repo.split(',') }}"
+    cache_src_local: "{{ playbook_dir }}/dist/{{ version }}/pigsty-{{ version }}.tgz"
+    cache_src_url: "https://repo.pigsty.io/src/pigsty-{{ version }}.tgz"
+    cache_src_target: "{{ repo_home }}/{{ repo_name }}/pigsty-{{ version }}.tgz"
 
 
 #--------------------------------------------------------------#
@@ -32,7 +35,32 @@
 
 
 #--------------------------------------------------------------#
-# 2. Create Repo                                  [cache_create]
+# 2. Include Pigsty Source                           [cache_src]
+#--------------------------------------------------------------#
+# Include the versioned Pigsty source artifact:
+#
+# - auto:   local release artifact, then URL
+# - local:  use the local release artifact when present
+# - remote: download the release artifact from the versioned URL
+#
+# Only include source when repo_name is archived by cache_repo.
+# SOW scans RPM/DEB files only and preserves this plain TGZ asset.
+#--------------------------------------------------------------#
+- name: cache_src
+  tags: cache_src
+  become: true
+  ignore_errors: true
+  when: repo_name in cache_repo_list
+  block:
+    - copy: { src: "{{ cache_src_local }}", dest: "{{ cache_src_target }}", mode: '0644' }
+      register: cache_src_copy
+      when: cache_src|lower in ['auto', 'local']
+    - get_url: { url: "{{ cache_src_url }}", dest: "{{ cache_src_target }}", mode: '0644', force: yes }
+      when: cache_src|lower == 'remote' or (cache_src|lower == 'auto' and cache_src_copy is failed)
+
+
+#--------------------------------------------------------------#
+# 3. Create Repo                                  [cache_create]
 #--------------------------------------------------------------#
 # Clean up dirty packages and recreate repository metadata
 #
@@ -59,7 +87,7 @@
 
 
 #--------------------------------------------------------------#
-# 3. Create Cache Tarball                            [cache_tgz]
+# 4. Create Cache Tarball                            [cache_tgz]
 #--------------------------------------------------------------#
 # Create compressed tarball of all repo directories
 # Tarball is created at /tmp/pkg.tgz on the target node
@@ -76,7 +104,7 @@
 
 
 #--------------------------------------------------------------#
-# 4. Fetch Cache Tarball                           [cache_fetch]
+# 5. Fetch Cache Tarball                           [cache_fetch]
 #--------------------------------------------------------------#
 # Pull tarball from target node to control node using rsync
 # Destination: {{ cache_dirname }}/{{ cache_filename }}