Repository navigation
Commit f4b8f37
authored
FEAT: add standalone conda build pipeline + build-validation tooling (#744)
### Work Item / Issue Reference
>
[AB#47315](https://sqlclientdrivers.visualstudio.com/c6d89619-62de-46a0-8b46-70b92a84d85e/_workitems/edit/47315)
### Summary
This pull request introduces a new standalone OneBranch pipeline for
building, validating, and consolidating self-contained `mssql-python`
conda packages across all supported platforms and Python versions. The
pipeline is carefully structured to ensure per-platform correctness,
best-effort artifact gathering, and robust isolation between build legs.
It also includes a template job for consolidating conda artifacts and a
`.gitattributes` update for shell scripts.
**New Standalone Conda Build Pipeline**
*Major features and structure:*
- **Adds `conda-build-pipeline.yml`**: Implements a full multi-stage
pipeline that builds and validates self-contained `mssql-python` conda
packages for Windows, macOS, and Linux, including cross-platform builds
(e.g., win-arm64, osx-arm64, linux-aarch64) using platform-appropriate
agents and emulation where needed. Each leg is carefully isolated to
prevent failures in one subdir from impacting others, and best-effort
logic is used for cross-arch builds.
- **Artifact Consolidation**:
- **Adds `consolidate-conda-artifacts-job.yml`**: Provides a job
template that collects all per-platform conda packages into a single
artifact, preserving the subdir structure. The consolidation is
best-effort in the build pipeline, with hard gating deferred to the
release pipeline to avoid blocking primary wheel deliverables.
**Build and Release Process Improvements**
- **Best-Effort and Gating Strategy**: The pipeline is designed so that
conda packaging failures do not block wheel releases. Completeness and
artifact presence are strictly enforced only at release time, not during
the build.
[[1]](diffhunk://#diff-e258bc0778f9b125cb5214aecba116312126f1a37e3db101606171b05a2b8302R1-R349)
[[2]](diffhunk://#diff-6a4150f95dfbb94f92e9ec3f674414deb459b23077ac34a99deb40b0479aee29R1-R129)
**Cross-Platform and Cross-Arch Handling**
- **Cross-Build Logic**: Implements cross-building for win-arm64,
osx-arm64, and linux-aarch64 using CONDA_SUBDIR and QEMU emulation as
appropriate, with architecture-specific validation and clear
blast-radius isolation to prevent a single platform???????????????s
failure from impacting others.
**Repository Hygiene**
- **Shell Script Line Endings**:
- Adds `OneBranchPipelines/scripts/.gitattributes` enforcing LF line
endings for `.sh` files under that folder (the conda recipe scripts are
covered separately by `conda/.gitattributes`).
**Build-Validation Tooling + Tests**
*The "build-validation tooling" named in the title -- the exact set of
files in this slice:*
- **`OneBranchPipelines/scripts/build_conda_packages.py`**: the single
cross-platform orchestrator (gather wheels -> Miniforge/conda-build ->
build the self-contained package -> masking-immune audit -> solve a
fresh env + import + driver-load + optional reachability gate -> stage).
Replaces the former per-OS PowerShell/bash build scripts.
- **`OneBranchPipelines/steps/conda-build-validate-step.yml` +
`-posix.yml`**: the Windows / POSIX step templates that invoke the
orchestrator.
- **`eng/scripts/audit_bundled_binaries.py`, `assert_pe_machine.py`,
`assert_macho_arch.py`, `_conda_pkg.py`**: masking-immune RUNPATH,
ELF/PE/Mach-O architecture, required Linux driver-inventory, and
package-reader validation.
- **`requirements.txt`**: declares `zstandard` so `.conda` payload tests
execute on Python versions without the stdlib `compression.zstd`
backend.
- **`conda/driver_load_probe.py`**: the DB-less driver-load proof.
- **`tests/test_029_bundled_binary_audit.py`,
`test_030_pe_machine_assert.py`, `test_033_driver_load_probe.py`,
`test_034_conda_verify_cwd.py`, `test_035_conda_macho_assert.py`**: unit
coverage for Linux payload completeness/RUNPATH, PE and Mach-O
architecture checks, the driver-load probe, neutral-cwd verification,
and blocking win-arm64 environment installation.
**Not in this slice (deliberately):** the live `Encrypt=yes` TLS gate
(`conda/tls_connect_probe.py` and its `test_028`/`test_031`) is a
*release*-pipeline concern -- the build pipeline never enables it (no
leg sets `CONDA_TLS_PROBE_CONN`), so it ships with the release slice,
not here. The masking-immune static RUNPATH audit already guards the
OpenSSL layout at build time.1 parent 4d029f8 commit f4b8f37
17 files changed
Lines changed: 4972 additions & 0 deletions
File tree
- OneBranchPipelines
- jobs
- scripts
- steps
- conda
- eng/scripts
- tests
Large diffs are not rendered by default.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
0 commit comments