Skip to content

Commit e2ea136

Browse files
deb: build the packages from debian/ with build-deb-action
debian/ becomes a Debian source package: the Build-Depends in control, a rules file with dh, a changelog and source/format. jtdor/build-deb-action installs the Build-Depends in a container of each release and runs dpkg-buildpackage. That replaces Dockerfile.linux, docker/build.sh and the buildx, cache and retry steps of the workflow, and dh computes the dependencies of the package, so nothing patches the control file or runs dpkg-shlibdeps by hand any more. debian/changelog does not repeat the version: debian/set-version.sh adds the entry for the version of pyproject.toml and the release, and the package is still named python3-monero_0.9.10-1noble1_amd64.deb. bin/build_monero_cpp.sh builds monero-project and monero-cpp, and debian/install-static-deps.sh the static expat and unbound, as the Dockerfile did. The flags of dpkg-buildflags are not passed to them. The packages are built on: - pull requests: amd64 on trixie and bookworm. - main: also amd64 on noble and jammy, and arm64 on trixie and bookworm. - tags and manual runs with full: the three architectures on the four releases. release.yml checks the tag and the runs of main that it needs first, then calls build-deb.yml and takes the packages from its own run, so a missing run does not show up after the hours of the build. arm64 and armhf are built natively on ubuntu-24.04-arm runners, which run 32 bit arm containers, instead of under qemu, where they took 76 to 136 minutes against 10 to 14 for amd64. A 32 bit container on a 64 bit arm kernel reports aarch64, which build_monero_cpp.sh would take for a 64 bit build and the configure scripts for the host, so debian/rules runs the steps of the build under linux32 there and the script knows armv7l and armv8l. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
1 parent e813f58 commit e2ea136

21 files changed

Lines changed: 336 additions & 332 deletions

‎.github/workflows/build-deb.yml‎

Lines changed: 41 additions & 101 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,10 @@
11
name: Build Debian Packages
22

3+
# Builds the package from debian/ with dpkg-buildpackage in a container of each release.
4+
# Pull requests build amd64 on trixie and bookworm. Main also builds amd64 on noble and jammy
5+
# and arm64 on trixie and bookworm. The release (called by release.yml) and the manual runs with
6+
# full build all of them. arm64 and armhf are built natively on arm runners, which run 32 bit
7+
# arm containers.
38
on:
49
push:
510
branches: [ main ]
@@ -8,10 +13,19 @@ on:
813
paths:
914
- '.github/workflows/build-deb.yml'
1015
- 'debian/**'
11-
- 'docker/**'
12-
- 'Dockerfile.linux'
16+
- 'bin/build_monero_cpp.sh'
1317
workflow_dispatch:
18+
inputs:
19+
full:
20+
description: Build every release and architecture
21+
type: boolean
22+
default: false
1423
workflow_call:
24+
inputs:
25+
full:
26+
description: Build every release and architecture
27+
type: boolean
28+
default: false
1529

1630
# A new push to a pull request cancels the previous run. Other events get their own
1731
# group so they are never cancelled or replaced.
@@ -26,60 +40,22 @@ permissions:
2640
jobs:
2741
build:
2842
name: ${{ matrix.distro_name }} (${{ matrix.name }} / ${{ matrix.arch }})
29-
runs-on: ubuntu-latest
43+
runs-on: ${{ matrix.arch == 'amd64' && 'ubuntu-latest' || 'ubuntu-24.04-arm' }}
44+
timeout-minutes: 360
3045

3146
strategy:
3247
fail-fast: false
3348
matrix:
49+
name: [noble, jammy, trixie, bookworm]
50+
arch: [amd64, arm64, armhf]
51+
# pull requests build amd64 on trixie and bookworm, main adds amd64 on the Ubuntu releases and
52+
# arm64 on the Debian ones, the tags and the manual runs with full build everything
53+
exclude: ${{ fromJSON((github.ref_type == 'tag' || inputs.full) && '[]' || (github.event_name == 'pull_request' && '[{"arch":"armhf"},{"arch":"arm64"},{"name":"noble"},{"name":"jammy"}]' || '[{"arch":"armhf"},{"arch":"arm64","name":"noble"},{"arch":"arm64","name":"jammy"}]')) }}
3454
include:
35-
- name: noble
36-
distro_name: ubuntu
37-
base_image: ubuntu:24.04
38-
arch: amd64
39-
- name: noble
40-
distro_name: ubuntu
41-
base_image: ubuntu:24.04
42-
arch: arm64
43-
- name: noble
44-
distro_name: ubuntu
45-
base_image: ubuntu:24.04
46-
arch: armhf
47-
- name: jammy
48-
distro_name: ubuntu
49-
base_image: ubuntu:22.04
50-
arch: amd64
51-
- name: jammy
52-
distro_name: ubuntu
53-
base_image: ubuntu:22.04
54-
arch: arm64
55-
- name: jammy
56-
distro_name: ubuntu
57-
base_image: ubuntu:22.04
58-
arch: armhf
59-
- name: trixie
60-
distro_name: debian
61-
base_image: debian:trixie
62-
arch: amd64
63-
- name: trixie
64-
distro_name: debian
65-
base_image: debian:trixie
66-
arch: arm64
67-
- name: trixie
68-
distro_name: debian
69-
base_image: debian:trixie
70-
arch: armhf
71-
- name: bookworm
72-
distro_name: debian
73-
base_image: debian:bookworm
74-
arch: amd64
75-
- name: bookworm
76-
distro_name: debian
77-
base_image: debian:bookworm
78-
arch: arm64
79-
- name: bookworm
80-
distro_name: debian
81-
base_image: debian:bookworm
82-
arch: armhf
55+
- { name: noble, distro_name: ubuntu, image: 'ubuntu:24.04' }
56+
- { name: jammy, distro_name: ubuntu, image: 'ubuntu:22.04' }
57+
- { name: trixie, distro_name: debian, image: 'debian:trixie-slim' }
58+
- { name: bookworm, distro_name: debian, image: 'debian:bookworm-slim' }
8359

8460
steps:
8561
- name: Checkout repository
@@ -90,43 +66,14 @@ jobs:
9066
- name: Determine package version
9167
id: version
9268
run: |
93-
VERSION=$(grep -m1 '^version' pyproject.toml | sed -E 's/version = "(.*)"/\1/')
69+
VERSION=$(sed -n 's/^version = "\(.*\)"$/\1/p' pyproject.toml | head -n 1)
9470
echo "version=${VERSION}" >> "$GITHUB_OUTPUT"
9571
96-
- name: Setup qemu
97-
if: matrix.arch == 'arm64' || matrix.arch == 'armhf'
98-
uses: docker/setup-qemu-action@96fe6ef7f33517b61c61be40b68a1882f3264fb8 # v4.2.0
99-
100-
- name: Setup docker buildx
101-
uses: docker/setup-buildx-action@bb05f3f5519dd87d3ba754cc423b652a5edd6d2c # v4.2.0
102-
10372
- name: Free up disk space
10473
run: |
10574
sudo rm -rf /usr/share/dotnet /usr/local/lib/android /opt/ghc || true
10675
docker system prune -af || true
10776
108-
- name: Prepare build environment
109-
uses: nick-fields/retry@ad984534de44a9489a53aefd81eb77f87c70dc60 # v4.0.0
110-
with:
111-
timeout_minutes: 60
112-
max_attempts: 3
113-
retry_wait_seconds: 15
114-
command: |
115-
PLATFORM=${{ matrix.arch == 'amd64' && 'linux/amd64' ||
116-
matrix.arch == 'armhf' && 'linux/arm/v7' ||
117-
'linux/arm64' }}
118-
119-
docker buildx build \
120-
--platform $PLATFORM \
121-
--build-arg BASE_IMAGE=${{ matrix.base_image }} \
122-
--build-arg THREADS=3 \
123-
-t monero-python:${{ matrix.name }}-${{ matrix.arch }} \
124-
-f Dockerfile.linux \
125-
--cache-from type=gha,scope=${{ matrix.name }}-${{ matrix.arch }} \
126-
--cache-to type=gha,mode=max,scope=${{ matrix.name }}-${{ matrix.arch }} \
127-
--load \
128-
.
129-
13077
- name: Restore ccache
13178
uses: actions/cache/restore@v4
13279
with:
@@ -135,26 +82,19 @@ jobs:
13582
restore-keys: |
13683
ccache-deb-${{ matrix.name }}-${{ matrix.arch }}_
13784
138-
- name: Build monero-python
139-
uses: nick-fields/retry@ad984534de44a9489a53aefd81eb77f87c70dc60 # v4.0.0
85+
# Installs the Build-Depends of debian/control in the container, sets the version of
86+
# debian/changelog from pyproject.toml and runs dpkg-buildpackage. The workspace is
87+
# mounted at /github/workspace in the container.
88+
- name: Build package
89+
uses: jtdor/build-deb-action@473136f14a42a231124c1f7cefaedfe941203c38 # v1.9.1
90+
env:
91+
CCACHE_DIR: /github/workspace/.ccache
92+
CCACHE_MAXSIZE: 500M
14093
with:
141-
timeout_minutes: 240
142-
max_attempts: 3
143-
retry_wait_seconds: 15
144-
command: |
145-
PLATFORM=${{ matrix.arch == 'amd64' && 'linux/amd64' ||
146-
matrix.arch == 'armhf' && 'linux/arm/v7' ||
147-
'linux/arm64' }}
148-
PACKAGE_NAME="python3-monero_${{ steps.version.outputs.version }}-1${{ matrix.name }}1_${{ matrix.arch }}"
149-
150-
docker run --rm \
151-
--platform $PLATFORM \
152-
-v $PWD:/monero-python -w /monero-python \
153-
-e PACKAGE_NAME="${PACKAGE_NAME}" \
154-
-e CCACHE_DIR=/monero-python/.ccache \
155-
-e CCACHE_MAXSIZE=500M \
156-
monero-python:${{ matrix.name }}-${{ matrix.arch }} \
157-
bash -euxc "./docker/build.sh"
94+
docker-image: ${{ matrix.image }}
95+
extra-docker-args: --platform ${{ matrix.arch == 'armhf' && 'linux/arm/v7' || format('linux/{0}', matrix.arch) }}
96+
before-build-hook: sh debian/set-version.sh ${{ matrix.name }}
97+
buildpackage-opts: --build=binary --no-sign --jobs=auto
15898

15999
- name: Save ccache
160100
# Pull requests only read the cache, pushes to main and manual runs replace it.
@@ -167,4 +107,4 @@ jobs:
167107
uses: actions/upload-artifact@v4
168108
with:
169109
name: python3-monero_${{ steps.version.outputs.version }}-1${{ matrix.name }}1_${{ matrix.arch }}
170-
path: build/*.deb
110+
path: debian/artifacts/python3-monero_*.deb

‎.github/workflows/release.yml‎

Lines changed: 31 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -10,12 +10,14 @@ permissions:
1010
actions: read
1111

1212
jobs:
13-
release:
14-
name: Publish draft release
13+
# the macos and windows builds and the tests are those of main, the deb ones are built here
14+
check:
15+
name: Check the tag and the builds of main
1516
runs-on: ubuntu-latest
16-
permissions:
17-
contents: write
18-
actions: read
17+
outputs:
18+
build_macos_run_id: ${{ steps.find_runs.outputs.build_macos_run_id }}
19+
build_windows_run_id: ${{ steps.find_runs.outputs.build_windows_run_id }}
20+
test_run_id: ${{ steps.find_runs.outputs.test_run_id }}
1921

2022
steps:
2123
- name: Check the tag against the version in pyproject.toml
@@ -40,7 +42,7 @@ jobs:
4042
env:
4143
GH_TOKEN: ${{ github.token }}
4244
run: |
43-
for wf in build-deb.yml build-macos.yml build-windows.yml test.yml; do
45+
for wf in build-macos.yml build-windows.yml test.yml; do
4446
key=$(echo "${wf%.*}" | tr '-' '_')
4547
run_id=$(gh run list --repo "${{ github.repository }}" --workflow "$wf" \
4648
--branch main --status success --json databaseId,headSha \
@@ -52,11 +54,29 @@ jobs:
5254
echo "${key}_run_id=${run_id}" >> "$GITHUB_OUTPUT"
5355
done
5456
57+
# every release and architecture, the builds of main only have some of them
58+
deb:
59+
name: Build the deb packages
60+
needs: check
61+
uses: ./.github/workflows/build-deb.yml
62+
permissions:
63+
contents: read
64+
actions: write # build-deb.yml replaces its ccache entries
65+
with:
66+
full: true
67+
68+
release:
69+
name: Publish draft release
70+
needs: [check, deb]
71+
runs-on: ubuntu-latest
72+
permissions:
73+
contents: write
74+
actions: read
75+
76+
steps:
5577
- name: Download deb packages
5678
uses: actions/download-artifact@v4
5779
with:
58-
github-token: ${{ github.token }}
59-
run-id: ${{ steps.find_runs.outputs.build_deb_run_id }}
6080
pattern: python3-monero_*
6181
path: release-assets
6282
merge-multiple: true
@@ -65,7 +85,7 @@ jobs:
6585
uses: actions/download-artifact@v4
6686
with:
6787
github-token: ${{ github.token }}
68-
run-id: ${{ steps.find_runs.outputs.build_macos_run_id }}
88+
run-id: ${{ needs.check.outputs.build_macos_run_id }}
6989
pattern: monero-macos-*
7090
path: release-assets
7191
merge-multiple: true
@@ -74,7 +94,7 @@ jobs:
7494
uses: actions/download-artifact@v4
7595
with:
7696
github-token: ${{ github.token }}
77-
run-id: ${{ steps.find_runs.outputs.build_windows_run_id }}
97+
run-id: ${{ needs.check.outputs.build_windows_run_id }}
7898
name: monero-python-win-amd64
7999
path: release-assets
80100
merge-multiple: true
@@ -83,7 +103,7 @@ jobs:
83103
uses: actions/download-artifact@v4
84104
with:
85105
github-token: ${{ github.token }}
86-
run-id: ${{ steps.find_runs.outputs.test_run_id }}
106+
run-id: ${{ needs.check.outputs.test_run_id }}
87107
name: coverage-reports-linux
88108
path: coverage-report
89109

‎.gitignore‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,3 +17,11 @@ coverage*
1717
monero.log
1818
monero_tests_*.log
1919
.ccache
20+
# left by dpkg-buildpackage
21+
debian/artifacts
22+
debian/python3-monero
23+
debian/.debhelper
24+
debian/debhelper-build-stamp
25+
debian/files
26+
debian/*.substvars
27+
debian/*.log

‎Dockerfile.linux‎

Lines changed: 0 additions & 46 deletions
This file was deleted.

‎README.md‎

Lines changed: 13 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -146,32 +146,30 @@ On anything else, build it from source.
146146

147147
> **Faster rebuilds:** if [ccache](https://ccache.dev/) is installed it is used automatically for monero and monero-python (pass `-DUSE_CCACHE=OFF` to cmake to disable it). To use it for monero-cpp as well, build that with `-DCMAKE_C_COMPILER_LAUNCHER=ccache -DCMAKE_CXX_COMPILER_LAUNCHER=ccache`.
148148

149-
### Linux Docker Build
149+
### Debian package
150+
151+
The package is built from [debian/](debian) with `dpkg-buildpackage`, the way CI does it. It installs the build dependencies and some static libraries system wide, so build it in a container of the release you want it for (`ubuntu:24.04`, `ubuntu:22.04`, `debian:trixie` or `debian:bookworm`).
150152

151153
1. Install [Docker](https://docs.docker.com/engine/install/)
152154
2. Clone the repository
153155
```sh
154156
git clone --recursive https://github.com/libmonero/monero-python.git
155157
```
156-
3. Prepare build environment
157-
```sh
158-
cd monero-python
159-
docker build --tag monero-python:build-linux --file Dockerfile.linux .
160-
```
161-
4. (Optional) Specify build image
162-
```sh
163-
cd monero-python
164-
docker build --tag monero-python:build-linux --build-arg BASE_IMAGE="ubuntu:22.04" --file Dockerfile.linux .
165-
```
166-
167-
5. Build
158+
3. Build, replacing `ubuntu:24.04` with the release you want
168159
```sh
169-
docker run --rm -it -v <MONERO_PYTHON_DIR_FULL_PATH>:/monero-python -w /monero-python monero-python:build-linux bash ./docker/build.sh
160+
docker run --rm -it -v <MONERO_PYTHON_DIR_FULL_PATH>:/monero-python -w /monero-python ubuntu:24.04 bash -c '
161+
apt-get update && apt-get install -y dpkg-dev &&
162+
apt-get build-dep -y ./ &&
163+
debian/set-version.sh &&
164+
dpkg-buildpackage --build=binary --no-sign &&
165+
mkdir -p debian/artifacts && mv ../python3-monero_* debian/artifacts/'
170166
```
171167

172168
* `MONERO_PYTHON_DIR_FULL_PATH` - full path to `monero-python` directory
169+
* `debian/set-version.sh` adds an entry for the version of `pyproject.toml` to `debian/changelog`, there is no need to commit it
170+
* add `--jobs=4` to `dpkg-buildpackage` to build faster, each compiler needs 1 to 2 GB of memory
173171

174-
6. Library build will be placed in `monero-python/build` directory
172+
4. The package is placed in `monero-python/debian/artifacts`
175173

176174
### Windows
177175

0 commit comments

Comments
 (0)