Skip to content

Commit 7938b3a

Browse files
jcorredor-specyenienserrano
jcorredor-spec
and
yenienserrano
committed
Migrate dashboard package-building workflow runners to AWS CodeBuild (#1373)
* Migrate dashboard package-building workflow runners to AWS CodeBuild Replace the GitHub-hosted `ubuntu-latest` runners in the 4.x package build workflows with AWS CodeBuild runners (codebuild-github-actions-codebuild-runner-dashboard-<arch>-<run_id>-<run_attempt>), backporting the runner migration done in #1372. - 4_builderpackage_dashboard.yml: setup-variables and validate-job to the amd runner; build-and-test-package to the arch-conditional runner. - 4_builderpackage_dashboard_core.yml: build job to the arch-conditional runner (matrix.ARCHITECTURE). - build_base.yml / build_wazuh_dashboard_with_plugins.yml: route arm builds to CodeBuild, keep amd on GitHub-hosted runners (mirrors #1372). Closes opensearch-project#5246 Signed-off-by: jcorredor-spec <juli.ricar0413@gmail.com> * Commit to test * Remove support for arm64 architecture in package build workflow * Remove arm64 runner configuration from dashboard build workflow * Refactor build workflow to create non-root user and streamline build process * Refactor dashboard build workflows to use github.workspace for artifact paths * Add Docker Hub credentials to build plugins job in workflow * Create non-root user for build and test steps in workflow * Add Docker Hub login step to avoid anonymous pull rate limits in workflow * Add Docker Hub login step to avoid pull rate limits in AWS CodeBuild workflow * Fix indentation for Docker Hub login step in AWS CodeBuild workflow * Refactor build workflow to create non-root user after Docker Hub login and copy credentials for authenticated image pulls * Revert commit to test --------- Signed-off-by: jcorredor-spec <juli.ricar0413@gmail.com> Co-authored-by: Ian Yenien Serrano <63758389+yenienserrano@users.noreply.github.com>
1 parent 868c488 commit 7938b3a

4 files changed

Lines changed: 61 additions & 38 deletions

File tree

‎.github/workflows/4_builderpackage_dashboard.yml‎

Lines changed: 47 additions & 26 deletions
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@
1111
#
1212
# - Allows customization of:
1313
# - Operating system (`deb` or `rpm`)
14-
# - Architecture (`amd64`, `x86_64`, `aarch64`, `arm64`)
14+
# - Architecture (`amd64`, `x86_64`)
1515
# - Package revision
1616
# - Plugin references (branches, tags, or commits)
1717
# - Staging, upload, and checksum options.
@@ -104,10 +104,9 @@ on:
104104

105105
jobs:
106106
setup-variables:
107-
runs-on: ubuntu-latest
107+
runs-on: codebuild-github-actions-codebuild-runner-dashboard-amd-${{ github.run_id }}-${{ github.run_attempt }}
108108
name: Setup variables
109109
outputs:
110-
CURRENT_DIR: ${{ steps.setup-variables.outputs.CURRENT_DIR }}
111110
VERSION: ${{ steps.setup-variables.outputs.VERSION }}
112111
REVISION: ${{ steps.setup-variables.outputs.REVISION }}
113112
COMMIT_SHA: ${{ steps.setup-variables.outputs.COMMIT_SHA }}
@@ -131,7 +130,6 @@ jobs:
131130
- name: Setup variables
132131
id: setup-variables
133132
run: |
134-
CURRENT_DIR=$(pwd -P)
135133
VERSION=$(tail -c +2 VERSION)
136134
REVISION=$(yarn --silent wzd-revision)
137135
COMMIT_SHA=$(git rev-parse --short HEAD)
@@ -158,7 +156,6 @@ jobs:
158156
PACKAGE_NAME=wazuh-dashboard_${VERSION}-${{ inputs.revision }}_${{ inputs.architecture }}_${COMMIT_SHA}.rpm
159157
fi
160158
fi
161-
echo "CURRENT_DIR=$CURRENT_DIR" >> $GITHUB_OUTPUT
162159
echo "VERSION=$VERSION" >> $GITHUB_OUTPUT
163160
echo "REVISION=$REVISION" >> $GITHUB_OUTPUT
164161
echo "COMMIT_SHA=$COMMIT_SHA" >> $GITHUB_OUTPUT
@@ -171,7 +168,7 @@ jobs:
171168
echo "PACKAGE_NAME=$PACKAGE_NAME" >> $GITHUB_OUTPUT
172169
173170
validate-job:
174-
runs-on: ubuntu-latest
171+
runs-on: codebuild-github-actions-codebuild-runner-dashboard-amd-${{ github.run_id }}-${{ github.run_attempt }}
175172
needs: setup-variables
176173
name: Validate inputs
177174
steps:
@@ -208,6 +205,9 @@ jobs:
208205
uses: wazuh/wazuh-dashboard-plugins/.github/workflows/4_builderpackage_plugins.yml@4.10.5
209206
with:
210207
reference: ${{ inputs.reference_wazuh_plugins }}
208+
secrets:
209+
DOCKERHUB_USERNAME: ${{ secrets.DOCKERHUB_USERNAME }}
210+
DOCKERHUB_TOKEN: ${{ secrets.DOCKERHUB_TOKEN }}
211211

212212
build-security-plugin:
213213
needs: [validate-job]
@@ -220,7 +220,7 @@ jobs:
220220

221221
build-and-test-package:
222222
needs: [setup-variables, build-main-plugins, build-base, build-security-plugin]
223-
runs-on: ubuntu-latest
223+
runs-on: 'codebuild-github-actions-codebuild-runner-dashboard-amd-${{ github.run_id }}-${{ github.run_attempt }}'
224224
name: Generate packages
225225
steps:
226226
- name: Checkout code
@@ -236,53 +236,74 @@ jobs:
236236
uses: actions/download-artifact@v4
237237
with:
238238
name: ${{ needs.setup-variables.outputs.WAZUH_DASHBOARD_SLIM }}
239-
path: ${{ needs.setup-variables.outputs.CURRENT_DIR }}/artifacts/dashboard
239+
path: ${{ github.workspace }}/artifacts/dashboard
240240

241241
- name: Download security plugin artifact
242242
uses: actions/download-artifact@v4
243243
with:
244244
name: ${{ needs.setup-variables.outputs.WAZUH_SECURITY_PLUGIN }}
245-
path: ${{ needs.setup-variables.outputs.CURRENT_DIR }}/artifacts/security-plugin
245+
path: ${{ github.workspace }}/artifacts/security-plugin
246246

247247
- name: Download main plugin's artifact
248248
uses: actions/download-artifact@v4
249249
with:
250250
name: ${{ needs.setup-variables.outputs.WAZUH_PLUGINS_WAZUH }}
251-
path: ${{ needs.setup-variables.outputs.CURRENT_DIR }}/artifacts/plugins
251+
path: ${{ github.workspace }}/artifacts/plugins
252252
- name: Download core plugin's artifact
253253
uses: actions/download-artifact@v4
254254
with:
255255
name: ${{ needs.setup-variables.outputs.WAZUH_PLUGINS_CORE }}
256-
path: ${{ needs.setup-variables.outputs.CURRENT_DIR }}/artifacts/plugins
256+
path: ${{ github.workspace }}/artifacts/plugins
257257
- name: Download check update plugin's artifact
258258
uses: actions/download-artifact@v4
259259
with:
260260
name: ${{ needs.setup-variables.outputs.WAZUH_PLUGINS_CHECK_UPDATES }}
261-
path: ${{ needs.setup-variables.outputs.CURRENT_DIR }}/artifacts/plugins
261+
path: ${{ github.workspace }}/artifacts/plugins
262262

263263
- name: Zip plugins
264264
run: |
265-
zip -r -j ${{ needs.setup-variables.outputs.CURRENT_DIR }}/artifacts/wazuh-package.zip ${{ needs.setup-variables.outputs.CURRENT_DIR }}/artifacts/plugins
266-
zip -r -j ${{ needs.setup-variables.outputs.CURRENT_DIR }}/artifacts/security-package.zip ${{ needs.setup-variables.outputs.CURRENT_DIR }}/artifacts/security-plugin
267-
zip -r -j ${{ needs.setup-variables.outputs.CURRENT_DIR }}/artifacts/dashboard-package.zip ${{ needs.setup-variables.outputs.CURRENT_DIR }}/artifacts/dashboard/${{ needs.setup-variables.outputs.WAZUH_DASHBOARD_SLIM }}
265+
zip -r -j ${{ github.workspace }}/artifacts/wazuh-package.zip ${{ github.workspace }}/artifacts/plugins
266+
zip -r -j ${{ github.workspace }}/artifacts/security-package.zip ${{ github.workspace }}/artifacts/security-plugin
267+
zip -r -j ${{ github.workspace }}/artifacts/dashboard-package.zip ${{ github.workspace }}/artifacts/dashboard/${{ needs.setup-variables.outputs.WAZUH_DASHBOARD_SLIM }}
268+
269+
270+
# Workaround: AWS CodeBuild runners hit Docker Hub anonymous pull rate limits
271+
- name: Login to Docker Hub
272+
uses: docker/login-action@v3
273+
with:
274+
username: ${{ secrets.DOCKERHUB_USERNAME }}
275+
password: ${{ secrets.DOCKERHUB_TOKEN }}
276+
277+
# Workaround: Docker containers run as root internally, so output files are root-owned on the host. Create a non-root user to run the build and test steps.
278+
- name: Create non-root build user
279+
run: |
280+
useradd -m builduser
281+
usermod -aG docker builduser
282+
chown -R builduser:builduser ${{ github.workspace }}
283+
# Copy Docker Hub credentials so builduser can pull authenticated images
284+
mkdir -p /home/builduser/.docker
285+
cp /root/.docker/config.json /home/builduser/.docker/config.json
286+
chown -R builduser:builduser /home/builduser/.docker
268287
269288
- name: Build package
270289
run: |
271-
cd ${{ needs.setup-variables.outputs.CURRENT_DIR }}/dev-tools/build-packages
272-
bash ./build-packages.sh \
290+
cd ${{ github.workspace }}/dev-tools/build-packages
291+
sudo -u builduser -H env PATH="$PATH" bash ./build-packages.sh \
273292
-v ${{ needs.setup-variables.outputs.VERSION }} \
274293
-r ${{ inputs.revision }} \
275-
-a file://${{needs.setup-variables.outputs.CURRENT_DIR}}/artifacts/wazuh-package.zip \
276-
-s file://${{needs.setup-variables.outputs.CURRENT_DIR}}/artifacts/security-package.zip \
277-
-b file://${{needs.setup-variables.outputs.CURRENT_DIR}}/artifacts/dashboard-package.zip \
294+
-a file://${{github.workspace}}/artifacts/wazuh-package.zip \
295+
-s file://${{github.workspace}}/artifacts/security-package.zip \
296+
-b file://${{github.workspace}}/artifacts/dashboard-package.zip \
278297
--${{ inputs.system }} ${{ needs.setup-variables.outputs.PRODUCTION }}
298+
# Workaround: Docker containers run as root internally, so output files are root-owned on the host.
299+
chown -R builduser:builduser output 2>/dev/null || true
279300
280301
- name: Test package
281302
run: |
282-
cd ${{ needs.setup-variables.outputs.CURRENT_DIR }}/dev-tools/test-packages
283-
ls -la ${{ needs.setup-variables.outputs.CURRENT_DIR }}/dev-tools/build-packages/output/${{ inputs.system }}
284-
cp ${{ needs.setup-variables.outputs.CURRENT_DIR }}/dev-tools/build-packages/output/${{ inputs.system }}/${{needs.setup-variables.outputs.PACKAGE_NAME}} ${{ needs.setup-variables.outputs.CURRENT_DIR }}/dev-tools/test-packages/${{ inputs.system }}
285-
bash ./test-packages.sh \
303+
cd ${{ github.workspace }}/dev-tools/test-packages
304+
ls -la ${{ github.workspace }}/dev-tools/build-packages/output/${{ inputs.system }}
305+
sudo -u builduser -H env PATH="$PATH" cp ${{ github.workspace }}/dev-tools/build-packages/output/${{ inputs.system }}/${{needs.setup-variables.outputs.PACKAGE_NAME}} ${{ github.workspace }}/dev-tools/test-packages/${{ inputs.system }}
306+
sudo -u builduser -H env PATH="$PATH" bash ./test-packages.sh \
286307
-p ${{needs.setup-variables.outputs.PACKAGE_NAME}}
287308
288309
- name: Set up AWS CLI
@@ -295,14 +316,14 @@ jobs:
295316
- name: Upload package
296317
run: |
297318
echo "Uploading package"
298-
aws s3 cp ${{ needs.setup-variables.outputs.CURRENT_DIR }}/dev-tools/build-packages/output/${{ inputs.system }}/${{needs.setup-variables.outputs.PACKAGE_NAME}} s3://xdrsiem-packages-dev-internal/development/wazuh/4.x/main/packages/
319+
aws s3 cp ${{ github.workspace }}/dev-tools/build-packages/output/${{ inputs.system }}/${{needs.setup-variables.outputs.PACKAGE_NAME}} s3://xdrsiem-packages-dev-internal/development/wazuh/4.x/main/packages/
299320
s3uri="s3://xdrsiem-packages-dev-internal/development/wazuh/4.x/main/packages/${{needs.setup-variables.outputs.PACKAGE_NAME}}"
300321
echo "S3 URI: ${s3uri}"
301322
302323
- name: Upload SHA512
303324
if: ${{ inputs.checksum }}
304325
run: |
305326
echo "Uploading checksum"
306-
aws s3 cp ${{ needs.setup-variables.outputs.CURRENT_DIR }}/dev-tools/build-packages/output/${{ inputs.system }}/${{needs.setup-variables.outputs.PACKAGE_NAME}}.sha512 s3://xdrsiem-packages-dev-internal/development/wazuh/4.x/main/packages/
327+
aws s3 cp ${{ github.workspace }}/dev-tools/build-packages/output/${{ inputs.system }}/${{needs.setup-variables.outputs.PACKAGE_NAME}}.sha512 s3://xdrsiem-packages-dev-internal/development/wazuh/4.x/main/packages/
307328
s3uri="s3://xdrsiem-packages-dev-internal/development/wazuh/4.x/main/packages/${{needs.setup-variables.outputs.PACKAGE_NAME}}.sha512"
308329
echo "S3 sha512 URI: ${s3uri}"

‎.github/workflows/4_builderpackage_dashboard_core.yml‎

Lines changed: 12 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -28,7 +28,7 @@ on:
2828

2929
jobs:
3030
build:
31-
runs-on: ubuntu-latest
31+
runs-on: 'codebuild-github-actions-codebuild-runner-dashboard-amd-${{ github.run_id }}-${{ github.run_attempt }}'
3232
name: Build
3333
defaults:
3434
run:
@@ -79,19 +79,21 @@ jobs:
7979
run: |
8080
echo "ARTIFACT_BUILD_NAME=wazuh-dashboard_${{ env.WZD_VERSION }}-${{ env.WZD_REVISION }}_${{ matrix.ARCHITECTURE }}.${{ matrix.DISTRIBUTION }}" >> $GITHUB_ENV
8181
82-
- name: Run bootstrap
83-
run: yarn osd bootstrap
82+
- name: Create non-root build user
83+
run: |
84+
useradd -m builduser
85+
chown -R builduser:builduser .
8486
85-
- name: Build linux-x64
86-
if: matrix.ARCHITECTURE == 'x64'
87-
run: yarn build-platform --linux --skip-os-packages --release
87+
- name: Run bootstrap
88+
run: |
89+
chown -R builduser:builduser "$YARN_CACHE_LOCATION" 2>/dev/null || true
90+
sudo -u builduser -H env PATH="$PATH" yarn osd bootstrap
8891
89-
- name: Build linux-arm64
90-
if: matrix.ARCHITECTURE == 'arm64'
91-
run: yarn build-platform --linux-arm --skip-os-packages --release
92+
- name: Build
93+
run: sudo -u builduser -H env PATH="$PATH" yarn build-platform --linux --skip-os-packages --release
9294

9395
- name: Rename artifact
94-
run: mv /home/runner/work/wazuh-dashboard/wazuh-dashboard/artifacts/target/opensearch-dashboards-${{ env.VERSION }}-linux-${{ matrix.ARCHITECTURE }}.${{ matrix.DISTRIBUTION }} /home/runner/work/wazuh-dashboard/wazuh-dashboard/artifacts/target/${{ env.ARTIFACT_BUILD_NAME }}
96+
run: mv target/opensearch-dashboards-${{ env.VERSION }}-linux-${{ matrix.ARCHITECTURE }}.${{ matrix.DISTRIBUTION }} target/${{ env.ARTIFACT_BUILD_NAME }}
9597

9698
- uses: actions/upload-artifact@v4
9799
if: success()

‎.github/workflows/build_base.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,7 @@ on:
2020

2121
jobs:
2222
build:
23-
runs-on: ubuntu-latest
23+
runs-on: ${{ matrix.ARCHITECTURE == 'arm64' && 'codebuild-github-actions-codebuild-runner-dashboard-arm-${{ github.run_id }}-${{ github.run_attempt }}' || 'ubuntu-latest' }}
2424
name: Build
2525
defaults:
2626
run:

‎.github/workflows/build_wazuh_dashboard_with_plugins.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -202,7 +202,7 @@ jobs:
202202

203203
build-and-test-package:
204204
needs: [setup-variables, build-main-plugins, build-base, build-security-plugin]
205-
runs-on: ubuntu-latest
205+
runs-on: ${{ (inputs.architecture == 'arm64' || inputs.architecture == 'aarch64') && 'codebuild-github-actions-codebuild-runner-dashboard-arm-${{ github.run_id }}-${{ github.run_attempt }}' || 'ubuntu-22.04' }}
206206
name: Generate packages
207207
steps:
208208
- name: Checkout code

0 commit comments

Comments
 (0)