Skip to content

Commit 7e9a192

Browse files
shchurAnirudhDagar
shchur
and
AnirudhDagar
authored
[AutoGluon] Release unified AutoGluon 1.6 SageMaker image (#6762)
* Release AutoGluon 1.6 Add unified AutoGluon 1.6.1 SageMaker CPU and GPU images on the PyTorch 2.13 AL2023 base, including training and inference serving, CI workflows, documentation, and test coverage. Co-authored-by: Anirudh Dagar <anirudhdagar6@gmail.com> * Simplify AutoGluon serving tests * Simplify AutoGluon serving configuration * Trim AutoGluon serving compatibility layer * Fix AutoGluon test lint * Pin AutoGluon 1.6.2 dependencies * Remove redundant AutoGluon sanity checks * Align AutoGluon CodeArtifact setup with Ray * Add AutoGluon time series SageMaker coverage * Fix AutoGluon Dockerfile formatting * Address AutoGluon release review feedback * Share AutoGluon Dockerfile across CPU and GPU * Lock AutoGluon dependencies per base image * Version AutoGluon dependency pins * Fix AutoGluon serving and telemetry Signed-off-by: Oleksandr Shchur <shchuro@amazon.com> * Pin AutoGluon 1.6.3 dependencies Signed-off-by: Oleksandr Shchur <shchuro@amazon.com> * Fix AutoGluon image vulnerability scan findings Signed-off-by: Oleksandr Shchur <shchuro@amazon.com> * Document temporary AutoGluon security overrides Signed-off-by: Oleksandr Shchur <shchuro@amazon.com> * Avoid hanging AutoGluon time series test workers Signed-off-by: Oleksandr Shchur <shchuro@amazon.com> --------- Signed-off-by: Oleksandr Shchur <shchuro@amazon.com> Co-authored-by: Anirudh Dagar <anirudhdagar6@gmail.com>
1 parent 3c5e385 commit 7e9a192

26 files changed

Lines changed: 1995 additions & 0 deletions

‎.github/config/image/autogluon/1.6-sagemaker-cpu.yml‎

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
image:
2+
name: "autogluon-1.6-sagemaker-cpu"
3+
description: "AutoGluon 1.6 CPU training and inference for SageMaker"
4+
5+
metadata:
6+
framework: "autogluon"
7+
framework_version: "1.6.3"
8+
os_version: "amzn2023"
9+
customer_type: "sagemaker"
10+
platform: "sagemaker"
11+
arch_type: "x86"
12+
device_type: "cpu"
13+
job_type: "general"
14+
prod_image: "autogluon:1.6-cpu-amzn2023"
15+
16+
build:
17+
dockerfile: "docker/autogluon/Dockerfile"
18+
target: "sagemaker"
19+
base_image: "public.ecr.aws/deep-learning-containers/pytorch:2.13-cpu-amzn2023-sagemaker"
20+
requirements_file: "docker/autogluon/1.6/requirements.cpu.in"
21+
python_version: "3.12"
22+
dlc_major_version: "1"
23+
dlc_minor_version: "0"
24+
25+
release:
26+
# Gamma-testing the unified image before production.
27+
# Follow-up: publish the AutoGluon docs only after the production release succeeds.
28+
release: true
29+
force_release: false
30+
public_registry: false
31+
private_registry: true
32+
enable_soci: false
33+
environment: "gamma"

‎.github/config/image/autogluon/1.6-sagemaker-cuda.yml‎

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
image:
2+
name: "autogluon-1.6-sagemaker-cuda"
3+
description: "AutoGluon 1.6 GPU training and inference for SageMaker"
4+
5+
metadata:
6+
framework: "autogluon"
7+
framework_version: "1.6.3"
8+
os_version: "amzn2023"
9+
customer_type: "sagemaker"
10+
platform: "sagemaker"
11+
arch_type: "x86"
12+
device_type: "gpu"
13+
job_type: "general"
14+
prod_image: "autogluon:1.6-cu133-amzn2023"
15+
16+
build:
17+
dockerfile: "docker/autogluon/Dockerfile"
18+
target: "sagemaker"
19+
base_image: "public.ecr.aws/deep-learning-containers/pytorch:2.13-cu133-amzn2023-sagemaker"
20+
requirements_file: "docker/autogluon/1.6/requirements.cuda.in"
21+
python_version: "3.12"
22+
cuda_version: "13.3.0"
23+
dlc_major_version: "1"
24+
dlc_minor_version: "0"
25+
26+
release:
27+
# Gamma-testing the unified image before production.
28+
# Follow-up: publish the AutoGluon docs only after the production release succeeds.
29+
release: true
30+
force_release: false
31+
public_registry: false
32+
private_registry: true
33+
enable_soci: false
34+
environment: "gamma"

‎.github/config/test-suites.yml‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -56,6 +56,11 @@ suites:
5656
code_paths:
5757
- test/pytorch/integration/sagemaker/**
5858

59+
autogluon/sagemaker:
60+
skip_eligible: true
61+
code_paths:
62+
- test/autogluon/sagemaker/**
63+
5964
tensorflow-inference/sagemaker:
6065
skip_eligible: true
6166
code_paths:

‎.github/workflows/autogluon.dispatch-release.yml‎

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
name: "Dispatch - AutoGluon Release"
2+
3+
# AutoGluon ships a few times a year, so releases are triggered by hand rather
4+
# than on a cron. Run once per config file from `main`; the pipeline's
5+
# release-gate reads release.environment (gamma / production) from the config.
6+
7+
on:
8+
workflow_dispatch:
9+
inputs:
10+
config-file:
11+
description: "Path to image config YAML (e.g., .github/config/image/autogluon/1.6-sagemaker-cuda.yml)"
12+
required: true
13+
type: string
14+
15+
concurrency:
16+
group: ${{ github.workflow }}
17+
cancel-in-progress: false
18+
19+
permissions:
20+
contents: read
21+
22+
jobs:
23+
pipeline:
24+
uses: ./.github/workflows/autogluon.pipeline.yml
25+
with:
26+
config-file: ${{ inputs.config-file }}
27+
tag-suffix: ${{ github.run_id }}
28+
release: true
29+
secrets: inherit

‎.github/workflows/autogluon.pipeline.yml‎

Lines changed: 201 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,201 @@
1+
name: "Pipeline - AutoGluon"
2+
3+
on:
4+
workflow_call:
5+
inputs:
6+
config-file:
7+
description: "Path to image config YAML"
8+
required: true
9+
type: string
10+
tag-suffix:
11+
description: "CI tag suffix (run_id or pr-NUMBER)"
12+
required: true
13+
type: string
14+
build:
15+
description: "Whether to build the image (false = skip build, test against prod)"
16+
required: false
17+
type: boolean
18+
default: true
19+
run-sanity-test:
20+
description: "Run sanity tests"
21+
required: false
22+
type: boolean
23+
default: true
24+
run-unit-test:
25+
description: "Run serving adapter unit tests"
26+
required: false
27+
type: boolean
28+
default: true
29+
run-security-test:
30+
description: "Run security tests"
31+
required: false
32+
type: boolean
33+
default: true
34+
run-telemetry-test:
35+
description: "Run telemetry tests"
36+
required: false
37+
type: boolean
38+
default: true
39+
run-sagemaker-test:
40+
description: "Run SageMaker training and endpoint tests"
41+
required: false
42+
type: boolean
43+
default: true
44+
release:
45+
description: "Run release job after tests pass"
46+
required: false
47+
type: boolean
48+
default: false
49+
outputs:
50+
image-uri:
51+
description: "Built image URI (empty if build was skipped)"
52+
value: ${{ jobs.build.outputs.image-uri }}
53+
54+
jobs:
55+
build:
56+
if: ${{ inputs.build }}
57+
concurrency:
58+
group: ${{ github.workflow }}-build-${{ inputs.config-file }}-${{ github.ref }}
59+
cancel-in-progress: true
60+
runs-on:
61+
- codebuild-runner-${{ github.run_id }}-${{ github.run_attempt }}
62+
fleet:x86-build-runner
63+
buildspec-override:true
64+
timeout-minutes: 120
65+
outputs:
66+
image-uri: ${{ steps.build.outputs.image-uri }}
67+
steps:
68+
- uses: actions/checkout@v6
69+
- id: build
70+
uses: ./.github/actions/build-image
71+
with:
72+
config-file: ${{ inputs.config-file }}
73+
aws-account-id: ${{ vars.CI_AWS_ACCOUNT_ID }}
74+
aws-region: ${{ vars.AWS_REGION }}
75+
tag-suffix: ${{ inputs.tag-suffix }}
76+
77+
check:
78+
if: ${{ always() && !failure() && !cancelled() }}
79+
needs: [build]
80+
concurrency:
81+
group: ${{ github.workflow }}-check-${{ inputs.config-file }}-${{ github.ref }}
82+
cancel-in-progress: true
83+
uses: ./.github/workflows/_reusable.check-test-pass.yml
84+
with:
85+
config-file: ${{ inputs.config-file }}
86+
image-uri: ${{ needs.build.outputs.image-uri || '' }}
87+
suites: '["sanity", "telemetry", "autogluon/sagemaker"]'
88+
89+
unit-tests:
90+
if: ${{ inputs.run-unit-test }}
91+
uses: ./.github/workflows/autogluon.tests-unit.yml
92+
93+
sanity-test:
94+
if: ${{ always() && !cancelled() && inputs.run-sanity-test && needs.build.result != 'failure' && fromJSON(needs.check.outputs.skips || '{}')['sanity'] != true }}
95+
needs: [build, check]
96+
concurrency:
97+
group: ${{ github.workflow }}-sanity-test-${{ inputs.config-file }}-${{ github.ref }}
98+
cancel-in-progress: true
99+
uses: ./.github/workflows/_reusable.sanity-tests.yml
100+
with:
101+
config-file: ${{ inputs.config-file }}
102+
image-uri: ${{ needs.build.outputs.image-uri || '' }}
103+
image-content-hash: ${{ needs.check.outputs.image-content-hash }}
104+
suite-code-hash: ${{ fromJSON(needs.check.outputs.suite-code-hashes || '{}')['sanity'] || '' }}
105+
106+
security-test:
107+
if: ${{ always() && !failure() && !cancelled() && inputs.run-security-test }}
108+
needs: [build]
109+
concurrency:
110+
group: ${{ github.workflow }}-security-test-${{ inputs.config-file }}-${{ github.ref }}
111+
cancel-in-progress: true
112+
uses: ./.github/workflows/_reusable.security-tests.yml
113+
with:
114+
config-file: ${{ inputs.config-file }}
115+
image-uri: ${{ needs.build.outputs.image-uri || '' }}
116+
117+
telemetry-test:
118+
if: ${{ always() && !cancelled() && inputs.run-telemetry-test && needs.build.result != 'failure' && fromJSON(needs.check.outputs.skips || '{}')['telemetry'] != true }}
119+
needs: [build, check]
120+
concurrency:
121+
group: ${{ github.workflow }}-telemetry-test-${{ inputs.config-file }}-${{ github.ref }}
122+
cancel-in-progress: false
123+
uses: ./.github/workflows/_reusable.telemetry-tests.yml
124+
with:
125+
config-file: ${{ inputs.config-file }}
126+
image-uri: ${{ needs.build.outputs.image-uri || '' }}
127+
image-content-hash: ${{ needs.check.outputs.image-content-hash }}
128+
suite-code-hash: ${{ fromJSON(needs.check.outputs.suite-code-hashes || '{}')['telemetry'] || '' }}
129+
130+
sagemaker-tests:
131+
if: >-
132+
${{ always() && !cancelled() && inputs.run-sagemaker-test &&
133+
needs.build.result != 'failure' &&
134+
fromJSON(needs.check.outputs.skips || '{}')['autogluon/sagemaker'] != true }}
135+
needs: [build, check]
136+
concurrency:
137+
group: ${{ github.workflow }}-sagemaker-tests-${{ inputs.config-file }}-${{ github.ref }}
138+
cancel-in-progress: false
139+
uses: ./.github/workflows/autogluon.tests-sagemaker.yml
140+
with:
141+
config-file: ${{ inputs.config-file }}
142+
image-uri: ${{ needs.build.outputs.image-uri || '' }}
143+
image-content-hash: ${{ needs.check.outputs.image-content-hash }}
144+
suite-code-hash: ${{ fromJSON(needs.check.outputs.suite-code-hashes || '{}')['autogluon/sagemaker'] || '' }}
145+
146+
release-gate:
147+
if: >-
148+
github.ref == 'refs/heads/main' &&
149+
(contains(github.workflow_ref, '.autorelease') || contains(github.workflow_ref, '.dispatch-release')) &&
150+
!contains(github.workflow_ref, '.pr') &&
151+
inputs.release &&
152+
!failure() && !cancelled() &&
153+
needs.build.result == 'success'
154+
needs: [build, unit-tests, sanity-test, security-test, telemetry-test, sagemaker-tests]
155+
runs-on: ubuntu-latest
156+
outputs:
157+
should-release: ${{ steps.check.outputs.should-release }}
158+
release-spec: ${{ steps.spec.outputs.release-spec }}
159+
environment: ${{ steps.check.outputs.environment }}
160+
steps:
161+
- uses: actions/checkout@v6
162+
- id: check
163+
run: |
164+
SHOULD_RELEASE=$(yq '.release.release' "${{ inputs.config-file }}")
165+
ENVIRONMENT=$(yq '.release.environment' "${{ inputs.config-file }}")
166+
echo "should-release=${SHOULD_RELEASE}" >> $GITHUB_OUTPUT
167+
echo "environment=${ENVIRONMENT}" >> $GITHUB_OUTPUT
168+
- id: spec
169+
if: steps.check.outputs.should-release == 'true'
170+
uses: ./.github/actions/generate-release-spec
171+
with:
172+
config-file: ${{ inputs.config-file }}
173+
174+
release:
175+
if: ${{ !failure() && !cancelled() && needs.release-gate.outputs.should-release == 'true' }}
176+
needs: [build, release-gate]
177+
uses: ./.github/workflows/_reusable.release-image.yml
178+
with:
179+
source-image-uri: ${{ needs.build.outputs.image-uri }}
180+
release-spec: ${{ needs.release-gate.outputs.release-spec }}
181+
environment: ${{ needs.release-gate.outputs.environment }}
182+
aws-region: ${{ vars.AWS_REGION }}
183+
secrets: inherit
184+
185+
notify-failure:
186+
if: >-
187+
failure() &&
188+
github.ref == 'refs/heads/main' &&
189+
(contains(github.workflow_ref, '.autorelease') || contains(github.workflow_ref, '.dispatch-release')) &&
190+
!contains(github.workflow_ref, '.pr') &&
191+
inputs.release
192+
needs: [build, unit-tests, sanity-test, security-test, telemetry-test, sagemaker-tests, release-gate, release]
193+
runs-on: ubuntu-latest
194+
steps:
195+
- uses: actions/checkout@v6
196+
- uses: ./.github/actions/notify-slack-failure
197+
with:
198+
webhook-url: ${{ secrets.SLACK_RELEASE_FAILURE_WEBHOOK_URL }}
199+
workflow-name: ${{ github.workflow }}
200+
config-file: ${{ inputs.config-file }}
201+
run-url: ${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}

0 commit comments

Comments
 (0)