Skip to content

Commit cdd40ec

Browse files
feat: bump deadline version to 0.60.4 and add optional console sign in deps (#525)
* feat: require deadline 0.60.4 with the console extra for AWS Console sign-in The submitter cannot use AWS Console sign-in. Attempting it fails with "Signing in to the AWS Console sign-in profile <name> requires an additional dependency". Two causes, both addressed here. The console extra was never requested, so awscrt was never installed. Console sign-in moves credential refresh into the calling process: the cached token is bound to a DPoP key and botocore's LoginProvider signs those proofs in-process, which needs awscrt. Requesting deadline[gui,console] pulls it in. The floor was also too low. Console sign-in landed in deadline 0.60.4 and nowhere earlier -- 0.60.1 through 0.60.3 have no AWS_CONSOLE_LOGIN credentials source and do not declare a console extra at all, so the previous ">= 0.60.1" range could resolve to a version where deadline[console] is not a valid request. Raised to ">= 0.60.4". Also added awscrt to the installer's NATIVE_DEPENDENCIES. Its wheels are not uniformly abi3: Python 3.10 gets _awscrt.cpython-310-<platform>.so while 3.11+ get _awscrt.abi3.so. Resolving it only in the base environment would ship whichever artifact the build host produced, so Cinema 4D 2024-2025 (Python 3.10) would have received an unloadable extension module and console sign-in would have broken there while working on 2026. Verified a built dependency bundle contains both awscrt artifacts, matching the existing xxhash pattern, and that a fresh resolve of .[gui] selects deadline 0.60.4, awscrt 0.36.0 and PySide6-Essentials 6.8.3. Note this raises the minimum deadline version from 0.60.1 to 0.60.4. Signed-off-by: andychoquette <78888816+andychoquette@users.noreply.github.com> * test: guard the AWS Console sign-in dependency requirements Console sign-in cannot be covered by the integration tests: it needs an interactive browser OAuth handshake and Deadline Cloud Monitor, while CI authenticates by assuming a role, so the credentials source is host-provided and the console path is never taken. What can break silently is the dependency requirements themselves, which is what these tests pin: - deadline is at least 0.60.4, below which there is no console sign-in support - AwsCredentialsSource.AWS_CONSOLE_LOGIN exists, which is how login_session profiles are recognised - awscrt is installed, and botocore.compat.EC is not None -- that symbol is awscrt.crypto.EC, or None when awscrt is missing or older than 0.28.4, and is the same check LoginProvider makes before accepting a login_session profile - _check_console_login_dependency does not raise, which is the guard the submitter actually hits when a user signs in Confirmed these catch the regression: reverting the dependency to "deadline[gui] >= 0.60.1" fails three of the five. Signed-off-by: andychoquette <78888816+andychoquette@users.noreply.github.com> * test: assert the declared requirement, not the resolved version Addresses review feedback on the console sign-in dependency guards. The floor test asserted on the installed version, which cannot detect the regression it existed to prevent: with the requirement loosened back to ">= 0.60.1", pip still resolves the newest 0.60.x, so version("deadline") stays above the floor and the test passes. It also duplicated the floor as a second constant that could drift from pyproject.toml. Replaced with two tests that read this package's own declared Requires-Dist and assert on the requirement itself -- that it asks for the console extra, and that its specifier excludes 0.60.3, the highest release without console sign-in support. Reading the distribution metadata rather than parsing pyproject.toml keeps this stdlib-only on Python 3.10, where tomllib is unavailable. Also reduced the exposure to private deadline-cloud internals, so a patch release inside the floating ">= 0.60.4, < 0.61" range cannot turn a rename into an unrelated CI failure: - botocore.compat.EC is replaced by awscrt.crypto.EC, which is the public symbol that undocumented re-export conditionally imports, plus an explicit awscrt version assertion for the 0.28.4 floor - the _check_console_login_dependency and AwsCredentialsSource checks now skip rather than error if those private names move Confirmed the guards still bite: reverting the requirement to "deadline[gui] >= 0.60.1" fails five of the six, including both declared requirement tests with messages naming the offending specifier. Signed-off-by: andychoquette <78888816+andychoquette@users.noreply.github.com> * fix: scope the console extra to the submitter, not the adaptor Addresses review feedback. The console extra was added to the base dependencies, which resolve into the adaptor package. That is both wrong in scope and actively harmful. scripts/create_adaptor_packaging_artifact.sh installs this package with --only-binary=:all: --platform <tag> and no --no-deps, for manylinux2014_x86_64, win_amd64 and macosx_10_9_x86_64. No awscrt wheel meeting the 0.28.4 floor exists for the macOS x86_64 tag, because current releases target macosx_10_15_universal2 -- so pip walks backwards and resolves 0.25.7. That has no crypto support botocore will accept, so the build succeeds and console sign-in is silently broken. Verified by resolving each of the three tags directly. The adaptor also has no use for it: it runs on a worker with host-provided credentials and never takes the interactive sign-in path. Upstream keeps console opt-in for this reason, noting awscrt is a compiled wheel and the library is imported from embedded application Pythons. So console and a direct "awscrt >= 0.28.4" now live on the gui extra, which is what the submitter resolves through. The floor is declared rather than left to whatever deadline[console] permits: botocore binds its EC symbol only when has_minimum_crt_version((0, 28, 4)) passes, so awscrt 0.28.3 imports awscrt.crypto.EC fine while console sign-in stays broken. Because get_dependencies reads project.dependencies only, the installer bundle would otherwise lose awscrt entirely. deps_bundle.py now installs it explicitly, in the same spirit as PySide6 -- another submitter-only compiled dependency declared in the gui extra. It runs before _download_native_dependencies, which pins native packages to the versions resolved in the base environment. The unit test environment gains the gui extra so it exercises the submitter's real dependency set, and tomli is declared for Python 3.10 where tomllib is absent. Also corrects the tests: - they read pyproject.toml rather than installed distribution metadata, which is captured at install time and therefore blind to the edit being guarded until the environment is reinstalled - the console-extra assertion is scoped to the gui extra, with a separate test asserting the base dependencies do NOT request it, rather than demanding it on every deadline requirement - botocore.compat.EC is restored as the load-bearing runtime check. The preflight helper returns normally both when awscrt works and when botocore.compat cannot be imported, so "did not raise" proved little; it is kept as a smoke test - the awscrt comment no longer claims crypto.EC first appears in 0.28.4; it exists from 0.28.3 and botocore's version gate is what matters Confirmed each guard bites independently: putting console back in the base list, dropping it from the gui extra, and lowering the awscrt floor each fail exactly one test. Signed-off-by: andychoquette <78888816+andychoquette@users.noreply.github.com> * docs: correct why the awscrt floor is declared directly The comments claimed the direct "awscrt >= 0.28.4" requirement exists because deadline[console] would otherwise permit something too low. That is not true: the console extra requires botocore[crt] >= 1.42.89, and botocore's crt extra pins awscrt to an exact version per release -- 1.42.89 pins 0.31.2, and every later release in the range pins newer. A normal install therefore satisfies the floor transitively, and this declaration is not what enforces it. It is still worth declaring, for a different reason: scripts/deps_bundle.py installs awscrt directly rather than through botocore[crt], so the bundled submitter has no transitive guarantee at all. Stating the same constraint in both places keeps the two install paths in agreement. Corrected in all three places that made the claim: the gui extra comment, the AWSCRT_REQUIREMENT comment, and the test docstring. Signed-off-by: andychoquette <78888816+andychoquette@users.noreply.github.com> * fix: resolve the console extra in the bundle instead of installing awscrt Addresses review feedback. Installing awscrt separately was the wrong mechanism. _build_base_environment is fed project.dependencies, which deliberately no longer request the console extra, so the bundle was resolving deadline[gui] and then installing awscrt in a second, unconstrained pip invocation. That hardcoded an assumption that the console extra's closure is exactly awscrt, and it was wrong in two observable ways: - deadline declares botocore ONLY in the console extra (botocore[crt] >= 1.42.89). Resolving deadline[gui] alone left botocore to arrive via boto3, so the floor the console login provider needs was satisfied by luck rather than by constraint. - botocore's crt extra pins awscrt to an exact version per release. Resolving awscrt separately drifted from it: a built bundle shipped awscrt 0.36.2 alongside a botocore that pins 0.36.0. The base environment now requests the console extra directly, inheriting the version specifier from the declared deadline requirement, so the bundle tracks the extra rather than a guess about it. A rebuilt bundle ships awscrt 0.36.0, matching what botocore pins, with both per-interpreter artifacts intact. That also removes the duplicated floor: AWSCRT_REQUIREMENT and the direct "awscrt >= 0.28.4" in the gui extra had to agree with nothing keeping them in sync, and the direct declaration could not do what its comment claimed anyway -- a >= floor cannot raise a version against botocore's exact ==, it can only turn a too-low pin into a resolution failure. Both are gone; the floor is guaranteed transitively. Also from review: - the base-dependency guard now also rejects a bare awscrt requirement, which is the likelier mistake than re-adding the extra and has the identical effect - test/installer/test_installer.py asserts awscrt reaches the shipped bundle, alongside the existing xxhash and psutil checks -- two steps could previously drop it without failing the build - packaging is declared in requirements-testing.txt, since the dependency tests import it unconditionally and it was only present transitively Signed-off-by: andychoquette <78888816+andychoquette@users.noreply.github.com> --------- Signed-off-by: andychoquette <78888816+andychoquette@users.noreply.github.com>
1 parent 4c5a6d4 commit cdd40ec

6 files changed

Lines changed: 196 additions & 4 deletions

File tree

hatch.toml

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,8 @@
11
[envs.default]
2+
# The unit tests exercise the submitter, which resolves through the gui extra -- that is
3+
# where the console extra and awscrt are declared, so the console sign-in dependency
4+
# tests need them present.
5+
features = ["gui"]
26
pre-install-commands = [
37
"pip install -r requirements-testing.txt"
48
]

pyproject.toml

Lines changed: 17 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -30,15 +30,30 @@ classifiers = [
3030
]
3131

3232
dependencies = [
33-
"deadline[gui] >= 0.60.1, < 0.61",
33+
# 0.60.4 is the floor because AWS Console sign-in landed there and nowhere earlier:
34+
# 0.60.1 through 0.60.3 have no AWS_CONSOLE_LOGIN credentials source, and do not
35+
# declare a `console` extra at all.
36+
#
37+
# The console extra is deliberately NOT requested here. It is only needed by the
38+
# submitter, and it pulls awscrt, a compiled wheel. These base dependencies are
39+
# resolved into the adaptor package by scripts/create_adaptor_packaging_artifact.sh
40+
# with --only-binary=:all: --platform <tag>, and no awscrt wheel satisfying the
41+
# 0.28.4 floor exists for the macosx_10_9_x86_64 tag that script uses -- pip walks
42+
# back to 0.25.7, which has no usable crypto support, so console sign-in would
43+
# appear installed and silently not work. The adaptor never takes the console
44+
# path anyway: it runs on a worker with host-provided credentials.
45+
"deadline[gui] >= 0.60.4, < 0.61",
3446
"openjd-adaptor-runtime >= 0.7,< 0.10",
3547
# fonttools is Windows-only due to Cinema 4D technical limitations
3648
"fonttools >=4.59.2, <4.64; sys_platform == 'win32'",
3749
]
3850

3951
[project.optional-dependencies]
4052
gui = [
41-
"deadline[gui] >= 0.60.1, < 0.61",
53+
# The console extra pulls awscrt, which AWS Console sign-in requires: botocore's
54+
# LoginProvider refreshes the DPoP-bound cached token in-process, and signing those
55+
# proofs needs awscrt. Without it, sign-in fails on a pre-flight dependency check.
56+
"deadline[gui,console] >= 0.60.4, < 0.61",
4257
"PySide6-Essentials == 6.8.3", # Use 6.8 to align with VFXP 2026: https://vfxplatform.com/#reference-platform
4358
]
4459

requirements-testing.txt

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -17,3 +17,6 @@ deadline-cloud-test-fixtures >= 0.18.18
1717
# Drives the real submitter dialog via the OS accessibility tree.
1818
# Used by test/integ/.
1919
xa11y >= 0.13.0
20+
# Both are used by test_console_signin_dependencies.py, which parses pyproject.toml.
21+
packaging >= 22
22+
tomli == 2.*; python_version < "3.11"

scripts/deps_bundle.py

Lines changed: 32 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,15 @@
1414

1515
SUPPORTED_PYTHON_VERSIONS = ["3.10", "3.11", "3.12", "3.13"]
1616
SUPPORTED_PLATFORMS = ["Windows", "Linux", "Darwin"]
17-
NATIVE_DEPENDENCIES = ["xxhash", "psutil"]
17+
# Packages with compiled extension modules, fetched once per supported Python version so the
18+
# bundle carries a loadable artifact for each interpreter.
19+
#
20+
# awscrt is here because its wheels are not uniformly abi3: Python 3.10 gets
21+
# _awscrt.cpython-310-<platform>.so while 3.11+ get _awscrt.abi3.so. Resolving it only in
22+
# the base environment would ship whichever the build host produced, so Cinema 4D 2024-2025
23+
# (Python 3.10) would fail to import awscrt and AWS Console sign-in would break there while
24+
# working on 2026.
25+
NATIVE_DEPENDENCIES = ["xxhash", "psutil", "awscrt"]
1826

1927
PYSIDE6_VERSION = "6.8.3"
2028
PYSIDE6_PACKAGES = [f"PySide6-Essentials=={PYSIDE6_VERSION}", f"shiboken6=={PYSIDE6_VERSION}"]
@@ -152,10 +160,32 @@ def _get_package_version(package: str, install_path: Path) -> str:
152160
raise RuntimeError(f"Could not find version for package {package}")
153161

154162

163+
def _add_console_extra(requirement: str) -> str:
164+
"""Add deadline's `console` extra to a requirement string, preserving its specifier."""
165+
match = re.fullmatch(
166+
r"(?P<name>[A-Za-z0-9._-]+)(?:\[(?P<extras>[^\]]*)\])?(?P<spec>.*)", requirement
167+
)
168+
if not match or match.group("name").lower() != "deadline":
169+
return requirement
170+
extras = [extra for extra in (match.group("extras") or "").split(",") if extra]
171+
if "console" not in extras:
172+
extras.append("console")
173+
return f"{match.group('name')}[{','.join(extras)}]{match.group('spec')}"
174+
175+
155176
def _build_base_environment(working_directory: Path, dependencies: list[Dependency]) -> Path:
156177
(working_directory / "base_env").mkdir()
157178
base_env_path = working_directory / "base_env"
158-
dependencies_for_pip = [d.for_pip() for d in dependencies]
179+
# The bundle is the submitter, which needs AWS Console sign-in. The console extra is
180+
# requested here rather than declared in project.dependencies, because those are also
181+
# resolved into the adaptor package, where a compiled awscrt wheel is both unusable and
182+
# unavailable for one of the platform tags that build targets (see pyproject.toml).
183+
#
184+
# Requesting the extra rather than installing awscrt directly means the bundle tracks
185+
# whatever the extra actually requires -- notably a botocore floor, since the console
186+
# login provider lives in botocore, not in deadline -- and takes awscrt from the exact
187+
# version botocore's crt extra pins, rather than resolving it independently and drifting.
188+
dependencies_for_pip = [_add_console_extra(d.for_pip()) for d in dependencies]
159189
base_env_pip_args = [
160190
"pip",
161191
"install",

test/installer/test_installer.py

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -92,6 +92,9 @@ def _validate_files(installation_path: Path) -> None:
9292
assert "qtpy" in top_level_dir
9393
assert "xxhash" in top_level_dir
9494
assert "psutil" in top_level_dir
95+
# awscrt reaches the bundle via deadline's console extra, requested by deps_bundle.py.
96+
# Without it botocore reports CRT as unavailable and AWS Console sign-in fails.
97+
assert "awscrt" in top_level_dir
9598

9699
# Verify PySide6/shiboken6 are bundled and stripped correctly
97100
assert "PySide6" in top_level_dir
Lines changed: 137 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,137 @@
1+
# Copyright Amazon.com, Inc. or its affiliates. All Rights Reserved.
2+
3+
"""Guards the dependency declarations that AWS Console sign-in depends on.
4+
5+
Console sign-in is not exercised by the integration tests: it needs an interactive
6+
browser OAuth handshake and Deadline Cloud Monitor, while CI authenticates by
7+
assuming a role, so credentials are host-provided and the console path is never
8+
taken. What can break silently is the dependency declaration, which is what these
9+
tests pin.
10+
11+
The declaration tests read ``pyproject.toml`` rather than installed distribution
12+
metadata. ``importlib.metadata`` reflects what was captured at install time, so an
13+
edit to ``pyproject.toml`` would not be seen until the environment is reinstalled --
14+
and "somebody edited that line" is precisely the regression being guarded.
15+
16+
Scope matters as much as the versions. ``console`` belongs on the ``gui`` extra and
17+
not on the base dependencies: the base list is resolved into the adaptor package by
18+
``scripts/create_adaptor_packaging_artifact.sh`` under ``--only-binary=:all:
19+
--platform <tag>``, and no awscrt wheel meeting the floor exists for the
20+
``macosx_10_9_x86_64`` tag that script uses, so pip would silently walk back to a
21+
release with no usable crypto support.
22+
"""
23+
24+
import sys
25+
from pathlib import Path
26+
27+
import pytest
28+
from packaging.requirements import Requirement
29+
30+
if sys.version_info >= (3, 11):
31+
import tomllib
32+
else: # pragma: no cover - exercised on Python 3.10 only
33+
import tomli as tomllib
34+
35+
PYPROJECT = Path(__file__).parents[3] / "pyproject.toml"
36+
37+
# Console sign-in landed in deadline 0.60.4 and nowhere earlier: 0.60.1 through
38+
# 0.60.3 have no AWS_CONSOLE_LOGIN credentials source and do not declare a
39+
# `console` extra at all. 0.60.3 is the highest version that must be excluded.
40+
HIGHEST_DEADLINE_WITHOUT_CONSOLE_SIGNIN = "0.60.3"
41+
42+
43+
def _requirements(*table_path: str) -> list[Requirement]:
44+
"""Parse a requirement list out of pyproject.toml by table path."""
45+
node = tomllib.loads(PYPROJECT.read_text(encoding="utf-8"))
46+
for key in table_path:
47+
assert key in node, f"pyproject.toml has no {'.'.join(table_path)}"
48+
node = node[key]
49+
return [Requirement(r) for r in node]
50+
51+
52+
def _named(requirements: list[Requirement], name: str) -> list[Requirement]:
53+
return [r for r in requirements if r.name == name]
54+
55+
56+
@pytest.fixture
57+
def base_dependencies() -> list[Requirement]:
58+
return _requirements("project", "dependencies")
59+
60+
61+
@pytest.fixture
62+
def gui_dependencies() -> list[Requirement]:
63+
return _requirements("project", "optional-dependencies", "gui")
64+
65+
66+
def test_gui_extra_requests_the_console_extra(gui_dependencies):
67+
"""The submitter resolves through the gui extra, so console belongs there."""
68+
deadline_reqs = _named(gui_dependencies, "deadline")
69+
assert deadline_reqs, "the gui extra declares no requirement on deadline"
70+
for req in deadline_reqs:
71+
assert "console" in req.extras, f"missing console extra in: {req}"
72+
73+
74+
def test_base_dependencies_do_not_request_the_console_extra(base_dependencies):
75+
"""Keeps awscrt out of the adaptor package.
76+
77+
The base list is resolved into the adaptor artifact for three platform tags under
78+
--only-binary=:all:. For macosx_10_9_x86_64 no awscrt wheel meets the floor, so pip
79+
resolves backwards to one whose crypto support botocore will not accept -- the build
80+
succeeds and console sign-in is quietly broken. The adaptor never signs in
81+
interactively, so it has no use for the extra.
82+
"""
83+
for req in _named(base_dependencies, "deadline"):
84+
assert (
85+
"console" not in req.extras
86+
), f"console extra leaks into the adaptor's dependency closure via: {req}"
87+
88+
# Copying the requirement in directly is the likelier mistake, and has the same effect.
89+
assert not _named(
90+
base_dependencies, "awscrt"
91+
), "awscrt must not be a base dependency; it would be resolved into the adaptor package"
92+
93+
94+
@pytest.mark.parametrize("table", ["base_dependencies", "gui_dependencies"])
95+
def test_deadline_floor_excludes_releases_without_console_signin(table, request):
96+
"""Guards the floor itself, not whatever a resolver happened to select.
97+
98+
An installed-version check cannot do this: with a loosened ">= 0.60.1"
99+
requirement, pip still resolves the newest 0.60.x, so the regression passes
100+
unnoticed.
101+
"""
102+
for req in _named(request.getfixturevalue(table), "deadline"):
103+
assert not req.specifier.contains(HIGHEST_DEADLINE_WITHOUT_CONSOLE_SIGNIN), (
104+
f"allows deadline {HIGHEST_DEADLINE_WITHOUT_CONSOLE_SIGNIN}, which has no "
105+
f"console sign-in support: {req}"
106+
)
107+
108+
109+
def test_botocore_sees_awscrt():
110+
"""The load-bearing runtime check.
111+
112+
botocore binds EC only when has_minimum_crt_version((0, 28, 4)) passes, and
113+
deadline.client refuses console sign-in when it is None. This is the exact
114+
condition upstream branches on, so it fails whenever awscrt is missing, too old,
115+
or disabled -- unlike importing awscrt.crypto.EC, which succeeds from 0.28.3 and
116+
would pass while sign-in is still broken.
117+
"""
118+
from botocore.compat import EC
119+
120+
assert EC is not None, "botocore does not see awscrt; console sign-in will fail"
121+
122+
123+
def test_console_login_preflight_is_reachable():
124+
"""Smoke test for the guard the submitter hits when a user signs in.
125+
126+
Deliberately not the file's guarantee: this helper returns normally both when
127+
awscrt works and when botocore.compat cannot be imported at all, so "did not
128+
raise" proves little. test_botocore_sees_awscrt carries that. Skipped rather than
129+
failed if the private helper is renamed upstream, which is not a breaking change
130+
within the floating requirement range.
131+
"""
132+
loginout = pytest.importorskip("deadline.client.api._loginout")
133+
check = getattr(loginout, "_check_console_login_dependency", None)
134+
if check is None:
135+
pytest.skip("upstream renamed the console-login preflight helper")
136+
137+
check("test-profile")

0 commit comments

Comments
 (0)