Commit cdd40ec
authored
feat: bump deadline version to 0.60.4 and add optional console sign in deps (#525)
* feat: require deadline 0.60.4 with the console extra for AWS Console sign-in
The submitter cannot use AWS Console sign-in. Attempting it fails with "Signing
in to the AWS Console sign-in profile <name> requires an additional dependency".
Two causes, both addressed here.
The console extra was never requested, so awscrt was never installed. Console
sign-in moves credential refresh into the calling process: the cached token is
bound to a DPoP key and botocore's LoginProvider signs those proofs in-process,
which needs awscrt. Requesting deadline[gui,console] pulls it in.
The floor was also too low. Console sign-in landed in deadline 0.60.4 and
nowhere earlier -- 0.60.1 through 0.60.3 have no AWS_CONSOLE_LOGIN credentials
source and do not declare a console extra at all, so the previous
">= 0.60.1" range could resolve to a version where deadline[console] is not a
valid request. Raised to ">= 0.60.4".
Also added awscrt to the installer's NATIVE_DEPENDENCIES. Its wheels are not
uniformly abi3: Python 3.10 gets _awscrt.cpython-310-<platform>.so while 3.11+
get _awscrt.abi3.so. Resolving it only in the base environment would ship
whichever artifact the build host produced, so Cinema 4D 2024-2025 (Python 3.10)
would have received an unloadable extension module and console sign-in would
have broken there while working on 2026.
Verified a built dependency bundle contains both awscrt artifacts, matching the
existing xxhash pattern, and that a fresh resolve of .[gui] selects deadline
0.60.4, awscrt 0.36.0 and PySide6-Essentials 6.8.3.
Note this raises the minimum deadline version from 0.60.1 to 0.60.4.
Signed-off-by: andychoquette <78888816+andychoquette@users.noreply.github.com>
* test: guard the AWS Console sign-in dependency requirements
Console sign-in cannot be covered by the integration tests: it needs an
interactive browser OAuth handshake and Deadline Cloud Monitor, while CI
authenticates by assuming a role, so the credentials source is host-provided
and the console path is never taken.
What can break silently is the dependency requirements themselves, which is what
these tests pin:
- deadline is at least 0.60.4, below which there is no console sign-in support
- AwsCredentialsSource.AWS_CONSOLE_LOGIN exists, which is how login_session
profiles are recognised
- awscrt is installed, and botocore.compat.EC is not None -- that symbol is
awscrt.crypto.EC, or None when awscrt is missing or older than 0.28.4, and is
the same check LoginProvider makes before accepting a login_session profile
- _check_console_login_dependency does not raise, which is the guard the
submitter actually hits when a user signs in
Confirmed these catch the regression: reverting the dependency to
"deadline[gui] >= 0.60.1" fails three of the five.
Signed-off-by: andychoquette <78888816+andychoquette@users.noreply.github.com>
* test: assert the declared requirement, not the resolved version
Addresses review feedback on the console sign-in dependency guards.
The floor test asserted on the installed version, which cannot detect the
regression it existed to prevent: with the requirement loosened back to
">= 0.60.1", pip still resolves the newest 0.60.x, so version("deadline") stays
above the floor and the test passes. It also duplicated the floor as a second
constant that could drift from pyproject.toml.
Replaced with two tests that read this package's own declared Requires-Dist and
assert on the requirement itself -- that it asks for the console extra, and that
its specifier excludes 0.60.3, the highest release without console sign-in
support. Reading the distribution metadata rather than parsing pyproject.toml
keeps this stdlib-only on Python 3.10, where tomllib is unavailable.
Also reduced the exposure to private deadline-cloud internals, so a patch release
inside the floating ">= 0.60.4, < 0.61" range cannot turn a rename into an
unrelated CI failure:
- botocore.compat.EC is replaced by awscrt.crypto.EC, which is the public symbol
that undocumented re-export conditionally imports, plus an explicit awscrt
version assertion for the 0.28.4 floor
- the _check_console_login_dependency and AwsCredentialsSource checks now skip
rather than error if those private names move
Confirmed the guards still bite: reverting the requirement to
"deadline[gui] >= 0.60.1" fails five of the six, including both declared
requirement tests with messages naming the offending specifier.
Signed-off-by: andychoquette <78888816+andychoquette@users.noreply.github.com>
* fix: scope the console extra to the submitter, not the adaptor
Addresses review feedback. The console extra was added to the base dependencies,
which resolve into the adaptor package. That is both wrong in scope and actively
harmful.
scripts/create_adaptor_packaging_artifact.sh installs this package with
--only-binary=:all: --platform <tag> and no --no-deps, for manylinux2014_x86_64,
win_amd64 and macosx_10_9_x86_64. No awscrt wheel meeting the 0.28.4 floor exists
for the macOS x86_64 tag, because current releases target
macosx_10_15_universal2 -- so pip walks backwards and resolves 0.25.7. That has no
crypto support botocore will accept, so the build succeeds and console sign-in is
silently broken. Verified by resolving each of the three tags directly.
The adaptor also has no use for it: it runs on a worker with host-provided
credentials and never takes the interactive sign-in path. Upstream keeps console
opt-in for this reason, noting awscrt is a compiled wheel and the library is
imported from embedded application Pythons.
So console and a direct "awscrt >= 0.28.4" now live on the gui extra, which is
what the submitter resolves through. The floor is declared rather than left to
whatever deadline[console] permits: botocore binds its EC symbol only when
has_minimum_crt_version((0, 28, 4)) passes, so awscrt 0.28.3 imports
awscrt.crypto.EC fine while console sign-in stays broken.
Because get_dependencies reads project.dependencies only, the installer bundle
would otherwise lose awscrt entirely. deps_bundle.py now installs it explicitly,
in the same spirit as PySide6 -- another submitter-only compiled dependency
declared in the gui extra. It runs before _download_native_dependencies, which
pins native packages to the versions resolved in the base environment.
The unit test environment gains the gui extra so it exercises the submitter's real
dependency set, and tomli is declared for Python 3.10 where tomllib is absent.
Also corrects the tests:
- they read pyproject.toml rather than installed distribution metadata, which is
captured at install time and therefore blind to the edit being guarded until the
environment is reinstalled
- the console-extra assertion is scoped to the gui extra, with a separate test
asserting the base dependencies do NOT request it, rather than demanding it on
every deadline requirement
- botocore.compat.EC is restored as the load-bearing runtime check. The preflight
helper returns normally both when awscrt works and when botocore.compat cannot be
imported, so "did not raise" proved little; it is kept as a smoke test
- the awscrt comment no longer claims crypto.EC first appears in 0.28.4; it exists
from 0.28.3 and botocore's version gate is what matters
Confirmed each guard bites independently: putting console back in the base list,
dropping it from the gui extra, and lowering the awscrt floor each fail exactly one
test.
Signed-off-by: andychoquette <78888816+andychoquette@users.noreply.github.com>
* docs: correct why the awscrt floor is declared directly
The comments claimed the direct "awscrt >= 0.28.4" requirement exists because
deadline[console] would otherwise permit something too low. That is not true: the
console extra requires botocore[crt] >= 1.42.89, and botocore's crt extra pins
awscrt to an exact version per release -- 1.42.89 pins 0.31.2, and every later
release in the range pins newer. A normal install therefore satisfies the floor
transitively, and this declaration is not what enforces it.
It is still worth declaring, for a different reason: scripts/deps_bundle.py
installs awscrt directly rather than through botocore[crt], so the bundled
submitter has no transitive guarantee at all. Stating the same constraint in both
places keeps the two install paths in agreement.
Corrected in all three places that made the claim: the gui extra comment, the
AWSCRT_REQUIREMENT comment, and the test docstring.
Signed-off-by: andychoquette <78888816+andychoquette@users.noreply.github.com>
* fix: resolve the console extra in the bundle instead of installing awscrt
Addresses review feedback. Installing awscrt separately was the wrong mechanism.
_build_base_environment is fed project.dependencies, which deliberately no longer
request the console extra, so the bundle was resolving deadline[gui] and then
installing awscrt in a second, unconstrained pip invocation. That hardcoded an
assumption that the console extra's closure is exactly awscrt, and it was wrong in
two observable ways:
- deadline declares botocore ONLY in the console extra (botocore[crt] >= 1.42.89).
Resolving deadline[gui] alone left botocore to arrive via boto3, so the floor the
console login provider needs was satisfied by luck rather than by constraint.
- botocore's crt extra pins awscrt to an exact version per release. Resolving awscrt
separately drifted from it: a built bundle shipped awscrt 0.36.2 alongside a
botocore that pins 0.36.0.
The base environment now requests the console extra directly, inheriting the version
specifier from the declared deadline requirement, so the bundle tracks the extra
rather than a guess about it. A rebuilt bundle ships awscrt 0.36.0, matching what
botocore pins, with both per-interpreter artifacts intact.
That also removes the duplicated floor: AWSCRT_REQUIREMENT and the direct
"awscrt >= 0.28.4" in the gui extra had to agree with nothing keeping them in sync,
and the direct declaration could not do what its comment claimed anyway -- a >=
floor cannot raise a version against botocore's exact ==, it can only turn a too-low
pin into a resolution failure. Both are gone; the floor is guaranteed transitively.
Also from review:
- the base-dependency guard now also rejects a bare awscrt requirement, which is the
likelier mistake than re-adding the extra and has the identical effect
- test/installer/test_installer.py asserts awscrt reaches the shipped bundle,
alongside the existing xxhash and psutil checks -- two steps could previously drop
it without failing the build
- packaging is declared in requirements-testing.txt, since the dependency tests import
it unconditionally and it was only present transitively
Signed-off-by: andychoquette <78888816+andychoquette@users.noreply.github.com>
---------
Signed-off-by: andychoquette <78888816+andychoquette@users.noreply.github.com>1 parent 4c5a6d4 commit cdd40ec
6 files changed
Lines changed: 196 additions & 4 deletions
File tree
- scripts
- test
- installer
- unit/deadline_submitter_for_cinema4d
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1 | 1 | | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
2 | 6 | | |
3 | 7 | | |
4 | 8 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
30 | 30 | | |
31 | 31 | | |
32 | 32 | | |
33 | | - | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
34 | 46 | | |
35 | 47 | | |
36 | 48 | | |
37 | 49 | | |
38 | 50 | | |
39 | 51 | | |
40 | 52 | | |
41 | | - | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
42 | 57 | | |
43 | 58 | | |
44 | 59 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
17 | 17 | | |
18 | 18 | | |
19 | 19 | | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
14 | 14 | | |
15 | 15 | | |
16 | 16 | | |
17 | | - | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
18 | 26 | | |
19 | 27 | | |
20 | 28 | | |
| |||
152 | 160 | | |
153 | 161 | | |
154 | 162 | | |
| 163 | + | |
| 164 | + | |
| 165 | + | |
| 166 | + | |
| 167 | + | |
| 168 | + | |
| 169 | + | |
| 170 | + | |
| 171 | + | |
| 172 | + | |
| 173 | + | |
| 174 | + | |
| 175 | + | |
155 | 176 | | |
156 | 177 | | |
157 | 178 | | |
158 | | - | |
| 179 | + | |
| 180 | + | |
| 181 | + | |
| 182 | + | |
| 183 | + | |
| 184 | + | |
| 185 | + | |
| 186 | + | |
| 187 | + | |
| 188 | + | |
159 | 189 | | |
160 | 190 | | |
161 | 191 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
92 | 92 | | |
93 | 93 | | |
94 | 94 | | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
95 | 98 | | |
96 | 99 | | |
97 | 100 | | |
| |||
Lines changed: 137 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| 13 | + | |
| 14 | + | |
| 15 | + | |
| 16 | + | |
| 17 | + | |
| 18 | + | |
| 19 | + | |
| 20 | + | |
| 21 | + | |
| 22 | + | |
| 23 | + | |
| 24 | + | |
| 25 | + | |
| 26 | + | |
| 27 | + | |
| 28 | + | |
| 29 | + | |
| 30 | + | |
| 31 | + | |
| 32 | + | |
| 33 | + | |
| 34 | + | |
| 35 | + | |
| 36 | + | |
| 37 | + | |
| 38 | + | |
| 39 | + | |
| 40 | + | |
| 41 | + | |
| 42 | + | |
| 43 | + | |
| 44 | + | |
| 45 | + | |
| 46 | + | |
| 47 | + | |
| 48 | + | |
| 49 | + | |
| 50 | + | |
| 51 | + | |
| 52 | + | |
| 53 | + | |
| 54 | + | |
| 55 | + | |
| 56 | + | |
| 57 | + | |
| 58 | + | |
| 59 | + | |
| 60 | + | |
| 61 | + | |
| 62 | + | |
| 63 | + | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
| 71 | + | |
| 72 | + | |
| 73 | + | |
| 74 | + | |
| 75 | + | |
| 76 | + | |
| 77 | + | |
| 78 | + | |
| 79 | + | |
| 80 | + | |
| 81 | + | |
| 82 | + | |
| 83 | + | |
| 84 | + | |
| 85 | + | |
| 86 | + | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
| 91 | + | |
| 92 | + | |
| 93 | + | |
| 94 | + | |
| 95 | + | |
| 96 | + | |
| 97 | + | |
| 98 | + | |
| 99 | + | |
| 100 | + | |
| 101 | + | |
| 102 | + | |
| 103 | + | |
| 104 | + | |
| 105 | + | |
| 106 | + | |
| 107 | + | |
| 108 | + | |
| 109 | + | |
| 110 | + | |
| 111 | + | |
| 112 | + | |
| 113 | + | |
| 114 | + | |
| 115 | + | |
| 116 | + | |
| 117 | + | |
| 118 | + | |
| 119 | + | |
| 120 | + | |
| 121 | + | |
| 122 | + | |
| 123 | + | |
| 124 | + | |
| 125 | + | |
| 126 | + | |
| 127 | + | |
| 128 | + | |
| 129 | + | |
| 130 | + | |
| 131 | + | |
| 132 | + | |
| 133 | + | |
| 134 | + | |
| 135 | + | |
| 136 | + | |
| 137 | + | |
0 commit comments