fix: bundle code reached through a symlinked root (FLYTE-SDK-7E) - #1441
Open
EngHabu wants to merge 1 commit into
Open
fix: bundle code reached through a symlinked root (FLYTE-SDK-7E)#1441EngHabu wants to merge 1 commit into
EngHabu wants to merge 1 commit into
Conversation
`flyte deploy` aborted with a raw ValueError out of pathlib when the bundle root
was reached through a symlink:
ValueError: '/tmp/flyte2-smoke.py' is not in the subpath of '/private/tmp'
On macOS `/tmp` is a symlink to `/private/tmp`, so the root resolves to
`/private/tmp` while the loaded module's `__file__` still reads `/tmp/task.py`.
`_is_user_file` checks containment on *resolved* paths, so the file is correctly
accepted, but discovery returns it with its original spelling — which is not a
literal subpath of the root. `ls_files` then takes it relative to the root and
raises.
The crash is only the visible half. `create_bundle` derives tar arcnames with
`os.path.relpath`, which is lexical: the same path yields `../link/task.py`,
trips the outside-the-root guard, and the file is *silently dropped*. A run that
got past the ValueError would upload an empty bundle and fail on the cluster
with ModuleNotFoundError instead.
Rebase discovered paths onto the root, so containment stays resolved-based while
the returned path keeps the `source_path` prefix that everything downstream
expects. This mirrors what the `additional_files` branch already does for
include paths. Paths that are already literal subpaths are left untouched — a
symlink *inside* the root that legitimately points outside it (`.venv/bin/python`)
must keep its in-root spelling, which `create_bundle` relies on.
A file that is still outside the root after resolving means the configured root
does not contain the code, so it now raises `CodeBundleError` naming the root and
pointing at `--root-dir`, instead of leaking a raw pathlib ValueError to Sentry.
fixes FLYTE-SDK-7E
Signed-off-by: Haytham Abuelfutuh <haytham@afutuh.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
flyte deployaborts with a rawValueErrorout ofpathlibwhen the bundle root is reached through a symlink (FLYTE-SDK-7E, release 2.6.0):Why
On macOS
/tmpis a symlink to/private/tmp. The bundle root resolves to/private/tmp, but the loaded module's__file__still reads/tmp/task.py._is_user_filechecks containment on resolved paths, so the file is correctly accepted — but discovery returns it with its original spelling, which is not a literal subpath of the root.ls_filesthen takes it relative to the root and raises.The crash is only the visible half.
create_bundlederives tar arcnames withos.path.relpath, which is purely lexical: the same path yields../link/task.py, trips the existing outside-the-root guard, and the file is silently dropped. A run that got past theValueErrorwould upload an empty bundle and fail on the cluster withModuleNotFoundErrorinstead. Verified both halves onmain— the tarball comes back empty.How
Rebase discovered paths onto the root, so containment stays resolved-based while the returned path keeps the
source_pathprefix that everything downstream (ls_files's digest,print_ls_tree,create_bundle's arcnames) expects. This mirrors what theadditional_filesbranch already does for include paths.Two deliberate constraints:
.venv/bin/python -> /usr/bin/python3) must keep its in-root spelling —create_bundledocuments that it relies on this and avoidsresolve()for exactly that reason. Pinned by a regression test.CodeBundleErrornaming the root and pointing at--root-dir, instead of leaking a raw pathlibValueErrorinto crash reporting.No digest change for setups that work today: the only paths whose spelling changes are the ones that previously crashed.
Tests
4 tests in
tests/flyte/code_bundle/test_code_bundle.py; 3 verified failing onmain. The 4th is the.venv/bin/pythonguard, which passes on both sides by design — it exists to catch a regression this change could otherwise introduce.Full
tests/flyte: 3597 passed, 7 failed — all 7 fail identically on cleanmainin this environment.fixes FLYTE-SDK-7E