Repository navigation
Task/10322 let the project declare its key prefix - #843
Merged
briemla merged 11 commits intoSep 30, 2026
Merged
Conversation
randy-seng
added this pull request to stack #837
September 12, 2026 07:45
randy-seng
force-pushed
the
task/10322-let-the-project-declare-its-key-prefix
branch
2 times, most recently
from
September 14, 2026 14:02
cbae070 to
b32f839
Compare
randy-seng
force-pushed
the
task/10322-let-the-project-declare-its-key-prefix
branch
from
September 14, 2026 18:37
b32f839 to
ee076aa
Compare
randy-seng
force-pushed
the
task/10322-let-the-project-declare-its-key-prefix
branch
2 times, most recently
from
September 21, 2026 12:46
522ddd1 to
2f254b4
Compare
randy-seng
force-pushed
the
task/10322-let-the-project-declare-its-key-prefix
branch
from
September 24, 2026 09:44
2f254b4 to
3e19b46
Compare
randy-seng
force-pushed
the
task/10322-let-the-project-declare-its-key-prefix
branch
3 times, most recently
from
September 29, 2026 09:18
5dfbf43 to
33a711a
Compare
Base automatically changed from
task/10321-do-not-silently-rebind-otconfig-file-references
to
main
September 29, 2026 09:44
randy-seng
force-pushed
the
task/10322-let-the-project-declare-its-key-prefix
branch
from
September 29, 2026 09:44
33a711a to
51c28b4
Compare
An instance reads from exactly one prefix, fixed at startup by S3_KEY_PREFIX, so every project it can open must live under that prefix. That is untenable for the UI Worker, which is handed a different project's otconfig per lease. The otconfig now carries a top-level `s3_key_prefix`, parsed into an S3KeyPrefix value object on OtConfig. Reading only, so far: the key is absent from every file written before this, so a missing key parses to None. Deciding whether a file may be loaded with a prefix -- or without one -- belongs to ValidateProjectLocation, which knows what this installation reads from. The field is defaulted, and has to stay so: OTCloud rebuilds an OtConfig by keyword and cannot name it until it bumps its OTAnalytics pin. See docs/adr/0004-project-declares-its-key-prefix.md.
ADR 0004 gives four cases: an s3-mode load needs a prefix, a local-mode load must not have one, and the two matching cases proceed. ValidateProjectLocation answers all four without any code asking what the transfer mode is -- the wiring picks RefuseAnyProjectLocation or RequireWellFormedKeyPrefix, and the implementation is the answer. The check sits in LoadOtconfig._begin, after the reset and parse but before anything is published. A refusal therefore leaves the application exactly as the reset left it, so all-or-nothing holds without _abort having to undo a partial load. A prefix that is absolute or bears `..` is refused too. The bucket is environment-only, so such a prefix cannot read across buckets, but it does not describe a location inside the configured one either. Nothing sets the prefix on save yet, so an s3-mode deployment can currently open no project at all.
Saving now records where the project's data lives, so a project saved in s3 mode can be reopened there. The prefix is held by CurrentKeyPrefix between loading a project and saving it: it arrives with the project, not with the process, and opening another project moves where this instance reads. The holder is seeded with None and reset with the application, so a prefix cannot outlive its project and let a save write the previous project's location. LoadOtconfig sets it in _publish_before_tracks, after _begin's reset cleared it and before the track files are read from under it. serialize and convert take the prefix as a required argument, not a defaulted one. SaveOtconfig and OtconfigHasChanged both call them and must pass the same value: if only the save side passed it, the written file would carry the key while convert omitted it, the comparison would never match, and every project would report itself as permanently unsaved. A default makes that a silent trap; requiring the argument makes both call sites decide. A project that names no location writes no key, so a local-mode otconfig is unchanged.
The providers now list under the prefix the loaded project declares, and S3_KEY_PREFIX is gone from the environment. An instance is no longer pinned to one prefix for its lifetime: opening another project moves where it reads, within the one bucket it may authenticate to. With no project loaded there is no prefix and nothing to list, so the time-range dialog is refused and says to load a project first. That is a behaviour change to slice 6, which offered the dialog at startup, and matches how the worker runs: a lease starts when an otconfig is delivered. The acceptance test preloads a project otconfig instead of setting S3_KEY_PREFIX, which is the same change seen from the outside. ADR 0003's variable list is marked as the list at the time of that ADR rather than rewritten: the decision to configure S3 through the environment stands, only the prefix left it.
The application is configured by environment variables only, and the test harness started it with `subprocess.Popen` without an `env`, so it inherited whatever the developer's shell exported. A machine set up for s3 work exports OTA_DATA_TRANSFER_MODE=s3, which started the local mode tests in s3 mode. Since `RequireWellFormedKeyPrefix` refuses a prefix-less otconfig, every config in tests/data was rejected, the load aborted, and four tests timed out waiting for a UI that never arrived -- while the same tests passed on CI, where no such variable exists. `local_filesystem_environment` drops the transfer mode variables and keeps the rest, so a local mode test says which mode it wants. The s3 fixture builds on it through `s3_environment`, which also stops a stray OTA_S3_REGION or OTA_S3_DOWNLOAD_CONCURRENCY from reaching an application the test believes it configured itself.
…ion` and update usages
…er serialization
randy-seng
force-pushed
the
task/10322-let-the-project-declare-its-key-prefix
branch
from
September 29, 2026 15:40
640af34 to
e9f2830
Compare
briemla
approved these changes
Sep 30, 2026
briemla
deleted the
task/10322-let-the-project-declare-its-key-prefix
branch
September 30, 2026 06:47
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
OP#10322