Repository navigation
Make offline/CI use robust: fetch timeout + JANELIA_PATHLIB_NO_FETCH - #2
Merged
Merged
Conversation
Two changes so the library can be used where the Janelia network is unreachable (e.g. CI on cloud runners), where the share-data auto-fetch otherwise raises or could hang: - Add a timeout to the Fileglancer fetch (default 10s, override via JANELIA_PATHLIB_FETCH_TIMEOUT) so an unreachable network can't hang the first import indefinitely. - Add JANELIA_PATHLIB_NO_FETCH: when set and no cache is present, _load_shares returns no shares and paths pass through untranslated instead of raising, so downstream code that constructs JaneliaPath runs offline (non-share paths behave exactly like pathlib.Path). Default behavior is unchanged: a missing cache with no opt-out still raises, now with a hint about the new flag. Adds tests for the timeout, the env override, the NO_FETCH degrade path, and that the raise-without-opt-out behavior is preserved. Documents both env vars. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
Downstream projects that construct
JaneliaPathin their test suites hit two rough edges when running on a host without access to the Janelia network (e.g. GitHub-hosted CI runners):_load_shares()raisesRuntimeErrorwhen there's no cachedshares.jsonand the auto-fetch fails — so anyJaneliaPathuse crashes offline, with no opt-out short of pre-seedingJANELIA_PATHLIB_SHARES_PATHwith a real file.urllib.request.urlopen(API_URL)), so on a network that routes to but can't reach the host it can hang the first import.(Context: adopting janelia-pathlib in a data-conversion project, we had to wrap every call in a try/except + availability probe to keep CI green. These changes would let downstream code use
JaneliaPathdirectly.)Changes
urlopen(API_URL, timeout=fetch_timeout()), default 10s, override viaJANELIA_PATHLIB_FETCH_TIMEOUT. Pure robustness; no behavior change on a reachable network.JANELIA_PATHLIB_NO_FETCHopt-out — when set and no cache is present,_load_shares()returns[](no known shares) so paths pass through untranslated instead of raising. Non-share paths already behave exactly likepathlib.Path, so offline code keeps working.Default behavior is unchanged: a missing cache with no opt-out still raises — the message now also mentions the new flag.
Tests
Added (44 passed, 3 skipped locally):
fetch_sharespasses a positivetimeouttourlopenJANELIA_PATHLIB_FETCH_TIMEOUToverride (incl. invalid value → default)NO_FETCH+ missing cache →_load_shares() == []and a share path is left untranslatedRuntimeError(behavior preserved)README documents both env vars under a new "Running offline (CI)" section.
🤖 Generated with Claude Code