Skip to content

Commit 5893fe8

Browse files
committed
fix: exclude .tmp-*.gtfs.zip from the GTFS archive scan
`scanGtfsArchives` matched any path ending in `.gtfs.zip`, including the `.tmp-<id>.gtfs.zip` files gtfsclean writes mid-cleanup before atomic-renaming to the published name. On a failed pipeline run the tmp file's recent mtime made the resume path classify it as "freshly downloaded" and upsert a `.tmp-<id>` entry into the dataset registry. After the eventual successful rename, the stale tmp entry lingered alongside the real one. Tighten the pattern to require a non-dot first character so dotfile-prefixed working files never enter the registry or the prune sweeps. Also closes a subtle data-loss window where pruneFeedsOutsideCountryFilter would wipe a `.tmp-de_DELFI.gtfs.zip` mid-rename if the country prefix didn't match.
1 parent 0389d8f commit 5893fe8

1 file changed

Lines changed: 6 additions & 1 deletion

File tree

services/data-manager/src/jobs/transitous-pipeline.ts

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,12 @@ export const DEFAULT_TRANSITOUS_REPO_URL = "https://github.com/public-transport/
1818
export const DEFAULT_TRANSITOUS_API_KEYS_PATH = "/config/transitous/api-keys.json";
1919
const TRANSITOUS_CATALOG_DIR = ".transitous-catalog";
2020
const TRANSITOUS_DOWNLOADS_DIR = ".transitous-downloads";
21-
const GTFS_ARCHIVE_RE = /\.(gtfs|netex)\.zip$/i;
21+
// Match published GTFS / NeTEx archives only. `.tmp-*.gtfs.zip` and any
22+
// other dotfile-prefixed name is excluded — gtfsclean writes its working
23+
// output as `.tmp-<id>.gtfs.zip` and renames atomically at the end, so a
24+
// half-written tmp file should never end up in the dataset registry or
25+
// the prune scans (where its mid-rename mtime would briefly look "fresh").
26+
const GTFS_ARCHIVE_RE = /^[^.][^/]*\.(gtfs|netex)\.zip$/i;
2227

2328
type CommandRunner = (
2429
command: string,

0 commit comments

Comments
 (0)