diff --git a/CHANGELOG.md b/CHANGELOG.md index 5485cff..259506a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,12 @@ All notable changes to this project are documented here. The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/) and the project uses [Semantic Versioning](https://semver.org/). +## [0.6.2] - 2026-10-05 + +### Added + +- `corrupt_source: skip | encode_anyway`, under `settings` or on one output (default `skip`, the 0.6.1 behaviour). Some damaged FLACs have no clean copy to replace them, and refusing them leaves the phone with no copy of the song at all. With `encode_anyway`, a source that fails the strict decode still gets the same ERROR line, then one more encode without `-xerror` and `-err_detect`, so FFmpeg conceals the damaged frames. That copy is logged with one WARNING saying it was made from a damaged source, and its manifest row has kind `tolerant`. The source is still remembered as failed, so the strict attempt is not repeated on every scan, and a source that fails even the tolerant encode stays refused. When the damaged file is later replaced by a clean one (a different size or modification time), the next scan rebuilds the tolerant copy with the strict encode. + ## [0.6.1] - 2026-10-04 ### Added @@ -53,5 +59,6 @@ All notable changes to this project are documented here. The format follows [Kee - Lossy copies are written through a temp file and renamed, like encodes, so a half-written copy is never visible. - `force_reencode: true` purges the outputs once at startup. It used to purge them again on every periodic sync, which re-encoded the whole library every five minutes. +[0.6.2]: https://github.com/GeiserX/audio-transcode-watcher/releases/tag/v0.6.2 [0.6.1]: https://github.com/GeiserX/audio-transcode-watcher/releases/tag/v0.6.1 [0.6.0]: https://github.com/GeiserX/audio-transcode-watcher/releases/tag/v0.6.0 diff --git a/README.md b/README.md index 8d19682..3898510 100644 --- a/README.md +++ b/README.md @@ -35,7 +35,7 @@ Keep one library in several formats: lossless for the archive, lossy for phones curl -fsSL -o config.yaml https://raw.githubusercontent.com/GeiserX/audio-transcode-watcher/main/config.example.yaml docker run -d --name audio_transcoder -e CONFIG_FILE=/app/config.yaml \ -v ./config.yaml:/app/config.yaml:ro -v /path/to/flac:/music/flac:ro -v /path/to/mp3:/music/mp3 \ - drumsergio/audio-transcoder:0.6.1 + drumsergio/audio-transcoder:0.6.2 ``` Edit `config.yaml` first so its outputs match the folders you mount; the example writes ALAC, MP3 and AAC. [Getting started](https://github.com/GeiserX/audio-transcode-watcher/blob/main/docs/getting-started.md) has Docker Compose and the full `docker run`. diff --git a/config.example.yaml b/config.example.yaml index dc76441..55fca9c 100644 --- a/config.example.yaml +++ b/config.example.yaml @@ -1,4 +1,4 @@ -# Audio Transcode Watcher Configuration (0.6.1) +# Audio Transcode Watcher Configuration (0.6.2) # ====================================== # # This configuration file defines the source folder and output destinations @@ -62,6 +62,13 @@ settings: # Seconds between periodic full syncs (default: 300) sync_interval_seconds: 300 + # A source that does not decode cleanly (a damaged FLAC): + # skip write nothing for it (default) + # encode_anyway log the error, then encode it with FFmpeg's error + # concealment, so a one-frame glitch is better than no copy + # An output can set its own corrupt_source to override this. + corrupt_source: skip + # whisper_fallback and whisper_model were removed in 0.6.0 and are ignored diff --git a/docs/ROADMAP.md b/docs/ROADMAP.md index b087bbf..29ebf15 100644 --- a/docs/ROADMAP.md +++ b/docs/ROADMAP.md @@ -1,6 +1,6 @@ # Roadmap -Current version: **0.6.1** +Current version: **0.6.2** ## Completed @@ -13,6 +13,7 @@ Current version: **0.6.1** - **Recursive directory support** -- mirror source folder hierarchy in outputs (v0.5.0) - **Lossy sources copied, not inflated**, corrupt sources fail loudly, ReplayGain and MusicBrainz tags kept in M4A outputs, configurable periodic sync (v0.6.0) - **Portable AAC**: stereo downmix and a 48 kHz cap for AAC outputs, per-output `channels` and `max_sample_rate` (v0.6.1) +- **Damaged sources**: `corrupt_source: encode_anyway` makes a concealed copy instead of none (v0.6.2) ## v0.7.0 -- Quality of Life diff --git a/docs/configuration.md b/docs/configuration.md index 4fec650..514682e 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -60,8 +60,30 @@ settings: # Seconds between periodic full syncs (default: 300) sync_interval_seconds: 300 + + # What to do with a source that does not decode cleanly: skip | encode_anyway + # (default: skip). An output can set its own corrupt_source. + corrupt_source: skip +``` + +### Damaged sources + +A source that fails the strict decode (see [Corrupt sources](how-it-works.md#corrupt-sources)) is handled by `corrupt_source`: + +- `skip` (the default) writes nothing for it. +- `encode_anyway` logs the same ERROR, then encodes it once more without `-xerror` and `-err_detect`, letting FFmpeg conceal the damaged frames. The copy exists, one WARNING line says it was made from a damaged source, and its manifest row has kind `tolerant`. A source that fails even that run stays refused. + +Set it under `settings` for every output, or on one output to override, for example only on the AAC output that a phone plays from: + +```yaml + - name: aac-256 + codec: aac + path: /music/aac + corrupt_source: encode_anyway ``` +Either way the source is remembered as failed, so the strict attempt is not repeated on every scan. Replacing the file with a clean copy (a new size or modification time) clears that, and the next scan rebuilds the tolerant copy with the strict encode. + `whisper_fallback` and `whisper_model` were removed in 0.6.0 along with the Whisper lyrics fallback. A config that still sets them loads, and the first load logs one warning that they are ignored. ## Source formats diff --git a/docs/getting-started.md b/docs/getting-started.md index ecccdb6..2998e7b 100644 --- a/docs/getting-started.md +++ b/docs/getting-started.md @@ -29,7 +29,7 @@ outputs: ```yaml services: audio-transcoder: - image: drumsergio/audio-transcoder:0.6.1 + image: drumsergio/audio-transcoder:0.6.2 container_name: audio_transcoder environment: - TZ=Europe/Madrid @@ -60,6 +60,6 @@ docker run -d \ -v /path/to/flac:/music/flac:ro \ -v /path/to/mp3:/music/mp3 \ --restart unless-stopped \ - drumsergio/audio-transcoder:0.6.1 + drumsergio/audio-transcoder:0.6.2 ``` diff --git a/docs/how-it-works.md b/docs/how-it-works.md index 09c193d..8da7839 100644 --- a/docs/how-it-works.md +++ b/docs/how-it-works.md @@ -17,7 +17,7 @@ Each output folder holds `.atw-manifest.json`, a map from each output file to th ## Corrupt sources -FFmpeg runs with `-xerror` and `-err_detect crccheck+explode`, so a frame whose checksum does not match stops the encode. If a source does not decode cleanly, the encode fails even when FFmpeg exits 0 but printed a decode error. The error is logged with the file name and no output is written. The file is not tried again until its modification time changes, or the service restarts. +FFmpeg runs with `-xerror` and `-err_detect crccheck+explode`, so a frame whose checksum does not match stops the encode. If a source does not decode cleanly, the encode fails even when FFmpeg exits 0 but printed a decode error. The error is logged with the file name and no output is written, unless `corrupt_source: encode_anyway` is set; then the file is encoded once more with FFmpeg's error concealment and marked `tolerant` in the manifest (see [Damaged sources](configuration.md#damaged-sources)). The strict attempt is not repeated until the file's modification time changes, or the service restarts. ## Recursive Directory Support diff --git a/pyproject.toml b/pyproject.toml index 11d62fe..edbbff0 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -5,7 +5,7 @@ build-backend = "hatchling.build" [project] name = "audio-transcode-watcher" -version = "0.6.1" +version = "0.6.2" description = "Watch a source folder and automatically transcode audio files to multiple formats" readme = "README.md" diff --git a/src/audio_transcode_watcher/__init__.py b/src/audio_transcode_watcher/__init__.py index 565c6ae..e0898b2 100644 --- a/src/audio_transcode_watcher/__init__.py +++ b/src/audio_transcode_watcher/__init__.py @@ -1,3 +1,3 @@ """Audio Transcode Watcher - Automatic audio file transcoding.""" -__version__ = "0.6.1" +__version__ = "0.6.2" diff --git a/src/audio_transcode_watcher/config.py b/src/audio_transcode_watcher/config.py index ac2dc66..0656d35 100644 --- a/src/audio_transcode_watcher/config.py +++ b/src/audio_transcode_watcher/config.py @@ -45,10 +45,19 @@ # Portable-playback limits applied when an output does not set them. # AAC goes to phones, AirPods and CarPlay: stereo, at most 48 kHz. # Any codec can opt in by setting channels / max_sample_rate; 0 = no limit. +# What to do with a source that does not decode cleanly: "skip" writes +# nothing; "encode_anyway" retries with ffmpeg's error concealment. +CORRUPT_SOURCE_MODES = ("skip", "encode_anyway") + DEFAULT_CHANNELS = {"aac": 2} DEFAULT_MAX_SAMPLE_RATE = {"aac": 48000} +def _check_corrupt_source(value: Any, label: str) -> None: + if value not in CORRUPT_SOURCE_MODES: + raise ValueError(f"{label} must be one of {', '.join(CORRUPT_SOURCE_MODES)}") + + @dataclass class OutputConfig: """Configuration for a single output destination.""" @@ -60,6 +69,7 @@ class OutputConfig: include_artwork: bool = True channels: int | None = None # Downmix above this many channels; 0 = keep max_sample_rate: int | None = None # Resample above this rate (Hz); 0 = keep + corrupt_source: str | None = None # "skip" | "encode_anyway"; None = global def __post_init__(self) -> None: """Validate and set defaults after initialization.""" @@ -83,6 +93,10 @@ def __post_init__(self) -> None: self.channels = DEFAULT_CHANNELS.get(self.codec, 0) if self.max_sample_rate is None: self.max_sample_rate = DEFAULT_MAX_SAMPLE_RATE.get(self.codec, 0) + if self.corrupt_source is not None: + _check_corrupt_source( + self.corrupt_source, f"Output '{self.name}': corrupt_source" + ) for key in ("channels", "max_sample_rate"): value = getattr(self, key) if isinstance(value, bool) or not isinstance(value, int) or value < 0: @@ -111,6 +125,7 @@ def from_dict(cls, data: dict[str, Any]) -> OutputConfig: include_artwork=data.get("include_artwork", True), channels=data.get("channels"), max_sample_rate=data.get("max_sample_rate"), + corrupt_source=data.get("corrupt_source"), ) @@ -127,12 +142,15 @@ class Config: min_stable_seconds: float = 1.0 fetch_lyrics: bool = True # Auto-fetch .lrc lyrics via syncedlyrics sync_interval_seconds: int = 300 # Seconds between periodic full syncs + corrupt_source: str = "skip" # Default for outputs that don't set it def __post_init__(self) -> None: """Validate configuration after initialization.""" if not self.source_path: raise ValueError("source_path is required") + _check_corrupt_source(self.corrupt_source, "settings.corrupt_source") + interval = self.sync_interval_seconds if ( isinstance(interval, bool) @@ -159,6 +177,10 @@ def output_paths(self) -> list[str]: """Get list of all output directory paths.""" return [o.path for o in self.outputs] + def corrupt_source_for(self, output: OutputConfig) -> str: + """The corrupt_source mode for *output*: its own, else the global one.""" + return output.corrupt_source or self.corrupt_source + def get_output_by_name(self, name: str) -> OutputConfig | None: """Get an output configuration by name.""" for output in self.outputs: @@ -183,6 +205,7 @@ def from_dict(cls, data: dict[str, Any]) -> Config: min_stable_seconds=settings.get("min_stable_seconds", 1.0), fetch_lyrics=settings.get("fetch_lyrics", True), sync_interval_seconds=settings.get("sync_interval_seconds", 300), + corrupt_source=settings.get("corrupt_source", "skip"), ) @classmethod diff --git a/src/audio_transcode_watcher/encoder.py b/src/audio_transcode_watcher/encoder.py index c6480c7..d08aba3 100644 --- a/src/audio_transcode_watcher/encoder.py +++ b/src/audio_transcode_watcher/encoder.py @@ -275,8 +275,31 @@ def _is_artwork_error(stderr: str) -> bool: return any(h in low for h in _ARTWORK_ERROR_HINTS) -def _run_ffmpeg(cmd: list[str]) -> tuple[int, str]: - """Run ffmpeg; a decode error on stderr counts as a failure even with rc 0.""" +def tolerant_command(cmd: list[str]) -> list[str]: + """*cmd* without -xerror and -err_detect, so ffmpeg conceals decode errors.""" + out: list[str] = [] + skip = False + for i, arg in enumerate(cmd): + if skip: + skip = False + continue + if arg == "-xerror": + continue + if arg == "-err_detect" and i + 1 < len(cmd): + skip = True + continue + out.append(arg) + return out + + +def _run_ffmpeg(cmd: list[str], strict: bool = True) -> tuple[int, str]: + """ + Run ffmpeg and return (rc, stderr). + + Strict: a decode error on stderr is a failure even with rc 0, and any + failure that reports a decode error returns DECODE_ERROR_RC. Not strict: + only the exit code counts (concealed errors still print to stderr). + """ try: proc = subprocess.run( cmd, capture_output=True, timeout=FFMPEG_TIMEOUT, check=False @@ -285,7 +308,7 @@ def _run_ffmpeg(cmd: list[str]) -> tuple[int, str]: return TIMEOUT_RC, f"ffmpeg timed out after {FFMPEG_TIMEOUT} s" stderr = proc.stderr.decode("utf-8", errors="ignore") if proc.stderr else "" rc = proc.returncode - if rc == 0 and _decode_error(stderr): + if strict and _decode_error(stderr): rc = DECODE_ERROR_RC return rc, stderr @@ -295,6 +318,7 @@ def atomic_ffmpeg_encode( final_dest: str, retry_without_artwork: bool = True, finalize: Callable[[str], None] | None = None, + strict: bool = True, ) -> int: """ Run FFmpeg with atomic output (write to temp, then rename). @@ -306,9 +330,12 @@ def atomic_ffmpeg_encode( failure points at the attached picture stream finalize: Optional callable run on the finished temp file before the rename (used to copy extra tags) + strict: If True, a decode error reported on stderr fails the encode + and the result is DECODE_ERROR_RC. False for the tolerant retry. Returns: - Return code (0 for success). No output is written on failure. + Return code (0 for success, DECODE_ERROR_RC when the source did not + decode cleanly). No output is written on failure. """ final_dest = nfc_path(final_dest) dest_dir = os.path.dirname(final_dest) @@ -329,7 +356,7 @@ def atomic_ffmpeg_encode( source = _source_from_cmd(cmd) logger.info("► %s", " ".join(cmd)) - rc, stderr = _run_ffmpeg(cmd) + rc, stderr = _run_ffmpeg(cmd, strict) if rc != 0 and retry_without_artwork and _is_artwork_error(stderr): _cleanup_temp(tmp_dest) @@ -337,7 +364,7 @@ def atomic_ffmpeg_encode( cmd = _remove_artwork_from_command(cmd) cmd[-1] = tmp_dest logger.info("► (retry) %s", " ".join(cmd)) - rc, stderr = _run_ffmpeg(cmd) + rc, stderr = _run_ffmpeg(cmd, strict) if rc != 0: detail = _decode_error(stderr) or (stderr.strip().splitlines() or [""])[-1] diff --git a/src/audio_transcode_watcher/sync.py b/src/audio_transcode_watcher/sync.py index cc640d0..721def2 100644 --- a/src/audio_transcode_watcher/sync.py +++ b/src/audio_transcode_watcher/sync.py @@ -14,10 +14,12 @@ from . import manifest from .config import Config, OutputConfig from .encoder import ( + DECODE_ERROR_RC, atomic_ffmpeg_encode, build_ffmpeg_command, copy_mp4_tags, portable_limit_args, + tolerant_command, ) from .lyrics import fetch_lyrics_for_file from .utils import ( @@ -320,6 +322,23 @@ def _made_from_lossy(output: OutputConfig, out_path: str) -> bool: return True +def _tolerant_source_changed( + output: OutputConfig, out_path: str, source_path: str +) -> bool: + """True if *out_path* is a tolerant copy and its source has since changed. + + A damaged source replaced by a clean copy gets a strict encode again. + """ + row = manifest.lookup(output.path, out_path) + if not row or row.get("kind") != "tolerant": + return False + try: + st = os.stat(source_path) + except OSError: + return False + return (st.st_size, st.st_mtime) != (row.get("size"), row.get("mtime")) + + def _process_outputs(source_path: str, config: Config, force: bool) -> bool: """ Process all outputs for a source file. @@ -347,13 +366,19 @@ def _process_outputs(source_path: str, config: Config, force: bool) -> bool: if siblings: _remove_lossy_copies(source_path, siblings, output, config) if not force and os.path.exists(out_path): - if not (is_lossless(source_path) and _made_from_lossy(output, out_path)): + if _tolerant_source_changed(output, out_path, source_path): + logger.info( + "Rebuilding %s: it was made from a damaged source that has changed", + out_path, + ) + elif is_lossless(source_path) and _made_from_lossy(output, out_path): + logger.info( + "Replacing %s: it was made from a lossy source, %s wins", + out_path, + source_path, + ) + else: continue - logger.info( - "Replacing %s: it was made from a lossy source, %s wins", - out_path, - source_path, - ) os.makedirs(os.path.dirname(out_path), exist_ok=True) if action == "copy": @@ -371,18 +396,36 @@ def finalize(tmp: str, src: str = source_path) -> None: cmd = build_ffmpeg_command(source_path, out_path, output) rc = atomic_ffmpeg_encode(cmd, out_path, finalize=finalize) - if rc == 0: - kind = "encode" if is_lossless(source_path) else "transcode" - manifest.record( - output.path, out_path, config.source_path, source_path, kind - ) - else: + kind = "encode" if is_lossless(source_path) else "transcode" + if rc != 0: logger.error( "%s encode failed for %s", output.name.upper(), source_path, ) + # Remembered even if the tolerant run below succeeds, so the + # strict attempt is not repeated on every scan. ok = False + if ( + rc == DECODE_ERROR_RC + and config.corrupt_source_for(output) == "encode_anyway" + ): + rc = atomic_ffmpeg_encode( + tolerant_command(cmd), out_path, finalize=finalize, strict=False + ) + if rc == 0: + kind = "tolerant" + logger.warning( + "%s copy %s was made from a damaged source %s; " + "it may glitch where the source is corrupt", + output.name.upper(), + out_path, + source_path, + ) + if rc == 0: + manifest.record( + output.path, out_path, config.source_path, source_path, kind + ) return ok diff --git a/tests/test_config.py b/tests/test_config.py index 109deec..d2253b6 100644 --- a/tests/test_config.py +++ b/tests/test_config.py @@ -338,3 +338,39 @@ def test_rejects_bad_values(self, key, bad): OutputConfig.from_dict( {"name": "aac", "codec": "aac", "path": "/a", key: bad} ) + + +class TestCorruptSourceSetting: + def test_defaults_to_skip(self): + cfg = Config.from_dict(_settings_config({})) + assert cfg.corrupt_source == "skip" + assert cfg.corrupt_source_for(cfg.outputs[0]) == "skip" + + def test_global_setting(self): + cfg = Config.from_dict(_settings_config({"corrupt_source": "encode_anyway"})) + assert cfg.corrupt_source_for(cfg.outputs[0]) == "encode_anyway" + + def test_output_overrides_global(self): + data = _settings_config({"corrupt_source": "skip"}) + data["outputs"].append( + { + "name": "aac", + "codec": "aac", + "path": "/music/aac", + "corrupt_source": "encode_anyway", + } + ) + cfg = Config.from_dict(data) + alac, aac = cfg.outputs + assert cfg.corrupt_source_for(alac) == "skip" + assert cfg.corrupt_source_for(aac) == "encode_anyway" + + def test_rejects_unknown_global_value(self): + with pytest.raises(ValueError, match="settings.corrupt_source"): + Config.from_dict(_settings_config({"corrupt_source": "ignore"})) + + def test_rejects_unknown_output_value(self): + with pytest.raises(ValueError, match="corrupt_source"): + OutputConfig.from_dict( + {"name": "aac", "codec": "aac", "path": "/a", "corrupt_source": "yes"} + ) diff --git a/tests/test_encoder.py b/tests/test_encoder.py index c7c7e70..868117f 100644 --- a/tests/test_encoder.py +++ b/tests/test_encoder.py @@ -13,6 +13,7 @@ from audio_transcode_watcher.config import OutputConfig from audio_transcode_watcher.encoder import ( + DECODE_ERROR_RC, MP4_TAG_ATOMS, _cleanup_temp, _remove_artwork_from_command, @@ -21,6 +22,7 @@ copy_mp4_tags, limit_args, target_sample_rate, + tolerant_command, ) @@ -996,3 +998,42 @@ def test_alac_keeps_96k_and_6_channels(self, tmp_path): assert self._encode( src, OutputConfig(name="alac", codec="alac", path="/x"), tmp_path ) == (96000, 6) + + +class TestTolerantRun: + """The relaxed encode used for corrupt_source: encode_anyway.""" + + def test_tolerant_command_drops_only_the_strict_flags(self, tmp_path): + cmd, _dest = _alac_cmd(tmp_path) + relaxed = tolerant_command(cmd) + assert "-xerror" not in relaxed and "-err_detect" not in relaxed + assert "crccheck+explode" not in relaxed + assert [ + a for a in cmd if a not in ("-xerror", "-err_detect", "crccheck+explode") + ] == relaxed + + def test_strict_failure_with_decode_error_reports_decode_rc( + self, fake_ffmpeg, tmp_path, monkeypatch + ): + monkeypatch.setenv("FAKE_RC", "183") + monkeypatch.setenv("FAKE_STDERR", CORRUPT_FLAC_STDERR) + cmd, dest = _alac_cmd(tmp_path) + assert atomic_ffmpeg_encode(cmd, dest) == DECODE_ERROR_RC + + def test_non_strict_run_ignores_concealed_errors( + self, fake_ffmpeg, tmp_path, monkeypatch + ): + monkeypatch.setenv("FAKE_RC", "0") + monkeypatch.setenv("FAKE_STDERR", CORRUPT_FLAC_STDERR) + cmd, dest = _alac_cmd(tmp_path) + assert atomic_ffmpeg_encode(tolerant_command(cmd), dest, strict=False) == 0 + assert os.path.isfile(dest) + + def test_non_strict_run_still_fails_on_exit_code( + self, fake_ffmpeg, tmp_path, monkeypatch + ): + monkeypatch.setenv("FAKE_RC", "1") + monkeypatch.setenv("FAKE_STDERR", CORRUPT_FLAC_STDERR) + cmd, dest = _alac_cmd(tmp_path) + assert atomic_ffmpeg_encode(tolerant_command(cmd), dest, strict=False) == 1 + assert not os.path.exists(dest) diff --git a/tests/test_sync.py b/tests/test_sync.py index c933709..92f2563 100644 --- a/tests/test_sync.py +++ b/tests/test_sync.py @@ -10,6 +10,7 @@ import audio_transcode_watcher.sync as sync_mod from audio_transcode_watcher import manifest from audio_transcode_watcher.config import Config, OutputConfig +from audio_transcode_watcher.encoder import DECODE_ERROR_RC from audio_transcode_watcher.sync import ( _cleanup_orphans, _has_lossless_source, @@ -2075,3 +2076,267 @@ def test_edited_output_is_not_replaced(self, _guard, temp_dir): enc.assert_not_called() assert (mp3 / "X.mp3").read_bytes() == b"edited by hand, longer" + + +@pytest.mark.usefixtures("fresh_manifests") +class TestCorruptSourceModes: + """corrupt_source: skip refuses a damaged source; encode_anyway conceals.""" + + def _config(self, source, out, mode, per_output=None): + return Config( + source_path=str(source), + outputs=[ + OutputConfig( + name="aac", codec="aac", path=str(out), corrupt_source=per_output + ) + ], + fetch_lyrics=False, + corrupt_source=mode, + ) + + @staticmethod + def _strict_fails_tolerant_works(calls): + def encode(cmd, dest, strict=True, **_kwargs): + calls.append((list(cmd), strict)) + if strict: + return DECODE_ERROR_RC + Path(dest).write_bytes(b"concealed") + return 0 + + return encode + + @patch("audio_transcode_watcher.sync.safety_guard_active", return_value=False) + def test_skip_writes_nothing_and_remembers(self, _guard, temp_dir): + source, out = _dirs(temp_dir, "source", "aac") + flac = source / "Radiohead - High and Dry.flac" + flac.write_bytes(b"flac") + calls = [] + with patch( + "audio_transcode_watcher.sync.atomic_ffmpeg_encode", + side_effect=self._strict_fails_tolerant_works(calls), + ): + process_source_file( + str(flac), self._config(source, out, "skip"), check_stable=False + ) + assert [strict for _cmd, strict in calls] == [True] + assert _visible(out) == [] + assert str(flac) in sync_mod._failed_sources + + @pytest.mark.parametrize( + "glob,per_output", [("encode_anyway", None), ("skip", "encode_anyway")] + ) + @patch("audio_transcode_watcher.sync.safety_guard_active", return_value=False) + def test_encode_anyway_makes_a_tolerant_copy( + self, _guard, temp_dir, caplog, glob, per_output + ): + source, out = _dirs(temp_dir, "source", "aac") + flac = source / "Radiohead - High and Dry.flac" + flac.write_bytes(b"flac") + config = self._config(source, out, glob, per_output) + calls = [] + with patch( + "audio_transcode_watcher.sync.atomic_ffmpeg_encode", + side_effect=self._strict_fails_tolerant_works(calls), + ): + process_source_file(str(flac), config, check_stable=False) + + # One strict run, then exactly one tolerant run without the strict flags. + assert [strict for _cmd, strict in calls] == [True, False] + strict_cmd, tolerant_cmd = calls[0][0], calls[1][0] + assert "-xerror" in strict_cmd and "-err_detect" in strict_cmd + assert "-xerror" not in tolerant_cmd and "-err_detect" not in tolerant_cmd + dest = out / "Radiohead - High and Dry.m4a" + assert dest.read_bytes() == b"concealed" + assert manifest.lookup(str(out), str(dest))["kind"] == "tolerant" + + errors = [r for r in caplog.records if r.levelname == "ERROR"] + warnings = [ + r + for r in caplog.records + if r.levelname == "WARNING" and "damaged source" in r.getMessage() + ] + assert any("encode failed" in r.getMessage() for r in errors) + assert len(warnings) == 1 + + # The source is remembered: the next scan does not retry it at all. + assert str(flac) in sync_mod._failed_sources + calls.clear() + dest.unlink() + with patch( + "audio_transcode_watcher.sync.atomic_ffmpeg_encode", + side_effect=self._strict_fails_tolerant_works(calls), + ): + process_source_file(str(flac), config, check_stable=False) + assert calls == [] + + @patch("audio_transcode_watcher.sync.safety_guard_active", return_value=False) + def test_source_failing_the_tolerant_run_stays_refused( + self, _guard, temp_dir, caplog + ): + source, out = _dirs(temp_dir, "source", "aac") + flac = source / "X.flac" + flac.write_bytes(b"flac") + calls = [] + + def encode(cmd, dest, strict=True, **_kwargs): + calls.append(strict) + return DECODE_ERROR_RC if strict else 1 + + with patch( + "audio_transcode_watcher.sync.atomic_ffmpeg_encode", side_effect=encode + ): + process_source_file( + str(flac), + self._config(source, out, "encode_anyway"), + check_stable=False, + ) + assert calls == [True, False] + assert _visible(out) == [] + assert str(flac) in sync_mod._failed_sources + assert "damaged source" not in caplog.text + + @patch("audio_transcode_watcher.sync.safety_guard_active", return_value=False) + def test_other_failures_get_no_tolerant_run(self, _guard, temp_dir): + source, out = _dirs(temp_dir, "source", "aac") + flac = source / "X.flac" + flac.write_bytes(b"flac") + calls = [] + + def encode(cmd, dest, strict=True, **_kwargs): + calls.append(strict) + return 124 # timed out, not a decode error + + with patch( + "audio_transcode_watcher.sync.atomic_ffmpeg_encode", side_effect=encode + ): + process_source_file( + str(flac), + self._config(source, out, "encode_anyway"), + check_stable=False, + ) + assert calls == [True] + + +@pytest.mark.skipif( + __import__("shutil").which("ffmpeg") is None, + reason="ffmpeg is not installed; the real-file checks need it", +) +@pytest.mark.usefixtures("fresh_manifests") +class TestCorruptSourceRealFile: + def test_damaged_flac_gets_a_tolerant_aac(self, temp_dir): + import subprocess + + source, out = _dirs(temp_dir, "source", "aac") + good = source / "good.tmp.flac" + subprocess.run( + [ + "ffmpeg", + "-loglevel", + "error", + "-y", + "-f", + "lavfi", + "-i", + "sine=frequency=440:duration=5", + "-c:a", + "flac", + str(good), + ], + check=True, + ) + data = bytearray(good.read_bytes()) + good.unlink() + for i in range(len(data) // 3, len(data) // 3 + 400): + data[i] ^= 0x5A + bad = source / "Scott McKenzie - San Francisco.flac" + bad.write_bytes(bytes(data)) + config = Config( + source_path=str(source), + outputs=[OutputConfig(name="aac", codec="aac", path=str(out))], + fetch_lyrics=False, + corrupt_source="encode_anyway", + ) + + process_source_file(str(bad), config, check_stable=False) + + dest = out / "Scott McKenzie - San Francisco.m4a" + assert dest.exists() and dest.stat().st_size > 0 + assert manifest.lookup(str(out), str(dest))["kind"] == "tolerant" + assert str(bad) in sync_mod._failed_sources + + +@pytest.mark.usefixtures("fresh_manifests") +class TestTolerantCopyReplacedByCleanSource: + """A clean replacement of a damaged source gets a strict encode on a plain scan.""" + + @patch("audio_transcode_watcher.sync.safety_guard_active", return_value=False) + def test_clean_replacement_rebuilds_the_tolerant_copy(self, _guard, temp_dir): + source, out = _dirs(temp_dir, "source", "aac") + flac = source / "Radiohead - High and Dry.flac" + flac.write_bytes(b"damaged flac") + config = Config( + source_path=str(source), + outputs=[OutputConfig(name="aac", codec="aac", path=str(out))], + fetch_lyrics=False, + corrupt_source="encode_anyway", + ) + dest = out / "Radiohead - High and Dry.m4a" + + def damaged(cmd, d, strict=True, **_kwargs): + if strict: + return DECODE_ERROR_RC + Path(d).write_bytes(b"concealed") + return 0 + + with patch( + "audio_transcode_watcher.sync.atomic_ffmpeg_encode", side_effect=damaged + ): + process_source_file(str(flac), config, check_stable=False) + assert dest.read_bytes() == b"concealed" + + # Unchanged source: the tolerant copy is left alone on later scans. + sync_mod._failed_sources.clear() + with patch("audio_transcode_watcher.sync.atomic_ffmpeg_encode") as enc: + process_source_file(str(flac), config, check_stable=False) + enc.assert_not_called() + + # The clean copy arrives (new size and mtime): strict encode replaces it. + flac.write_bytes(b"clean flac, a different size") + later = flac.stat().st_mtime + 10 + os.utime(flac, (later, later)) + calls = [] + + def clean(cmd, d, strict=True, **_kwargs): + calls.append(strict) + Path(d).write_bytes(b"clean encode") + return 0 + + with patch( + "audio_transcode_watcher.sync.atomic_ffmpeg_encode", side_effect=clean + ): + process_source_file(str(flac), config, force=False, check_stable=False) + assert calls == [True] + assert dest.read_bytes() == b"clean encode" + assert manifest.lookup(str(out), str(dest))["kind"] == "encode" + + @patch("audio_transcode_watcher.sync.atomic_ffmpeg_encode") + @patch("audio_transcode_watcher.sync.safety_guard_active", return_value=False) + def test_only_tolerant_rows_are_rebuilt_on_a_plain_scan( + self, _guard, mock_encode, temp_dir + ): + source, out = _dirs(temp_dir, "source", "aac") + flac = source / "X.flac" + flac.write_bytes(b"flac") + dest = out / "X.m4a" + dest.write_bytes(b"strict encode") + manifest.record(str(out), str(dest), str(source), str(flac), "encode") + flac.write_bytes(b"flac with edited tags") # source changed after the encode + config = Config( + source_path=str(source), + outputs=[OutputConfig(name="aac", codec="aac", path=str(out))], + fetch_lyrics=False, + ) + + process_source_file(str(flac), config, force=False, check_stable=False) + + mock_encode.assert_not_called() diff --git a/uv.lock b/uv.lock index 9553676..71bef3a 100644 --- a/uv.lock +++ b/uv.lock @@ -4,7 +4,7 @@ requires-python = ">=3.14" [[package]] name = "audio-transcode-watcher" -version = "0.6.1" +version = "0.6.2" source = { editable = "." } dependencies = [ { name = "mutagen" },