From 47720b3111884293c9b109a19ee5bfd4101a2fcb Mon Sep 17 00:00:00 2001 From: GeiserX <9169332+GeiserX@users.noreply.github.com> Date: Mon, 5 Oct 2026 00:36:57 +0200 Subject: [PATCH 1/3] feat: let a damaged source still get a concealed copy with corrupt_source: encode_anyway (0.6.2) Nine FLACs in the library fail the strict decode and have no clean copy anywhere, so 0.6.1 leaves the phone with no copy of those songs at all. The new corrupt_source setting (global under settings, or per output) keeps "skip" as the default. With "encode_anyway", a decode failure logs the same ERROR, then encodes once more without -xerror and -err_detect so ffmpeg conceals the damaged frames; the copy gets one WARNING line and the manifest kind "tolerant". The source is still remembered as failed, so the strict attempt is not repeated every scan, and a source that fails the tolerant run too stays refused. atomic_ffmpeg_encode now reports any failure that printed a decode error as DECODE_ERROR_RC, so the caller can tell it from a timeout or a disk error, and takes strict=False for the tolerant run. --- CHANGELOG.md | 7 + README.md | 2 +- config.example.yaml | 9 +- docs/ROADMAP.md | 3 +- docs/configuration.md | 22 +++ docs/getting-started.md | 4 +- docs/how-it-works.md | 2 +- pyproject.toml | 2 +- src/audio_transcode_watcher/__init__.py | 2 +- src/audio_transcode_watcher/config.py | 23 +++ src/audio_transcode_watcher/encoder.py | 39 ++++- src/audio_transcode_watcher/sync.py | 32 +++- tests/test_config.py | 36 +++++ tests/test_encoder.py | 41 ++++++ tests/test_sync.py | 188 ++++++++++++++++++++++++ uv.lock | 2 +- 16 files changed, 393 insertions(+), 21 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 5485cff..2c48254 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,12 @@ All notable changes to this project are documented here. The format follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/) and the project uses [Semantic Versioning](https://semver.org/). +## [0.6.2] - 2026-10-05 + +### Added + +- `corrupt_source: skip | encode_anyway`, under `settings` or on one output (default `skip`, the 0.6.1 behaviour). Some damaged FLACs have no clean copy to replace them, and refusing them leaves the phone with no copy of the song at all. With `encode_anyway`, a source that fails the strict decode still gets the same ERROR line, then one more encode without `-xerror` and `-err_detect`, so FFmpeg conceals the damaged frames. That copy is logged with one WARNING saying it was made from a damaged source, and its manifest row has kind `tolerant`. The source is still remembered as failed, so the strict attempt is not repeated on every scan, and a source that fails even the tolerant encode stays refused. + ## [0.6.1] - 2026-10-04 ### Added @@ -53,5 +59,6 @@ All notable changes to this project are documented here. The format follows [Kee - Lossy copies are written through a temp file and renamed, like encodes, so a half-written copy is never visible. - `force_reencode: true` purges the outputs once at startup. It used to purge them again on every periodic sync, which re-encoded the whole library every five minutes. +[0.6.2]: https://github.com/GeiserX/audio-transcode-watcher/releases/tag/v0.6.2 [0.6.1]: https://github.com/GeiserX/audio-transcode-watcher/releases/tag/v0.6.1 [0.6.0]: https://github.com/GeiserX/audio-transcode-watcher/releases/tag/v0.6.0 diff --git a/README.md b/README.md index 8d19682..3898510 100644 --- a/README.md +++ b/README.md @@ -35,7 +35,7 @@ Keep one library in several formats: lossless for the archive, lossy for phones curl -fsSL -o config.yaml https://raw.githubusercontent.com/GeiserX/audio-transcode-watcher/main/config.example.yaml docker run -d --name audio_transcoder -e CONFIG_FILE=/app/config.yaml \ -v ./config.yaml:/app/config.yaml:ro -v /path/to/flac:/music/flac:ro -v /path/to/mp3:/music/mp3 \ - drumsergio/audio-transcoder:0.6.1 + drumsergio/audio-transcoder:0.6.2 ``` Edit `config.yaml` first so its outputs match the folders you mount; the example writes ALAC, MP3 and AAC. [Getting started](https://github.com/GeiserX/audio-transcode-watcher/blob/main/docs/getting-started.md) has Docker Compose and the full `docker run`. diff --git a/config.example.yaml b/config.example.yaml index dc76441..55fca9c 100644 --- a/config.example.yaml +++ b/config.example.yaml @@ -1,4 +1,4 @@ -# Audio Transcode Watcher Configuration (0.6.1) +# Audio Transcode Watcher Configuration (0.6.2) # ====================================== # # This configuration file defines the source folder and output destinations @@ -62,6 +62,13 @@ settings: # Seconds between periodic full syncs (default: 300) sync_interval_seconds: 300 + # A source that does not decode cleanly (a damaged FLAC): + # skip write nothing for it (default) + # encode_anyway log the error, then encode it with FFmpeg's error + # concealment, so a one-frame glitch is better than no copy + # An output can set its own corrupt_source to override this. + corrupt_source: skip + # whisper_fallback and whisper_model were removed in 0.6.0 and are ignored diff --git a/docs/ROADMAP.md b/docs/ROADMAP.md index b087bbf..29ebf15 100644 --- a/docs/ROADMAP.md +++ b/docs/ROADMAP.md @@ -1,6 +1,6 @@ # Roadmap -Current version: **0.6.1** +Current version: **0.6.2** ## Completed @@ -13,6 +13,7 @@ Current version: **0.6.1** - **Recursive directory support** -- mirror source folder hierarchy in outputs (v0.5.0) - **Lossy sources copied, not inflated**, corrupt sources fail loudly, ReplayGain and MusicBrainz tags kept in M4A outputs, configurable periodic sync (v0.6.0) - **Portable AAC**: stereo downmix and a 48 kHz cap for AAC outputs, per-output `channels` and `max_sample_rate` (v0.6.1) +- **Damaged sources**: `corrupt_source: encode_anyway` makes a concealed copy instead of none (v0.6.2) ## v0.7.0 -- Quality of Life diff --git a/docs/configuration.md b/docs/configuration.md index 4fec650..d04fc9a 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -60,8 +60,30 @@ settings: # Seconds between periodic full syncs (default: 300) sync_interval_seconds: 300 + + # What to do with a source that does not decode cleanly: skip | encode_anyway + # (default: skip). An output can set its own corrupt_source. + corrupt_source: skip +``` + +### Damaged sources + +A source that fails the strict decode (see [Corrupt sources](how-it-works.md#corrupt-sources)) is handled by `corrupt_source`: + +- `skip` (the default) writes nothing for it. +- `encode_anyway` logs the same ERROR, then encodes it once more without `-xerror` and `-err_detect`, letting FFmpeg conceal the damaged frames. The copy exists, one WARNING line says it was made from a damaged source, and its manifest row has kind `tolerant`. A source that fails even that run stays refused. + +Set it under `settings` for every output, or on one output to override, for example only on the AAC output that a phone plays from: + +```yaml + - name: aac-256 + codec: aac + path: /music/aac + corrupt_source: encode_anyway ``` +Either way the source is remembered as failed, so the strict attempt is not repeated on every scan. Replacing the file with a clean copy (a new modification time) clears that, and the next encode is strict again. + `whisper_fallback` and `whisper_model` were removed in 0.6.0 along with the Whisper lyrics fallback. A config that still sets them loads, and the first load logs one warning that they are ignored. ## Source formats diff --git a/docs/getting-started.md b/docs/getting-started.md index ecccdb6..2998e7b 100644 --- a/docs/getting-started.md +++ b/docs/getting-started.md @@ -29,7 +29,7 @@ outputs: ```yaml services: audio-transcoder: - image: drumsergio/audio-transcoder:0.6.1 + image: drumsergio/audio-transcoder:0.6.2 container_name: audio_transcoder environment: - TZ=Europe/Madrid @@ -60,6 +60,6 @@ docker run -d \ -v /path/to/flac:/music/flac:ro \ -v /path/to/mp3:/music/mp3 \ --restart unless-stopped \ - drumsergio/audio-transcoder:0.6.1 + drumsergio/audio-transcoder:0.6.2 ``` diff --git a/docs/how-it-works.md b/docs/how-it-works.md index 09c193d..8da7839 100644 --- a/docs/how-it-works.md +++ b/docs/how-it-works.md @@ -17,7 +17,7 @@ Each output folder holds `.atw-manifest.json`, a map from each output file to th ## Corrupt sources -FFmpeg runs with `-xerror` and `-err_detect crccheck+explode`, so a frame whose checksum does not match stops the encode. If a source does not decode cleanly, the encode fails even when FFmpeg exits 0 but printed a decode error. The error is logged with the file name and no output is written. The file is not tried again until its modification time changes, or the service restarts. +FFmpeg runs with `-xerror` and `-err_detect crccheck+explode`, so a frame whose checksum does not match stops the encode. If a source does not decode cleanly, the encode fails even when FFmpeg exits 0 but printed a decode error. The error is logged with the file name and no output is written, unless `corrupt_source: encode_anyway` is set; then the file is encoded once more with FFmpeg's error concealment and marked `tolerant` in the manifest (see [Damaged sources](configuration.md#damaged-sources)). The strict attempt is not repeated until the file's modification time changes, or the service restarts. ## Recursive Directory Support diff --git a/pyproject.toml b/pyproject.toml index 11d62fe..edbbff0 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -5,7 +5,7 @@ build-backend = "hatchling.build" [project] name = "audio-transcode-watcher" -version = "0.6.1" +version = "0.6.2" description = "Watch a source folder and automatically transcode audio files to multiple formats" readme = "README.md" diff --git a/src/audio_transcode_watcher/__init__.py b/src/audio_transcode_watcher/__init__.py index 565c6ae..e0898b2 100644 --- a/src/audio_transcode_watcher/__init__.py +++ b/src/audio_transcode_watcher/__init__.py @@ -1,3 +1,3 @@ """Audio Transcode Watcher - Automatic audio file transcoding.""" -__version__ = "0.6.1" +__version__ = "0.6.2" diff --git a/src/audio_transcode_watcher/config.py b/src/audio_transcode_watcher/config.py index ac2dc66..0656d35 100644 --- a/src/audio_transcode_watcher/config.py +++ b/src/audio_transcode_watcher/config.py @@ -45,10 +45,19 @@ # Portable-playback limits applied when an output does not set them. # AAC goes to phones, AirPods and CarPlay: stereo, at most 48 kHz. # Any codec can opt in by setting channels / max_sample_rate; 0 = no limit. +# What to do with a source that does not decode cleanly: "skip" writes +# nothing; "encode_anyway" retries with ffmpeg's error concealment. +CORRUPT_SOURCE_MODES = ("skip", "encode_anyway") + DEFAULT_CHANNELS = {"aac": 2} DEFAULT_MAX_SAMPLE_RATE = {"aac": 48000} +def _check_corrupt_source(value: Any, label: str) -> None: + if value not in CORRUPT_SOURCE_MODES: + raise ValueError(f"{label} must be one of {', '.join(CORRUPT_SOURCE_MODES)}") + + @dataclass class OutputConfig: """Configuration for a single output destination.""" @@ -60,6 +69,7 @@ class OutputConfig: include_artwork: bool = True channels: int | None = None # Downmix above this many channels; 0 = keep max_sample_rate: int | None = None # Resample above this rate (Hz); 0 = keep + corrupt_source: str | None = None # "skip" | "encode_anyway"; None = global def __post_init__(self) -> None: """Validate and set defaults after initialization.""" @@ -83,6 +93,10 @@ def __post_init__(self) -> None: self.channels = DEFAULT_CHANNELS.get(self.codec, 0) if self.max_sample_rate is None: self.max_sample_rate = DEFAULT_MAX_SAMPLE_RATE.get(self.codec, 0) + if self.corrupt_source is not None: + _check_corrupt_source( + self.corrupt_source, f"Output '{self.name}': corrupt_source" + ) for key in ("channels", "max_sample_rate"): value = getattr(self, key) if isinstance(value, bool) or not isinstance(value, int) or value < 0: @@ -111,6 +125,7 @@ def from_dict(cls, data: dict[str, Any]) -> OutputConfig: include_artwork=data.get("include_artwork", True), channels=data.get("channels"), max_sample_rate=data.get("max_sample_rate"), + corrupt_source=data.get("corrupt_source"), ) @@ -127,12 +142,15 @@ class Config: min_stable_seconds: float = 1.0 fetch_lyrics: bool = True # Auto-fetch .lrc lyrics via syncedlyrics sync_interval_seconds: int = 300 # Seconds between periodic full syncs + corrupt_source: str = "skip" # Default for outputs that don't set it def __post_init__(self) -> None: """Validate configuration after initialization.""" if not self.source_path: raise ValueError("source_path is required") + _check_corrupt_source(self.corrupt_source, "settings.corrupt_source") + interval = self.sync_interval_seconds if ( isinstance(interval, bool) @@ -159,6 +177,10 @@ def output_paths(self) -> list[str]: """Get list of all output directory paths.""" return [o.path for o in self.outputs] + def corrupt_source_for(self, output: OutputConfig) -> str: + """The corrupt_source mode for *output*: its own, else the global one.""" + return output.corrupt_source or self.corrupt_source + def get_output_by_name(self, name: str) -> OutputConfig | None: """Get an output configuration by name.""" for output in self.outputs: @@ -183,6 +205,7 @@ def from_dict(cls, data: dict[str, Any]) -> Config: min_stable_seconds=settings.get("min_stable_seconds", 1.0), fetch_lyrics=settings.get("fetch_lyrics", True), sync_interval_seconds=settings.get("sync_interval_seconds", 300), + corrupt_source=settings.get("corrupt_source", "skip"), ) @classmethod diff --git a/src/audio_transcode_watcher/encoder.py b/src/audio_transcode_watcher/encoder.py index c6480c7..d08aba3 100644 --- a/src/audio_transcode_watcher/encoder.py +++ b/src/audio_transcode_watcher/encoder.py @@ -275,8 +275,31 @@ def _is_artwork_error(stderr: str) -> bool: return any(h in low for h in _ARTWORK_ERROR_HINTS) -def _run_ffmpeg(cmd: list[str]) -> tuple[int, str]: - """Run ffmpeg; a decode error on stderr counts as a failure even with rc 0.""" +def tolerant_command(cmd: list[str]) -> list[str]: + """*cmd* without -xerror and -err_detect, so ffmpeg conceals decode errors.""" + out: list[str] = [] + skip = False + for i, arg in enumerate(cmd): + if skip: + skip = False + continue + if arg == "-xerror": + continue + if arg == "-err_detect" and i + 1 < len(cmd): + skip = True + continue + out.append(arg) + return out + + +def _run_ffmpeg(cmd: list[str], strict: bool = True) -> tuple[int, str]: + """ + Run ffmpeg and return (rc, stderr). + + Strict: a decode error on stderr is a failure even with rc 0, and any + failure that reports a decode error returns DECODE_ERROR_RC. Not strict: + only the exit code counts (concealed errors still print to stderr). + """ try: proc = subprocess.run( cmd, capture_output=True, timeout=FFMPEG_TIMEOUT, check=False @@ -285,7 +308,7 @@ def _run_ffmpeg(cmd: list[str]) -> tuple[int, str]: return TIMEOUT_RC, f"ffmpeg timed out after {FFMPEG_TIMEOUT} s" stderr = proc.stderr.decode("utf-8", errors="ignore") if proc.stderr else "" rc = proc.returncode - if rc == 0 and _decode_error(stderr): + if strict and _decode_error(stderr): rc = DECODE_ERROR_RC return rc, stderr @@ -295,6 +318,7 @@ def atomic_ffmpeg_encode( final_dest: str, retry_without_artwork: bool = True, finalize: Callable[[str], None] | None = None, + strict: bool = True, ) -> int: """ Run FFmpeg with atomic output (write to temp, then rename). @@ -306,9 +330,12 @@ def atomic_ffmpeg_encode( failure points at the attached picture stream finalize: Optional callable run on the finished temp file before the rename (used to copy extra tags) + strict: If True, a decode error reported on stderr fails the encode + and the result is DECODE_ERROR_RC. False for the tolerant retry. Returns: - Return code (0 for success). No output is written on failure. + Return code (0 for success, DECODE_ERROR_RC when the source did not + decode cleanly). No output is written on failure. """ final_dest = nfc_path(final_dest) dest_dir = os.path.dirname(final_dest) @@ -329,7 +356,7 @@ def atomic_ffmpeg_encode( source = _source_from_cmd(cmd) logger.info("► %s", " ".join(cmd)) - rc, stderr = _run_ffmpeg(cmd) + rc, stderr = _run_ffmpeg(cmd, strict) if rc != 0 and retry_without_artwork and _is_artwork_error(stderr): _cleanup_temp(tmp_dest) @@ -337,7 +364,7 @@ def atomic_ffmpeg_encode( cmd = _remove_artwork_from_command(cmd) cmd[-1] = tmp_dest logger.info("► (retry) %s", " ".join(cmd)) - rc, stderr = _run_ffmpeg(cmd) + rc, stderr = _run_ffmpeg(cmd, strict) if rc != 0: detail = _decode_error(stderr) or (stderr.strip().splitlines() or [""])[-1] diff --git a/src/audio_transcode_watcher/sync.py b/src/audio_transcode_watcher/sync.py index cc640d0..5e3b970 100644 --- a/src/audio_transcode_watcher/sync.py +++ b/src/audio_transcode_watcher/sync.py @@ -14,10 +14,12 @@ from . import manifest from .config import Config, OutputConfig from .encoder import ( + DECODE_ERROR_RC, atomic_ffmpeg_encode, build_ffmpeg_command, copy_mp4_tags, portable_limit_args, + tolerant_command, ) from .lyrics import fetch_lyrics_for_file from .utils import ( @@ -371,18 +373,36 @@ def finalize(tmp: str, src: str = source_path) -> None: cmd = build_ffmpeg_command(source_path, out_path, output) rc = atomic_ffmpeg_encode(cmd, out_path, finalize=finalize) - if rc == 0: - kind = "encode" if is_lossless(source_path) else "transcode" - manifest.record( - output.path, out_path, config.source_path, source_path, kind - ) - else: + kind = "encode" if is_lossless(source_path) else "transcode" + if rc != 0: logger.error( "%s encode failed for %s", output.name.upper(), source_path, ) + # Remembered even if the tolerant run below succeeds, so the + # strict attempt is not repeated on every scan. ok = False + if ( + rc == DECODE_ERROR_RC + and config.corrupt_source_for(output) == "encode_anyway" + ): + rc = atomic_ffmpeg_encode( + tolerant_command(cmd), out_path, finalize=finalize, strict=False + ) + if rc == 0: + kind = "tolerant" + logger.warning( + "%s copy %s was made from a damaged source %s; " + "it may glitch where the source is corrupt", + output.name.upper(), + out_path, + source_path, + ) + if rc == 0: + manifest.record( + output.path, out_path, config.source_path, source_path, kind + ) return ok diff --git a/tests/test_config.py b/tests/test_config.py index 109deec..d2253b6 100644 --- a/tests/test_config.py +++ b/tests/test_config.py @@ -338,3 +338,39 @@ def test_rejects_bad_values(self, key, bad): OutputConfig.from_dict( {"name": "aac", "codec": "aac", "path": "/a", key: bad} ) + + +class TestCorruptSourceSetting: + def test_defaults_to_skip(self): + cfg = Config.from_dict(_settings_config({})) + assert cfg.corrupt_source == "skip" + assert cfg.corrupt_source_for(cfg.outputs[0]) == "skip" + + def test_global_setting(self): + cfg = Config.from_dict(_settings_config({"corrupt_source": "encode_anyway"})) + assert cfg.corrupt_source_for(cfg.outputs[0]) == "encode_anyway" + + def test_output_overrides_global(self): + data = _settings_config({"corrupt_source": "skip"}) + data["outputs"].append( + { + "name": "aac", + "codec": "aac", + "path": "/music/aac", + "corrupt_source": "encode_anyway", + } + ) + cfg = Config.from_dict(data) + alac, aac = cfg.outputs + assert cfg.corrupt_source_for(alac) == "skip" + assert cfg.corrupt_source_for(aac) == "encode_anyway" + + def test_rejects_unknown_global_value(self): + with pytest.raises(ValueError, match="settings.corrupt_source"): + Config.from_dict(_settings_config({"corrupt_source": "ignore"})) + + def test_rejects_unknown_output_value(self): + with pytest.raises(ValueError, match="corrupt_source"): + OutputConfig.from_dict( + {"name": "aac", "codec": "aac", "path": "/a", "corrupt_source": "yes"} + ) diff --git a/tests/test_encoder.py b/tests/test_encoder.py index c7c7e70..868117f 100644 --- a/tests/test_encoder.py +++ b/tests/test_encoder.py @@ -13,6 +13,7 @@ from audio_transcode_watcher.config import OutputConfig from audio_transcode_watcher.encoder import ( + DECODE_ERROR_RC, MP4_TAG_ATOMS, _cleanup_temp, _remove_artwork_from_command, @@ -21,6 +22,7 @@ copy_mp4_tags, limit_args, target_sample_rate, + tolerant_command, ) @@ -996,3 +998,42 @@ def test_alac_keeps_96k_and_6_channels(self, tmp_path): assert self._encode( src, OutputConfig(name="alac", codec="alac", path="/x"), tmp_path ) == (96000, 6) + + +class TestTolerantRun: + """The relaxed encode used for corrupt_source: encode_anyway.""" + + def test_tolerant_command_drops_only_the_strict_flags(self, tmp_path): + cmd, _dest = _alac_cmd(tmp_path) + relaxed = tolerant_command(cmd) + assert "-xerror" not in relaxed and "-err_detect" not in relaxed + assert "crccheck+explode" not in relaxed + assert [ + a for a in cmd if a not in ("-xerror", "-err_detect", "crccheck+explode") + ] == relaxed + + def test_strict_failure_with_decode_error_reports_decode_rc( + self, fake_ffmpeg, tmp_path, monkeypatch + ): + monkeypatch.setenv("FAKE_RC", "183") + monkeypatch.setenv("FAKE_STDERR", CORRUPT_FLAC_STDERR) + cmd, dest = _alac_cmd(tmp_path) + assert atomic_ffmpeg_encode(cmd, dest) == DECODE_ERROR_RC + + def test_non_strict_run_ignores_concealed_errors( + self, fake_ffmpeg, tmp_path, monkeypatch + ): + monkeypatch.setenv("FAKE_RC", "0") + monkeypatch.setenv("FAKE_STDERR", CORRUPT_FLAC_STDERR) + cmd, dest = _alac_cmd(tmp_path) + assert atomic_ffmpeg_encode(tolerant_command(cmd), dest, strict=False) == 0 + assert os.path.isfile(dest) + + def test_non_strict_run_still_fails_on_exit_code( + self, fake_ffmpeg, tmp_path, monkeypatch + ): + monkeypatch.setenv("FAKE_RC", "1") + monkeypatch.setenv("FAKE_STDERR", CORRUPT_FLAC_STDERR) + cmd, dest = _alac_cmd(tmp_path) + assert atomic_ffmpeg_encode(tolerant_command(cmd), dest, strict=False) == 1 + assert not os.path.exists(dest) diff --git a/tests/test_sync.py b/tests/test_sync.py index c933709..51456a7 100644 --- a/tests/test_sync.py +++ b/tests/test_sync.py @@ -10,6 +10,7 @@ import audio_transcode_watcher.sync as sync_mod from audio_transcode_watcher import manifest from audio_transcode_watcher.config import Config, OutputConfig +from audio_transcode_watcher.encoder import DECODE_ERROR_RC from audio_transcode_watcher.sync import ( _cleanup_orphans, _has_lossless_source, @@ -2075,3 +2076,190 @@ def test_edited_output_is_not_replaced(self, _guard, temp_dir): enc.assert_not_called() assert (mp3 / "X.mp3").read_bytes() == b"edited by hand, longer" + + +@pytest.mark.usefixtures("fresh_manifests") +class TestCorruptSourceModes: + """corrupt_source: skip refuses a damaged source; encode_anyway conceals.""" + + def _config(self, source, out, mode, per_output=None): + return Config( + source_path=str(source), + outputs=[ + OutputConfig( + name="aac", codec="aac", path=str(out), corrupt_source=per_output + ) + ], + fetch_lyrics=False, + corrupt_source=mode, + ) + + @staticmethod + def _strict_fails_tolerant_works(calls): + def encode(cmd, dest, strict=True, **_kwargs): + calls.append((list(cmd), strict)) + if strict: + return DECODE_ERROR_RC + Path(dest).write_bytes(b"concealed") + return 0 + + return encode + + @patch("audio_transcode_watcher.sync.safety_guard_active", return_value=False) + def test_skip_writes_nothing_and_remembers(self, _guard, temp_dir): + source, out = _dirs(temp_dir, "source", "aac") + flac = source / "Radiohead - High and Dry.flac" + flac.write_bytes(b"flac") + calls = [] + with patch( + "audio_transcode_watcher.sync.atomic_ffmpeg_encode", + side_effect=self._strict_fails_tolerant_works(calls), + ): + process_source_file( + str(flac), self._config(source, out, "skip"), check_stable=False + ) + assert [strict for _cmd, strict in calls] == [True] + assert _visible(out) == [] + assert str(flac) in sync_mod._failed_sources + + @pytest.mark.parametrize( + "glob,per_output", [("encode_anyway", None), ("skip", "encode_anyway")] + ) + @patch("audio_transcode_watcher.sync.safety_guard_active", return_value=False) + def test_encode_anyway_makes_a_tolerant_copy( + self, _guard, temp_dir, caplog, glob, per_output + ): + source, out = _dirs(temp_dir, "source", "aac") + flac = source / "Radiohead - High and Dry.flac" + flac.write_bytes(b"flac") + config = self._config(source, out, glob, per_output) + calls = [] + with patch( + "audio_transcode_watcher.sync.atomic_ffmpeg_encode", + side_effect=self._strict_fails_tolerant_works(calls), + ): + process_source_file(str(flac), config, check_stable=False) + + # One strict run, then exactly one tolerant run without the strict flags. + assert [strict for _cmd, strict in calls] == [True, False] + strict_cmd, tolerant_cmd = calls[0][0], calls[1][0] + assert "-xerror" in strict_cmd and "-err_detect" in strict_cmd + assert "-xerror" not in tolerant_cmd and "-err_detect" not in tolerant_cmd + dest = out / "Radiohead - High and Dry.m4a" + assert dest.read_bytes() == b"concealed" + assert manifest.lookup(str(out), str(dest))["kind"] == "tolerant" + + errors = [r for r in caplog.records if r.levelname == "ERROR"] + warnings = [ + r + for r in caplog.records + if r.levelname == "WARNING" and "damaged source" in r.getMessage() + ] + assert any("encode failed" in r.getMessage() for r in errors) + assert len(warnings) == 1 + + # The source is remembered: the next scan does not retry it at all. + assert str(flac) in sync_mod._failed_sources + calls.clear() + dest.unlink() + with patch( + "audio_transcode_watcher.sync.atomic_ffmpeg_encode", + side_effect=self._strict_fails_tolerant_works(calls), + ): + process_source_file(str(flac), config, check_stable=False) + assert calls == [] + + @patch("audio_transcode_watcher.sync.safety_guard_active", return_value=False) + def test_source_failing_the_tolerant_run_stays_refused( + self, _guard, temp_dir, caplog + ): + source, out = _dirs(temp_dir, "source", "aac") + flac = source / "X.flac" + flac.write_bytes(b"flac") + calls = [] + + def encode(cmd, dest, strict=True, **_kwargs): + calls.append(strict) + return DECODE_ERROR_RC if strict else 1 + + with patch( + "audio_transcode_watcher.sync.atomic_ffmpeg_encode", side_effect=encode + ): + process_source_file( + str(flac), + self._config(source, out, "encode_anyway"), + check_stable=False, + ) + assert calls == [True, False] + assert _visible(out) == [] + assert str(flac) in sync_mod._failed_sources + assert "damaged source" not in caplog.text + + @patch("audio_transcode_watcher.sync.safety_guard_active", return_value=False) + def test_other_failures_get_no_tolerant_run(self, _guard, temp_dir): + source, out = _dirs(temp_dir, "source", "aac") + flac = source / "X.flac" + flac.write_bytes(b"flac") + calls = [] + + def encode(cmd, dest, strict=True, **_kwargs): + calls.append(strict) + return 124 # timed out, not a decode error + + with patch( + "audio_transcode_watcher.sync.atomic_ffmpeg_encode", side_effect=encode + ): + process_source_file( + str(flac), + self._config(source, out, "encode_anyway"), + check_stable=False, + ) + assert calls == [True] + + +@pytest.mark.skipif( + __import__("shutil").which("ffmpeg") is None, + reason="ffmpeg is not installed; the real-file checks need it", +) +@pytest.mark.usefixtures("fresh_manifests") +class TestCorruptSourceRealFile: + def test_damaged_flac_gets_a_tolerant_aac(self, temp_dir): + import subprocess + + source, out = _dirs(temp_dir, "source", "aac") + good = source / "good.tmp.flac" + subprocess.run( + [ + "ffmpeg", + "-loglevel", + "error", + "-y", + "-f", + "lavfi", + "-i", + "sine=frequency=440:duration=5", + "-c:a", + "flac", + str(good), + ], + check=True, + ) + data = bytearray(good.read_bytes()) + good.unlink() + for i in range(len(data) // 3, len(data) // 3 + 400): + data[i] ^= 0x5A + bad = source / "Scott McKenzie - San Francisco.flac" + bad.write_bytes(bytes(data)) + config = Config( + source_path=str(source), + outputs=[OutputConfig(name="aac", codec="aac", path=str(out))], + fetch_lyrics=False, + corrupt_source="encode_anyway", + ) + + process_source_file(str(bad), config, check_stable=False) + + dest = out / "Scott McKenzie - San Francisco.m4a" + assert dest.exists() and dest.stat().st_size > 0 + assert manifest.lookup(str(out), str(dest))["kind"] == "tolerant" + assert str(bad) in sync_mod._failed_sources diff --git a/uv.lock b/uv.lock index 9553676..71bef3a 100644 --- a/uv.lock +++ b/uv.lock @@ -4,7 +4,7 @@ requires-python = ">=3.14" [[package]] name = "audio-transcode-watcher" -version = "0.6.1" +version = "0.6.2" source = { editable = "." } dependencies = [ { name = "mutagen" }, From 79bf9da00a4f919236dd47eff9a67f366d9dceb2 Mon Sep 17 00:00:00 2001 From: GeiserX <9169332+GeiserX@users.noreply.github.com> Date: Mon, 5 Oct 2026 00:48:16 +0200 Subject: [PATCH 2/3] fix: rebuild a tolerant copy strictly once its damaged source is replaced From the CodeRabbit review: a clean replacement cleared the failure memory, but a plain scan still saw the existing tolerant copy and kept it. A tolerant row now carries the source's size and mtime at encode time; when the source no longer matches, the output is rebuilt with the strict encode. --- src/audio_transcode_watcher/sync.py | 35 ++++++++++--- tests/test_sync.py | 77 +++++++++++++++++++++++++++++ 2 files changed, 106 insertions(+), 6 deletions(-) diff --git a/src/audio_transcode_watcher/sync.py b/src/audio_transcode_watcher/sync.py index 5e3b970..721def2 100644 --- a/src/audio_transcode_watcher/sync.py +++ b/src/audio_transcode_watcher/sync.py @@ -322,6 +322,23 @@ def _made_from_lossy(output: OutputConfig, out_path: str) -> bool: return True +def _tolerant_source_changed( + output: OutputConfig, out_path: str, source_path: str +) -> bool: + """True if *out_path* is a tolerant copy and its source has since changed. + + A damaged source replaced by a clean copy gets a strict encode again. + """ + row = manifest.lookup(output.path, out_path) + if not row or row.get("kind") != "tolerant": + return False + try: + st = os.stat(source_path) + except OSError: + return False + return (st.st_size, st.st_mtime) != (row.get("size"), row.get("mtime")) + + def _process_outputs(source_path: str, config: Config, force: bool) -> bool: """ Process all outputs for a source file. @@ -349,13 +366,19 @@ def _process_outputs(source_path: str, config: Config, force: bool) -> bool: if siblings: _remove_lossy_copies(source_path, siblings, output, config) if not force and os.path.exists(out_path): - if not (is_lossless(source_path) and _made_from_lossy(output, out_path)): + if _tolerant_source_changed(output, out_path, source_path): + logger.info( + "Rebuilding %s: it was made from a damaged source that has changed", + out_path, + ) + elif is_lossless(source_path) and _made_from_lossy(output, out_path): + logger.info( + "Replacing %s: it was made from a lossy source, %s wins", + out_path, + source_path, + ) + else: continue - logger.info( - "Replacing %s: it was made from a lossy source, %s wins", - out_path, - source_path, - ) os.makedirs(os.path.dirname(out_path), exist_ok=True) if action == "copy": diff --git a/tests/test_sync.py b/tests/test_sync.py index 51456a7..92f2563 100644 --- a/tests/test_sync.py +++ b/tests/test_sync.py @@ -2263,3 +2263,80 @@ def test_damaged_flac_gets_a_tolerant_aac(self, temp_dir): assert dest.exists() and dest.stat().st_size > 0 assert manifest.lookup(str(out), str(dest))["kind"] == "tolerant" assert str(bad) in sync_mod._failed_sources + + +@pytest.mark.usefixtures("fresh_manifests") +class TestTolerantCopyReplacedByCleanSource: + """A clean replacement of a damaged source gets a strict encode on a plain scan.""" + + @patch("audio_transcode_watcher.sync.safety_guard_active", return_value=False) + def test_clean_replacement_rebuilds_the_tolerant_copy(self, _guard, temp_dir): + source, out = _dirs(temp_dir, "source", "aac") + flac = source / "Radiohead - High and Dry.flac" + flac.write_bytes(b"damaged flac") + config = Config( + source_path=str(source), + outputs=[OutputConfig(name="aac", codec="aac", path=str(out))], + fetch_lyrics=False, + corrupt_source="encode_anyway", + ) + dest = out / "Radiohead - High and Dry.m4a" + + def damaged(cmd, d, strict=True, **_kwargs): + if strict: + return DECODE_ERROR_RC + Path(d).write_bytes(b"concealed") + return 0 + + with patch( + "audio_transcode_watcher.sync.atomic_ffmpeg_encode", side_effect=damaged + ): + process_source_file(str(flac), config, check_stable=False) + assert dest.read_bytes() == b"concealed" + + # Unchanged source: the tolerant copy is left alone on later scans. + sync_mod._failed_sources.clear() + with patch("audio_transcode_watcher.sync.atomic_ffmpeg_encode") as enc: + process_source_file(str(flac), config, check_stable=False) + enc.assert_not_called() + + # The clean copy arrives (new size and mtime): strict encode replaces it. + flac.write_bytes(b"clean flac, a different size") + later = flac.stat().st_mtime + 10 + os.utime(flac, (later, later)) + calls = [] + + def clean(cmd, d, strict=True, **_kwargs): + calls.append(strict) + Path(d).write_bytes(b"clean encode") + return 0 + + with patch( + "audio_transcode_watcher.sync.atomic_ffmpeg_encode", side_effect=clean + ): + process_source_file(str(flac), config, force=False, check_stable=False) + assert calls == [True] + assert dest.read_bytes() == b"clean encode" + assert manifest.lookup(str(out), str(dest))["kind"] == "encode" + + @patch("audio_transcode_watcher.sync.atomic_ffmpeg_encode") + @patch("audio_transcode_watcher.sync.safety_guard_active", return_value=False) + def test_only_tolerant_rows_are_rebuilt_on_a_plain_scan( + self, _guard, mock_encode, temp_dir + ): + source, out = _dirs(temp_dir, "source", "aac") + flac = source / "X.flac" + flac.write_bytes(b"flac") + dest = out / "X.m4a" + dest.write_bytes(b"strict encode") + manifest.record(str(out), str(dest), str(source), str(flac), "encode") + flac.write_bytes(b"flac with edited tags") # source changed after the encode + config = Config( + source_path=str(source), + outputs=[OutputConfig(name="aac", codec="aac", path=str(out))], + fetch_lyrics=False, + ) + + process_source_file(str(flac), config, force=False, check_stable=False) + + mock_encode.assert_not_called() From e21ff491efaeb5945af19242e3366f4cce8a77b1 Mon Sep 17 00:00:00 2001 From: GeiserX <9169332+GeiserX@users.noreply.github.com> Date: Mon, 5 Oct 2026 00:48:48 +0200 Subject: [PATCH 3/3] docs: say a tolerant copy is rebuilt once its source is replaced --- CHANGELOG.md | 2 +- docs/configuration.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 2c48254..259506a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,7 +6,7 @@ All notable changes to this project are documented here. The format follows [Kee ### Added -- `corrupt_source: skip | encode_anyway`, under `settings` or on one output (default `skip`, the 0.6.1 behaviour). Some damaged FLACs have no clean copy to replace them, and refusing them leaves the phone with no copy of the song at all. With `encode_anyway`, a source that fails the strict decode still gets the same ERROR line, then one more encode without `-xerror` and `-err_detect`, so FFmpeg conceals the damaged frames. That copy is logged with one WARNING saying it was made from a damaged source, and its manifest row has kind `tolerant`. The source is still remembered as failed, so the strict attempt is not repeated on every scan, and a source that fails even the tolerant encode stays refused. +- `corrupt_source: skip | encode_anyway`, under `settings` or on one output (default `skip`, the 0.6.1 behaviour). Some damaged FLACs have no clean copy to replace them, and refusing them leaves the phone with no copy of the song at all. With `encode_anyway`, a source that fails the strict decode still gets the same ERROR line, then one more encode without `-xerror` and `-err_detect`, so FFmpeg conceals the damaged frames. That copy is logged with one WARNING saying it was made from a damaged source, and its manifest row has kind `tolerant`. The source is still remembered as failed, so the strict attempt is not repeated on every scan, and a source that fails even the tolerant encode stays refused. When the damaged file is later replaced by a clean one (a different size or modification time), the next scan rebuilds the tolerant copy with the strict encode. ## [0.6.1] - 2026-10-04 diff --git a/docs/configuration.md b/docs/configuration.md index d04fc9a..514682e 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -82,7 +82,7 @@ Set it under `settings` for every output, or on one output to override, for exam corrupt_source: encode_anyway ``` -Either way the source is remembered as failed, so the strict attempt is not repeated on every scan. Replacing the file with a clean copy (a new modification time) clears that, and the next encode is strict again. +Either way the source is remembered as failed, so the strict attempt is not repeated on every scan. Replacing the file with a clean copy (a new size or modification time) clears that, and the next scan rebuilds the tolerant copy with the strict encode. `whisper_fallback` and `whisper_model` were removed in 0.6.0 along with the Whisper lyrics fallback. A config that still sets them loads, and the first load logs one warning that they are ignored.