Skip to content

Fix UnicodeEncodeError that kills the CLI after a successful run - #1282

Merged
harry0703 merged 1 commit into
harry0703:mainfrom
housine35:fix/windows-console-encoding
Aug 30, 2026
Merged

Fix UnicodeEncodeError that kills the CLI after a successful run#1282
harry0703 merged 1 commit into
harry0703:mainfrom
housine35:fix/windows-console-encoding

Conversation

@housine35

Copy link
Copy Markdown

What happens

On a Windows console the default code page is a legacy one (cp1252 in Western Europe). Generating a video in French produces U+202F, the narrow no-break space French typography places before : and !, and Loguru's own progress lines carry circled digits such as U+2464.

Either character raises UnicodeEncodeError when the CLI prints its result — which happens after the pipeline has finished writing the video:

SUCCESS | "./app/services/task.py:1446": _run_pipeline - task 62f8f42a-... finished, generated 1 videos.
Traceback (most recent call last):
  File "cli.py", line 1495, in run_cli
    print(json.dumps({"task_id": task_id, "result": result}, ensure_ascii=False))
UnicodeEncodeError: 'charmap' codec can't encode character ' ' in position 1116

The video is intact on disk. But the process exits non-zero and never prints where the file is, so the run reads as a failure and the path has to be hunted for under storage/tasks/.

The same thing happens inside Loguru's handler during the run:

--- Logging error in Loguru Handler #1 ---
UnicodeEncodeError: 'charmap' codec can't encode character '⑤'

Reproducing

python cli.py --video-subject "Les 18 heures de Pompéi" --video-language fr-FR --stop-at script

Windows 11, Python 3.13, chcp 1252. Any non-ASCII target language reaches it; French does so reliably because of the narrow no-break space.

The change

Reconfigure stdout and stderr to UTF-8 at the entry point, before anything is written. Streams that do not support reconfigure() — a wrapper, a pipe under some runners — are skipped rather than made to crash on the fix itself.

22 lines, cli.py only. No behaviour change on platforms that already default to UTF-8.

Verified

Same command after the patch prints the JSON result intact, with , °C and the narrow no-break space preserved, and exits 0.

🤖 Generated with Claude Code

On a Windows console the default code page is a legacy one (cp1252 in
Western Europe). Generating a video in French produces U+202F, the narrow
no-break space French typography places before ':' and '!', and Loguru's
own progress lines carry circled digits such as U+2464.

Either character raises UnicodeEncodeError when the CLI prints its result,
which happens *after* the pipeline has finished writing the video:

    SUCCESS | task ... finished, generated 1 videos
    Traceback (most recent call last):
      File "cli.py", line 1495, in run_cli
        print(json.dumps({"task_id": task_id, "result": result}, ...))
    UnicodeEncodeError: 'charmap' codec can't encode character ' '

The video is fine on disk, but the process exits non-zero and never prints
where the file is, so the run looks like a failure and the path has to be
hunted for under storage/tasks/.

Reconfigure stdout and stderr to UTF-8 at the entry point, before anything
is written. Streams that do not support reconfigure() -- a wrapper, a pipe
under some runners -- are left alone rather than crashing on the fix.

Reproduced and verified on Windows 11, Python 3.13, with
--video-language fr-FR.
@harry0703
harry0703 merged commit 1cfc203 into harry0703:main Aug 30, 2026
@harry0703

Copy link
Copy Markdown
Owner

Thank you for the clear report and the focused fix. I reproduced the issue with a cp1252 output stream and confirmed that the CLI now prints the Unicode result correctly and exits successfully. This has been merged. Thanks again for your contribution!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants