Skip to content

[Bugfix] Reinitialize logger backends after fork - #1313

Open
dante159753 wants to merge 1 commit into
ModelEngine-Group:developfrom
dante159753:bugfix-logger-fork-reinit
Open

[Bugfix] Reinitialize logger backends after fork#1313
dante159753 wants to merge 1 commit into
ModelEngine-Group:developfrom
dante159753:bugfix-logger-fork-reinit

Conversation

@dante159753

@dante159753 dante159753 commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

Why

vLLM initializes UCM logging before spawning the V1 EngineCore. With the fork multiprocessing method, the vLLM EngineCore process inherits spdlog objects, mutexes, and async queue state, but it does not inherit the logger worker threads. EngineCore and tensor-parallel workers can therefore execute normally while their UCM-managed file logs are never drained.

The child now detects the PID mismatch on its first logger operation and initializes its own backend. The parent does not need to stop or reinitialize its logger.

What changed

  • Replace the process-global spdlog state with a lazily created per-process backend identified by PID.
  • When a forked child observes an inherited backend, create independent UCM and vLLM sinks, an async queue, a worker thread, source-location storage, and rate-limit state for that child.
  • Avoid touching or destroying the inherited backend in the child because its copied mutexes and background-thread state may be invalid after fork().
  • Use fork-safe stdout output and child formatting paths that do not depend on locks potentially retained by vanished parent threads.
  • Clear Python *_once logger caches through os.register_at_fork() so child logging is not suppressed by parent cache entries.
  • Add Python coverage for fork cache cleanup and C++ coverage that verifies a child creates and writes its own ucm-<pid>.log and vllm-<pid>.log files.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant