Skip to content

Commit 673b220

Browse files
DeusDataLA-10dpersek
committed
fix(py-lsp): memoize + guard expression evaluation (exponential re-eval hang)
py_eval_expr_type evaluated a call node's attribute receiver twice (once in the container special-case, once in the general attribute path), so an N-link chained call cost O(2^N) evaluations — real-world ~65-link builder chains hung indexing for hours (DeusData#710). The evaluator also recursed once per expression nesting level with no depth guard, so a pathologically deep expression overflowed the native stack (DeusData#720). Three guard layers, mirroring c_eval_expr_type's design in c_lsp.c: 1. Memoization: py_eval_expr_type is now a wrapper around the renamed py_eval_expr_type_uncached. Results are cached per node in an open-addressing table on PyLSPContext (arena-allocated, cold per file), keyed by node identity (TSNode.id) — NOT the start byte, since every leftmost descendant of a chain shares it and byte-keyed entries alias distinct nodes into silently wrong resolutions. Entries carry a scope generation; every scope bind/restore bumps it (O(1) whole-cache flush), so re-evaluation under changed bindings (lambda re-walks, isinstance narrowing) behaves exactly as before. Inserts are load-bounded (<75%) and rejected when growth fails, so a full table can never spin. 2. Depth cap: PY_LSP_MAX_EVAL_DEPTH 256 (same as C_EVAL_DEPTH_LIMIT); past the cap the evaluator returns unknown instead of overflowing the stack. Truncated results — including any ancestor of a truncation — are never cached, so a depth- or budget-limited `unknown` cannot poison post-truncation reuse. 3. Step budget: PY_EVAL_MAX_STEPS_PER_FILE 10000 (mirrors C_EVAL_MAX_STEPS_PER_FILE); pathological files degrade to unknown instead of stalling indexing. Cache hits don't consume budget. Distills PR DeusData#732 (memoization; fixed the start-byte cache key and the missing staleness handling) and PR DeusData#758 (depth cap; raised 64 -> 256 to match the sibling LSPs, added the never-cache-truncated ordering), and adds the step budget plus reproduce-first tests: a 40-link chain that must resolve its final link, a heterogeneous-receiver chain that fails under byte keying, a 30k-deep expression crash guard, and a budget degradation guard. Fixes DeusData#710. Fixes DeusData#720. Co-authored-by: LA-10 <lashuha@uwaterloo.com> Co-authored-by: Dustin Persek <dustin.persek@protonmail.com> Signed-off-by: Martin Vogel <martin.vogel.tech@gmail.com>
1 parent 09148ab commit 673b220

4 files changed

Lines changed: 400 additions & 29 deletions

File tree

0 commit comments

Comments
 (0)