Repository navigation
Fix runtime memory lifetimes and validate loaded bytecode - #391
Merged
Merged
Conversation
Detach closed captures, clear reference-containing pooled buffers and released compiler sources, and share terminal cleanup for wrapped coroutine failures. Initialize loaded upvalues according to Lua 5.2 and name the closure factory operand bx. Add 20 regression cases. Validation: 1,686 runtime tests pass on each of .NET 8 and .NET 10; all four library targets build cleanly. Allocations are unchanged across ten measured workloads.
Apply terminal cleanup before propagating coroutine cancellation, and release completed task references. Scope tail-call diagnostic references and program counters to each RunAsync invocation so nested executions restore their caller. Clear removed exception frames, preserve frame positions when growing reversed stacks, and correct the upvalue and traceback indices used by error reporting. Add 24 regression cases covering cancellation, retained values, mixed Lua/C# tracebacks, and captured environment errors in both string modes. All 1,710 runtime tests pass on each of .NET 8 and .NET 10, and all four library targets build without warnings. Eleven measured workloads stay within about 3% of baseline; allocations are unchanged except for 16 bytes of coroutine diagnostic bookkeeping.
Give each UTF-8 reader a pooled buffer for its full lifetime, release it on disposal, and bound BOM inspection by the bytes read. Reject unknown read formats with Lua errors while preserving host stack prefixes. Finalize cancelled coroutine execution even when no resume call is active, and coordinate completion with copying yielded values. Allow top-level managed coroutine bodies to yield while keeping the nested C# boundary restriction. Release handled pcall exception state while preserving Lua error values. Add 23 regression cases for buffer ownership, cross-thread disposal, pooled buffer bounds, idle cancellation, managed yields, handled-error lifetimes, and invalid read formats. All 1,733 runtime tests pass on each of .NET 8 and .NET 10; all four library targets build without warnings. Fourteen measured workloads show unchanged coroutine and normal stream-read allocations; overlapping readers save 1,048 bytes per cycle at about 76 ns extra for three in-memory readers, and handled-error allocations decrease.
Clear failed RunAsync calls back to their return base, including cancelled calls and argument/frame setup failures. Preserve caller stack values and release discarded locals without adding checks to the VM execution loop. Drain the UTF-8 reader before raw byte and bulk reads, and report EOF without accessing stale buffer contents. Keep fresh raw reads allocation-free apart from their existing stream/writer allocations and inline byte reads. Return nil for parameter names missing from stripped debug metadata while preserving ordinary Lua 5.2 parameter lookup. Add 24 regression cases covering synchronous/asynchronous failures, cancellation, caller arguments, retained locals, mixed buffered/raw reads, EOF, and stripped parameter metadata in both string modes. Validation: 1,757 tests pass on each of net8.0 and net10.0; final stream regressions pass on both; all four library targets build without warnings. Seventeen before/after workloads retain identical allocations. Lua/text workloads remain within 4%; raw reads add about 0.24 ns per byte and 47 ns per bulk memory-stream read in the local probe.
Release deleted hash keys through weak records while preserving Lua 5.2 next(t, key). Reclaim empty hashes, compact deleted entries in place, and reuse a bounded set of cleared weak records. Keep normal key lookup intact and find live or deleted iteration keys in one hash lookup. Always dispose the underlying stream when flushing fails, and release the reader buffer even if underlying disposal also fails. Handle missing debug line metadata and function-only getinfo queries, and compare local indices against frame size without overflowing an addition. Give pending yields ownership of cancellation registration and arbitrate resume/cancellation completion. Respect later resume tokens, link distinct tokens only when required, and reject cancelled yields before publishing. Add 40 regression cases covering key collection and iteration, compaction, cleanup failures, stripped metadata, oversized local indices, and coroutine cancellation at the first and subsequent yields. Validation: 1,797 tests pass on each of net8.0 and net10.0. All four library targets build without warnings, and formatting checks pass. Local 21-workload comparisons show hash churn about 3% faster with 28% less allocation and cancellable coroutines allocating 64 bytes less. The normal coroutine workload adds 24 bytes. Deleting during iteration costs about 95 ns more per key (45%) to preserve weak identity without retaining keys.
Pad missing Lua parameters at call boundaries, including fixed-parameter functions and tail calls. Clear discarded tail-call slots before padding, retain native callback argument counts, and keep exact-arity calls fast. Route SetTableAsync writes through the existing deletion bookkeeping so reference keys can be collected and hash counts and iteration remain valid. Restore the array-returning CallAsync stack region on errors, cancellation, and frame-setup failure. Append the callable and arguments with one capacity reservation before mutation, copying borrowed arguments before their source can be overwritten. Preserve buffered unread bytes when a seek fails and invalidate the reader only after successful positioning. Add 51 regression cases for missing arguments in both string modes and with hooks, host deletion and key collection, failed and nested calls, borrowed arguments, and successful and failed seeks. Validation: 1,848 tests pass on each of net8.0 and net10.0. All four library targets build without warnings; formatting and diff checks pass. Eighteen before/after workloads were measured. Normal call allocations are unchanged. A paired check shows host-call time unchanged and the tail-call probe about 1.5% slower. Correct host deletion churn costs about 30 ns more per key (41%) while allocating about 7% less than the broken path.
Align hook frame bases after vararg rearrangement and treat stripped line metadata as unknown. Reuse the outgoing vararg area for tail calls and close captures from the original argument base when execution fails. Retain caller prototypes for diagnostics without rooting old closures. Add 62 regression cases covering both string modes, hook signatures and events, stripped chunks, constant-space tail calls, suspended lifetimes, and capture unwinding. Validation: 1,910 tests pass on each of net8.0 and net10.0; all four library targets build without warnings; CSharpier and diff checks pass. Paired performance runs show unchanged allocations and timings within 3%.
Use the original call boundary for caller-local access across vararg and callable frames. Reuse saved callable slots on tail calls and release the outgoing VM closure before a hook or native callee can suspend. Trim Lua arguments outside the register window at frame entry, preserving open results and native callback argument counts. Release coroutine entry functions on completion and retain the normal dead-resume errors for wrapped coroutines. Document CoroutineFunction's completed-state behavior. Add 46 regression cases covering both string modes, managed and Lua caller boundaries, async tail calls with hooks, surplus arguments, open results, and completed coroutine captures. Validation: all 1,956 Lua tests pass on each of net8.0 and net10.0; all four library targets build without warnings; formatting and diff checks pass. Paired runs of ten workloads show unchanged allocations and no measured performance regression.
Detach captured registers into their existing upvalue cells while a coroutine is yielded, weakly reference the suspended owner, and restore registers before execution resumes. Escaped closures retain their shared values without retaining the abandoned thread and its unrelated locals. Route debug-local access through the same cells and preserve explicit close, cancellation, and stack-growth behavior. Remove the synthetic Lua frame around coroutine.wrap so parent-frame inspection sees the real native wrapper and cannot index past its frames. Add 30 regression cases covering both string modes, synchronous and async wrappers, abandoned self-referencing coroutines, and capture updates. Validation: all 1,986 tests pass on both net8.0 and net10.0; all four library targets build without warnings; formatting and diff checks pass. Paired Release measurements keep ordinary capture allocations unchanged. The captured-yield workload costs about 5.7% and 32 bytes more per coroutine; capture-free coroutines add 8 bytes with approximately unchanged timing.
Dispatch Lua return hooks before moving results and reclaiming registers. Preserve locals, varargs, and open results across synchronous and async hooks, including stack growth and hooks enabled during execution. Keep hook bookkeeping outside the main dispatch method and preserve existing host-hook exception propagation. Own the shared file metatable per Lua global state and store replacements per handle. Independent states and discarded metatable values no longer remain reachable through a process-wide static table. Reuse the state-owned table for every standard-library file creation path. Replace explicitly closed upvalues when a local is captured again, while keeping existing closed closures independent. Reject nil table keys at the write boundary so rawset cannot retain an inaccessible value. Add 26 regression cases across both string modes. The full suite passes 2,012 tests on each of net8.0 and net10.0; 115 focused hook cases also pass after the final dispatch refactor. All four library targets build without warnings, and formatting and diff checks pass. Paired Release measurements leave ordinary capture timings within about 2% of baseline with unchanged execution allocations. Return-hook-heavy calls cost about 6.6% more; opening standard libraries adds 664 bytes per state for isolated file metatables and per-handle ownership.
Hash reference-valued Lua keys by identity, matching their existing reference equality. Mutable host GetHashCode overrides can no longer make userdata entries inaccessible to lookup and deletion. Return stable opaque upvalue IDs through a weak-key table. Retaining an ID no longer retains the upvalue or captured payload. Keep the mapping on the debug API path so ordinary captures gain no field or allocation; preserve shared and joined upvalue identity across collection. Release a file handle's stream when close reports an error after actually closing it. Keep still-open streams available for retry after cancellation or failure. Clear reference-containing pooled arrays on return while leaving primitive buffers unchanged. Add 28 regression cases covering both string modes, full and light userdata, closed and suspended captures, cleanup failures and retries, and pooled object/LuaValue buffers. All 2,040 tests pass on each of net8.0 and net10.0; all four library targets build without warnings. Formatting and diff checks pass. Paired Release samples keep ordinary execution allocations unchanged. Isolated close and pooled-byte timings stay near baseline. Debug-ID lookup workloads cost about 6% more; the first-ID microbenchmark adds about 0.6 us and 51 allocated bytes per operation for the token and weak mapping.
Preserve the completed traceback before detaching the state's pending exception and clearing residual frames. Keep collecting frames across nested invocations. Add lifetime regressions for execution errors, direct calls, metamethod closures, cancellation, and early traceback reads in both string modes. Verify retained exceptions preserve their error objects and nested traces across later executions. Validation: 2,066 tests passed on each of .NET 8 and .NET 10; Release library build passed for all four target frameworks with no warnings or errors.
Restore the original operand base when host concatenation fails, including after partial reductions and asynchronous metamethods. Keep its result on the stack after success and leave invalid-count validation outside cleanup. Register pending exceptions only when the state has call frames to unwind. This releases pre-frame cancellation sources while preserving nested caller tracebacks and the original exception payloads and tokens. Add regression coverage for both string modes, synchronous and asynchronous concatenation, pre-frame errors and cancellation, and retained diagnostics. Validation: 2,092 tests passed on each of .NET 8 and .NET 10. Release library build passed for all four target frameworks with no warnings or errors.
Discard native comparison results after converting them to booleans so loops use constant stack space and release returned objects during execution. Cache generated default metatables with weak main-state keys. Threads share their defaults while independent states do not retain or modify each other's Lua values. Store explicit replacements with weak userdata keys, including nil removal, without adding instance fields or changing record equality. Add regressions for comparison stack growth, collection during execution, generated metatable lifetimes and isolation, coroutine sharing, per-object replacement, nil removal, concurrent initialization, and record equality. Validation: 2,122 runtime tests passed on each of .NET 8 and .NET 10; all 69 source-generator tests passed. Release builds passed for all four runtime targets without warnings or errors. Steady-state allocations and generated object size were unchanged in the before/after benchmarks. State isolation adds about 7-16% in the isolated generated-userdata lookup loops; numeric loops and normal calls were near baseline.
Share one frozen default metatable per generated C# type and use direct userdata metatable dispatch. Keep explicit per-object replacements weakly held and mutable without changing record equality or instance size. Reject mutations at table writes and writable-view boundaries, detach retained array views when freezing, and read module searchers through a read-only view. Cover raw, VM, host and table-library writes, collection, sharing, replacement and ordinary Lua behavior with regression tests. Add reusable generated-userdata benchmarks and document the contract. Validation: 2,164 runtime tests pass on each of .NET 8 and .NET 10; all 69 source-generator tests pass. Release builds pass for all four library targets and the benchmark project. Pinned .NET 10 comparisons against the previous commit show 15% faster field reads, 14% faster method calls and 22% faster userdata comparisons with unchanged managed allocations; n-body, hash writes and host setters remain within 3% of the baseline.
Restore the original argument base when Lua metamethods return so their vararg prefixes cannot accumulate on the caller's stack or retain discarded operands. Preserve the reversed <= comparison flag when tail calls replace a metamethod frame. Use existing frame metadata without adding checks. Cover every affected arithmetic, unary, comparison, index and assignment operation, including global accesses, fixed parameters, suspension, tail calls and discarded extra results. Add GC regressions for operand lifetime and controls for method lookup, concatenation and callable tables in both string modes. Validation: all 2,402 runtime tests pass on each of .NET 8 and .NET 10; all four Release library targets build successfully. Pinned .NET 10 benchmarks preserve allocations, with ordinary workloads within 2% of baseline and no slowdown in the focused variadic comparison rerun.
Release yielded values once their transfer finishes, including when the receiving stack is full. Capture the result range so cleanup reclaims the transferred slots while preserving live locals and suspended captures. Separate coroutine-body failures from receiving-stack failures. Report "too many results to resume" with Lua 5.2 protected/wrapped behavior, keep failed yield transfers resumable and cancellable, and release completed coroutines even when their final results cannot be transferred. Push the success flag and results together with one capacity check. Add 36 regression cases covering both string modes, resume/wrap, nested resumers, payload collection, captures, retries, cancellation, asynchronous bodies and oversized final returns. All 2,438 runtime tests pass on each of .NET 8 and .NET 10, and all four Release library targets build successfully. Paired .NET 10 yield benchmarks show unchanged allocations and 0.4-2.3% additional time for prompt result cleanup.
Defer cancellation unwind until the resumer releases the yielded stack range, including when result transfer overflows. Preserve an unstarted coroutine and report Lua 5.2's argument-transfer error if its initial arguments do not fit. Add 24 regression cases covering protected and unprotected resumes, both string modes, cancellation from either token, result overflow, and entry retries. All new cases fail against the previous assembly. The full runtime suite passes 2,462 tests on each of .NET 8 and .NET 10; all four runtime targets build. Paired coroutine benchmarks show unchanged allocations and 0.5-3% more time.
Reserve a suspended yield while preparing incoming argument capacity so a failed later resume remains retryable without losing its frames or captures. Defer cancellation during preparation and release the reservation on failure. Append complete metamethod argument packets atomically, including callable receivers. Share synchronous setup across arithmetic, unary and comparison helpers while keeping operand copies out of the async invocation method. Add 120 regression cases covering all affected operations, both string modes, callable receivers, capacity boundaries, captured locals and external yield stacks. Validate the runtime tests on .NET 8 and .NET 10, all four build targets, formatting, cancellation handoffs and paired Release performance benchmarks.
Keep a stable execution token for native callbacks and VM executions across resumes, and retire replaced resume-token registrations. Preserve the original cancellation token in errors and release cancellation sources on completion. Use one handoff state to protect stack transfers and consume function completion once. Defer cancellation until published packets have been copied, including callbacks awaiting YieldAsync through WaitAsync. Share synchronous metamethod preparation across VM and managed operations, and copy wrapped coroutine results directly while preserving Lua 5.2 slot and error behavior. Add dispatch, lifetime, transfer, and wrapped-packet regressions plus parameterized coroutine transfer benchmarks. Validation: 2,850 runtime tests pass on each of .NET 8 and .NET 10; the final rollback cleanup also passes all 448 coroutine cases on each runtime. All four runtime targets and the benchmark project build. One existing .NET 8 async GC assertion was intermittent; three focused reruns and the full retry passed. Nine paired samples on .NET 10 against 26e85a0 show Lua coroutine workloads 1-15% faster, native coroutine yields 2.3% slower, and numeric/metamethod workloads broadly unchanged. The stable token adds 64 bytes per uncancellable coroutine (152 bytes with an external token), with no allocation added per yield.
Use the handoff state as the resume guard, reuse unchanged resume-token registrations, and share the lifetime registration when a callback yields with that token. Remove the cancellation scope's duplicate execution token and callback forwarding method, and check external yield tokens once. Trim 224 redundant Cartesian test combinations while retaining every callback dispatch, sync/async path, string mode, and distinct transfer failure. Extend the existing token tests through repeated resumes, retirement, and a shared resume/yield token instead of adding another test fixture. Validation: all 2,627 tests pass on both .NET 8 and .NET 10, all four runtime targets and the benchmark project build, and formatting checks pass. Nine paired .NET 10 samples against 6a3578d show repeated-token and lifetime-token native workloads about 7.7% faster, with 8 fewer bytes allocated per cancellable coroutine. Other changed coroutine workloads remain within roughly 2%.
Keep internal VM writes separate from writable views that escape to host code. Use a bulk array setter for SETLIST, retain nil-write metadata updates, and check table mutability once before growing arrays. Share ContainsKey lookup and dictionary key comparison logic. Mix numeric hashes with HashCode.Combine so negative and large integer keys spread across the power-of-two buckets instead of forming long chains. Extend existing mutation tests to cover literal/proxy writes, numeric reinsertion, fractional and large keys, and signed zero. Paired .NET 10 measurements against 25a7b02: repeated length queries are about 86-90% faster; collision-heavy numeric lookups and writes are about 99% faster. Plain VM array/string writes remain within 2%, with allocations unchanged.
Decode complete buffered lines directly, and accumulate only lines that cross buffers. Reuse PooledList for multi-buffer lines and read-all operations, share text refill handling, and remove the duplicate byte-buffer growth code. Consume CRLF as one terminator even when it crosses a buffer boundary, while preserving raw CR bytes in Lua byte-string mode. Cover short, boundary-sized, and multi-buffer lines, kept endings, Unicode tails, and EOF in both modes. Paired .NET 10 measurements against 25a7b02 show short-line reads 36-44% faster without additional allocation; longer UTF-16 reads remain broadly unchanged.
Keep one dispatch case for each arithmetic operator and use addition for their shared variadic, async, and tail-call matrix. Keep representative generated metatable mutation checks; the general frozen-table tests cover the remaining write mechanisms. Remove 58 redundant cases while retaining all metamethods. Combined validation: 2,581 tests pass on each of .NET 8 and .NET 10. The final VM assignment shape also passes all 494 table/metamethod cases on both runtimes. All four runtime targets and the benchmark project build with no warnings; formatting and diff checks pass.
Reuse CreateCallStackFrame from RunAsync so argument padding, vararg rotation, and hook flags have one implementation. Inline that builder to preserve the cost of short calls. Share call/return hook invocation and its cleanup while retaining the per-instruction hook ownership rules. Push hook argument pairs atomically through the existing fixed-arity stack helper. Trim 12 repeated argument-normalization cases while retaining every hook event and all adjustment branches. The 600 focused cases pass on both runtimes. Paired .NET 10 samples against 3be7afc show managed-call workloads 2-6% faster; hook workloads range from 5% faster to 2.4% slower, with allocation unchanged.
Parse the pooled character span before returning its buffer, and return the nullable number directly to LuaStream. This removes one temporary string and the redundant null/parse handling while preserving the existing scanner, numeric parser, and stream positioning behavior. Paired .NET 10 samples against 3be7afc show numeric reads about 9-10% faster and 16 KiB less allocation per 512 short numbers in either string mode. Combined validation: all 2,569 tests pass on each of .NET 8 and .NET 10, all four runtime targets and the benchmark project build without warnings, and formatting and diff checks pass.
Read instruction and line-info arrays through one checked block copy, then reverse their words only for the opposite byte order. Keep all serialized length and bytecode validation at the input boundary. Avoid invoking the same operand verifier through two APIs for every invalid instruction case. Keep both caller contexts for inferred upvalue initialization and test the context-independent explicit environment once per string mode. Paired .NET 10 samples against 2d82534 show medium/large bytecode loads about 19-30% faster, with unchanged allocation. Tiny chunks remain broadly unchanged. The attempted upvalue accessor merge was discarded after profiling; final upvalue controls stay within 1% of baseline. Validation: all 2,567 runtime tests pass on each of .NET 8 and .NET 10, including both byte orders and malformed chunks. All four runtime targets and the benchmark project build without warnings; formatting and diff checks pass.
Make metatable emission a void operation, remove its unused production context, and call it directly. The routine always succeeded and emitted no diagnostics; this preserves its generated code while simplifying the caller. Validation: all 69 source-generator tests pass. The full runtime suite passes on .NET 8 and .NET 10, including generated userdata behavior and lifetime tests.
Format the benchmark source-generator reference for the CI formatter and document the coroutine transfer benchmark alongside generated userdata. Validation: Release solution build succeeds without warnings; all 2,567 runtime tests pass on each of .NET 8 and .NET 10, and all 69 generator tests pass. CSharpier checks all 340 files and benchmark --verify passes.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Calls, metamethods, coroutine transfers and error paths can retain discarded values or leave invalid runtime state. This change reclaims their argument/result ranges, closes captures when execution unwinds, and releases references held by pools, exceptions, deleted table keys and closed streams.
Validate serialized lengths, operands and control-flow targets when loading bytecode so the VM can retain its unchecked register access. Loaded chunks initialize their own closed upvalues instead of capturing the active caller, and closure instructions encode prototype indices as unsigned operands.
Coroutine handoffs preserve suspended locals and give argument/result transfers exclusive stack ownership while cancellation is pending. Each started coroutine owns a stable cancellation source so host callbacks can retain their execution token across resumes; completion releases the source and registrations. Debug hooks, local inspection and traceback handling preserve the appropriate callee frames.
Generated C# userdata share an immutable default metatable per type. Explicit per-object replacements use weak keys, and file-handle metatables belong to their Lua global state. Table length caching, numeric hash distribution, buffered stream reads and bytecode-array reads also avoid unnecessary work and allocations. Reusable generated-userdata and coroutine-transfer benchmarks are included.
Compatibility
Validation
Checks were run locally on Windows. Hosted Linux CI and Unity execution remain to be confirmed by their respective environments.