Skip to content

Prototype private runtime references with System.Reflection.Metadata (#560) - #568

Draft
jogibear9988 wants to merge 1 commit into
dadhi:masterfrom
jogibear9988:experiment/metadata-private-access-560
Draft

jogibear9988 wants to merge 1 commit into
dadhi:masterfrom
jogibear9988:experiment/metadata-private-access-560

Conversation

@jogibear9988

Copy link
Copy Markdown
Contributor

Experimental follow-up to #567 / investigation for #560

This draft records a working private-access prototype based on IgnoresAccessChecksToGenerator, replacing its Cecil backend with System.Reflection.Metadata and System.Reflection.PortableExecutable. It is an independent PR based on master 734314ed; it does not modify #567 or FEC's production build/source. It does not close #560.

The local fork/backend experiment lives under BuildScripts/IgnoresAccessChecksToGenerator. It retains upstream MIT licensing and source/targets snapshots at revision 882b30bf82fbb72f3bee8878ad9fd31f5c93c875. The new generator has no Cecil or NuGet package dependency. It is not a complete drop-in replacement for the upstream package.

Findings and implementation

  1. Why the stock package could not resolve the type: SDK reference assemblies expose the supported API and omit implementation types such as RuntimeILGenerator. A publicizer cannot expose a type absent from its input.
  2. Why supplying CoreLib failed: in the isolated net10.0 probe, stock package 0.8.0 tried to rewrite the installed .NET 10 implementation and Cecil rejected its non-IL-only image with Writing mixed-mode assemblies is not supported. No .NET Core DLL was added to a .NET Framework project; that error describes the writer's binary-format limitation.
  3. Private access is feasible: the stock policy publicizes internal members but leaves private ones private. The new backend optionally exposes private fields, methods and nested types in a generated build-time reference. The actual runtime remains private; IgnoresAccessChecksTo authorizes the compiled consumer's access.
  4. Avoid rewriting native code: read the implementation using PEReader, retain assembly identity and compiler-relevant metadata, and use MetadataBuilder / ManagedPEBuilder to produce a new IL-only reference marked ReferenceAssemblyAttribute. Method bodies, native sections, field RVA data and resources are omitted. Verification checks identity, definition tokens/ranges/signatures, visibility and absence of executable method bodies.
  5. There is an additional reference-set problem: a full CoreLib facade conflicts with SDK System.Runtime definitions. The prototype therefore compiles against one consistent set of managed implementation references from the selected runtime, replacing CoreLib with the generated reference. Native DLLs are filtered out with HasMetadata. This is runtime-version/OS-specific, not a portable targeting pack; compiler validation against one version does not guarantee compatibility with other consumer runtimes. Windows implementation references also carry Windows platform annotations.
  6. MSBuild integration works: a new PublicizeInternals task generates the facade and access attribute and supplies the matching compiler references. Generated files stay under obj; the generated CoreLib is not copied into application output or used as the executing CoreLib.

Results

Windows ARM64, SDK 10.0.301; executing runtimes are pinned with roll-forward disabled and checked against the selected input path.

Check .NET 8.0.28 .NET 10.0.9
Direct access to private fields and private IncreaseCapacity, emitted method returns 42 Pass Pass
Direct reference to genuinely private nested GC.StartNoGCRegionStatus Pass Pass
MSBuild task generates reference and access attribute; consumer executes Pass Pass
Standard and LightExpression: 100 repeated local/try-finally compilations each, plus nested captures Pass Pass
Runtime contracts from #567, compiled into the isolated FEC host 1 per API passed 3 per API passed
Private exposure disabled / access attribute removed / missing field / wrong field type Expected failures detected Expected failures detected

25/25 replay checks passed, including the expected-negative cases; these are replay steps, not 25 independent unit tests. Across four FEC configurations, the smoke tests perform 400 repeated expression compilations plus nested captures and eight runtime-contract test cases. A throwaway FEC source snapshot changes only the local-allocation helper to direct typed field access; the production source is untouched. Both installed CoreLib SHA-256 hashes are unchanged before/after and the facade does not appear in consumer output.

The negative cases establish that disabled private exposure fails C# compilation, missing access permission fails at runtime with TypeAccessException, a nonexistent field fails with CS1061, and a wrong field type fails with CS0029. An exploratory Array.ArrayAssignType reference compiled on .NET 10 but failed on .NET 8 because that nested type is absent there; the retained cross-version test uses the private GC enum. This illustrates the versioning concern directly.

Reproduce from BuildScripts/IgnoresAccessChecksToGenerator with python replay.py. See the report and instructions and recorded commands, exit codes and input hashes. Generated binaries/logs are ignored and regenerated locally.

Performance and limitations

No performance or allocation benefit has been measured. Direct typed helpers might reduce reflection/generated-method setup at initialization or delegate dispatch during compilation; replacing an already efficient UnsafeAccessor field access is not an established improvement. Specialized/batched emission would be another experiment. Emitted application code would need to change to establish an application-runtime benefit.

The prototype requires .NET 10 MSBuild and supports CoreLib only. General package support, upstream exclusion/legacy options, older hosts, packaging and incremental-generation caching are not implemented. There is no x64, Linux, Framework 4.7.2, preview-runtime or complete FEC regression-suite claim. The metadata verifier is focused, not certification of a general metadata rewriter.

Merge recommendation

Do not merge this follow-up into master in its current form; keep it as an experimental draft. It proves feasibility, but adds runtime-specific reference machinery without demonstrated performance benefit or sufficient portability/metadata validation.

Before considering production integration: measure cold initialization, warm compilation, allocations, generated-delegate runtime and build cost against existing helpers; narrow exposure/reference handling; decide supported runtime-version behavior; and add broader architecture/OS and regression coverage. Review the smaller runtime-contract tests in #567 independently. This draft supplies evidence for the #560 design discussion, rather than replacing those tests or claiming the issue is solved.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Explore internal accessor with compile checks https://github.com/aelij/IgnoresAccessChecksToGenerator

1 participant