Parallelize build-time shader compilation in the descriptor generators - #932
Draft
rickbrew wants to merge 1 commit into
Draft
Parallelize build-time shader compilation in the descriptor generators#932rickbrew wants to merge 1 commit into
rickbrew wants to merge 1 commit into
Conversation
Move FXC/DXC compilation out of the ForAttributeWithMetadataName transform callbacks (which the incremental driver invokes sequentially) into a dedicated node that warms the shared bytecode cache for all shaders in parallel. A join node (ordered after the warm-up by its dataflow edge) then folds each compiled shader back into its model and synthesizes the compile diagnostics, using location info captured by value during the transform. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
rickbrew
marked this pull request as draft
August 31, 2026 15:15
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
via Claude. I have not yet reviewed this code yet.
Closes #931
Description
Moves FXC/DXC compilation out of the
ForAttributeWithMetadataNametransform callbacks (which the incremental driver invokes sequentially, serializing all shader compilations) and into a dedicated node per generator that compiles every shader in the project in parallel. Applies to bothD2DPixelShaderDescriptorGeneratorandComputeShaderDescriptorGenerator; the sharedHlslBytecodeSyntaxProcessorcarries the new logic for both.Pipeline shape (identical in both generators):
HlslBytecodeInfoKey, but no longer compiles. It also captures the info needed to later synthesize the compile diagnostics (HlslBytecodeDiagnosticsInfo: type name, type location,[D2DRequiresDoublePrecisionSupport]presence + location), since symbols must not escape the transform.Select(key).Collect()→CompileAllInParallel(keys, token), aParallel.ForEachover the distinct keys into the existingDynamicCache(which is already thread-safe and needed no changes; the FXC and DXC wrappers are also both safe for concurrent use, DXC via its existing[ThreadStatic]instance).Combinewith the warm-up node's output (this edge is what guarantees the ordering, via the dataflow graph rather than driver scheduling assumptions), then aSelectthat fetches the bytecode (a guaranteed cache hit), synthesizes the deferred diagnostics, and drops the captured diagnostics info from the model.Locations are captured by value in the new
LocationInfomodel (file path +TextSpan+LinePositionSpan, materialized viaLocation.Create(path, span, lineSpan)), deliberately not as aSyntaxTreereference: this keeps transform outputs equatable across unrelated edits (comment-only edits still produce value-equal models), and avoids ever holding stale trees in cached models. All existing incrementality tests pass unchanged.Numbers from Paint.NET (warm
--no-incrementalRelease x64 rebuilds, second of two runs, vs. stock 3.2.0):API breakdown
No public API changes — internal generator restructuring only.
Additional context
.g.csfiles compared, generator version stamp aside), and the full Paint.NET solution builds clean against a locally packed build of this branch.ComputeSharp.D2D1.Tests.SourceGenerators(113) andComputeSharp.Tests.SourceGenerators(110) all pass. The WinUI source generator test project currently fails torestorefor me on stockmainas well (NU1603 warnings-as-errors from feed version drift), so I couldn't run that suite locally — but it only exercisesCanvasEffectPropertyGenerator, which this PR doesn't touch.D2DPixelShaderSourceGenerator(raw HLSL string literals) intentionally keeps compiling inline; it can get the same treatment in a follow-up if you want it.🤖 Generated with Claude Code