[TRTLLM-16722][feat] Dflash Unified KVCache by allisonlim-nv · Pull Request #20486 · NVIDIA/TensorRT-LLM

allisonlim-nv · 2026-10-16T17:35:24Z

Dev Engineer Review

Eligible DFlash runs can place target and draft layers in one KVCacheManagerV2. Other runs retain separate draft-cache handling.
Per-layer specifications affect ownership, dtype, buffer roles, page indexing, cache views, allocation, and cost estimates. Verify mixed cache formats and compatibility with legacy manager behavior.
Peer registration now rejects peers with different unified draft-layer counts. This adds a compatibility constraint for disaggregated peers.
Buffer coalescing now considers both coalescing group and expanded size. Non-FP4 attention pointer handling also supports 2D and 3D tables.
The author reports 24 comparisons. Aggregated acceptance length was unchanged in 23; one attention-data-parallel comparison changed by −0.20% in a run with a timing-dependent concurrent client. Disaggregated acceptance length increased in all 24. The author reports exact agreement when prefill and decode use the same tensor-parallel size without attention data parallelism. Other configurations could generate different text; for identical outputs, the reported agreement was within about ±1%, except mixed precision at prefill TP2/decode TP4, which differed by −1.89%.
The author reports that mixed-precision logs confirmed an FP8 target cache and a BF16 draft cache on every rank, with no fallback to a private draft cache. Transfer unit coverage checked two-byte draft pages when the advertised target element size was one byte.
PR_Github #78565 completed with FAILURE for commit b6db612. PR_Github #78633 completed with SUCCESS for commit b757065, but its triggered L0 Merge Request pipeline reported FAILURE. Failure details are unavailable. Current review severity counts are unavailable.

QA Engineer Review

Modified unit tests cover unified-cache selection, cache-cost estimation, mixed-layout pools and metadata, MiniMax M3 cache indexing, peer compatibility, and rank-info serialization.
The new TestQwen3_8BDFlash::test_gsm8k_1p1d_unified_draft_kv integration test checks acceptance length and a shared-manager log marker. The test is listed in tests/integration/test_lists/test-db/l0_dgx_b200.yml.
Coverage verdict: needs follow-up. The supplied evidence describes test coverage but does not establish successful test execution. The reported pipeline failure details are unavailable.

Per-File QA Perspective

cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/config.h: Verify that the default coalescingGroup preserves existing behavior.
cpp/tensorrt_llm/batch_manager/kv_cache_manager_v2/storage/config.cpp: Verify that buffers coalesce only when both group and expanded size match.
cpp/tensorrt_llm/nanobind/batch_manager/kvCacheManagerV2.cpp: Verify that the optional coalescing_group argument and writable property match the Python stub.
cpp/tensorrt_llm/thop/attentionOp.cpp: Verify pointer-table handling for 2D and 3D non-FP4 inputs, including mixed-format tables. The FP4 path is unchanged.
tensorrt_llm/_torch/attention/backends/flashinfer.py: Verify that layers with different KV factors receive separate page-index spaces.
tensorrt_llm/_torch/attention/backends/sparse/minimax_m3/cache_manager.py: Verify compatibility with the base method’s optional kv_factor argument.
tensorrt_llm/_torch/disaggregation/native/peer.py: Verify that matching draft-layer counts allow peer registration and mismatches raise ValueError.
tensorrt_llm/_torch/disaggregation/native/rank_info.py: Verify draft-layer counts with and without layer specs, plus compatibility with older serialized payloads.
tensorrt_llm/_torch/pyexecutor/_util.py: Verify unified-cache selection gates, fallback behavior, and consistent layer specs in manager construction and cost estimation. Verify draft quantization handling for DFlash and DSpark.
tensorrt_llm/_torch/pyexecutor/kv_cache/kv_cache_manager_v2.py: Verify per-layer ownership, dtype, buffer roles, KV factors, page indexing, cache views, and cost calculations. Verify legacy behavior when layer specs are absent.
tensorrt_llm/_torch/pyexecutor/scheduler/scheduler_v2.py: Verify disaggregated generation initialization when only the target manager receives prompt KV.
tensorrt_llm/_torch/speculative/dflash.py: Verify unified-pool binding, metadata validation, block offsets, and context-length handling. Unsupported unified layouts should fail rather than use a private arena.
tensorrt_llm/_torch/speculative/interface.py: Verify that a shared target and draft manager bypass separate-cache handling.
tensorrt_llm/_torch/speculative/utils.py: Verify that a draft view identical to the target manager is not returned as a separate manager.
tensorrt_llm/runtime/kv_cache_manager_v2/__init__.pyi: Verify that the coalescing_group stub matches the bound constructor and property.
tests/integration/defs/accuracy/references/acceptance_length.yaml: Adds the unified DFlash GSM8K acceptance-length reference. Verify the reference and minimum against the intended B200 run.
tests/integration/defs/accuracy/test_disaggregated_serving.py: Adds a Qwen3-8B DFlash test with an acceptance-length check and shared-manager log assertion. Its CI list entry is in tests/integration/test_lists/test-db/l0_dgx_b200.yml.
tests/integration/test_lists/test-db/l0_dgx_b200.yml: Adds the unified DFlash test to the two-GPU B200 pre-merge MPI/PyTorch list with a 60-second timeout.
tests/unittest/_torch/attention/sparse/glm_kpool/test_glm_kpool.py: Initializes layer_specs in a cache-cost test. Verify the legacy configuration path.
tests/unittest/_torch/attention/sparse/msa/test_minimax_m3_shared_draft_layers.py: Initializes layer_specs in NVFP4 role tests for MiniMax M3 and generic managers. Verify both paths.
tests/unittest/_torch/executor/kv_cache/test_dflash_unified_kv_gate.py: Covers aggregated and disaggregated unification gates, head-layout comparisons, and per-layer head-count rejection.
tests/unittest/_torch/executor/kv_cache/test_kv_cache_estimation.py: Covers cache-cost calculations for sink tokens and reuse policies, quota conversion, per-request layer-token values, and draft-layer rounding.
tests/unittest/_torch/executor/kv_cache/test_kvcm2_integration.py: Covers mixed-role pools, NVFP4 scale-page indexing, metadata pointer views, and cold-codec dtype selection and rejection.
tests/unittest/_torch/executor/kv_cache/test_mamba_cache_manager.py: Initializes layer_specs in three hybrid-manager setups. Verify quota and pool-ratio checks.
tests/unittest/disaggregated/test_minimax_m3_kv_transfer.py: Covers cache-index arguments and results, and initializes manager fields in the block-count-limit test.
tests/unittest/disaggregated/test_peer.py: Covers peer registration with matching and mismatched unified draft-layer counts.
tests/unittest/disaggregated/test_rank_info.py: Covers draft-layer counts, serialization, and defaulting older payloads to zero.
tensorrt_llm/commands/serve.py: Sets TRTLLM_DISAGG_SHARED_DRAFT_KV based on speculative configuration and matching tensor-parallel sizes. Verify that other configurations disable shared draft KV.

Description

Follow-up to #20178. Integrate DFlash with the unified key-value cache manager so target and draft layers share one manager while retaining their respective cache data types.

Test Coverage

The following 24 comparisons cover tensor parallel configurations, CUDA graphs, overlap scheduling, attention data parallelism, and mixed cache precision.

All result columns report AL. Measurements come from the generation server’s /metrics, except the attention data parallelism comparisons, which use per-request statistics from a concurrent client. Both configuration columns show tensor parallel sizes; the disaggregated configuration lists prefill → decode.

Test configuration	Aggregated configuration	Disaggregated configuration	Aggregated before	Aggregated after	Disaggregated before	Disaggregated after
CUDA graphs enabled; overlap scheduler enabled	1	1 → 1	3.8180	3.8180	3.1793	3.8180
CUDA graphs enabled; overlap scheduler enabled	2	2 → 2	3.8646	3.8646	3.3140	3.8646
CUDA graphs enabled; overlap scheduler enabled	1	2 → 1	3.8180	3.8180	3.2437	3.8284
CUDA graphs enabled; overlap scheduler enabled	2	1 → 2	3.8646	3.8646	3.0522	3.6005
CUDA graphs enabled; overlap scheduler enabled	4	2 → 4	3.5928	3.5928	2.9582	3.4645
CUDA graphs enabled; overlap scheduler disabled	1	1 → 1	3.6360	3.6360	3.0302	3.6360
CUDA graphs enabled; overlap scheduler disabled	2	1 → 2	3.6973	3.6973	2.9202	3.4406
CUDA graphs enabled; overlap scheduler disabled	1	2 → 1	3.6360	3.6360	3.0928	3.6395
CUDA graphs enabled; overlap scheduler disabled	4	2 → 4	3.4499	3.4499	2.8491	3.3349
CUDA graphs disabled; overlap scheduler enabled	1	1 → 1	3.7645	3.7645	3.2793	3.7645
CUDA graphs disabled; overlap scheduler enabled	2	1 → 2	3.5380	3.5380	3.3256	3.8240
CUDA graphs disabled; overlap scheduler enabled	1	2 → 1	3.7645	3.7645	3.0041	3.4615
CUDA graphs disabled; overlap scheduler enabled	4	2 → 4	3.5565	3.5565	3.3079	3.8810
CUDA graphs disabled; overlap scheduler disabled	1	1 → 1	3.5856	3.5856	3.1083	3.5856
CUDA graphs disabled; overlap scheduler disabled	2	1 → 2	3.3938	3.3938	3.1842	3.6538
CUDA graphs disabled; overlap scheduler disabled	1	2 → 1	3.5856	3.5856	2.8926	3.3511
CUDA graphs disabled; overlap scheduler disabled	4	2 → 4	3.4255	3.4255	3.1556	3.7135
Attention data parallelism	2	2 → 2	3.7766	3.7766	3.1109	3.6687
Attention data parallelism	4	2 → 4	3.5341	3.5269	2.9660	3.5495
Mixed cache precision	1	1 → 1	3.7690	3.7690	3.2181	3.7690
Mixed cache precision	2	2 → 2	3.8874	3.8874	3.3286	3.8874
Mixed cache precision	2	1 → 2	3.8874	3.8874	3.2567	3.7878
Mixed cache precision	1	2 → 1	3.7690	3.7690	3.2178	3.7714
Mixed cache precision	4	2 → 4	3.6799	3.6799	3.0542	3.4708

The mixed cache precision runs use the following configuration. Here, bfloat16 denotes the 16-bit brain floating-point format.

Model	Weights	Key-value cache
Target: Qwen3-8B	bfloat16	8-bit floating point
Drafter: Qwen3-8B-DFlash-b16	bfloat16	bfloat16

Only the target cache uses 8-bit floating point; target weights and activations remain bfloat16. Runtime logs confirm the intended target and draft cache types on every rank, with the drafter using the shared bfloat16 pool and no fallback to its private cache. Transfer unit coverage verifies that draft pages use two-byte elements even when the servers advertise the target cache’s one-byte element size.

Results:

Aggregated AL is unchanged in 23 of 24 comparisons. The attention data parallelism comparison at tensor parallel size 4 changes by −0.20%; this run uses a timing-dependent concurrent client.
Disaggregated AL increases in all 24 comparisons.
After the change, aggregated and disaggregated AL match exactly whenever prefill and decode use the same tensor parallel size without attention data parallelism.
Other configurations can produce different generated text. Among requests with identical output, aggregated and disaggregated AL agree within approximately ±1%, except the mixed cache precision configuration with prefill size 2 and decode size 4, where the difference is −1.89%.

PR Checklist

Please review the following before submitting your PR:

PR description clearly explains what and why. If using CodeRabbit's summary, please make sure it makes sense.
PR Follows TRT-LLM CODING GUIDELINES to the best of your knowledge.
Test cases are provided for new code paths (see test instructions)
If PR introduces API changes, an appropriate PR label is added - either api-compatible or api-breaking. For api-breaking, include BREAKING in the PR title.
Any new dependencies have been scanned for license and vulnerabilities
CODEOWNERS updated if ownership changes
Documentation updated as needed
Update tava architecture diagram if there is a significant design change in PR.
The reviewers assigned automatically/manually are appropriate for the PR.
Please check this after reviewing the above items as appropriate for this PR.

GitHub Bot Help

To see a list of available CI bot commands, please comment /bot help.

Signed-off-by: Allison Lim <allim@nvidia.com>

Separate NVFP4 data and block scales from other equal-size buffers while keeping shared lifecycle slot allocation. Preserve matching role order and require matching scale roles. Add one regression test for mixed NVFP4 and BF16 storage, checking scale addresses at slot 1 and the final slot. Signed-off-by: Allison Lim <allim@nvidia.com>

Read the data-only view of three-dimensional pointer tables for non-FP4 layers while preserving NVFP4 scale handling and legacy two-dimensional tables. Add one native-binding regression covering mixed BF16/NVFP4 pointer tables and legacy BF16 tables. Signed-off-by: Allison Lim <allim@nvidia.com>

Signed-off-by: Allison Lim <allim@nvidia.com>

Preserve explicit and legacy layer configuration while applying upstream pool rebalancing settings to both paths. Signed-off-by: Allison Lim <allim@nvidia.com>

Preserve unified DFlash storage specs across model geometry refactoring, combine per-layer cache layouts with mixed-precision pool handling, and retain both acceptance references. Add CPU regression coverage for unification selection and storage-spec forwarding. Signed-off-by: Allison Lim <allim@nvidia.com>

NVIDIA#20178 foundation) Squashed net change of PR NVIDIA#20486 at head 714916d relative to the main it last merged (0829689), ported onto feat/musespark-open-agentx; unrelated main commits are not included. NVIDIA#20486 carries the unified KV cache manager foundation of PR NVIDIA#20178 through eea6dcb (NVIDIA#20178's later 61248ec, 77d4eae and 636715a are not part of the NVIDIA#20709 stack and are not ported). Commits: 3bc36a5 [None][feat] add unified KV cache layout foundation dad9f1f [None][refactor] simplify unified KV cache layer specs 6ea5ad8 [None][test] remove unified KV cache foundation tests 090cbc0 [None][docs] remove unified KV cache design document 511975f [None][feat] support per-layer storage layouts in KVCacheManagerV2 bfc8bc7 fix: normalize explicit KV cache spec windows 18c71c1 fix: use explicit KV cache dtype for cold-page codec ae64dca test: cover explicit KV cache dtype for cold-page codec 701abea fix: honor physical slot strides in KV cache views 623816c test: cover mixed KV cache odd-slot addressing 7cde28e fix: restore KV cache subclass and test compatibility 4a3bb25 [None][test] fix mixed KV cache slot address arithmetic 9d0107b [None][fix] align NVFP4 data and scale page layouts e273a98 [None][fix] accept mixed KV cache pointer table layouts f6b1d08 [None][fix] preserve sink and reuse policy in KV cache costing b6db612 [TRTLLM-20434][feat] Dflash Unified KVCache 39d814d [None][fix] share the DFlash draft KV cache in disagg only with speculative_config in both sections 22cf25c [None][fix] gate the disagg shared draft KV cache on matching tensor_parallel_size feed06a [None][test] cover the disagg shared draft KV gate for generation-only speculative_config and mismatched TP 0319dc3 test: use default timeout for unified DFlash KV test 1f39128 fix: restore DFlash private arena fallback for separate KV caches 97fb214 fix: seed unified DFlash context lengths from bounded token counts caf4c45 test: cover DFlash mixed-precision block tables 875eff2 fix: require matching attention DP for shared draft KV 212fdab test: enable shared draft KV in DFlash integration test 1302d43 fix: keep mixed-window DFlash caches separate Port conflict resolutions (base = main 0829689, ours = agentx b7979ca + !10912): - kv_cache_manager_v2.py: our joint_kv_cache_reuse kwarg and the PR's layer_specs kwarg land at the same spot in _get_per_request_layer_tokens, get_cache_bytes_per_request and get_cache_quota_per_request (5 hunks): keep both. The generic layer_configs loop moves under the PR's `else:`; main's kv_pool_rebalance_config (NVIDIA#19077) is not on agentx and is dropped. - _util.py: agentx's _per_manager_cache_cost has no tokens_per_block parameter (main-only); keep the PR's unified layer_specs injection. - dflash.py: keep agentx's windowed VANILLA arena (_ctx_base) branch; ctx_len_gen (PR) equals self._ctx_len[slots]. - rank_info.py: drop main's bounded_replay_on_generation (not on agentx); keep num_draft_layers. - tests: drop main-only imports (PyTorchModelEngine, KvCacheRetentionConfig). !10912 stays intact: validate_block_attention_windows() is not re-added. Signed-off-by: Allison Lim <allim@nvidia.com> Signed-off-by: Zheyu Fu <zheyuf@nvidia.com>

Squashed increment of draft PR NVIDIA#20709 at head 1703851 over NVIDIA#20486 at 714916d (both on main 0829689; the NVIDIA#20709 tree contains all of NVIDIA#20486), ported onto feat/musespark-open-agentx. Commits: 7a85a73 [None][feat] unify the standalone DSpark target and draft KV cache d492b52 [None][feat] reshard unified target/draft KV by per-view attention heads Port conflict resolutions (base = NVIDIA#20486 on main 0829689, ours = the NVIDIA#20486 port): - kv_extractor.py: agentx lacks main's view_partition hook (NVIDIA#19727); add only kv_heads_for_layer and the Callable import its annotation needs (Muse target and draft buffers differ in size, so they never share a physical pool/view). - kv_cache_manager_v2.py add_dummy_requests: keep agentx's stop_committing() for the dummy and size it with the PR's _kv_alloc_extra_tokens. - test_kvcm2_integration.py: keep agentx's DecoderRunner import and add the PR's tests only (main's generation-block reuse tests, NVIDIA#19710, are not on agentx). Signed-off-by: Allison Lim <allim@nvidia.com> Signed-off-by: Zheyu Fu <zheyuf@nvidia.com>

…indings The ported NVIDIA#20178/NVIDIA#20486/NVIDIA#20709 Python assumes main's KV cache manager bindings. Two pieces are not on agentx: - Per-layer AttentionReusePolicy (main NVIDIA#19727 C++). The stack only ever uses REQUIRED, which is what agentx does for every layer, so the reuse_policy field, its AttentionLayerConfig argument and its place in the scratch and pool-group keys are dropped. The one test that needs PRIVATE is skipped. - BufferConfig.coalescing_group (the stack's own C++, which the agentx image does not carry). Group 0 is what the prebuilt bindings do implicitly, so the argument is passed only for nonzero (NVFP4) groups, which fail loudly unless the bindings are rebuilt from this branch's cpp/. Signed-off-by: Zheyu Fu <zheyuf@nvidia.com>

Keep upstream cache-layer validation and preserve DFlash mixed-dtype coverage. Update unified-cache dtype access for the model engine refactor and distinguish model geometry from storage layer specs in tests. Signed-off-by: Allison Lim <allim@nvidia.com>

allisonlim-nv · 2026-10-26T07:07:48Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-10-26T07:15:30Z

PR_Github #80565 [ run ] triggered by Bot. Commit: 02951f9 Link to invocation

tensorrt-cicd · 2026-10-26T12:54:42Z

PR_Github #80565 [ run ] completed with state SUCCESS. Commit: 02951f9
/LLM/main/L0_MergeRequest_PR pipeline #66326 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

Signed-off-by: Allison Lim <allim@nvidia.com>

allisonlim-nv · 2026-10-26T17:46:06Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-10-26T17:52:31Z

PR_Github #80661 [ run ] triggered by Bot. Commit: 2648d18 Link to invocation

allisonlim-nv · 2026-10-26T21:19:03Z

The improvement is good.

May you share measurements upon this improvement with respect to e2e performance, ,latency, TTFT, or time per output token, or throughput at a fixed GPU budget and latency target which would take account for the additional memory and transfer overhead.

Yes, here are the stats for the improvements: perf numbers

tensorrt-cicd · 2026-10-26T23:03:06Z

PR_Github #80661 [ run ] completed with state SUCCESS. Commit: 2648d18
/LLM/main/L0_MergeRequest_PR pipeline #66416 completed with status: 'FAILURE'

CI Report

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

CI Agent Failure Analysis

Link to invocation

allisonlim-nv · 2026-10-26T23:40:39Z

/bot run --reuse-test 66416

tensorrt-cicd · 2026-10-26T23:51:52Z

PR_Github #80722 [ run ] triggered by Bot. Commit: 2648d18 Link to invocation

tensorrt-cicd · 2026-10-27T01:58:57Z

PR_Github #80722 [ run ] completed with state SUCCESS. Commit: 2648d18
/LLM/main/L0_MergeRequest_PR pipeline #66468 completed with status: 'UNSTABLE'

CI Report

⚠️ Multi-GPU Label Required:
Multi-GPU tests require the ci: full pre-merge approved label on this PR. Either:

Wait for the PR to be fully approved — the label is added automatically once approval is complete. Having unresolved open comments is fine, or
If needed, ask a member of NVIDIA/trt-llm-ci-approvers to add the label manually.
Then re-trigger CI with the same bot command (no rebase needed).

⚠️ Action Required:

Please check the failed tests and fix your PR
If you cannot view the failures, ask the CI triggerer to share details
Once fixed, request an NVIDIA team member to trigger CI again

Link to invocation

mikeiovine · 2026-10-27T16:09:24Z

Should be reviewed by kv cache manager owners before merging

…#20486 and #20709) (#20806) Signed-off-by: Zheyu Fu <zheyuf@nvidia.com> Co-authored-by: Allison Lim <allim@nvidia.com>

SimengLiu-nv

Approve to unblock. Left comments to address.

Preserve unified DFlash cache layer offsets in the fused block-table decoder and cover nonzero offsets in the existing kernel regression test. Retain the ModelEngine rename alongside cache storage layer specifications. Signed-off-by: Allison Lim <allim@nvidia.com>

Require compatible target and draft attention backends, static drafting, and supported page sizes on every disaggregated peer before enabling shared draft KV. Keep separate caches when the effective page size is unsupported by DFlash, and cover the launcher and local fallback decisions. Signed-off-by: Allison Lim <allim@nvidia.com>

allisonlim-nv · 2026-10-27T22:12:49Z

/bot run --disable-fail-fast

tensorrt-cicd · 2026-10-27T22:20:43Z

PR_Github #80992 [ run ] triggered by Bot. Commit: 219e26e Link to invocation

allisonlim-nv added 16 commits October 12, 2026 16:01

[None][feat] add unified KV cache layout foundation

3bc36a5

Signed-off-by: Allison Lim <allim@nvidia.com>

[None][refactor] simplify unified KV cache layer specs

dad9f1f

Signed-off-by: Allison Lim <allim@nvidia.com>

[None][test] remove unified KV cache foundation tests

6ea5ad8

Signed-off-by: Allison Lim <allim@nvidia.com>

[None][docs] remove unified KV cache design document

090cbc0

Signed-off-by: Allison Lim <allim@nvidia.com>

[None][feat] support per-layer storage layouts in KVCacheManagerV2

511975f

Signed-off-by: Allison Lim <allim@nvidia.com>

fix: normalize explicit KV cache spec windows

bfc8bc7

Signed-off-by: Allison Lim <allim@nvidia.com>

fix: use explicit KV cache dtype for cold-page codec

18c71c1

Signed-off-by: Allison Lim <allim@nvidia.com>

test: cover explicit KV cache dtype for cold-page codec

ae64dca

Signed-off-by: Allison Lim <allim@nvidia.com>

fix: honor physical slot strides in KV cache views

701abea

Signed-off-by: Allison Lim <allim@nvidia.com>

test: cover mixed KV cache odd-slot addressing

623816c

Signed-off-by: Allison Lim <allim@nvidia.com>

fix: restore KV cache subclass and test compatibility

7cde28e

Signed-off-by: Allison Lim <allim@nvidia.com>

[None][test] fix mixed KV cache slot address arithmetic

4a3bb25

Signed-off-by: Allison Lim <allim@nvidia.com>

[None][fix] preserve sink and reuse policy in KV cache costing

f6b1d08

Signed-off-by: Allison Lim <allim@nvidia.com>

Merge main into unified KV cache manager foundation

eea6dcb

Preserve explicit and legacy layer configuration while applying upstream pool rebalancing settings to both paths. Signed-off-by: Allison Lim <allim@nvidia.com>

github-actions Bot assigned allisonlim-nv Oct 16, 2026

allisonlim-nv changed the title ~~Feat/dflash unified kv~~ [TRTLLM-20434][feat] Dflash Unified KVCache Oct 16, 2026

allisonlim-nv marked this pull request as ready for review October 16, 2026 18:55

allisonlim-nv requested review from a team as code owners October 16, 2026 18:55

allisonlim-nv requested review from VALLIS-NERIA, lfr-0531 and lowsfer October 16, 2026 18:55

Tabrizian mentioned this pull request Oct 21, 2026

[None][feat] Migrate DFlash2 sliding-window drafters to the TRTLLM attention backend #20776

Merged

1 task done

zheyuf mentioned this pull request Oct 22, 2026

[None][feat] Unified target/draft KV cache for DFlash (port of #20178, #20486 and #20709) #20806

Merged

1 task done

allisonlim-nv added 2 commits October 26, 2026 10:40

[TRTLLM-16722][fix] align DFlash fixtures and skip empty updates

f8e216c

Signed-off-by: Allison Lim <allim@nvidia.com>

Merge branch 'main' into feat/dflash-unified-kv

2648d18

allisonlim-nv requested a review from eopXD October 26, 2026 22:39

mikeiovine added the ci: full pre-merge approved label Oct 27, 2026

mikeiovine approved these changes Oct 27, 2026

View reviewed changes

zheyuf added a commit that referenced this pull request Oct 27, 2026

[None][feat] Unified target/draft KV cache for DFlash (port of #20178, …

0f6cff3

…#20486 and #20709) (#20806) Signed-off-by: Zheyu Fu <zheyuf@nvidia.com> Co-authored-by: Allison Lim <allim@nvidia.com>

SimengLiu-nv approved these changes Oct 27, 2026

View reviewed changes

Comment thread tensorrt_llm/commands/serve.py

Comment thread tensorrt_llm/_torch/pyexecutor/_util.py

allisonlim-nv added 2 commits October 27, 2026 11:08

allisonlim-nv enabled auto-merge (squash) October 27, 2026 22:24

Conversation

allisonlim-nv commented Oct 16, 2026 • edited by coderabbitai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Dev Engineer Review

QA Engineer Review

Per-File QA Perspective

Description

Test Coverage

PR Checklist

GitHub Bot Help

Uh oh!

allisonlim-nv commented Oct 26, 2026

Uh oh!

tensorrt-cicd commented Oct 26, 2026

Uh oh!

tensorrt-cicd commented Oct 26, 2026

Uh oh!

allisonlim-nv commented Oct 26, 2026

Uh oh!

tensorrt-cicd commented Oct 26, 2026

Uh oh!

allisonlim-nv commented Oct 26, 2026

Uh oh!

tensorrt-cicd commented Oct 26, 2026

Uh oh!

allisonlim-nv commented Oct 26, 2026

Uh oh!

tensorrt-cicd commented Oct 26, 2026

Uh oh!

tensorrt-cicd commented Oct 27, 2026

Uh oh!

mikeiovine commented Oct 27, 2026

Uh oh!

SimengLiu-nv left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

allisonlim-nv commented Oct 27, 2026

Uh oh!

tensorrt-cicd commented Oct 27, 2026

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

9 participants

allisonlim-nv commented Oct 16, 2026 •

edited by coderabbitai Bot

Loading