Skip to content

riscv: add int8 convolution winograd - #7093

Merged
nihui merged 14 commits into
Tencent:masterfrom
nihui:optimize-rvv-convolution-winograd-int8
Oct 14, 2026
Merged

nihui merged 14 commits into
Tencent:masterfrom
nihui:optimize-rvv-convolution-winograd-int8

Conversation

@nihui

@nihui nihui commented Oct 13, 2026 •

Copy link
Copy Markdown
Member

Add tiled INT8 Winograd F(2,3) and F(4,3) for RISC-V Convolution.

Implementation:

  • INT16 kernel/input transforms and widening INT32 GEMM, using eight m2 accumulators with explicit 8/4/2/1 spatial blocks and scalar 2x2/tail kernels. No fractional LMUL or microkernel templates.
  • Fuse input transformation with packing, reuse input panels across output-channel tiles, and retain cache-sized split-K partial sums until the final K tile.
  • Normalize once after the full reduction: /4 for F23 and /576 for F43, following the ARM/x86 arithmetic organization. No bounded normalization groups, special modular recovery, or additional input-channel safety limits.
  • Use the x86 INT8 entry predicate (input channels > 8 or output channels > 8) consistently in pipeline creation and execution. Prefer F43 when enabled, otherwise F23; retain the existing geometry, options, packing and fallback behavior.
  • Use an explicit RTZ conversion on XTHeadVector, preserving the caller's rounding mode. Standard RVV retains its dedicated RTZ instruction.

Related correctness fixes:

  • Align both K tile selection and balancing to eight in the SSE2, MSA, LSX and ARM32 NEON Winograd helpers, matching pack8 input addressing. The ARM32 case is source-proven; it was not a reproduced ARM CI failure.
  • Restore the enclosing thread ID/team size in both GNU and Clang SIMPLEOMP paths after a serialized inner region, and keep Clang's pending thread-count request separate from active team state. Cover this in the existing test_cpu regression.

Tests and validation:

  • Run the unmodified pre-PR convolution tests against the new implementation before selecting additions. Preserve all 134 original cases and their data-generation sequence, ranges and tolerances. Keep only one targeted F23 case (7x3, K384, out9, unpacked with threading) in test_convolution_3a; remove the obsolete F43 grouping cases. No new test file.
  • The measured standard-RVV/scalar executable-line union improves from 672/749 to 748/749 with that one case; only the nT==0 default-thread fallback is not exercised.
  • Final optimized existing-suite matrix: 30/30 passes for convolution_3a, convolution_3b and convolution_oom across VLEN128/256/512/1024, runtime dispatch ON/OFF, scalar and V-disabled execution. Native x86 convolution_3a also passes.
  • 6,636 bounded-data production pipeline/forward cases and 491,520 selector combinations pass, covering options, actual packing, tails, lightmode, thread changes and normal/balanced AVL. These do not claim universal overflow safety or resolve existing heterogeneous-scale postprocessing issues.
  • Compile the actual runtime-ON RVV/XTHeadVector and runtime-OFF XTHeadVector convolution translation units. Check generated normalization instructions and rounding-mode restoration; standard-RVV preprocessing is unchanged by the XTHead compatibility branch. Local XTHead validation is compile/assembly-only.

Latest-head CI for 35ede47 is still running at this update. No Vulkan shader or Quantize/Dequantize changes, and no RVV hardware-performance claim.

Use exact scaled-integer F(2,3) transforms with tiled int16-to-int32
RVV widening GEMM, explicit 4/2/1 spatial blocks, and scalar 2x2 tails.
Fuse input transformation into packed panels and retain split-K sums in
transform-domain scratch until the final output transform.

Bound transformed accumulators and inverse intermediates by limiting
Winograd to 3640 input channels; retain SGEMM/direct fallback above it.
Reuse existing convolution cases for packed/unpacked, threaded split-K
and odd Winograd tile coverage without adding a standalone test.
@tencent-adm

Copy link
Copy Markdown
Member

CLA assistant check
Thank you for your submission, we really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

@codecov-commenter

codecov-commenter commented Oct 13, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 98.01829% with 13 lines in your changes missing coverage. Please review.
✅ Project coverage is 95.28%. Comparing base (b0f97f4) to head (1bbee07).

Files with missing lines Patch % Lines
src/layer/riscv/convolution_3x3_winograd_int8.h 98.24% 11 Missing ⚠️
src/layer/riscv/convolution_riscv.cpp 90.00% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master    #7093      +/-   ##
==========================================
+ Coverage   95.26%   95.28%   +0.01%     
==========================================
  Files        1476     1477       +1     
  Lines      503457   504776    +1319     
==========================================
+ Hits       479643   480999    +1356     
+ Misses      23814    23777      -37     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@github-actions

github-actions Bot commented Oct 13, 2026 •

Copy link
Copy Markdown

Binary size comparison

The binary size change of libncnn.so (bytes)

architecture base size pr size difference
x86_64 27751048 27751048 0 😘
armhf 7710292 7710292 0 😘
aarch64 11849384 11849384 0 😘

Commit: 1bbee071a1c446673748ba19495e8dd5936d6274 · Workflow run

@github-actions github-actions Bot added the core label Oct 13, 2026
@nihui
nihui marked this pull request as ready for review October 14, 2026 04:39
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 14, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-14T04:53:06.170092Z 1bbee07 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1bbee071a1

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/layer/riscv/convolution_3x3_winograd_int8.h
@nihui
nihui merged commit fcde729 into Tencent:master Oct 14, 2026
111 of 113 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants