Qualcomm AI Engine Direct - Support QAT for concat observer

[shewu-quic]

Summary

See PR #23299 for the root-cause writeup — ConcatObserver is PTQ-only (plain observer, pokes other observers' min_val/max_val directly), which breaks under QAT since the output never gets fake-quantized and the write lands on a dead attribute on FusedMovingAvgObsFakeQuantize inputs.

This PR takes a different fix than #23299's SharedQuantizationSpec approach: that shares qparams only with the first cat input, which clips any other input with a wider range (verified with a cat([x+1, y*4]) toy model — the first input's calibrated range doesn't cover the second). Instead, concat_observer_ctr() keeps ConcatObserver unchanged for PTQ, and adds ConcatFakeQuantize for QAT — a FakeQuantize paired with ConcatObserver, same pattern as the existing PerBlockParamObserver/PerBlockParamFakeQuantize — so the output is still fake-quantized while ConcatObserver keeps aggregating the true max range across every input and the output, not just the first.

Test plan

Added to backends/qualcomm/tests/rework/utils/test.py (no device required):

• test_qat_concat — QAT-prepared cat([x+1, y*4]): output is ConcatFakeQuantize wrapping ConcatObserver, its observed range covers both inputs within one quant step, and convert_pt2e gives inputs and output one shared qparam.
• test_concat_observer_deepcopy — cat at the end of a 300-node chain: copy.deepcopy + convert_pt2e doesn't raise RecursionError, and the copy's input_observers point at the copy's own submodules.

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/23347

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

⚠️ 15 Awaiting Approval

As of commit a4f43ad with merge base 91a918e ():

AWAITING APPROVAL - The following workflows need approval before CI can run:

• Build documentation (gh)
• Build Presets (gh)
• Cadence Build & Test (gh)
• Lint (gh)
• periodic (gh)
• pull (gh)
• Test ARM Backend (gh)
• Test CoreML Backend (gh)
• Test Cortex-M Backend (gh)
• Test NXP Backend (gh)
• Test QNN Backend (gh)
• Test QNN Windows MSVC build (gh)
• Test Vulkan Backend (gh)
• Test WebGPU Backend (gh)
• Test XNNPACK Backend (gh)

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[github-actions]

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

[meta-codesync]

@billmguo has imported this pull request. If you are a Meta employee, you can view this in D122948809.

[billmguo]

@shewu-quic seems this cover#23298 and use different method to fix concat as #23299

1 With SharedQuantizationSpec in pt2e, every input edge and the output get one shared observer instance. Your approach does observe all of them, not just the first input.The real weakness is in the QAT observer, which uses a moving average. Calls on x+1, y*4 and the cat output are blended rather than combined into one overall min/max. So the wider input can be partly clipped. Your test only checked that there is one qparam, not that the range covers every input, which is why it didn't catch this. #23347's shared_range_covers_every_input subtest does check it.
2. The range only ever grows under QAT. ConcatObserver keeps an all-time min/max, not a moving average. An outlier early in training fixes the cat range for good, while every other activation uses MovingAverageMinMaxObserver.
3. The QAT deepcopy case isn't tested. ConcatObserverDeepcopy uses PTQ only. The original vizard failure was a QAT copy.deepcopy before convert. It now goes through ConcatFakeQuantize, and it should work, since the copy recurses into ConcatObserver.deepcopy. But no test covers it.
4. Inputs lag by one step during training. Each input reads the overwritten range only on its next forward pass. At convert_pt2e the qparams do line up, and the test checks that.

[billmguo]

Two cats sharing an input fight over its range (S4). Both ConcatObservers overwrite the observer on a+1, and the last one to write wins. So cat0 ends up with input qparams that don't match its output. I haven't checked whether QNN's Concat accepts mixed input encodings or adds a requantize.

do you able to try a model on device or simulator and see whether this case is covered

[billmguo]

@shewu-quic when will u mainline this PR?

[shewu-quic]

@shewu-quic when will u mainline this PR?

I have test with test_qat_concat for multi-step QAT process and test_concat_observer_deepcopy for deep copy issue.

[billmguo]

@shewu-quic when can u mainline on this I will ask our model team use this PR once mainline.

[shewu-quic]

Hi @psiddh,
This PR addresses the concat observer issue in QAT and the deep-copy issue.
Could you please take a look?