feat: layerwise decompression/compression support - #811
Conversation
Add leave_decompressed parameter to decompress_module() — when False, strips all quantization metadata (scheme, status, qparams, forward hooks), leaving the module as if quantization had never been applied. This enables decompress-then-reapply workflows for layerwise recalibration. Add allowed_modules parameter to apply_quantization_config() and _apply_kv_cache_scheme() to restrict quantization initialization to a subset of modules (e.g., a single decoder layer during sequential processing). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
Important Review skippedAuto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Merge Protections🔴 1 of 1 protections blocking · waiting on 👀 reviews
🔴 Require one maintainer reviewWaiting for any of
This rule is failing.All PRs must have at least one approving review from a maintainer before merging.
|
|
This pull request has merge conflicts that must be resolved before it can be |
Summary
leave_decompressedparameter todecompress_module()— whenFalse, strips all quantization metadata (scheme, status, qparams, forward hooks), enabling decompress-then-reapply workflowsallowed_modulesparameter toapply_quantization_config()and_apply_kv_cache_scheme()to restrict quantization initialization to a subset of modules (e.g., a single decoder layer)These APIs enable layerwise decompression and compression in the sequential pipeline, where each layer can be decompressed, recalibrated, and re-compressed independently.
Companion PR: llm-compressor (TBD) integrates these APIs into the sequential pipeline via
layerwise_decompression/layerwise_compressiondata argumentsTest plan
decompress_module(leave_decompressed=True)preserves quantization metadata (existing behavior)decompress_module(leave_decompressed=False)fully strips quantizationapply_quantization_config(allowed_modules=...)only quantizes specified modules🤖 Generated with Claude Code