Skip to content

feat: layerwise decompression/compression support - #811

Open
kylesayrs wants to merge 1 commit into
mainfrom
layerwise-decompression-compression
Open

feat: layerwise decompression/compression support#811
kylesayrs wants to merge 1 commit into
mainfrom
layerwise-decompression-compression

Conversation

@kylesayrs

Copy link
Copy Markdown
Collaborator

Summary

  • Adds leave_decompressed parameter to decompress_module() — when False, strips all quantization metadata (scheme, status, qparams, forward hooks), enabling decompress-then-reapply workflows
  • Adds allowed_modules parameter to apply_quantization_config() and _apply_kv_cache_scheme() to restrict quantization initialization to a subset of modules (e.g., a single decoder layer)

These APIs enable layerwise decompression and compression in the sequential pipeline, where each layer can be decompressed, recalibrated, and re-compressed independently.

Companion PR: llm-compressor (TBD) integrates these APIs into the sequential pipeline via layerwise_decompression / layerwise_compression data arguments

Test plan

  • Verify decompress_module(leave_decompressed=True) preserves quantization metadata (existing behavior)
  • Verify decompress_module(leave_decompressed=False) fully strips quantization
  • Verify apply_quantization_config(allowed_modules=...) only quantizes specified modules
  • Verify round-trip: decompress(leave_decompressed=False) → apply_quantization_config(allowed_modules) → calibrate → compress

🤖 Generated with Claude Code

Add leave_decompressed parameter to decompress_module() — when False,
strips all quantization metadata (scheme, status, qparams, forward
hooks), leaving the module as if quantization had never been applied.
This enables decompress-then-reapply workflows for layerwise
recalibration.

Add allowed_modules parameter to apply_quantization_config() and
_apply_kv_cache_scheme() to restrict quantization initialization to
a subset of modules (e.g., a single decoder layer during sequential
processing).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Aug 2, 2026

Copy link
Copy Markdown
Contributor

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 9da4778e-d28d-4045-826b-4d81024c54ee

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@mergify

mergify Bot commented Aug 17, 2026

Copy link
Copy Markdown

Merge Protections

🔴 1 of 1 protections blocking · waiting on 👀 reviews

Protection Waiting on
🔴 Require one maintainer review 👀 reviews

🔴 Require one maintainer review

Waiting for any of

  • approved-reviews-by=HDCharles
  • approved-reviews-by=brian-dellabetta
  • approved-reviews-by=dsikka
  • approved-reviews-by=kylesayrs
This rule is failing.

All PRs must have at least one approving review from a maintainer before merging.

  • any of:
    • approved-reviews-by=HDCharles
    • approved-reviews-by=brian-dellabetta
    • approved-reviews-by=dsikka
    • approved-reviews-by=kylesayrs
  • #changes-requested-reviews-by = 0

@mergify

mergify Bot commented Aug 25, 2026

Copy link
Copy Markdown

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @kylesayrs.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant