Add a compact plugin that does auto compact on context like claude code #249

Activity

1. codelion

   added

   enhancementNew feature or requestNew feature or request

   help wantedExtra attention is neededExtra attention is needed

   good first issueGood for newcomersGood for newcomers

   on Sep 26, 2025
2. GoDiao commented on May 5, 2026

   

   GoDiao

   on May 5, 2026

   Contributor

   More actions

   Hi! I'd love to work on this. I've implemented a similar auto-compact system in MiniCode (a lightweight Claude Code clone in TypeScript), which includes:

   • Auto-compact: detects when context approaches token limit and triggers summarization (src/compact/auto-compact.ts)
   • Micro-compact: lightweight compression for partial context reduction (src/compact/microcompact.ts)
   • Manual compact: user-triggered full compaction (src/compact/manual-compact.ts)
   • Compact boundaries: session persistence marks where compaction happened for replay (src/session.ts)

   For this plugin, I'm thinking of the following approach:

   1. Token budget tracking — count conversation tokens, trigger compact when exceeding a configurable threshold (e.g. 80% of model context window)
   2. Summarization — use the LLM itself to produce a compact summary of the conversation so far, then replace older messages with the summary
   3. Plugin interface — follow the existing pattern (SLUG + run() function), configurable via environment variables or the existing optillm config

   A few questions before I start:

   1. Should this be a standalone plugin (like memory_plugin.py) or integrated into the proxy layer to intercept all requests regardless of approach?
   2. Is there a preferred way to get the model's context window size, or should it be a config parameter?
   3. Should the compact trigger be token-count-based, message-count-based, or both?

   Happy to implement once we align on the direction. Thanks!
3. codelion commented on May 5, 2026

   

   codelion

   on May 5, 2026

   MemberAuthor

   More actions

   1 -> We can make it a separate plugin but let's try and make it easy to add on to existing approaches or plugins so someone can do a requests/response handover transparently using existing operator like &compact&moa .

   2 -> We can make it a config but if the upstream provider provides this info we should try and get that first form the /models or similar endpoint.

   3 -> make it configurable with some good default.
4. codelion

   added a commit that references this issue on May 7, 2026

   Add compact plugin for auto context compression (closes #249)

   This issue will close once commit 5c07f46 is merged into the 'main' branch.

   5c07f46
5. codelion

   closed this as completedin #305on May 7, 2026