Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
46 changes: 20 additions & 26 deletions astrbot/core/agent/context/compressor.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,16 +7,6 @@
from ..message import Message
from .token_counter import EstimateTokenCounter, TokenCounter

if TYPE_CHECKING:
from astrbot import logger
else:
try:
from astrbot import logger
except ImportError:
import logging

logger = logging.getLogger("astrbot")

if TYPE_CHECKING:
from astrbot.core.provider.provider import Provider

Expand Down Expand Up @@ -207,8 +197,16 @@ async def __call__(self, messages: list[Message]) -> list[Message]:
"""Use LLM to generate a summary of the conversation history.

Uses round-based splitting to preserve user-assistant turn boundaries.
On LLM failure, returns the original messages unchanged (caller should
fall back to truncation).

Args:
messages: The original message list.

Returns:
System messages, the generated summary, and recent exact context.

Raises:
RuntimeError: If the provider returns an error or an empty summary.
Exception: If the summary request fails.
"""
from .round_utils import split_into_rounds

Expand Down Expand Up @@ -271,22 +269,18 @@ async def __call__(self, messages: list[Message]) -> list[Message]:
)
log_context_sanitize_stats(sanitize_stats)

# Generate summary. A single attempt is enough: on failure the caller
# falls back to truncation, so retrying (e.g. on an exhausted quota)
# would only hold up the reply.
try:
response = await self.provider.text_chat(
contexts=sanitized_summary_contexts,
request_max_retries=1,
# Try once and let the runner decide whether the uncompressed request fits.
response = await self.provider.text_chat(
contexts=sanitized_summary_contexts,
request_max_retries=1,
)
if response.role == "err":
raise RuntimeError(
f"LLM context compression failed: {response.completion_text}"
)
summary_content = (response.completion_text or "").strip()
except Exception as e:
logger.error(f"Failed to generate summary: {e}")
return messages

summary_content = (response.completion_text or "").strip()
if not summary_content:
logger.warning("LLM context compression returned an empty summary.")
return messages
raise RuntimeError("LLM context compression returned an empty summary.")

# Build result: system messages + summary pair + recent rounds
result = _extract_system_messages(messages)
Expand Down
26 changes: 21 additions & 5 deletions astrbot/core/agent/context/manager.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
from astrbot import logger
from astrbot.core.exceptions import ContextCompressionError

from ..message import Message
from .compressor import LLMSummaryCompressor, TruncateByTurnsCompressor
Expand Down Expand Up @@ -50,9 +51,13 @@ async def process(

Args:
messages: The original message list.
trusted_token_usage: Provider-reported usage for the request context.

Returns:
The processed message list.

Raises:
ContextCompressionError: If the compressor fails.
"""
try:
result = messages
Expand All @@ -77,6 +82,8 @@ async def process(
result = await self._run_compression(result, total_tokens)

return result
except ContextCompressionError:
raise
except Exception as e:
logger.error(f"Error during context processing: {e}", exc_info=True)
return messages
Expand All @@ -87,21 +94,30 @@ async def _run_compression(
"""
Compress/truncate the messages.

If the context still exceeds the limit after compression (for example
because the summary request failed), whole rounds are dropped, oldest
first, until it fits. The latest round is always kept, even when it
alone exceeds the limit.
If successful compression leaves the context above the threshold, drop
whole rounds, oldest first. Always keep the latest round. Compression
failures propagate so the runner can check the model's hard limit.

Args:
messages: The original message list.
prev_tokens: The token count before compression.

Returns:
The compressed/truncated message list.

Raises:
ContextCompressionError: If the compressor fails.
"""
logger.debug("Compress triggered, starting compression...")

messages = await self.compressor(messages)
try:
messages = await self.compressor(messages)
except Exception as e:
raise ContextCompressionError(
f"Context compression failed: {e}",
messages=messages,
Comment on lines +114 to +118

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Medium · Original conversation history is lost

When a custom compressor mutates its input list in place and then raises, _run_compression() stores the same mutated messages list in ContextCompressionError, so the runner uses altered history instead of the pre-compression context and the caller loses history.

Preserve a copy of the messages before calling the compressor and store that copy in ContextCompressionError if compression fails.

Prompt for AI agents
In `astrbot/core/agent/context/manager.py` at lines 114-118:

**Original conversation history is lost**

When a custom compressor mutates its input list in place and then raises, `_run_compression()` stores the same mutated `messages` list in `ContextCompressionError`, so the runner uses altered history instead of the pre-compression context and the caller loses history.

Preserve a copy of the messages before calling the compressor and store that copy in `ContextCompressionError` if compression fails.

token_count=prev_tokens,
) from e

# double check
tokens_after_summary = self.token_counter.count_tokens(messages)
Expand Down
29 changes: 23 additions & 6 deletions astrbot/core/agent/runners/tool_loop_agent_runner.py
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@
from astrbot.core.agent.message import ImageURLPart, TextPart, ThinkPart
from astrbot.core.agent.tool import FunctionTool, ToolSet
from astrbot.core.agent.tool_image_cache import tool_image_cache
from astrbot.core.exceptions import EmptyModelOutputError
from astrbot.core.exceptions import ContextCompressionError, EmptyModelOutputError
from astrbot.core.message.components import Json
from astrbot.core.message.message_event_result import (
MessageChain,
Expand Down Expand Up @@ -827,12 +827,29 @@ async def step(self):
# Process request-time context before sending it to the provider.
token_usage = self.req.conversation.token_usage if self.req.conversation else 0
self._simple_print_message_role("[BefCompact]", self.run_context.messages)
processed_messages = await self._await_or_stop(
self.request_context_manager.process(
self.run_context.messages,
trusted_token_usage=token_usage,
try:
processed_messages = await self._await_or_stop(
self.request_context_manager.process(
self.run_context.messages,
trusted_token_usage=token_usage,
)
)
)
except ContextCompressionError as e:
max_tokens = self.provider.provider_config.get("max_context_tokens", 0)
# Keep the count from before compression; a fresh local estimate may
# undercount the unchanged history and conceal a context overflow.
if max_tokens > 0 and e.token_count > max_tokens:
self._transition_state(AgentState.ERROR)
self.stats.end_time = time.time()
raise RuntimeError(
f"Context compression failed and the context exceeds the model's "
f"maximum context length ({e.token_count} > {max_tokens} tokens): {e}"
) from e
Comment on lines +842 to +847

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Medium · Overflow leaves no final response

When a caller relies on the runner’s final response after run_agent() handles a context-overflow exception, the overflow branch raises without assigning self.final_llm_resp, unlike the runner’s ordinary provider-error path. Callers that inspect get_final_llm_resp() after run_agent() handles the exception receive None instead of the terminal error response, so they cannot recognize or process the failure through the runner result.

Store the overflow error as the runner’s final response before raising, or route it through the runner’s normal error-response finalization path.

Prompt for AI agents
In `astrbot/core/agent/runners/tool_loop_agent_runner.py` at lines 842-847:

**Overflow leaves no final response**

When a caller relies on the runner’s final response after `run_agent()` handles a context-overflow exception, the overflow branch raises without assigning `self.final_llm_resp`, unlike the runner’s ordinary provider-error path. Callers that inspect `get_final_llm_resp()` after `run_agent()` handles the exception receive `None` instead of the terminal error response, so they cannot recognize or process the failure through the runner result.

Store the overflow error as the runner’s final response before raising, or route it through the runner’s normal error-response finalization path.

logger.warning(
f"{e}. Attempting the uncompressed request "
f"({e.token_count} tokens, context limit: {max_tokens})."
)
processed_messages = e.messages
if processed_messages is None:
yield await self._finalize_aborted_step()
return
Expand Down
24 changes: 24 additions & 0 deletions astrbot/core/exceptions.py
Original file line number Diff line number Diff line change
@@ -1,5 +1,10 @@
from __future__ import annotations

from typing import TYPE_CHECKING

if TYPE_CHECKING:
from astrbot.core.agent.message import Message


class AstrBotError(Exception):
"""Base exception for all AstrBot errors."""
Expand All @@ -13,6 +18,25 @@ class EmptyModelOutputError(AstrBotError):
"""Raised when the model response contains no usable assistant output."""


class ContextCompressionError(AstrBotError):
"""A compression failure with the context and count used to trigger it."""

def __init__(
self, message: str, *, messages: list[Message], token_count: int
) -> None:
"""Preserve the request context for the runner's context-limit check.

Args:
message: Description of the compression failure.
messages: Request messages before the failed compression attempt.
token_count: Token count that triggered compression, including any
provider-reported usage.
"""
super().__init__(message)
self.messages = messages
self.token_count = token_count


class KnowledgeBaseUploadError(AstrBotError):
"""Raised when knowledge base upload fails with a user-facing message."""

Expand Down
10 changes: 8 additions & 2 deletions docs/en/use/context-compress.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,15 +36,15 @@ LLM compression adds a model request, which takes time and may incur a charge. S
| Recent Context Token Ratio to Keep | Defaults to `0.15`: a budget of 15% of the current context's tokens is used to retain recent content unchanged. The range is `0–0.3`. AstrBot keeps whole turns rather than cutting at an exact token boundary; a positive ratio keeps at least the latest turn. |
| Context Compression Instruction | Leave empty for the default prompt. A custom prompt can ask the summary to preserve the task goal, completed steps, key conclusions, file paths, and next actions. |

The default summary focuses on the user's goal, task progress, conclusions, useful tool results, and materials already read so work can continue. If the selected compression model is unavailable, AstrBot tries the current chat model. If it cannot generate a summary, it falls back to removing older turns.
The default summary focuses on the user's goal, task progress, conclusions, useful tool results, and materials already read so work can continue. If the selected compression model is unavailable, AstrBot tries the current chat model. If summary generation fails, AstrBot checks the uncompressed context against the chat model's configured context limit. It attempts the uncompressed request if the context is within that limit, or stops with an error if it exceeds the limit.

## When Context Is Processed

### 1. A Request Approaches the Model's Context Limit

AstrBot checks context **before every model request**, including follow-up requests after an Agent calls tools. The selected strategy is triggered when context usage exceeds approximately **82%** of the model's context window.

AstrBot checks again after processing. If the context is still too long, it removes complete turns from the oldest first while keeping the latest turn. Tool calls and their results are handled together to avoid leaving incomplete call records.
AstrBot checks again after successful compression. If the context is still too long, it removes complete turns from the oldest first while keeping the latest turn. Tool calls and their results are handled together to avoid leaving incomplete call records.

If the latest turn alone is too large, automatic compression may still be unable to fit the request into the model's window. Reduce the size of the input or tool output, or choose a model with a larger context window.

Expand Down Expand Up @@ -82,6 +82,12 @@ Check that you edited the profile used by the current session and saved it. Toke

Check the window sizes of both the chat model and the compression model. One large message, file, or tool result can make the latest turn exceed the limit on its own. Compression does not discard the current task indefinitely; reduce the input size and try again.

### What If Compression Fails and the Context Exceeds the Limit?

An error containing `Context compression failed and the context exceeds the model's maximum context length` means summary generation failed and the uncompressed context exceeds the chat model's configured context limit. AstrBot therefore stops the current request.

Check the logs for the compression failure's cause; the underlying exception is also included at the end of the error message. Check the compression model's service availability, quota, and context window. Try selecting an available model with a large enough context window under **Model Provider ID for Context Compression**, save the configuration, and retry.

### How Do I Start Fresh?

Send `/reset` to clear the current conversation context or `/new` to create a new conversation. See [Built-in Commands](./command.md) for commands and wake-prefix settings.
10 changes: 8 additions & 2 deletions docs/zh/use/context-compress.md
Original file line number Diff line number Diff line change
Expand Up @@ -36,15 +36,15 @@ LLM 压缩会额外调用模型,因此会增加一次请求的时间和费用
| 压缩时保留最近上下文比例 | 默认 `0.15`,表示以当前上下文 Token 数的 15% 为预算保留最近的原始内容,范围为 `0–0.3`。按完整对话轮次保留,不会精确切到某个 Token;比例大于 0 时至少保留最后一轮。 |
| 上下文压缩提示词 | 留空使用默认提示词。需要定制摘要时,可以要求保留任务目标、已完成的步骤、关键结论、文件路径和下一步计划。 |

默认摘要会关注用户目标、任务进度、关键结论、工具结果以及读过的资料,方便继续后续工作。如果压缩模型不可用,AstrBot 会尝试使用当前聊天模型;如果无法生成摘要,则回退为截断旧对话。
默认摘要会关注用户目标、任务进度、关键结论、工具结果以及读过的资料,方便继续后续工作。如果压缩模型不可用,AstrBot 会尝试使用当前聊天模型;如果摘要生成失败,AstrBot 会检查压缩前的上下文是否超过聊天模型配置的窗口上限,未超过时尝试使用未压缩的上下文继续请求,超过时停止并报错。

## 什么情况下会处理上下文

### 1. 请求接近模型上下文窗口上限

AstrBot 会在**每次向模型发送请求之前**检查上下文,包括 Agent 调用工具后再次请求模型的步骤。当上下文 Token 用量超过模型窗口的约 **82%** 时,会触发所选处理方式。

处理后会再次检查。如果仍然过长,AstrBot 会从最旧的完整对话轮次开始移除内容,并保留最后一轮。工具调用与对应结果会一起处理,避免只剩下一半的调用记录。
压缩成功后会再次检查。如果仍然过长,AstrBot 会从最旧的完整对话轮次开始移除内容,并保留最后一轮。工具调用与对应结果会一起处理,避免只剩下一半的调用记录。

如果最新一轮本身就过大,自动压缩仍可能无法使请求符合模型限制。此时需要减少单次输入或工具返回内容,或者使用上下文窗口更大的模型。

Expand Down Expand Up @@ -82,6 +82,12 @@ AstrBot 会在**每次向模型发送请求之前**检查上下文,包括 Agen

检查聊天模型和压缩模型的窗口大小是否准确。单次长消息、大文件或工具返回的内容可能使最新一轮独自超限,自动压缩不会无限删去当前任务。减少本次输入规模后重试。

### 出现上下文压缩失败且超限的报错怎么办?

如果报错包含 `Context compression failed and the context exceeds the model's maximum context length`,表示摘要压缩失败,且压缩前的上下文已超过聊天模型配置的窗口上限,AstrBot 因此停止本次请求。

请先检查日志中的压缩失败原因,具体异常也会附在报错末尾。确认压缩模型的服务可用性、额度和上下文窗口是否满足要求。可以在「用于上下文压缩的模型提供商 ID」中更换为可用且窗口足够大的模型,保存配置后重试。

### 想从头开始一段对话怎么办?

在聊天中使用 `/reset` 清空当前对话上下文,或使用 `/new` 创建新对话。指令及唤醒前缀说明见[内置指令](./command.md)。
Loading
Loading