[BUG]: AI Chat Fails Due to Context Window Overflow (Improper Token Handling) #407

Description

The current AI chat system limits conversation context by storing only the last 8 messages, assuming this prevents exceeding the model’s context window. However, this approach does not account for token size variability, where longer messages can significantly increase token count.

As a result, even with fewer messages, the total token size can exceed the model’s limit, causing API failures such as “Context Window Exceeded” errors. This leads to unstable chat behavior and failed responses during extended or detailed conversations.

Steps to Reproduce

Open the AI chat feature
Enter multiple long or detailed messages (e.g., large paragraphs or code blocks)
Continue the conversation beyond a few exchanges
Observe API failure or missing response

Expected Behavior

The chat system should dynamically manage context to stay within token limits
AI responses should consistently generate without errors
Long conversations should be handled gracefully

Actual Behavior

API requests fail with context window exceeded errors
Chat responses stop working after several long messages
No fallback or recovery mechanism is triggered

Screenshots / Logs

N/A

OS

Windows

Browser

Chrome

Version

No response