You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
[ENHANCEMENT]: [Performance] Add missing cache decorator to get_category_books #576
Currently, the get_category_books() function in backend/ai_service.py handles the generation of relevant book lists based on specific shelf themes and vibes. However, other expensive LLM operations in this service (such as get_ai_recommendations or generate_chat_response), this function completely lacks a caching decorator.
As a result, every time a user refreshes the virtual library or navigates the landing page shelves, a raw, LLM network request is fired to generate the data from scratch.
in a nutshell, The current behavior forces the backend to call the AI from scratch every single time a user loads or refreshes a book shelf. This slows down the app to a crawl and burns through your API limits on identical requests that could easily be saved and reused.
kindly consider assigning it to me under GSSOC'26 .
Suggested Improvement
Apply Decorator-Based Caching: Leverage the existing caching ecosystem established in backend/cache_service.py to seamlessly wrap get_category_books().
Category-Specific Cache Keys: Ensure the underlying cache key dynamically hashes both the category string and the requested count so that varying limits don't serve mismatched cached arrays
Benefits
Drops response times for shelf fetching from several seconds (waiting on LLM generation) to milliseconds (reading from local cache memory/Redis).
Drastically minimizes the number of duplicate API tokens consumed from upstream providers (Groq, Gemini, or OpenAI), preventing rapid quota exhaustion during multi-user testing.
Prevents the frontend virtual library shelves from looking blank or displaying skeleton loaders repeatedly on minor page update
Activity
ghost-exodus commentedon May 16, 2026
kindly consider assigning it to me under GSSOC'26 .
@devanshi14malhotra
ghost-exodus commentedon May 16, 2026
@devanshi14malhotra can you please add the Tags for GSSoC'26 and level:advance
devanshi14malhotra commentedon May 16, 2026
will add while reviewing the pr, dw
Merge pull request #593 from ghost-exodus/feature/cache-category-books