Skip to content

[ENHANCEMENT]: [Performance] Add missing cache decorator to get_category_books #576

Closed

Description

@ghost-exodus

Current Behavior

Currently, the get_category_books() function in backend/ai_service.py handles the generation of relevant book lists based on specific shelf themes and vibes. However, other expensive LLM operations in this service (such as get_ai_recommendations or generate_chat_response), this function completely lacks a caching decorator.

As a result, every time a user refreshes the virtual library or navigates the landing page shelves, a raw, LLM network request is fired to generate the data from scratch.

in a nutshell,
The current behavior forces the backend to call the AI from scratch every single time a user loads or refreshes a book shelf. This slows down the app to a crawl and burns through your API limits on identical requests that could easily be saved and reused.

kindly consider assigning it to me under GSSOC'26 .

Suggested Improvement

  • Apply Decorator-Based Caching: Leverage the existing caching ecosystem established in backend/cache_service.py to seamlessly wrap get_category_books().

  • Category-Specific Cache Keys: Ensure the underlying cache key dynamically hashes both the category string and the requested count so that varying limits don't serve mismatched cached arrays

Benefits

  • Drops response times for shelf fetching from several seconds (waiting on LLM generation) to milliseconds (reading from local cache memory/Redis).

  • Drastically minimizes the number of duplicate API tokens consumed from upstream providers (Groq, Gemini, or OpenAI), preventing rapid quota exhaustion during multi-user testing.

  • Prevents the frontend virtual library shelves from looking blank or displaying skeleton loaders repeatedly on minor page update

Possible Implementation

No response

Activity

  1. ghost-exodus commented on May 16, 2026

    @ghost-exodus
    ContributorAuthor

    kindly consider assigning it to me under GSSOC'26 .

    @devanshi14malhotra

  2. ghost-exodus commented on May 16, 2026

    @ghost-exodus
    ContributorAuthor

    @devanshi14malhotra can you please add the Tags for GSSoC'26 and level:advance

  3. devanshi14malhotra commented on May 16, 2026

    @devanshi14malhotra
    Owner

    @devanshi14malhotra can you please add the Tags for GSSoC'26 and level:advance

    will add while reviewing the pr, dw

  4. added
    level:advancedGSSoC'26 advanced level PR/issue
    GSSoC'26For all issues/PRs related to GSSoC'2026 open source event
    on May 17, 2026
  5. added a commit that references this issue on May 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

GSSoC'26For all issues/PRs related to GSSoC'2026 open source eventlevel:advancedGSSoC'26 advanced level PR/issue

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions