XHS Topic Intelligence Bot
A compliance-first assistant that converts one or multiple Xiaohongshu links plus user-provided content into structured topic intelligence reports and pushes them to WeCom / Enterprise WeChat.
Xiaohongshu content is often fragmented, experience-based, and mixed with personal opinions, product recommendations, and commercial promotion. Users may collect multiple posts around one scenario, such as maternal and infant care, product selection, hospital preparation, travel planning, or consumer research. However, manually extracting reusable topics, concerns, risks, and action items is time-consuming.
This project aims to turn those fragmented notes into standardized, reviewable, and traceable topic reports.
Build a lightweight personal / small-team intelligence assistant that can:
- Accept Xiaohongshu links and manually pasted content.
- Extract topics, subtopics, questions, keywords, and risks.
- Merge multiple links into one structured report.
- Push the final report to Enterprise WeChat.
- Gradually evolve into a domain-specific knowledge base.
- Accept a JSON input file containing multiple content items.
- Support manual content input as the primary ingestion method.
- Clean and normalize text.
- Extract structured topics from each item.
- Cluster similar topics across multiple items.
- Identify potential risks, unsupported claims, medical/health-sensitive content, and advertising bias.
- Generate:
- Structured JSON result.
- Markdown report.
- Save generated outputs locally.
- Prepare interfaces for future WeCom integration.
- Add FastAPI service endpoints.
- Add WeCom push capability.
- Add report history and topic trend tracking.
- Add knowledge base integration.
- Add interactive query support through WeCom.
- Support domain-specific templates, such as maternal and infant care, IT product research, consumer products, and travel planning.
The MVP will not:
- Automatically scrape Xiaohongshu at scale.
- Bypass login, anti-bot mechanisms, CAPTCHA, paywalls, private APIs, or access controls.
- Collect private user data.
- Reproduce full original posts in the final report.
- Provide medical, legal, or financial advice as authoritative conclusions.
- Replace professional judgment for health-related decisions.
A personal knowledge worker who collects Xiaohongshu links and wants structured insights, especially for practical decision-making scenarios.
Example scenarios:
- Preparing for childbirth and newborn care.
- Summarizing product reviews.
- Extracting consumer pain points.
- Building a personal knowledge base.
- Monitoring recurring questions in a topic area.
- Small teams that need lightweight social content summaries.
- Product researchers.
- Content planners.
- Knowledge base maintainers.
The user provides one Xiaohongshu link and manually pasted content. The system extracts key topics, summary, keywords, and risks.
The user provides 5-20 links with pasted content. The system merges similar topics, removes duplicates, and generates a consolidated report.
The user collects posts about hospital bag preparation, newborn feeding, postpartum recovery, and baby skincare. The system converts them into a structured checklist and flags health-sensitive claims.
After generating a report, the system sends a Markdown summary to the user or a WeCom group.
Reports are stored and can be queried later by topic, keyword, source link, or date.
The system shall accept a JSON file containing multiple content items.
Required fields:
{
"items": [
{
"url": "https://www.xiaohongshu.com/explore/example",
"title": "Example title",
"content": "Manually pasted content",
"comments": ["optional comment 1", "optional comment 2"]
}
]
}The system shall validate:
urlis not empty.titleis not empty.contentis not empty.commentsis optional.
The system shall treat manually provided content as the primary source in MVP.
The system may keep a placeholder module for public page reading, but it must not implement bypass or anti-bot circumvention in MVP.
The system shall normalize:
- Extra spaces.
- Repeated line breaks.
- Duplicate hashtags.
- Common platform noise.
- Excessive emoji and decorative symbols.
The system shall preserve:
- Product names.
- Brand names.
- Prices.
- Dates.
- Quantities.
- Locations.
- Medical terms.
- Source URLs.
Each cleaned content item shall retain its original URL and title.
For each item, the system shall generate a concise summary.
For each item, the system shall extract:
- Topics.
- Subtopics.
- Keywords.
- User pain points.
- Frequently asked questions.
- Evidence snippets.
- Confidence score.
Each extracted topic should include evidence from the input content, but evidence should be short and not reproduce large portions of original content.
Topic extraction output shall be saved as structured JSON.
The system shall merge semantically similar topics across multiple input items.
The system shall remove duplicate or near-duplicate keywords and questions.
Each consolidated topic shall include the number of related source links.
The system shall identify potentially conflicting opinions, such as:
- Different product recommendations.
- Conflicting health claims.
- Contradictory preparation checklists.
- Overgeneralized personal experience.
The system shall flag content involving:
- Medical or health claims.
- Infant care.
- Pregnancy and postpartum recovery.
- Medication or supplement use.
- Financial claims.
- Safety-critical recommendations.
The system shall identify possible commercial or promotional bias, such as:
- Strong brand claims without evidence.
- Repeated purchase links or coupon language.
- Absolute claims like “must buy” or “best”.
The system shall mark claims that require further verification.
For medical or baby-care related content, the system shall include a warning that the output is for information organization only and should not replace professional advice.
The system shall generate a Markdown report with the following sections:
- Input summary.
- Executive summary.
- Core topics.
- Subtopics.
- Keywords.
- Pain points.
- Frequently asked questions.
- Conflicting opinions.
- Risk warnings.
- Actionable recommendations.
- Suggested follow-up questions.
- Source links.
The system shall generate a structured JSON result for machine processing.
The system shall save generated reports locally.
Suggested output files:
outputs/topic_result_YYYYMMDD_HHMMSS.json
outputs/report_YYYYMMDD_HHMMSS.md
The system shall include a wecom_sender.py module for future integration.
In MVP 1, WeCom sending can be a stub or optional module.
In later milestones, the system shall support:
- Send Markdown report to a WeCom group.
- Send summary to a specific user through self-built application.
- Send report file as attachment if needed.
- No unauthorized scraping.
- No bypassing access controls.
- No private user data extraction.
- Keep source URLs for traceability.
- Do not reproduce full original content.
- Store API keys in
.env, not source code. - Never commit secrets.
- Mask secrets in logs.
- Validate external input.
- Invalid input should produce clear error messages.
- LLM failures should not crash the whole process.
- Partial results should be saved when possible.
- Modular architecture.
- Type hints required.
- Pydantic models required.
- Unit tests for core modules.
- Prompt templates stored separately.
The architecture should support:
- Multiple LLM providers.
- Future web UI.
- Future WeCom interaction.
- Future knowledge base.
- Future topic trend analysis.
{
"url": "string",
"title": "string",
"content": "string",
"comments": ["string"]
}{
"topic": "string",
"confidence": 0.0,
"summary": "string",
"subtopics": ["string"],
"keywords": ["string"],
"pain_points": ["string"],
"questions": ["string"],
"evidence": ["string"],
"source_urls": ["string"]
}{
"risk_type": "medical|advertising|unsupported_claim|privacy|other",
"severity": "low|medium|high",
"description": "string",
"related_topic": "string",
"recommendation": "string"
}{
"report_id": "string",
"created_at": "string",
"input_count": 0,
"topics": [],
"risks": [],
"markdown_report_path": "string",
"json_result_path": "string"
}Deliverables:
- Project skeleton.
- JSON input parser.
- Content cleaner.
- LLM client abstraction.
- Prompt templates.
- Topic extractor.
- Topic clusterer.
- Risk analyzer.
- Markdown report generator.
- Local output files.
- Basic tests.
Deliverables:
POST /analyze.GET /reports/{report_id}.POST /reports/{report_id}/send-wecom.- API request validation.
Deliverables:
- WeCom webhook sender.
- WeCom self-built application sender.
- Access token cache.
- Error handling and retry.
Deliverables:
- SQLite report history.
- Topic search.
- Keyword search.
- Source link search.
- Trend summary.
Deliverables:
- WeCom interaction.
- Ask follow-up questions.
- Generate checklist from selected topic.
- Generate domain-specific reports.
- The system can process at least 3 manually provided Xiaohongshu content items.
- The system outputs a valid JSON result.
- The system outputs a readable Markdown report.
- Each topic contains source link references.
- Medical/infant-care claims are flagged when present.
- No scraping or bypass logic is implemented.
- Unit tests pass for core modules.
- Secrets are not hardcoded.
- Which LLM provider will be used first?
- Should the report language be Chinese only or bilingual?
- Should comments be treated equally with post content or as weaker evidence?
- Should the system support multiple report templates by domain?
- Should WeCom push be group-based first or personal app-based first?