Skip to content

README.md

Latest commit

 

History

History
462 lines (337 loc) · 12.3 KB

README.md

File metadata and controls

462 lines (337 loc) · 12.3 KB
Raw

Marketing Analyst Agent

Python 3.11+ FastAPI LangGraph

Production-ready marketing analysis AI agent built with LangGraph and FastAPI. Analyzes market trends, competitor data, consumer sentiment, generates reports, and provides strategic recommendations using real data sources.

🚀 Features

  • 📊 Market Trend Analysis - Analyze market trends, growth rates, and key market dynamics using Google Trends and real news data
  • 🏢 Competitor Analysis - Evaluate competitor positioning, market share, strengths, and weaknesses
  • 💭 Consumer Sentiment Analysis - Analyze consumer sentiment across social media (Hacker News), news articles, and reviews
  • 📄 Report Generation - Generate comprehensive marketing reports in multiple formats
  • 🎯 Strategy Recommendations - Get actionable strategic recommendations based on business objectives
  • 🔌 FastAPI REST API - Production-ready API with authentication, rate limiting, and comprehensive error handling
  • 🐳 Docker Support - Full Docker and Docker Compose support for easy deployment
  • 🆓 Free Data Sources - Uses only free/open-source services (no paid APIs required)

📋 Table of Contents

🏗️ Architecture

The agent uses a node-based architecture where each analysis type is a dedicated node that executes based on boolean flags. This provides explicit control over which analyses run.

Node-Based Execution

Request with boolean flags
         ↓
[Market Trend Node] → [Competitor Analysis Node] → [Consumer Sentiment Node] 
         ↓                      ↓                           ↓
   (if flag=true)          (if flag=true)             (if flag=true)
         ↓
[Report Generation Node] → [Strategy Recommendation Node] → [Synthesis Node]
         ↓                            ↓                          ↓
   (if flag=true)              (if flag=true)              (always runs)

Key Components

  • Nodes: Analysis components (market_trend, competitor_analysis, etc.)
  • Tools: Helper utilities used by nodes (web search, scraping, data sources)
  • State Management: TypedDict-based state passing between nodes
  • LLM Integration: OpenAI GPT-4o for query analysis and synthesis

📦 Installation

Prerequisites

  • Python 3.11 or higher
  • OpenAI API key

Install Dependencies

pip install -r requirements.txt

Install Playwright (Optional but Recommended)

For advanced web scraping capabilities:

playwright install chromium

⚙️ Configuration

Required Environment Variables

OpenAI API Key (Required)

export OPENAI_API_KEY=your_openai_api_key_here

Or create a .env file:

OPENAI_API_KEY=your_openai_api_key_here

Optional Environment Variables

Self-Hosted SearXNG Search Service (Required for full functionality)

You need to self-host your own SearXNG instance. The agent uses SearXNG's API to fetch news articles and search results.

Option 1: Docker (Recommended)

docker run -d -p 8080:8080 \
  -e SEARXNG_HOSTNAME=localhost \
  searxng/searxng:latest

Then set in your .env:

SEARCH_SERVICE_URL=http://localhost:8080

Option 2: Docker Compose

Create a docker-compose.searxng.yml:

version: '3.8'
services:
  searxng:
    image: searxng/searxng:latest
    ports:
      - "8080:8080"
    environment:
      - SEARXNG_HOSTNAME=localhost

Run: docker-compose -f docker-compose.searxng.yml up -d

Option 3: Manual Installation

Follow the SearXNG installation guide.

Configuration

Once SearXNG is running, set the URL in your .env:

SEARCH_SERVICE_URL=http://your-searxng-instance:8080

Self-Hosted Scraper (Optional)

SCRAPER_SERVICE_URL=https://your-scraper-url.com

YouTube Data API (Optional)

YOUTUBE_API_KEY=your_youtube_api_key

API Security (For FastAPI)

API_KEY=your_secret_api_key
RATE_LIMIT_PER_MINUTE=10
CORS_ORIGINS=*

Server Configuration

PORT=8000
HOST=0.0.0.0
DEBUG=false

Data Sources

The agent uses only free/open-source services:

  • ✅ Google Trends (pytrends) - Market trends and search interest (no setup required)
  • ✅ Self-hosted SearXNG - News articles and web search (you must self-host)
  • ✅ Hacker News API - Social media sentiment (no auth required)
  • ✅ YouTube Data API - Video content analysis (free tier, optional)
  • ✅ RSS Feeds - News articles (fallback, no setup required)
  • ✅ Playwright Scraper - Advanced web scraping (optional, install Playwright)

🚀 Usage

FastAPI Server

Start the Server

# Development
python main.py

# Or with uvicorn
uvicorn main:app --reload --host 0.0.0.0 --port 8000

# Production (with Gunicorn)
gunicorn -w 5 -k uvicorn.workers.UvicornWorker -b 0.0.0.0:8000 main:app

API Endpoints

Health Check

curl http://localhost:8000/health

Perform Analysis

curl -X POST "http://localhost:8000/api/v1/analyze" \
  -H "X-API-Key: your_secret_api_key" \
  -H "Content-Type: application/json" \
  -d '{
    "query": "Analyze market trends for AI tools in last year",
    "market_trend": true,
    "competitor_analysis": true,
    "consumer_sentiment": true,
    "report_generation": false,
    "strategy_recommendation": false
  }'

Interactive API Documentation

Python Library

from marketing_analyst_agent import create_agent

# Create agent
agent = create_agent()

# Process analysis
result = await agent.process(
    query="Analyze market trends for AI agents in 2025",
    run_market_trend=True,
    run_competitor_analysis=True,
    run_consumer_sentiment=True,
    run_report_generation=False,
    run_strategy_recommendation=False
)

# Access results
print(result["analysis_result"]["content"])
print(result["data_logs"])

📚 API Documentation

Request Format

{
  "query": "Analyze market trends for AI tools in last year",
  "market_trend": true,
  "competitor_analysis": true,
  "consumer_sentiment": true,
  "report_generation": true,
  "strategy_recommendation": true
}

Response Format

{
  "status": "success",
  "query": "Analyze market trends for AI tools in last year",
  "result": {
    "status": "success",
    "processing_status": "completed",
    "analysis_content": "# Marketing Analysis Report\n\n...",
    "market_data": {...},
    "competitor_data": {...},
    "sentiment_data": {...},
    "data_logs": [...],
    "metadata": {
      "summary": "...",
      "key_insights": [...],
      "recommendations": [...],
      "confidence_score": 85
    }
  },
  "output_files": {
    "markdown": "output/analysis_20251206_120000.md",
    "info": "output/analysis_20251206_120000_info.txt"
  }
}

Authentication

The API uses API key authentication via the X-API-Key header:

curl -H "X-API-Key: your_secret_api_key" ...

If API_KEY is not set in environment variables, the API is accessible without authentication (not recommended for production).

Rate Limiting

Default: 10 requests per minute (configurable via RATE_LIMIT_PER_MINUTE)

📊 Data Sources

Market Trend Analysis

  • Google Trends - Search interest and trend data (no setup required)
  • Self-hosted SearXNG - News articles about market segment (requires self-hosting)
  • RSS Feeds - Additional news sources (fallback, no setup required)

Competitor Analysis

  • Self-hosted SearXNG - Competitor news and articles (requires self-hosting)
  • Google Trends - Search interest per competitor (no setup required)
  • Playwright Scraper - Competitor website analysis (optional, install Playwright)

Consumer Sentiment Analysis

  • Hacker News - Social media sentiment (primary, no auth required)
  • YouTube - Video content sentiment (optional, requires API key)
  • Self-hosted SearXNG - News sentiment (requires self-hosting)
  • RSS Feeds - Additional news sources (fallback, no setup required)

🐳 Docker Deployment

Using Docker Compose

  1. Create .env file:
OPENAI_API_KEY=your_openai_api_key
API_KEY=your_secret_api_key
SEARCH_SERVICE_URL=http://your-searxng-instance:8080

Note: Make sure you have SearXNG running (see Self-Hosted SearXNG Setup above).

  1. Build and run:
docker-compose up -d
  1. Check logs:
docker-compose logs -f

Manual Docker Build

docker build -t marketing-analyst-agent .
docker run -p 8000:8000 \
  -e OPENAI_API_KEY=your_key \
  -e API_KEY=your_secret \
  -v $(pwd)/output:/app/output \
  marketing-analyst-agent

📁 Output

Results are saved to the output folder:

  • analysis_[timestamp].md - Comprehensive analysis report in markdown
  • analysis_[timestamp]_info.txt - Analysis metadata, insights, and data logs

Report Contents

Markdown File:

  • Executive Summary
  • Key Findings
  • Detailed Analysis
  • Strategic Recommendations
  • Next Steps

Info File:

  • Analysis summary
  • Key insights
  • Recommendations
  • Confidence score
  • Data sources used
  • Detailed data fetching logs

🛠️ Development

Project Structure

marketing_analyst_agent/
├── main.py                      # FastAPI application
├── marketing_analyst_agent.py    # Core agent implementation
├── tools/
│   ├── market_data.py           # Market analysis tools
│   ├── data_sources.py          # Data source clients
│   ├── report.py                # Report generation
│   ├── strategy.py              # Strategy recommendations
│   └── advanced_scraper.py      # Playwright scraper (optional)
├── output/                      # Generated reports
├── requirements.txt             # Python dependencies
├── Dockerfile                   # Docker configuration
├── docker-compose.yml           # Docker Compose setup
└── README.md                    # This file

Running Tests

# Check syntax
python -m py_compile marketing_analyst_agent.py main.py

# Run with example query
python -c "from marketing_analyst_agent import create_agent; import asyncio; asyncio.run(create_agent().process('test query'))"

Code Quality

The project follows production-ready practices:

  • ✅ Type hints throughout
  • ✅ Comprehensive error handling
  • ✅ Input validation and sanitization
  • ✅ Security best practices (API keys, rate limiting)
  • ✅ Docker support for deployment
  • ✅ Proper logging and monitoring

🔒 Security

  • API Key Authentication - Configurable API key protection
  • Rate Limiting - Prevents abuse
  • Input Validation - Pydantic models for request validation
  • CORS Configuration - Configurable cross-origin policies
  • Error Handling - Comprehensive error handling without exposing internals

📝 Example Queries

  • "Analyze market trends in the mobile gaming industry over the last 3 months"
  • "Compare our competitors: Apple, Samsung, and Google in the smartphone market"
  • "What is the consumer sentiment around our new product launch?"
  • "Generate a comprehensive market overview report for Q1 2024"
  • "Provide strategic recommendations to increase market share in the SaaS industry"

🤝 Contributing

This project is part of an agent automation suite. Contributions are welcome!

📄 License

This project is part of the agent automation suite.

🙏 Acknowledgments

Marketing-Analyst-AI-Agent