Skip to content

README.md

Latest commit

 

History

History
142 lines (106 loc) · 5.17 KB

README.md

File metadata and controls

142 lines (106 loc) · 5.17 KB
Raw

autonomous-job-scout

An autonomous, multi-source job monitoring agent built with a LangGraph-orchestrated pipeline, hybrid LLM + keyword filtering, and real-time Telegram delivery. Designed for discoverability and separation of concerns — each stage of the pipeline lives in its own module and can be extended or swapped independently.

Architecture

The agent runs as a stateful LangGraph graph where each node is a discrete, testable unit:

[Collector] → [Processor] → [Decision] → [Analyzer] → [Discoverer] → [Notifier]
Node Module Responsibility
Collector collector/ Fetches HTML & RSS feeds from multiple job sources
Processor processor/ Normalizes and cleans raw text for downstream evaluation
Decision decision/ Hybrid filter — fast keyword pass, then LLM (Groq / llama3) for semantic relevance
Analyzer analysis/ Extracts structured metadata (role, company, location)
Discoverer discovery/ Enriches matches with additional context
Notifier notifier/ Formats and pushes alerts via Telegram Bot API

Memory layer (memory/): SQLite-backed deduplication — every alerted job is hashed and stored, preventing repeat notifications across cycles.

Key Design Decisions

  • Hybrid filtering over pure LLM: Keyword pre-screening short-circuits expensive API calls. The LLM (llama3-3.1-8b-instant via Groq) is only invoked when a keyword match warrants deeper semantic evaluation.
  • LangGraph for orchestration: The agent's control flow is a compiled, stateful graph — conditional edges handle the relevant/irrelevant branch without nested if chains in application code.
  • Rate-limit guards built into config: MAX_LLM_CALLS_PER_CYCLE and LLM_TIMEOUT prevent runaway API spend or latency spikes from stalling the loop.
  • Firebase for cloud persistence: When deployed, the SQLite memory layer is backed by Firebase Storage, enabling stateful restarts on ephemeral hosts (Render, Railway, etc.).

Tech Stack

Layer Technology
Orchestration LangGraph, LangChain Core
LLM Groq API (llama-3.1-8b-instant)
Backend API FastAPI + Uvicorn
Memory SQLite (local) / Firebase Admin SDK (cloud)
Notifications Telegram Bot API
Containerization Docker
Config .env / python-dotenv

Project Structure

.
├── collector/        # Data fetching — HTML scraping & RSS parsing
├── processor/        # Text normalization pipeline
├── decision/         # Hybrid keyword + LLM relevance filter
├── analysis/         # Structured metadata extraction
├── discovery/        # Job enrichment and context expansion
├── notifier/         # Telegram alert formatting & delivery
├── memory/           # SQLite deduplication store
├── graph.py          # LangGraph graph definition and compilation
├── main.py           # Entry point — polling loop
├── server.py         # FastAPI server exposing agent state
├── state.py          # Shared AgentState TypedDict
├── config.py         # Centralized settings & env-var loading
├── Dockerfile        # Container build
└── .env.example      # Required environment variable template

Running Locally

1. Clone and install dependencies

git clone https://github.com/<your-username>/autonomous-job-scout
cd autonomous-job-scout
pip install -r requirements.txt

2. Configure environment

cp .env.example .env
# Fill in GROQ_API_KEY, TELEGRAM_TOKEN, TELEGRAM_CHAT_ID

3. Start the agent

python main.py

4. (Optional) Start the FastAPI server

python server.py
# API available at http://localhost:8000

Running with Docker

docker build -t autonomous-job-scout .
docker run --env-file .env autonomous-job-scout

Configuration

All tunable parameters live in config.py and are overridable via .env:

Variable Default Description
LLM_PROVIDER groq LLM backend (groq or ollama)
GROQ_MODEL llama-3.1-8b-instant Groq model identifier
CHECK_INTERVAL 60 Polling interval in seconds
MAX_ALERTS_PER_CYCLE 5 Max Telegram notifications per cycle
MAX_LLM_CALLS_PER_CYCLE 10 LLM call cap per cycle
LLM_TIMEOUT 30 Seconds before falling back to keyword-only mode

Data sources are set in SOURCE_URLS inside config.py. The agent currently monitors:

Keyword targeting is controlled by the KEYWORDS list — add role titles, company names, or locations to tune the pre-screening pass.

Environment Variables

See .env.example for the full list. Required variables:

GROQ_API_KEY=
TELEGRAM_TOKEN=
TELEGRAM_CHAT_ID=

Firebase variables (FIREBASE_SERVICE_ACCOUNT, FIREBASE_STORAGE_BUCKET) are only required for cloud deployments.