autonomous-job-scout

An autonomous, multi-source job monitoring agent built with a LangGraph-orchestrated pipeline, hybrid LLM + keyword filtering, and real-time Telegram delivery. Designed for discoverability and separation of concerns — each stage of the pipeline lives in its own module and can be extended or swapped independently.

---

Architecture

The agent runs as a stateful LangGraph graph where each node is a discrete, testable unit:

[Collector] → [Processor] → [Decision] → [Analyzer] → [Discoverer] → [Notifier]

Node	Module	Responsibility
Collector	collector/	Fetches HTML & RSS feeds from multiple job sources
Processor	processor/	Normalizes and cleans raw text for downstream evaluation
Decision	decision/	Hybrid filter — fast keyword pass, then LLM (Groq / llama3) for semantic relevance
Analyzer	analysis/	Extracts structured metadata (role, company, location)
Discoverer	discovery/	Enriches matches with additional context
Notifier	notifier/	Formats and pushes alerts via Telegram Bot API

Memory layer (memory/): SQLite-backed deduplication — every alerted job is hashed and stored, preventing repeat notifications across cycles.

---

Key Design Decisions

• Hybrid filtering over pure LLM: Keyword pre-screening short-circuits expensive API calls. The LLM (llama3-3.1-8b-instant via Groq) is only invoked when a keyword match warrants deeper semantic evaluation.
• LangGraph for orchestration: The agent's control flow is a compiled, stateful graph — conditional edges handle the relevant/irrelevant branch without nested if chains in application code.
• Rate-limit guards built into config: MAX_LLM_CALLS_PER_CYCLE and LLM_TIMEOUT prevent runaway API spend or latency spikes from stalling the loop.
• Firebase for cloud persistence: When deployed, the SQLite memory layer is backed by Firebase Storage, enabling stateful restarts on ephemeral hosts (Render, Railway, etc.).

---

Tech Stack

Layer	Technology
Orchestration	LangGraph, LangChain Core
LLM	Groq API (llama-3.1-8b-instant)
Backend API	FastAPI + Uvicorn
Memory	SQLite (local) / Firebase Admin SDK (cloud)
Notifications	Telegram Bot API
Containerization	Docker
Config	.env / python-dotenv

---

Project Structure

.
├── collector/        # Data fetching — HTML scraping & RSS parsing
├── processor/        # Text normalization pipeline
├── decision/         # Hybrid keyword + LLM relevance filter
├── analysis/         # Structured metadata extraction
├── discovery/        # Job enrichment and context expansion
├── notifier/         # Telegram alert formatting & delivery
├── memory/           # SQLite deduplication store
├── graph.py          # LangGraph graph definition and compilation
├── main.py           # Entry point — polling loop
├── server.py         # FastAPI server exposing agent state
├── state.py          # Shared AgentState TypedDict
├── config.py         # Centralized settings & env-var loading
├── Dockerfile        # Container build
└── .env.example      # Required environment variable template

---

Running Locally

1. Clone and install dependencies

git clone https://github.com/<your-username>/autonomous-job-scout
cd autonomous-job-scout
pip install -r requirements.txt

2. Configure environment

cp .env.example .env
# Fill in GROQ_API_KEY, TELEGRAM_TOKEN, TELEGRAM_CHAT_ID

3. Start the agent

python main.py

4. (Optional) Start the FastAPI server

python server.py
# API available at http://localhost:8000

---

Running with Docker

docker build -t autonomous-job-scout .
docker run --env-file .env autonomous-job-scout

---

Configuration

All tunable parameters live in config.py and are overridable via .env:

Variable	Default	Description
LLM_PROVIDER	groq	LLM backend (groq or ollama)
GROQ_MODEL	llama-3.1-8b-instant	Groq model identifier
CHECK_INTERVAL	60	Polling interval in seconds
MAX_ALERTS_PER_CYCLE	5	Max Telegram notifications per cycle
MAX_LLM_CALLS_PER_CYCLE	10	LLM call cap per cycle
LLM_TIMEOUT	30	Seconds before falling back to keyword-only mode

Data sources are set in SOURCE_URLS inside config.py. The agent currently monitors:

• Hacker News Jobs
• Remotive AI/ML RSS
• We Work Remotely — Programming

Keyword targeting is controlled by the KEYWORDS list — add role titles, company names, or locations to tune the pre-screening pass.

---

Environment Variables

See .env.example for the full list. Required variables:

GROQ_API_KEY=
TELEGRAM_TOKEN=
TELEGRAM_CHAT_ID=

Firebase variables (FIREBASE_SERVICE_ACCOUNT, FIREBASE_STORAGE_BUCKET) are only required for cloud deployments.