cat-agent-stack

Local HashiCorp stack for cat-agent deploy: Consul + Vault (-dev) + Nomad (server+client) + LiteLLM gateway (+ optional Zot).

Agent packaging / CLI live in the sibling cat-agent library (pip install 'cat-agent[platform]'). This repo is only the compose fixture and sample operator config.

Prerequisites

Docker Desktop (macOS) or Docker Engine (Linux)
pip install 'cat-agent[platform]' (provides cat-agent stack / deploy)
A real LLM backend for agent calls (default: host Ollama on Mac)

Quick start

cd /path/to/cat-agent-stack
cp .env.example .env
# edit .env — VAULT_TOKEN=root + Ollama/OpenAI

export CAT_AGENT_STACK_DIR=$PWD
export CAT_AGENT_CONFIG=$PWD/cat-agent.config.toml

cat-agent stack bootstrap     # compose up --build -d + Vault seed + demo team key
cat-agent doctor              # must print docker_network: cat-agent-stack_hashicorp

# deploy an agent from the cat-agent checkout:
cat-agent deploy --dir /path/to/cat-agent/examples/serve_fastapi
curl -sS http://demo-calculator.localhost:8088/readyz

Optional Zot registry profile:

cat-agent stack bootstrap --profile registry --registry

Tear down

cat-agent rm demo/calculator --yes   # optional
cat-agent stack down

Do not run bare docker compose — HOST_NOMAD_DATA / HOST_ZOT_DATA must be absolute paths; cat-agent stack sets them.

Keys: two layers (this is the confusing part)

There are two different kinds of secrets. Mixing them up feels like “aptallık”; they are not the same thing.

  laptop .env  ──seed──►  Vault platform LLM secret
                              │
                              ▼
                         LiteLLM gateway  ◄── uses Ollama + OpenAI provider keys
                              ▲
                              │  (virtual key as OPENAI_API_KEY)
  deployed agent job ─────────┘
         ▲
         └── reads ONE team virtual key from Vault
             secret/.../llm/teams/{team}

Layer A — provider keys (platform, shared)

Written by seed into secret/data/platform/llm.

Field	Role
`OLLAMA_API_BASE` / `OLLAMA_API_KEY`	Upstream Ollama (host or cloud)
`OPENAI_API_BASE` / `OPENAI_API_KEY`	Upstream OpenAI (optional; can coexist with Ollama)
`LITELLM_MASTER_KEY`	Admin key for LiteLLM (mint/delete virtual keys)

Yes: Ollama and OpenAI keys can both live here at once. LiteLLM picks which upstream to call from the model id in agent.yaml (model.alias), not from “which person deployed”.

These are org/stack credentials — not Ali’s vs Ayşe’s personal OpenAI keys.

Layer B — team virtual key (one per team)

Minted by LiteLLM (key/generate), stored only as api_key at secret/data/platform/llm/teams/{team} (default team: demo).

Every agent job for that team injects this same key as OPENAI_API_KEY.
The job’s OPENAI_BASE_URL points at the gateway, not at Ollama/OpenAI.
Purpose: gateway auth + TPM/RPM budget for that team, not identity of a person.

So today: 1 team → 1 virtual key → all that team’s deployed agents share it.

What we do not do today

Expectation	Reality
Each person on the team has their own OpenAI/Ollama key in Vault	Not supported for deploy. Provider keys are platform-shared (Layer A).
Each person has their own LiteLLM virtual key	Not the default. Seed mints per team, not per user.
Laptop `.env` is what the Nomad job uses	No. Jobs only see Vault (Layer B for the call key; LiteLLM uses Layer A).

If you need per-person billing / keys, that is a different product shape, e.g.:

mint LiteLLM keys per user (--team is a stand-in; you’d use teams/{user} or a users path), or
run cat-agent serve locally with that person’s own env keys (no Nomad), or
put provider keys behind a corporate gateway account and keep Layer B as team quotas only.

Local demo intent: one shared Ollama/OpenAI for the stack + one demo virtual key.

What is `cat-agent stack seed`?

Containers alone are not enough. Vault starts empty (-dev); seed fills Layer A

Layer B from this repo’s .env. It does not start/stop Docker.

Step	What it does
1. Layer A	Writes provider + master key into `secret/data/platform/llm`
2. Layer B	Mints LiteLLM virtual key for `--team` (default `demo`) → `…/teams/{team}` + Vault read policy
3. Optional	`--registry` → Zot push/pull secrets

Re-run seed after changing .env, recreating Vault, or adding another --team.

Commands (related)

Command	Meaning
`cat-agent stack up`	Start compose only
`cat-agent stack seed`	Fill Vault + mint team key from `.env` (stack already up)
`cat-agent stack bootstrap`	First-time: `up --build -d` then `seed`
`cat-agent stack down`	Stop compose

cat-agent stack seed                 # refresh Layer A + demo team key
cat-agent stack seed --team billing  # also mint a second team’s virtual key

Secrets are never printed; seed only logs bases / key lengths.

Config

File	Role
`cat-agent.config.toml`	Sample for `CAT_AGENT_CONFIG` (Nomad, gateway, Mac `docker_network`)
`.env` / `.env.example`	Provider keys + `VAULT_TOKEN=root` for seed
`docker-compose.yml`	Project name pinned: `cat-agent-stack` → network `cat-agent-stack_hashicorp`

macOS / Docker Desktop

Bridge CNI fails with unknown FS magic on "/var/run/docker/netns/…". This stack sets:

docker_network = "cat-agent-stack_hashicorp"
consul_dns = "10.32.0.2"

cat-agent doctor must show those lines before deploy.

Config loading (`cat-agent`)

Platform config lives here (not in the cat-agent library repo):

CAT_AGENT_CONFIG
$CAT_AGENT_STACK_DIR/cat-agent.config.toml
cwd only if this directory is the stack root (docker-compose.yml present)
sibling ../cat-agent-stack/cat-agent.config.toml next to a cat-agent checkout
~/.cat-agent/config.toml

Shared access (LAN / corp) — not `*.localhost`

*.localhost resolves to 127.0.0.1 on the client machine. Other PCs and phones on the same network cannot use http://demo-calculator.localhost:8088.

Traefik + Consul Catalog already front agents. For a corporate deployment, use your internal DNS (Active Directory / Bind / CoreDNS / cloud private zone) — do not rely on public wildcard DNS helpers.

# Must match what clients resolve via company DNS
ingress_host_template = "{team}-{name}.agents.example.internal"
public_url_template   = "https://{team}-{name}.agents.example.internal"

Then:

Create a DNS record or wildcard zone so *.agents.example.internal → the Traefik / load-balancer VIP that publishes the agents (HTTP/HTTPS).
Redeploy agents so Nomad job tags pick up the new Host(…) rule (cat-agent deploy again).
Clients open the URL from public_url_template (any machine on the corp network that can resolve that name).

Placeholders: {team}, {name}, {agent} (agent == name).
Env overrides: CAT_AGENT_INGRESS_HOST_TEMPLATE, CAT_AGENT_PUBLIC_URL_TEMPLATE.

Local laptop default stays {team}-{name}.localhost — no DNS required on that Mac.

Day-2

cat-agent ls
cat-agent status demo/calculator
cat-agent logs demo/calculator
cat-agent rm demo/calculator --yes

Traefik (host :8088): hostname from ingress_host_template
(local default http://{team}-{name}.localhost:8088)
Dashboard: http://127.0.0.1:8089 · Nomad: http://127.0.0.1:4646

Contract with cat-agent

Knob	Local value
`llm_gateway`	`http://llm-gateway.service.consul:4000/v1`
`vault_addr`	`http://127.0.0.1:8200`
`VAULT_TOKEN`	`root` (`-dev` only)
Team key	`secret/data/platform/llm/teams/{team}`
`docker_network`	`cat-agent-stack_hashicorp`
`consul_dns`	`10.32.0.2`
`ingress_host_template`	`{team}-{name}.localhost`
`public_url_template`	`http://{team}-{name}.localhost:8088`
`base_image`	`cat-agent-runtime:latest` (publish/pull in a later phase)

Layout

docker-compose.yml          # pinned name: cat-agent-stack
cat-agent.config.toml       # sample CAT_AGENT_CONFIG
.env.example
config/                     # consul, nomad, vault, litellm, zot
jobs/                       # gate Nomad jobs
Dockerfile.*                # consul / nomad / vault images
docker-entrypoint-*.sh      # container entrypoints (not operator scripts)

Linux notes

Omit or retarget Mac-only knobs (docker_network, consul_dns=10.32.0.2, host.docker.internal). See comments in cat-agent.config.toml and compose.

Name		Name	Last commit message	Last commit date
config		config
jobs		jobs
.env.example		.env.example
.gitignore		.gitignore
Dockerfile.consul		Dockerfile.consul
Dockerfile.nomad		Dockerfile.nomad
Dockerfile.vault		Dockerfile.vault
README.md		README.md
cat-agent.config.toml		cat-agent.config.toml
docker-compose.yml		docker-compose.yml
docker-entrypoint-litellm.sh		docker-entrypoint-litellm.sh
docker-entrypoint-nomad.sh		docker-entrypoint-nomad.sh

Navigation Menu

Repository files navigation

cat-agent-stack

Prerequisites

Quick start

Tear down

Keys: two layers (this is the confusing part)

Layer A — provider keys (platform, shared)

Layer B — team virtual key (one per team)

What we do not do today

What is `cat-agent stack seed`?

Config

macOS / Docker Desktop

Config loading (`cat-agent`)

Shared access (LAN / corp) — not `*.localhost`

Day-2

Contract with cat-agent

Layout

Linux notes

About

Releases

Packages

Contributors

Languages

Latest commit

History

Folders and files

Repository files navigation

cat-agent-stack

Prerequisites

Quick start

Tear down

Keys: two layers (this is the confusing part)

Layer A — provider keys (platform, shared)

Layer B — team virtual key (one per team)

What we do not do today

What is cat-agent stack seed?

Config

macOS / Docker Desktop

Config loading (cat-agent)

Shared access (LAN / corp) — not *.localhost

Day-2

Contract with cat-agent

Layout

Linux notes

About

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages

What is `cat-agent stack seed`?

Config loading (`cat-agent`)

Shared access (LAN / corp) — not `*.localhost`