Skip to content

✨ feat: collaboration substrate for RubyLLM multi-agent runs - #3

Merged
pftg merged 1 commit into
masterfrom
vcr-failure-integration
Aug 28, 2026
Merged

✨ feat: collaboration substrate for RubyLLM multi-agent runs#3
pftg merged 1 commit into
masterfrom
vcr-failure-integration

Conversation

@pftg

@pftg pftg commented Aug 27, 2026

Copy link
Copy Markdown
Member

What this is

RubyLLM::Team coordinates several RubyLLM agents through one auditable run: named
artifact handoffs, a shared call budget, safe fan-out, and a trace showing the exact prompt
every agent received.

It is a small library, not a framework — 658 lines, one runtime dependency, and your workflow
stays ordinary Ruby.

Why you would install it

You do not need it for one agent, or two in a straight line. You start needing it when several
agents share work and you have to answer "which version did the editor actually review?" and
"why did this run cost 40 calls?"

Hand-rolled Provided
A result hash plus rules for which version is current Named artifact versions, as: / from: handoffs
A counter so one runaway loop cannot bill you forever One atomic budget, bounded by default
Thread.new per agent and a join that swallows a failure Fan-out that settles every sibling before raising
rescue => e in six shapes One CollaborationError, typed BudgetExceededError
A logger you grep afterwards A trace with the verbatim prompt per call

Review guide

Four commits, smallest surface first:

  1. 93c3b67 library + packaging — the whole public API. Start at lib/ruby_llm/team.rb.
  2. 76d4190 specs — what a reader cannot verify by inspection: budget reservation across
    threads, artifact ordering under out-of-order completion, crash handling on both
    schedulers, re-entrancy, handoff forgery.
  3. fa65680 examples — read examples/code_review/ first (110 lines, exercises the whole
    library). examples/blog/ is production-scale and deliberately the largest.
  4. 32ed0ea docsdocs/DECISIONS.md is the fastest way to judge scope: it carries the
    measurement behind every refusal, and a known-weaknesses section.

Correctness worth a second look

Each of these was reproduced before it was fixed, not inferred:

  • Artifact versions are reserved at submission under the session lock, so artifact(name)
    is deterministic when parallel work completes out of order.
  • Re-entrancy protection was absent on the path every example uses — a class-registered
    coworker delegating into itself recursed 51 levels deep. Now a property of the role,
    tracked fiber-locally so concurrent same-role work stays legal.
  • Handoffs were forgeable: results were relayed under a fixed literal header, so a coworker
    (or a fetched web page speaking through one) could fabricate a handoff from a coworker that
    never ran. Each result now carries a per-session random fence.
  • max_calls is bounded by default. Every framework that loops on paid calls ships a cap;
    this one claimed spend-bounding as its headline while shipping unbounded.

Deliberately not here

No graph DSL, YAML workflows, or role/backstory metaphors. No memory, RAG, MCP, or search. No
dashboards or persistence — ruby_llm-agents owns that Rails layer and Team composes inside
it. Quality loops stay in application Ruby: refine/repair were built, measured, and removed
because upstream teaches the same loop in fewer lines and only one domain ever needed them.

Verification

  • 98 examples, 0 failures; RuboCop clean; Ruby 3.1–4.0 in CI
  • Three VCR cassettes replay in CI with no API key
  • gem build produces exactly 7 files (lib, README, CHANGELOG, LICENSE)
  • All five examples run; the live ones were exercised against real providers during development

Known weaknesses

Recorded in docs/DECISIONS.md rather than left for a reviewer to find: a budget bounds
delegation hops rather than provider spend (ruby_llm has no internal tool-call cap);
exhaustion raises on the Ruby path but returns a hash on the tool path; mutual recursion across
threads defeats the re-entrancy guard; run-total token usage sums across models; and the blog
example is larger than the code it saves.

The API is pre-1.0 and moved late — artifact ordering, error types, trace serialization, and
the read-side accessors all changed shortly before this branch settled.

@pftg pftg changed the title Add local blog workflow example ✨ feat: add reliable multi-agent blog workflow Aug 27, 2026
@pftg pftg changed the title ✨ feat: add reliable multi-agent blog workflow ✨ feat: add artifact-centered team collaboration Aug 27, 2026
@pftg
pftg force-pushed the vcr-failure-integration branch from d58bf59 to 32ed0ea Compare August 28, 2026 12:05
@pftg pftg changed the title ✨ feat: add artifact-centered team collaboration ✨ feat: collaboration substrate for RubyLLM multi-agent runs Aug 28, 2026
Team coordinates several RubyLLM agents through one auditable run:
named immutable artifact versions with explicit as:/from: handoffs, an
atomic call budget bounded by default, thread and fiber fan-out,
normalised failures, and a trace recording the exact prompt every
coworker received. It stays a small library — your workflow remains
ordinary Ruby, and models, prompts, schemas and retries stay with
RubyLLM.

Correctness the library owns because hand-rolled versions get it wrong,
each reproduced before it was fixed:

- Artifact versions are reserved at submission under the session lock,
  so artifact(name) is deterministic when parallel work completes out
  of order, and a failed call leaves a gap rather than renumbering.
- Non-StandardError crashes finalise their call and re-raise instead of
  leaving it :running with a burned budget slot; fiber siblings settle
  before a crash propagates; thread joins cannot mask the first failure.
- A coworker delegating into its own call fails clearly rather than
  recursing 51 levels deep, whether registered as a class or instance.
- Relayed results carry a per-session random fence, so a coworker — or
  a fetched web page speaking through one — cannot forge a handoff from
  a coworker that never ran.
- A run is bounded by default. Every framework that loops on paid calls
  ships a cap; this one claimed spend-bounding as its headline while
  shipping unbounded.

Five examples, smallest first: simple_team (no API key), code_review
(parallel fan-out, verdict computed in Ruby because a free model will
write "approve" above the injection it just reported), topic_analyst,
decision_panel (no Ruby orchestration — a lead model chooses whom to
consult), and the seven-pass blog with bounded gates, escalation to a
stronger writer, a cold reader that sees only the finished article, and
citation provenance checked by comparing strings rather than asking a
model. editorial_pipeline composes two teams in plain Ruby and lets the
panel argue the analyst's shortlist.

Quality policy stays in the examples: deterministic checks are limited
to defects a reader cannot forgive, because failing a run over a word
count discards writing that cost real money.

Docs collapse six roadmap drafts and three contradictory ADRs into
docs/ROADMAP.md and docs/DECISIONS.md, which carries the measurement
behind every refusal and a known-weaknesses section.

98 examples, RuboCop clean, Ruby 3.1-4.0 in CI, three VCR cassettes
replayed without an API key, and a gem package of exactly seven files.

Claude-Session: https://claude.ai/code/session_01EtHLQMFyPQEQHwwvVhUdGK
@pftg
pftg force-pushed the vcr-failure-integration branch from 27c679e to 9a4dcf2 Compare August 28, 2026 12:30
@pftg
pftg merged commit 915abb0 into master Aug 28, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant