Merge pull request #3 from jetthoughts/vcr-failure-integration

✨ feat: collaboration substrate for RubyLLM multi-agent runs

Author: pftg

.commandcode/taste/taste.md

@@ -28,6 +28,11 @@ jobs:
       - name: Run specs
         run: bundle exec rake spec
 
+      - name: Replay recorded workflow cassettes
+        env:
+          VCR_RECORD_MODE: none
+        run: bundle exec rspec --tag live
+
       - name: Run RuboCop
         run: bundle exec rubocop
 

.github/workflows/ci.yml

@@ -10,3 +10,35 @@
 .bundle
 vendor/bundle
 Gemfile.lock
+
+# Local agent-tooling state
+/.claude/
+/.claude-flow/
+/.junie/
+
+# Local secrets
+.env
+.env.local
+.env.*.local
+
+# Codex local configuration
+.codex/
+
+# Local agent tooling (kept on disk, never packaged)
+/.agents/
+/.swarm/
+/.mcp.json
+/AGENTS.md
+/CLAUDE.md
+examples/blog/pipeline_trace.md
+.commandcode/
+
+# Generated by the examples on every run; regenerate rather than track
+examples/*/trace.md
+examples/*/pipeline_trace.md
+examples/*/choice_trace.md
+examples/blog/output.md
+examples/blog/eval.json
+examples/code_review/review.md
+examples/decision_panel/decision.md
+examples/topic_analyst/plan.md

.gitignore

@@ -0,0 +1,40 @@
+# Changelog
+
+## 0.1.0 — experimental
+
+First public release. The API is deliberately small but not yet stable: artifact ordering,
+error types, and trace serialization all changed shortly before this release. Pin an exact
+version, and read [docs/DECISIONS.md](docs/DECISIONS.md) for what the gem refuses to do and
+why — those refusals are the stable part.
+
+### Added
+
+- Named immutable artifact versions with `as:`/`from:` handoffs and the thin `Team#run` API.
+- Machine-readable traces: `Session#to_h`/`#to_json` with per-call and run-total best-known
+  token usage; prompts and results export only with `include_content: true`.
+- Typed `BudgetExceededError < CollaborationError` for budget exhaustion.
+- `examples/code_review/` — parallel fan-out/fan-in with a VCR-replayed spec and a
+  line comparison against the upstream plain-Ruby pattern.
+
+### Fixed
+
+- Artifact versions are reserved in submission order, so `artifact(name)` is deterministic
+  when parallel work completes out of order.
+- Non-`StandardError` crashes finalize their call as `:failed` and re-raise instead of
+  leaving it `:running` with a burned budget slot.
+- Fiber siblings settle before a crash propagates; thread joins no longer mask the first crash.
+- A coworker instance delegating back into its own call fails with a clear error instead of
+  `deadlock; recursive locking`.
+- Duplicate coworkers in one `parallel` batch are rejected before reserving budget instead of
+  silently dropping results.
+- `share_context: false` sessions no longer record handoff inputs the coworker never received.
+
+### Changed
+
+- `Run#step` omitted `from:` now hands over every completed artifact, matching `Session#ask`.
+- The published gem contains only `lib/`, README, CHANGELOG, and LICENSE.
+
+### Foundation
+
+- Coworker registry, `delegate_work`/`ask_question` tools, session call budgets,
+  thread/fiber `parallel`, selected handoffs, and the Markdown trace.

CHANGELOG.md

@@ -5,7 +5,13 @@ source 'https://rubygems.org'
 gemspec
 
 group :development, :test do
+  gem 'async', require: false
   gem 'rake'
   gem 'rspec', '~> 3.0'
   gem 'rubocop', require: false
+  gem 'ruby_llm-mcp', '~> 1.0', require: false
+  gem 'vcr'
+  gem 'webmock', '~> 3.18'
+
+  gem 'ruby_llm-tribunal', '~> 0.1', require: false if Gem::Version.new(RUBY_VERSION) >= Gem::Version.new('3.2')
 end

Gemfile

@@ -1,159 +1,98 @@
-# PITCH: `ruby_llm-team` — the delegation primitive RubyLLM won't ship, as a gem
+# PITCH: `ruby_llm-team`
 
-One sentence: **a `RubyLLM::Team` is the one multi-agent primitive that plain Ruby cannot
-express on its own — letting the *model* choose which named coworker to route to at
-runtime — and it belongs in a small extension gem, not in RubyLLM core and not inside a
-CrewAI-style orchestration framework.**
+`ruby_llm-team` packages the repeated plumbing around named RubyLLM coworkers:
+delegation tools, exact result handoffs, bounded calls, concurrent review, and an
+inspectable collaboration record. It is a small extension to RubyLLM, not a process
+engine.
 
-This document grounds that claim in the rejection of PR #891, the maintainer's stated
-philosophy, the CrewAI model and its community pain, and RubyLLM's own issue history.
+## Why a separate gem
 
----
+RubyLLM's Agentic Workflows guidance intentionally uses ordinary Ruby for sequencing,
+routing, parallel work, fan-in, and evaluator/reviser loops. The maintainer closed the
+original Team contribution to RubyLLM core for that reason: the demonstrated workflows
+did not justify another core abstraction.
 
-## 1. The seed: why this gem exists
+That decision sets the boundary for this gem. Team must remove repeated integration code
+without taking workflow policy away from the application.
 
-PR [#891](https://github.com/crmne/ruby_llm/pull/891) ("Add RubyLLM::Team for multi-agent
-collaboration") was closed by the maintainer. The rejection is the single most important
-fact about this gem, because it defines the *correct* scope:
+## What the CrewAI review actually shows
 
-> "This does not belong in the library. The existing Agentic Workflows documentation
-> already shows multi-agent sequencing, routing, handoffs, parallel work, and fan-in
-> using plain Ruby and the existing Agent and Tool APIs. Those examples are clearer, use
-> less code, and do not require adding another abstraction or public API to RubyLLM."
-> — [crmne, PR #891 comment](https://github.com/crmne/ruby_llm/pull/891#issuecomment-5435551502)
+CrewAI offers two related layers:
 
-Read carefully. The maintainer did **not** say the idea is bad. He said:
+- **Crews** organize autonomous agents into sequential or hierarchical processes.
+- **Flows** give the application explicit, event-driven control over state, branches, and
+  execution paths.
 
-1. The **workflow patterns** (sequencing, routing, handoffs, parallel, fan-in) are already
-   well served by plain Ruby + Agent + Tool. Adding a process engine would be worse.
-2. A new abstraction does not belong in **core**.
-3. If a concrete capability can't be expressed with existing APIs, open an issue first.
+CrewAI tasks can name an agent, expected output, prior task context, guardrails, and
+asynchronous execution. A later task that depends on asynchronous tasks forms a clear
+fan-out/fan-in boundary. These are useful collaboration mechanics, independent of
+CrewAI's larger framework.
 
-`ruby_llm-team` answers all three. It adds **no process, no scheduler, no graph**. It is a
-thin tool-layer. And it is a **gem**, not a core addition — precisely the escape hatch the
-maintainer's "does not belong in the library" leaves open.
+The review does **not** support describing CrewAI as simply rigid or claiming that its
+framework decides every execution path. CrewAI itself recommends Crews for autonomous
+work, Flows for deterministic work, and a hybrid for applications needing both.
 
----
+## What Team adopts
 
-## 2. The two ways, compared
+- Named specialists with explicit responsibilities.
+- Exact outputs from completed work as inputs to dependent work.
+- Concurrent execution for independent tasks, followed by a synchronization barrier.
+- Artifact-preserving reviewer handoffs with application-owned revision limits.
+- Visible call limits, errors, inputs, and results.
+- A choice between application-directed calls and model-directed delegation tools.
 
-### The RubyLLM way (maintainer's stance, current docs)
+These mechanics map naturally to Ruby agents, tools, threads, and fibers. They do not
+require a second workflow language.
 
-Orchestration is **ordinary Ruby**. `Agent` is a configured chat; `Tool` is a capability;
-the docs show sequential, routing, handoff, parallel, fan-in, and evaluator-optimizer as
-small plain-Ruby classes. Applications own task order, dependencies, persistence, and
-resume. `RubyLLM.workflow` only adds instrumentation correlation — it does not take over
-execution.
+## What Team deliberately leaves out
 
-Strengths: total flexibility, nothing hidden, debuggable, idiomatic, durable (the loop is
-interruptible and resumable). Weakness: every team re-writes the same delegation boundary
-by hand.
+- `Crew` / `Task` / `Process` or graph DSLs.
+- YAML workflow definitions and generated project structure.
+- A built-in hierarchical manager or automatic planner.
+- Framework-owned state persistence, scheduling, deployment, or remote transport.
+- Built-in memory, knowledge stores, RAG, or MCP clients.
+- Hidden retry, model-selection, or concurrency policy.
+- Runtime quality claims based only on an LLM judge.
 
-### The CrewAI way
+Those capabilities can be valuable, but RubyLLM, ordinary Ruby, and focused ecosystem
+gems already provide composition points for them. Adding them to Team would turn a small
+collaboration primitive into a competing agent platform.
 
-`Crew` / `Agent` / `Task` / `Process` abstractions. Agents declare role/goal/backstory;
-tasks declare expected output; the crew runs a `sequential` or `hierarchical` process
-(hierarchical needs a manager LLM). Simple to explain, fast to prototype.
+## The product boundary
 
-Strengths: approachable, opinionated, quick demos; CrewAI reports enterprise adoption and
-"14x less code" vs graph frameworks. Weakness: a fixed process model that is **not very
-flexible** — the exact tradeoff the user named.
+Team owns:
 
-### The decisive evidence
+- a named coworker registry;
+- `delegate_work` and `ask_question` tools;
+- per-run collaboration state and exact handoffs;
+- immutable named artifacts, revision lineage, and a thin Run API;
+- atomic call budgets;
+- thread or fiber fan-out/fan-in;
+- normalized errors, results, and collaboration traces.
 
-CrewAI's own engineering blog, after "2 billion agentic workflows", lands on the **RubyLLM
-position**, not the Crew abstraction:
+The application owns task dependencies, conditional policy, quality gates, revision and
+escalation limits, persistence, authorization, cancellation, and approvals. A model may
+choose coworkers through `session.tools`; explicit workflows may call `session.ask` and
+`session.parallel` directly.
 
-> "Architecture choices compound fast... separating the predictable from the
-> unpredictable. Having deterministic workflows handling the structure, and agents
-> deployed strategically where judgment actually matters."
->
-> "Many engineers regret graph-based architectures... too many abstraction layers stacked
-> on top of each other... when something breaks, the engineers dig through multiple
-> indirections just to try finding which prompt or tool caused it."
-> — [Lessons From 2 Billion Agentic Workflows](https://blog.crewai.com/lessons-from-2-billion-agentic-workflows/)
+## Evaluation boundary
 
-The community reports the same friction:
+Runtime validators protect production invariants such as Ruby syntax and real APIs.
+Tribunal is an optional test-time grader for relevance, faithfulness, hallucination, and
+regression evaluation. Its report is evidence about a saved artifact; it is not a Team
+coworker, runtime gate, or telemetry system.
 
-- r/crewai: "Overwhelmed with limitations... outdated dependencies, slow performance."
-- r/AI_Agents: "it gets fragile and you lose fine-grained control" in production.
-- r/LangChain: "LangGraph and CrewAI are overcomplicating agents... So I abandoned these
-  libraries, as a bonus dropped the necessity to use Python in production."
-- r/AI_Agents "Who's using CrewAI really?": few teams report production use.
+## Evidence standard
 
-**Conclusion: do not ship a CrewAI-style `Crew`/`Task`/`Process` abstraction.** It would
-contradict the maintainer's philosophy, the docs, and CrewAI's own hard-won lessons. The
-gem's job is the *opposite*: give Ruby developers the one missing low-level primitive and
-let them keep orchestration in plain Ruby.
+Product decisions should rely on current primary documentation, source code, and observed
+behavior. Anonymous community complaints may suggest questions to investigate, but they
+are not sufficient evidence for permanent scope decisions.
 
----
+## Sources
 
-## 3. Shared requests and pains (grounding)
-
-These are the signals that a delegation primitive is genuinely wanted, from RubyLLM's own
-issue tracker and the broader community:
-
-| Signal | Source | What it says |
-|---|---|---|
-| Multi-agent is wanted, but not as transport | [#670 A2A protocol](https://github.com/crmne/ruby_llm/issues/670) (declined, `not_planned`) | People want multi-agent; maintainer drew the line at external transport. **Local delegation is the acceptable scope.** |
-| Team idea itself | [#891](https://github.com/crmne/ruby_llm/pull/891) | Rejected on process/scope, not on value. The code was correct and fully tested (17 specs, 97.74% coverage). |
-| Long-running, resumable work | [#635 "Interrupting the agentic loop"](https://github.com/crmne/ruby_llm/issues/635) (completed) | Real pain: multi-step loops that must pause/resume across deploys. A team that composes with durable agents fits this. |
-| The "one more abstraction" fatigue | Maintainer's rejection; CrewAI blog; community threads | Nobody wants another rigid framework. The gem must stay a primitive, not a platform. |
-| Ruby landscape gap | langchain.rb (huge/complex), FlowNodes (minimalist) | No Ruby-idiomatic, provider-agnostic multi-agent delegation primitive exists. RubyLLM is the natural host ecosystem. |
-
-The through-line: **Ruby developers want multi-agent capability without sacrificing
-control.** CrewAI-style frameworks sell the former and tax the latter. A small gem that
-sells the primitive and leaves control alone is the gap.
-
----
-
-## 4. What the gem is, and is not
-
-### Is
-
-- `RubyLLM::Team` — a named coworker registry.
-- Two ordinary `RubyLLM::Tool`s the model calls at runtime: `delegate_work` and
-  `ask_question`. The model, not the developer, picks the coworker.
-- Recoverable error contract (`{ error: ... }`), shared `context:`, attachment
-  round-tripping, class-vs-instance lifecycle, snapshot concurrency safety.
-- Composes with the existing RubyLLM way: plain-Ruby workflow classes, durable agents,
-  `RubyLLM.workflow` instrumentation.
-
-### Is not
-
-- Not a `Crew`/`Task`/`Process` engine. No sequential/hierarchical process, no scheduler,
-  no executor, no graph.
-- Not an A2A transport. No host-boundary communication.
-- Not a replacement for the Agentic Workflows patterns — those stay in plain Ruby, per the
-  maintainer.
-
-The differentiator in one line: **Team is where the model decides; the workflow is where
-the developer decides.** CrewAI lets the framework decide both; plain Ruby leaves both to
-you; this gem takes only the part the model must own.
-
----
-
-## 5. The pitch
-
-> RubyLLM gives you one beautiful API for every provider, and says "orchestrate with
-> plain Ruby." That's right — until you want the *model* to route work to a named
-> specialist at runtime. Today every team hand-rolls that delegation boundary: a registry,
-> two tool classes, an error contract, attachment handling. `ruby_llm-team` is that
-> boundary, extracted, tested, and composable — so your orchestration stays plain Ruby and
-> the model's delegation stays first-class.
->
-> - **For RubyLLM users:** one line of setup, two tools, no framework.
-> - **For the maintainer's philosophy:** no new abstraction in core, no process engine,
->   no transport — just a Tool boundary that plain Ruby couldn't express.
-> - **For the market:** the Ruby gap between "single agent" and "crew" — without the crew's
->   rigidity.
-
----
-
-## 6. Sources
-
-- Maintainer rejection: https://github.com/crmne/ruby_llm/pull/891#issuecomment-5435551502
-- RubyLLM Agentic Workflows docs: `docs/_advanced/agentic-workflows.md` (working tree)
-- CrewAI lessons: https://blog.crewai.com/lessons-from-2-billion-agentic-workflows/
-- RubyLLM issues: #670 (A2A, declined), #635 (interrupt loop), #891 (Team), #889 (JSON format)
-- Community: r/crewai, r/AI_Agents, r/LangChain threads (linked in section 2)
+- [RubyLLM Agentic Workflows](https://rubyllm.com/agentic-workflows/)
+- [RubyLLM Team PR discussion](https://github.com/crmne/ruby_llm/pull/891)
+- [CrewAI introduction: Crews and Flows](https://docs.crewai.com/core-concepts/Agents)
+- [CrewAI tasks and asynchronous context](https://docs.crewai.com/en/concepts/tasks)
+- [CrewAI source](https://github.com/crewAIInc/crewAI)
+- [RubyLLM ecosystem](https://rubyllm.com/ecosystem/)