feat(mcp): add stdio_mcp_tools and http_mcp_tools convenience adapters

[ahmeshaf]

Simplifying MCP integration: stdio_mcp_tools and http_mcp_tools convenience adapters

Summary

Two async context managers that collapse the MCP boilerplate (connect → initialize → list → convert) into a single async with statement. Each addresses the friction of manually wiring up ClientSession, list_tools(), and Tool.from_mcp_tool every time you use MCP with DSPy. All changes are backwards-compatible — convert_mcp_tool / Tool.from_mcp_tool still work as before.

+375 lines, −126 lines across 6 files (excluding uv.lock churn).

The changes

1. stdio_mcp_tools — async context manager for stdio MCP servers

Problem: Connecting to a local MCP server via stdio requires 3 nested context managers, manual initialize(), list_tools(), and a list comprehension to convert tools — ~12 lines of boilerplate every time.

Solution: stdio_mcp_tools(server_params) handles the full lifecycle and yields a ready-to-use list[dspy.Tool].

from mcp import StdioServerParameters
from dspy.utils.mcp import stdio_mcp_tools

params = StdioServerParameters(command="python", args=["server.py"])
async with stdio_mcp_tools(params, include_tools=["search"], timeout=30) as tools:
    react = dspy.ReAct(MySignature, tools=tools)

2. http_mcp_tools — async context manager for streamable-HTTP MCP servers

Problem: Same boilerplate issue for remote MCP servers, plus the streamablehttp_client import and the 3-tuple unpacking.

Solution: http_mcp_tools(url) wraps the HTTP transport with the same clean interface.

from dspy.utils.mcp import http_mcp_tools

async with http_mcp_tools("http://localhost:8000/mcp", exclude_tools=["dangerous"]) as tools:
    react = dspy.ReAct(MySignature, tools=tools)

3. Tool filtering with validation

Problem: Users often want a subset of tools from an MCP server but have to filter manually after conversion.

Solution: Both adapters accept include_tools / exclude_tools with strict validation — unknown tool names raise ValueError immediately instead of silently returning fewer tools.

4. Timeout propagation

timeout (in seconds) is forwarded as read_timeout_seconds to ClientSession, so long-running tool calls don't hang indefinitely.

5. Dependency fix: weaviate-client / mcp httpx conflict

Problem: weaviate-client~=4.5.4 pinned httpx<=0.27.0, conflicting with mcp>=1.20.0 which requires httpx>=0.27.1.

Solution: Widen to weaviate-client>=4.5.4 and pin mcp>=1.20.0.

File-by-file

File	Lines	What
dspy/utils/mcp.py	+141/−3	stdio_mcp_tools, http_mcp_tools, shared helpers (_build_tools_from_session, _filter_mcp_tools, _validate_tool_names), __all__ export
tests/utils/test_mcp.py	+157/−1	Tests for both adapters: basic usage, include/exclude, conflict, timeout, unknown name validation
tests/utils/resources/mcp_server.py	+10/−2	Support --transport and --port CLI args for streamable-HTTP test fixture
docs/docs/learn/programming/mcp.md	+49/−77	Rewrite examples to use new adapters; add filtering and timeout sections
docs/docs/tutorials/mcp/index.md	+15/−40	Update airline booking tutorial to use stdio_mcp_tools
pyproject.toml	+3/−3	Widen weaviate-client constraint, pin mcp>=1.20.0
uv.lock	+460/−296	Lock file regeneration from dependency changes

Test summary

Test Area	Count
stdio_mcp_tools (basic, include, exclude, conflict, timeout, unknown names)	6
http_mcp_tools (basic, include, exclude, unknown names)	4
Existing convert_mcp_tool (unchanged)	1

Docs updates

• docs/docs/learn/programming/mcp.md: Rewrote HTTP and stdio examples to use the new adapters. Replaced "Tool Conversion" section with "Filtering Tools" and "Timeout" sections.
• docs/docs/tutorials/mcp/index.md: Updated the airline booking tutorial to use stdio_mcp_tools, removing manual session management code.

Breaking change: mcp minimum version bumped to 1.20.0

The mcp optional dependency is pinned to >=1.20.0 (was unpinned). This is required because mcp 1.20.0 renamed the streamable-HTTP client import:

• Before (mcp<1.20): from mcp.client.streamable_http import streamablehttp_client
• After (mcp>=1.20): from mcp.client.streamable_http import streamable_http_client

http_mcp_tools uses the new streamable_http_client name, so older mcp versions will raise ImportError. Users on older versions will need to upgrade: pip install --upgrade "mcp>=1.20.0". The stdio_mcp_tools adapter is unaffected by this rename.

Implementation notes

• All changes additive — convert_mcp_tool and Tool.from_mcp_tool remain available for advanced use cases
• Lazy imports: mcp dependencies are imported inside the context managers, so the module loads without mcp installed
• include_tools and exclude_tools are mutually exclusive (raises ValueError if both specified)
• Unknown tool names in either filter raise ValueError with the available tool set listed in the message
• Uses is not None checks (not truthiness) so empty lists are handled correctly

🤖 Generated with Claude Code

---

PR format inspired by #9295.

[chatgpt-codex-connector]

To use Codex here, create a Codex account and connect to github.

[ahmeshaf]

@codex can you review this

[chatgpt-codex-connector]

To use Codex here, create a Codex account and connect to github.