Repository navigation
test: cover URL elicitation and user actions through the gateway - #7279
Draft
yashwant938 wants to merge 3308 commits into
Draft
yashwant938 wants to merge 3308 commits into
yashwant938 wants to merge 3308 commits into
Conversation
* fix: use opaque dataplane subject ids Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com> * fix: resolve uuid jwt subjects for auth contexts Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com> * fix: preserve public-only jwt fallback Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com> * fix: resolve uuid subjects in auth checks Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com> * test: cover uuid subject ci paths Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com> * fix: resolve reverse proxy uuid subject ownership Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com> * fix: use canonical jwt email resolution Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com> * docs: clarify opaque jwt subjects Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com> * test: align streamable uuid subject resolver coverage Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com> * fix: resolve ci lint failures Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com> * docs: clarify api token subject compatibility Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com> * fix: add missing docstring to resolve_uuid_subject in streamable transport interrogate --fail-under 100 blocked merge queue; sibling copies in auth.py, verify_credentials.py, and token_scoping.py already had it. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> --------- Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com> Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> Co-authored-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
The OAuth Dynamic Client Registration management endpoints gated only on the `is_admin` flag and ignored the `token_teams` narrowing carried by scoped tokens. A team-narrowed admin API token could therefore list, read, and delete globally-stored `RegisteredOAuthClient` records outside its intended team scope. Registered clients have no team column - their only tenancy anchor is the `gateway_id` foreign key - so a narrowed admin has no coherent scope over them. Scoping via gateway visibility is not viable either, since `Gateway.visibility` defaults to `public` and the gateway-access helpers short-circuit on that, which would have left the endpoints effectively unprotected on a default deployment. Add `_require_unnarrowed_admin()`, which runs the existing admin check and then rejects any caller whose scope resolves to a team list, whether narrowed or public-only. Scope resolution goes through the existing fail-closed `_resolve_token_teams_for_scope_check()` rather than reading `request.state.token_teams` directly, which reads as un-narrowed whenever that attribute was never set. Only API and legacy tokens carrying an explicit `teams` array are affected. Admin session tokens resolve their teams from the database and cannot be narrowed, so interactive Admin UI access is unchanged. Closes GHSA-gj7g-7r6g-jc8v Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> Co-authored-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
* fix: harden REST tool outbound validation Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com> * test: align REST validation coverage Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com> * test: cover outbound validation branches Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com> * fix: tighten REST URL pinning validation Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com> * fix: isolate REST URL pinning requests Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com> * test: align REST invocation mocks with isolated client Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com> * fix: block shared address space in URL validation Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com> --------- Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com> Co-authored-by: Lang-Akshay <akshay.shinde26@ibm.com>
…zer (IBM#5961) * fix: generate strong per-run JWT secret in plugin integration harness The plugin E2E harness defaulted JWT_SECRET_KEY to 'my-test-key-but-now-longer-than-32-bytes', which is now on the known-weak blocklist rejected by Settings validation in every environment — both the test gateway and the pytest legs failed at startup/import. - run_plugin_tests.sh: generate a strong random secret per run via 'openssl rand -base64 48' when none is supplied; gateway and pytest share the same value so tokens stay consistent within a run. - Makefile.plugin-integration: drop the weak default so the runner fallback applies; explicit JWT_SECRET_KEY overrides still work. - Stale-gateway cleanup now also kills uvicorn-launched gateways (uvicorn mcpgateway.main:app), which previously kept the port and fooled wait_for_health into a false 'gateway is up' while the real test gateway was crashing. Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> * refactor(plugins): replace in-tree sql_sanitizer with cpex-sql-sanitizer package Consume the Rust-backed SQL sanitizer from the published cpex-sql-sanitizer package instead of the in-tree Python implementation, matching how PIIFilter, SecretsDetection and the other managed plugins are wired. - Add cpex-sql-sanitizer>=0.1.0 to the [plugins] extra and pin its exclude-newer-package cutoff to 2026-07-21. The pin is required, not cosmetic: exclude-newer = "10 days" would otherwise resolve the only release away and the gateway would fail to load SQLSanitizer at boot. - Repoint kind to cpex_sql_sanitizer.sql_sanitizer.SQLSanitizerPlugin in plugins/config.yaml, config-pii-guardian-policy.yaml, the performance config and the init-hooks test fixture. - Remove plugins/sql_sanitizer/ and its unit test. The unit test only exercised plugin internals (nested traversal, fields: None), which the crate covers upstream; the gateway-facing contract is covered by tests/live_gateway/plugins/test_sql_sanitizer_e2e.py. - Drop the now-dead plugins/sql_sanitizer/** CI path triggers and correct the "native plugin, not a cpex wheel" comments. Behaviour was compared against the removed Python implementation across 530 cases (10 configs x 30 argument shapes x both hooks): no divergence in block/allow decisions, modified payloads, or violation reason/description/ code. Three diagnostic-only differences are documented inline in plugins/config.yaml: - issue strings no longer carry the "<field>: " prefix - monitoring mode now reports sql_issues alongside sql_sanitized - require_parameterization (off by default) masks quoted string literals before its interpolation heuristic, removing a false positive Verified: make test-plugin-sql-sanitizer passes 2/2 (static + binding); test_init_hooks_plugins.py 104 passed; plugins config validation passes. Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> * chore(tests): allowlist detect-secrets false positive on generated JWT secret The plugin E2E harness derives its per-run secret from an openssl call: JWT_SECRET="${JWT_SECRET_KEY:-$(openssl rand -base64 48 | tr -d '\n')}" detect-secrets' Secret Keyword heuristic fires on the JWT_SECRET= prefix even though the value is a runtime command substitution, not a literal. Suppress it at source with a pragma rather than carrying an audited baseline entry. make detect-secrets-scan now reports no unaudited, live, or real secrets with .secrets.baseline unchanged. Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> --------- Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com>
…dpoints (IBM#5959) (IBM#5979) * docs: document CSRF protection settings (IBM#5959) Add comprehensive CSRF Protection section to configuration.md documenting: - All 12 CSRF_* environment variables with code defaults and descriptions - Two independent CSRF implementations (middleware vs. admin routes) - Comparison table showing which settings apply where - Four admonitions for critical operator traps - Cross-link to middleware-ordering.md architecture doc - Default exempt paths reference Addresses findings IBM#12 and IBM#13 from issue IBM#5959. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * fix: correct CSRF documentation - enforce_admin_csrf scope and exempt paths list Fix two important findings from task review: 1. enforce_admin_csrf applies to BOTH legacy (/admin/*, /admin/llm/*) AND versioned (/v1/admin/*, /v1/admin/llm/*) mounts, not just legacy. Updated the 'Two Independent CSRF Implementations' overview to clarify this matches the later admonition about double validation on /v1/admin/*. 2. Default Exempt Paths JSON was incomplete, missing 6 entries: - /auth/email/login, /auth/email/register - /auth/email/forgot-password, /auth/email/reset-password - /admin/forgot-password, /admin/reset-password Expanded to full 28-entry list from mcpgateway/config.py:418-448. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * docs: document middleware execution ordering (IBM#5959) Create docs/docs/architecture/middleware-ordering.md to document: - Starlette's reverse-registration rule with worked example - Complete ordered middleware stack from main.py, including DBQueryLoggingMiddleware and the DeprecationHeadersMiddleware outlier registered after routers - CSRF-before-auth ordering constraint (issue IBM#5739, fixed in IBM#5780) - PasswordChangeEnforcementMiddleware same-shape dependency - Guidance for adding new middleware with regression test pattern Register page in .pages nav next to Security Features, and add cross-link from security-features.md to explain the CSRF/auth ordering constraint that prevents Admin UI write failures. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * docs: document LLM Settings Management endpoints and CSRF schemes (IBM#5959) Add comprehensive section to api-usage.md covering both route families (/v1/llm/* and /v1/admin/llm/*), CSRF protection schemes, and curl examples for bearer-token and cookie+X-CSRF-Token authentication. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * docs: fix LLM Settings Management documentation (5 reviewer findings) Fix the following issues in the LLM Settings Management section: 1. Correct CSRFMiddleware origin check: use CSRF_CHECK_REFERER setting + CSRF_TRUSTED_ORIGINS (not settings.allowed_origins) 2. Clarify that CSRF only applies to write operations; GET/HEAD/OPTIONS are safe methods and skip CSRF validation 3. Remove jq pipes from 4 HTML-returning endpoints (set_provider_state, delete_provider, set_model_state, delete_model) and replace JSON response blocks with descriptions 4. Fix /test endpoint example: accepts JSON only, not form-encoded; provide correct cookie+CSRF JSON example 5. Add per-path CSRF validation matrix distinguishing legacy /admin/llm/* (1 scheme, middleware-exempt) from /v1/admin/llm/* (2 schemes, both apply) Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * test: add wrong-session_id CSRF binding coverage (IBM#5959) Add test_csrf_token_bound_to_wrong_session_id_fails_validation to verify CSRFMiddleware rejects tokens when session_id (jti) doesn't match, even with the correct email. Also add test_validate_csrf_token_session_id_binding_differs to unit-test the CSRFService binding behavior directly. Complements existing test_csrf_token_bound_to_user_id_fails_middleware_validation by testing the other half of the (user_id, session_id) HMAC binding pair. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * test: expand JS missing-CSRF-cookie coverage to all 11 write sites (IBM#5959) Add the 3 missing imports (syncLLMProviderModels, fetchModelsForModelModal, toggleLLMModel) and rewrite the missing-cookie describe block as a table-driven test.each over all 11 llmModels.js write call sites, closing the coverage gap IBM#5739 originally enumerated. Update the header docstring to reflect that all 11 sites are now covered for the no-cookie regression. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * test: add e2e CSRF coverage for /admin/llm/* enforce_admin_csrf path (IBM#5959) Adds real-request tests against enforce_admin_csrf (admin.py), the second, independent CSRF implementation guarding /admin/llm/* and /v1/admin/llm/* routes -- distinct from CSRFMiddleware, which the existing e2e tests in this file already exercise via /llm/providers. Covers the full deny-path matrix (missing header, header/cookie mismatch, missing cookie, bad origin, form-encoded fallback), the happy path, and the bearer-token no-session-cookie bypass. Also pins the versioned-mount divergence: /admin/llm/* is exempt from CSRFMiddleware (its /admin prefix match) while /v1/admin/llm/* is not, so the latter is checked by both CSRFMiddleware and enforce_admin_csrf. In the window between /admin/login and the first dashboard load, the CSRF cookie is an opaque (non-HMAC) placeholder that satisfies enforce_admin_csrf's plain double-submit check but fails CSRFMiddleware's HMAC validation -- so the exact same session's write to /admin/llm/* succeeds while the identical write to /v1/admin/llm/* 403s. A dedicated test pins this divergence rather than asserting the two mounts agree. Extends the shared _main_app_with_llm_routes/e2e_client fixtures to also mount llm_admin_router under /v1/admin/llm (mirroring production's dual mount in mcpgateway/api/v1/__init__.py) and to override the deprecated rbac.get_db dependency that llm_admin_router.py uses instead of mcpgateway.db.get_db -- without it, /admin/llm/* requests silently bypass the isolated test database. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * test: add expired (two-windows-old) CSRF token rejection coverage (IBM#5959) Add test_two_windows_old_token_returns_false in test_csrf_service.py and the middleware-level equivalent asserting 403 CSRF_TOKEN_INVALID, using deterministic mid-window timestamps to pin the exact window distance. Also fix test_httponly_always_false, which only proved settings.csrf_cookie_httponly was echoed with False and never exercised True; rename it and add a companion case with csrf_cookie_httponly=True. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * test: pin CSRF cookie-name consistency across JS and admin constants (IBM#5959) llmModels.js hardcodes the mcpgateway_csrf_token cookie name and admin.py/oauth_router.py each hardcode a duplicate ADMIN_CSRF_COOKIE_NAME constant, none of which track settings.csrf_cookie_name. Pin all of these against the config default so any future drift (the class of bug behind IBM#5739 and IBM#4712) fails a test instead of silently breaking CSRF validation. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * fix: correct final-review findings across CSRF docs and tests (IBM#5959) Cross-task cleanup for the CSRF docs+tests chore closing IBM#5959: - api-usage.md: stop recommending the versioned /v1/admin/llm/* mount as strictly preferable and document the known post-login CSRF timing gap between it and the legacy /admin/llm/* mount (IBM#5978), including that the Admin UI itself only calls the legacy mount. - configuration.md: same IBM#5978 note on the CSRF_EXEMPT_PATHS admonition; reword the CSRF_COOKIE_NAME warning to correctly identify the middleware as reading the setting (not hardcoding it) and list all three actual hardcoded copies, including oauth_router.py; correct the CSRF_TRUSTED_ORIGINS admonition to cite .env.example's actual values. - middleware-ordering.md: remove the Execution Order column, which mislabeled both row 1 (CORSMiddleware) and row 27 (DeprecationHeadersMiddleware) as "Outermost" and asserted AuthContextMiddleware was the innermost of rows 18-20 when it is actually the outermost of that group; replace with verified prose covering the same ordering facts. Also de-confuse the self-link in the "adding new middleware" guidance. - test_admin_csrf_binding.py: add a docstring reference to IBM#5978 on the divergence-pinning test, and add a test covering a present but non-matching Origin header (previously only the missing-header case was covered, which short-circuits before the allowed_origins fallback branch). Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * fix: correct broken in-page anchor in middleware-ordering.md The link to the State Dependencies section used a double hyphen (#state-dependencies-the-csrf--auth-middleware-ordering-bug), but python-markdown's default heading slugifier collapses the '+' in 'CSRF + Auth' and the surrounding whitespace to a single hyphen, producing #state-dependencies-the-csrf-auth-middleware-ordering-bug. Verified against the slugify algorithm directly. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * fix: reword to avoid CI's AI-transition-phrase lint on 'remember that' The repo's pre-commit warn-ai-transitions hook flags 'remember that' among other stock phrases. Reworded two sentences in middleware-ordering.md without changing their meaning. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * chore: trim trailing whitespace and regenerate secrets baseline Mechanical fixes from running pre-commit locally: trailing-whitespace hook auto-fixed middleware-ordering.md, and .secrets.baseline needed its usual line-number regeneration after the doc edits (no new secrets, timestamp + line-number churn only). Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * chore: regenerate secrets baseline (line-number drift from full-repo scan) Prior local pre-commit runs were scoped to specific touched files and missed line-number drift in docs/docs/architecture/security-features.md (this branch added a cross-link there in an earlier commit, shifting line numbers for pre-existing, already-audited baseline entries below it). A full-repo detect-secrets-scan against origin/main catches this; no new secrets, timestamp and line-number churn only. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * docs: fold in the third CSRF implementation and fix an incomplete file list Address PR review feedback on IBM#5979: - configuration.md and api-usage.md both framed CSRF protection as two independent implementations. There is a third: enforce_fetch_tools_csrf (oauth_router.py), which guards POST /oauth/fetch-tools/{gateway_id} with its own duplicated ADMIN_CSRF_COOKIE_NAME/ADMIN_CSRF_HEADER_NAME constants and its own origin-check logic, independent of both CSRFMiddleware and enforce_admin_csrf. This is why /oauth/fetch-tools is in CSRF_EXEMPT_PATHS. - The CSRF_COOKIE_NAME warning enumerated three hardcoded-copy locations, but there are 11: admin.py, oauth_router.py, four Admin UI JS files (auth.js, formHandlers.js, utils.js, llmModels.js), and five server-rendered templates (admin.html, login.html, forgot-password.html, reset-password.html, change-password-required.html). Reworded to state the setting only governs CSRFMiddleware and every other consumer hardcodes the name, rather than an enumerated list that will drift as consumers are added or removed. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * chore: regenerate secrets baseline after rebase Mechanical line-number/timestamp churn from the rebase onto latest origin/main; no new secrets. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * docs: narrow HTTPONLY warning scope, fix 404 wording, add pagination fields Address CHANGES_REQUESTED review feedback on IBM#5979: - CSRF_COOKIE_HTTPONLY warning claimed setting=true 'breaks every CSRF-protected write'. csrf_cookie_httponly is only read by CSRFMiddleware's set_csrf_cookie() (csrf_service.py:258) -- admin route CSRF (enforce_admin_csrf, enforce_fetch_tools_csrf) hardcodes httponly=False unconditionally, unaffected by this setting. Narrowed the claim to CSRFMiddleware-protected writes specifically. - LLM Settings intro said feature-disabled routes 'return 404 Not Found', implying an active feature-gate check. The LLM routers are actually never mounted when LLMCHAT_ENABLED=false (api/v1/__init__.py's _assemble_routers skips the include_router calls entirely) -- the 404 comes from the route simply not existing. Reworded to state the mechanism accurately. - List LLM Providers/Models response examples omitted the total/page/ page_size fields that LLMProviderListResponse and LLMModelListResponse (llm_schemas.py) actually include. Added them. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * chore: regenerate secrets baseline after rebase Mechanical line-number/timestamp churn from the rebase onto latest origin/main (IBM#5961, IBM#5925); no new secrets. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> --------- Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> Co-authored-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
Downgrade Vite to go back to Rollup rather than Rolldown for s390x compatibility Signed-off-by: Gabriel Costa <gabrielcg@proton.me>
…M#5997) * fix: remove Python comment from JavaScript code in OAuth callback The OAuth callback endpoint was embedding a Python pragma comment (# pragma: allowlist secret) inside JavaScript code at line 838. This invalid syntax prevented the JavaScript fetch() call from executing, causing the 'Fetch Tools from MCP Server' button to silently fail after OAuth authorization. Removed the Python comment to fix JavaScript parsing. The button should now properly trigger the POST request to /oauth/fetch-tools. Signed-off-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com> * update .secrets.baseline Signed-off-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com> --------- Signed-off-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com> Co-authored-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com>
…5924) * feat(dataplane): publish per-backend add_headers/remove_headers The dataplane consumer (contextforge-gateway-rs IBM#4544) now accepts per-backend `add_headers` (map, overrides passthrough) and `remove_headers` (list, applied last) alongside the existing `passthrough_headers`. Thread both fields through the publisher so the published BackendConfig matches the consumer's contract. Sourced via getattr with the consumer-side defaults ({} / []) since the Gateway model has no columns for these yet; the publisher lights up automatically once the columns are added, with no further change here. Signed-off-by: Lang-Akshay <akshay.shinde26@ibm.com> * feat: added imgration and updated tests Signed-off-by: Lang-Akshay <akshay.shinde26@ibm.com> --------- Signed-off-by: Lang-Akshay <akshay.shinde26@ibm.com>
Align domain blocklist evaluation with the host used for the connection. Canonicalize both the parsed host and the configured entries through one helper so equivalent spellings compare equal: percent-encoding, case, IPv6 brackets, trailing dots, uncompressed address literals, and Unicode versus punycode domains. Add regression coverage. Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com>
Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com>
…BM#5631) * feat(observability): improve upstream MCP session error diagnostics Enhance error handling in upstream_session_registry to provide actionable, specific error messages when upstream MCP session creation fails. Replaces generic 'unhandled errors in a TaskGroup' message with categorized errors. Key improvements: - Unwrap ExceptionGroup before string conversion to preserve root cause - Categorize errors into 13 distinct types (connection_refused, timeout, ssl_tls, auth_unauthorized, auth_forbidden, dns_resolution, etc.) - Add structured logging with error_category, exception_type, and metadata for correlation across log aggregation systems - Include full traceback via exc_info for deep diagnosis - Surface error category in RuntimeError message for user visibility This enables operators to quickly identify whether failures are due to: - Network issues (connection refused/reset, DNS, timeouts) - Authentication problems (401/403) - SSL/TLS certificate issues - Upstream server errors (5xx) Error messages now display as: [connection_refused] ConnectionRefusedError: Connection refused [auth_unauthorized] HTTPStatusError: 401 Unauthorized [ssl_tls] SSLError: certificate verify failed [timeout] TimeoutError: Session initialization timeout Benefits: - Faster MTTR for production incidents - Actionable error messages without requiring verbose logging - Consistent error handling across MCP session modes - Better alerting and monitoring capabilities Testing: - All 68 existing upstream_session_registry tests pass - 10 new tests validate error categorization for all failure modes - Backward compatible (still raises RuntimeError) Closes IBM#5608 Signed-off-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com> * test(observability): improve structured logger exception handling test coverage Enhance test_structured_logger_exception_handling to ensure lines 462-465 are covered by properly mocking get_structured_logger at the module level where it's imported. The test now verifies that: - The structured logger is actually invoked (confirming except block is hit) - Primary error message is preserved when structured logging fails - Structured logger failures don't disrupt the main error flow Also fix f-string formatting in upstream_session_registry.py per ruff format. This brings coverage of the structured logging exception handler to 100%. Signed-off-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com> * update .secrets.baseline Signed-off-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com> * update .secrets.baseline Signed-off-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com> * feat(observability): address code review findings for upstream session error diagnostics Address all blocking issues and warnings from code review: 1. **Type Safety Enhancement** (Issue IBM#5) - Improve type: ignore comment with detailed rationale - Explain why type checker cannot infer concrete Exception after unwrapping - Lines: upstream_session_registry.py:387-391 2. **Logging Diagnostics Test Coverage** (Issue IBM#2) - Add test_logger_error_call_with_exc_info: verify logger.error called with exc_info - Add test_structured_logger_metadata_payload: verify structured logger metadata - Tests validate both standard logging (with traceback) and structured logging paths - Lines: test_upstream_session_error_categories.py:459-543 3. **Cross-Layer Consistency Regression Test** (Issue IBM#3) - Add test_cross_layer_error_message_consistency - Verify registry RuntimeError surfaces categorized text through consuming layer - Ensures fix remains effective across error propagation chain - Lines: test_upstream_session_error_categories.py:547-599 4. **Feature Documentation** (Issue IBM#1 - Blocking) - Add comprehensive "Upstream Session Error Diagnostics" section to observability-otel.md - Document all 13 error categories with descriptions and common causes - Include error message format examples - Provide structured logging metadata schema - Add monitoring/alerting examples (Prometheus, Datadog, Splunk) - Explain ExceptionGroup unwrapping behavior - Lines: observability-otel.md:651-756 All tests pass (18/18). Code review findings fully addressed. Related: IBM#5608 Signed-off-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com> * fix(observability): address code review findings for upstream session error diagnostics This commit implements all feedback from code review, addressing both blocking issues and suggested improvements to the upstream MCP session error diagnostics enhancement. ## Blocking Issues Fixed ### 1. Credential Sanitization (Security) - **Problem**: Exception messages with URLs containing secrets (API keys, tokens) were flowing unsanitized to client-facing RuntimeError messages, logs, and structured logging sinks - **Fix**: Use `sanitize_exception_message()` in `_categorize_upstream_error()` to redact sensitive query params before returning sanitized message - **Coverage**: Added regression tests for API key and Bearer token redaction ### 2. httpx Exception Type Categorization - **Problem**: Real httpx timeout/connection exceptions (httpx.ConnectTimeout, httpx.ReadTimeout, httpx.ConnectError) fell through to 'unknown' category - **Fix**: Check `isinstance(root_cause, httpx.TimeoutException)` to catch all httpx timeout types; handle httpx.ConnectError with message inspection for 'refused' vs generic connection errors - **Coverage**: Added regression tests for httpx.ConnectTimeout, httpx.ReadTimeout, and httpx.ConnectError with/without "refused" message ## Improvements Implemented ### Refactoring - Extracted error categorization logic into pure function `_categorize_upstream_error()` - Returns: (error_category, exception_type, sanitized_message, exception_count) - Makes taxonomy testable as pure function (no async task/transport mocking needed) - Enables future code reuse by tool_service.py for consistency ### Structured Logging Enhancements - Added `correlation_id` from request context for cross-layer correlation - Changed `metadata={...}` to `error_details={...}` for consistency with tool_service.py - `error_details` maps to dedicated column; `metadata` contains non-error context - Added debug logging for structured logger failures (was silent `except: pass`) ### Error Category Additions - Added `mcp_protocol_error` category for McpError (failed session.initialize()) - Tightened `ssl.SSLError` check to use isinstance() before string matching - Fixed httpx.ConnectError to categorize as 'connection_refused' when message contains "refused" ### Log Level Handling - Downgraded post-ready errors (teardown races) to WARNING level - Pre-ready failures remain ERROR (blocks session creation) - Only ERROR-level failures trigger structured logging (reduces alert noise) ### Exception Group Metadata - Track and log `exception_count` when BaseExceptionGroup contains multiple exceptions - Append " (N exceptions in group)" to log messages when count > 1 - Include `exception_count` in structured logging `error_details` ### Documentation Updates - Removed obsolete `MCP_SESSION_POOL_ENABLED` references from observability-otel.md - Updated to describe actual trigger: Mcp-Session-Id header presence - Added `mcp_protocol_error` to error categories table ## Test Coverage ### New Regression Tests (9 added) 1. `test_httpx_connect_timeout_category` - httpx.ConnectTimeout → timeout 2. `test_httpx_read_timeout_category` - httpx.ReadTimeout → timeout 3. `test_httpx_connect_error_with_refused_message` - "refused" → connection_refused 4. `test_httpx_connect_error_generic` - no "refused" → connection_error 5. `test_credential_sanitization_in_http_error` - API key redaction 6. `test_credential_sanitization_with_bearer_token` - Bearer token redaction 7. `test_mcp_protocol_error_category` - McpError → mcp_protocol_error 8. `test_ssl_error_category_with_isinstance_check` - ssl.SSLError via isinstance 9. `test_exception_group_with_multiple_exceptions_logged` - exception_count > 1 ### Modified Tests (2 updated) - `test_structured_logger_metadata_payload` - validates error_details structure - `test_post_ready_error_is_warning_level` - documents WARNING downgrade behavior ### Test Results - All 28 tests in test_upstream_session_error_categories.py pass - All 68 tests in test_upstream_session_registry.py pass (no regressions) - Total: 96 tests passing ## Security Impact **CRITICAL**: This fix prevents credential disclosure that was introduced by the original PR. Before this fix, URLs with secrets in query params (e.g., `?apiKey=secret123`) would leak to client-facing error messages. The sanitization now redacts all sensitive query params using static fallback patterns (api_key, token, password, etc.) and supports gateway-specific param names when available. ## Implementation Notes ### auth_query_params Threading Decision The review suggested threading gateway `auth_query_params_decrypted` through SessionCreateRequest for full credential redaction. However: - Static fallback in `sanitize_exception_message()` already covers common cases (api_key, token, password, Bearer tokens, etc.) - Threading through would require larger changes (SessionCreateRequest fields, all call sites, decryption at registry level) - Current implementation documents this trade-off in code comments ### Story 2 Consistency (tool_service.py) The refactored `_categorize_upstream_error()` function is now available for tool_service.py to call in a future PR, which would deliver full consistency across both session paths. This PR focuses on the registry path where the issue was reported. Closes feedback items from code review on IBM#5608 Signed-off-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com> * fix linter issues Signed-off-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com> * fix(upstream): handle real-world httpx exception chains and timeout paths Fixes three blocking issues in upstream session error categorization identified during real-socket e2e verification: 1. **connection_refused** detection: httpx.ConnectError wraps ConnectionRefusedError deep in __context__/__cause__ chain. Added _find_in_chain() helper to unwrap exception chains so real refused connections produce correct category instead of generic connection_error. 2. **ssl_tls** detection: Moved ssl.SSLError check to unwrap exception chains (httpx.ConnectError wrapping SSLError) and added fallback chain walk at end of categorization logic. 3. **timeout** at asyncio.wait_for call site: Owner task receives CancelledError (BaseException, excluded from except Exception), so asyncio.wait_for timeout never hit the owner's exception handler. Added explicit except asyncio.TimeoutError block at call site with categorization, sanitization, structured logging, and RuntimeError wrapping to match owner-task error path. 4. **Credential leak via exc_info**: Removed exc_info=exc from all logger.error/warning calls. Python's traceback formatter renders the raw exception __str__, bypassing sanitized message strings and reintroducing credential disclosure for HTTPStatusError. Added end-to-end integration tests (test_upstream_session_error_e2e.py) that validate fixes against real TCP sockets, blackhole listeners, and HTTP servers with no mocking. Updated test expectations: - test_logger_error_call_with_exc_info renamed to test_logger_error_call_without_exc_info and flipped assertion - test_default_session_factory_cancelled_path_runs_on_ready_timeout now expects RuntimeError wrapping TimeoutError with categorization All unit tests (86) and e2e tests (4) passing. Closes IBM#5608 Signed-off-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com> * fix(tests): filter caplog records by ERROR level in test_remove_root_generic_exception The test was failing after rebase because it captured all mcpgateway logs, including DEBUG level logs from get_db and user request logging. Updated the assertion to filter caplog records to only include ERROR level logs, ensuring only the expected "Failed to remove root" error message is validated. Signed-off-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com> * fix linter issues Signed-off-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com> * test: add coverage for uncovered lines in upstream_session_registry.py Add tests to cover previously uncovered lines: - Line 311: _find_in_chain return current path - Line 329: connection_refused fallback message check - Lines 375-377: SSL error detection via _find_in_chain - Line 542: WARNING level log post-ready - Lines 665-666: structured logging exception during timeout Coverage improved from 91.7% to higher coverage for upstream session registry. Tests added: - test_categorize_upstream_error_ssl_error_in_exception_chain - test_categorize_upstream_error_connection_refused_message_fallback - test_categorize_upstream_error_find_in_chain_returns_current - test_default_session_factory_logs_warning_on_post_ready_failure - test_default_session_factory_timeout_structured_logging_failure Signed-off-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com> * fixed pre-commit Signed-off-by: Jitesh Nair <jiteshnair@ibm.com> * fixed pre-commit Signed-off-by: Jitesh Nair <jiteshnair@ibm.com> --------- Signed-off-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com> Signed-off-by: Jitesh Nair <jiteshnair@ibm.com> Co-authored-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com> Co-authored-by: Jitesh Nair <jiteshnair@ibm.com>
…o-derived validation (IBM#4476) * fix(oauth): harden audience learning with issuer pinning and shape checks Address follow-up review feedback for the audience-learning fix to make the trust model defensible end-to-end and the data path defensive against unexpected IdP responses. Production code changes: - Factor the unverified JWT decode into _decode_token_claims_unverified() with a docstring explicitly documenting the trust model: signatures and exp/iss/aud are NOT verified; this path is for non-authoritative metadata extraction only; authoritative validation happens upstream when the token is presented to the protected resource. - Add _extract_token_issuer() paralleling _extract_token_audience(); both now reuse the shared decoder. complete_authorization_code_flow returns token_iss alongside token_aud. - _persist_learned_audience() now runs two defensive checks before writing. Shape validation: token_aud must be a non-empty string or non-empty list of non-empty strings; numbers, empty containers, mixed lists, and dicts are silently dropped so a malformed IdP response cannot pollute persisted state. Issuer pinning: when oauth_config has an 'issuer' configured, the token's iss must match it -- prevents a stale or misrouted token from a different AS injecting an audience for the wrong IdP. When no issuer is configured, behavior is unchanged. - Tighten _extract_token_audience return type from Any to Optional[Union[str, List[str]]]; reject claim values whose type doesn't match (numbers, mixed-type lists, dicts). Test changes: - TestExtractTokenClaims (test_oauth_manager.py): 13 unit tests covering the audience and issuer extractors -- string, list, missing claim, unexpected types, mixed list, opaque token, malformed JWT, empty input. - TestPersistLearnedAudience (test_oauth_router.py): parametrized shape-validation test plus 4 issuer-pinning tests (no issuer configured, matching iss, mismatching iss, missing iss). - test_token_validation_service.py: 2 round-trip tests confirming an opaque learned aud (string and list) successfully validates against the persisted opaque resource (the flagship PR scenario). - test_oauth_manager.py: existing assertion on complete_authorization_code_flow result dict updated to expect the new token_iss field. Signed-off-by: Jonathan Springer <jps@s390x.com> * fix(oauth): add diagnostic logs and multi-user regression test for audience learning Final follow-up bundle from review feedback: - Diagnosability (S4): emit DEBUG breadcrumbs on each silent skip path so operators chasing 'audience never learned' reports can distinguish between the causes: * _decode_token_claims_unverified logs when an exception is caught -- distinguishes 'token was opaque' (normal) from 'JWT library raised something unexpected' (worth investigating). DEBUG level keeps it out of production logs by default but available when troubleshooting. * _persist_learned_audience logs when token_aud is malformed/absent and when the resource is already set (first-write skip). The issuer-mismatch breadcrumb already existed. - Docstring (M5): add explicit Returns: section to _persist_learned_audience documenting the side-effect-only contract (oauth_config mutation + db.flush). - Multi-user regression test (M2): explicit guard for first-write-only semantics under the multi-user scenario. Two users complete the callback against the same shared gateway with different audience values; the second user's callback must observe the already-set resource and skip its write entirely. This keeps the protection visible even if the short-circuit logic is later refactored. - Test coverage for the new DEBUG logs: parametrized shape-validation test now asserts the breadcrumb is emitted; new test confirms decode failure logs at DEBUG; complement test confirms empty-input short-circuit does NOT log (would be noise). Signed-off-by: Jonathan Springer <jps@s390x.com> * feat(oauth): expose Resource field in UI, derive origin fallback, soften auto-derived validation Implement the maintainer-recommended Option-2 hybrid approach for issue IBM#4384 on top of the auto-learn fix: UI: OAuth Resource (Audience) field - Add 'Resource (Audience)' input next to Issuer URL in all four OAuth configuration forms (gateway create + edit, A2A agent create + edit) in templates/admin.html. - Wire up form parsing in admin.py to extract oauth_resource and merge it into oauth_config['resource']; supports comma-separated multi-value input per RFC 8707, stored as str when one value or list when multiple. - Pre-populate the field on edit dialogs from gateway.oauthConfig.resource in admin_ui/gateways.js and admin_ui/a2aAgents.js (handles both string and list shapes). Origin-extraction fallback (replaces full-URL fallback) - Add _derive_resource_origin() in oauth_router.py: returns scheme+netloc only, dropping path/query/fragment. Returns None for URN / scheme-less inputs so the caller falls back to auto-learn. - /oauth/authorize and /oauth/callback now use this helper instead of the full-path _normalize_resource_url(gateway.url) when no resource is configured. This matches real OAuth provider behavior (Salesforce, Azure AD, Okta typically issue origin-level audiences). Softened validation for auto-derived resources - TokenValidationResult gains audience_authoritative: bool = True. When False, audience mismatch is dropped from blocking_errors so the token still forwards (the upstream MCP server is the authoritative validator). - _validate_audience() sets audience_authoritative = bool(configured_resource) on mismatch. Configured (admin-set or learned) -> blocking; gateway-URL fallback -> advisory. Tests - Origin-extraction parametrized test (5 hierarchical URLs + 6 non-URL inputs) covers the new helper. - Validator authoritative/advisory split: 4 new tests on TokenValidationResult.blocking_errors and validate_oauth_token_claims covering configured-resource (blocking) and no-resource fallback (advisory) paths. Closes the remaining IBM#4384 acceptance criteria gaps: - 'OAuth Resource field is visible in Admin UI' - 'Auto-derives origin if not set' - 'Auto-derived resource logs warnings but doesn't block' - 'Existing gateways without resource field continue working' Signed-off-by: Jonathan Springer <jps@s390x.com> * fix(oauth): address PR IBM#4476 review findings from msureshkumar88 Blocking findings: - admin.py: Add Gateway form Resource field was a no-op — extract shared _parse_oauth_resource() helper (URI-aware, comma-in-URI safe) and wire it into _parse_gateway_data_from_request plus the three edit sites, eliminating a 9-line duplicated block copy-pasted 3x. - templates/admin.html: A2A edit dialog Discover button lost status feedback after the Resource-field restructuring — move the oauth-discover-status div back into the Issuer wrapper. Hardening (deep review): - config.py + token_validation_service.py: Add OAUTH_REQUIRE_CONFIGURED_RESOURCE setting (default false). - encryption_service.py: Add _OAUTH_CONFIG_RACE_PRONE_KEYS backfill so a stale UI save that omits resource does not silently wipe a callback-learned audience. Cleanup: - oauth_router.py: Remove dead _normalize_resource_url() (zero production call sites) plus its urlunparse import; delete both associated test classes. - oauth_manager.py: Consolidate JWT decode into _extract_aud_and_iss() single-decode helper. Tests: - test_gateway_service.py: TestFetchToolsAfterOAuthEnforcementPoint (4). - test_admin.py: TestParseOAuthResource + TestParseGatewayDataOAuthResource (20). - test_encryption_service.py: TestOAuthConfigRaceProneKeyBackfill (7). - test_token_validation_service.py: TestOAuthRequireConfiguredResourceSetting (4). Documentation: - templates/admin.html: Rewrite Resource-field help text to reflect the race-safe UI behavior. Signed-off-by: Jonathan Springer <jps@s390x.com> * docs(oauth): audience handling, resource configuration, and troubleshooting Absorb PR IBM#4477's docs onto this branch, with one accuracy correction to reflect the race-safe UI behavior added in the preceding commit. New file: - docs/docs/manage/oauth-resource-configuration.md — end-user guide to the RFC 8707 resource / audience field with provider patterns, multi- resource configs, and the API-based re-learn workflow. architecture/oauth-design.md: - Add 'Audience handling and RFC 8707' section documenting three-stage resolution, first-write-only + shape validation + issuer pinning, authoritative vs. advisory split, OAUTH_REQUIRE_CONFIGURED_RESOURCE escape hatch, race-safe backfill, and implementation references. - Remove obsolete 'Known Gap: No local audience/scope pre-validation' admonition. - Update 'Forwarded as-is' bullet to describe actual pre-validation. manage/oauth-troubleshooting.md: - Add 'Token audience mismatch' section: symptom, diagnosis, common causes and fixes, debug logs, gateway vs. upstream responsibility. - Cross-link to oauth-resource-configuration.md. manage/oauth.md: - Add Resource field to Admin UI field list with cross-link. - Add tip admonition explaining zero-config default. - Add 'See also' pointer in existing Resource Parameter section. manage/.pages: - Wire oauth-resource-configuration.md into nav. Accuracy correction vs IBM#4477: 'Forcing a Re-learn' via UI-clear no longer applies after the encryption backfill. Re-learn now requires a PUT with explicit resource: null. Applied consistently across all three touched files. Function/log/setting/error-string references verified against code. Signed-off-by: Jonathan Springer <jps@s390x.com> * fix(oauth): per-user learned audience (MEDIUM security fix) Reviewer's MEDIUM finding: the OAuth callback path enforces gateway access but not gateways.update, so allowing every authenticated callback to write shared gateway.oauth_config.resource let any user with gateway access mutate global state on behalf of all users, AND created a cross-tenant DoS in multi-tenant IdP setups where whichever user authenticated first pinned the audience for everyone else. Redesign: move callback-driven audience persistence from shared gateway.oauth_config.resource to per-user OAuthToken.learned_aud / learned_iss columns. Each user's OAuth callback only writes to their own row, and each user's tool-fetch validation only reads their own learned value. Schema: - Add learned_aud (JSON) and learned_iss (String) to OAuthToken - Alembic migration c9f8e7d6a4b3 (idempotent) Write path: - TokenStorageService.store_tokens accepts learned_aud/learned_iss; None-valued args preserve existing values. - OAuthManager.complete_authorization_code_flow forwards _extract_aud_and_iss output to store_tokens. - oauth_router.oauth_callback no longer calls _persist_learned_audience. Read path: - TokenStorageService.get_user_learned_audience returns tuple. - gateway_service.fetch_tools_after_oauth passes learned_aud to validator. - token_validation_service._validate_audience precedence: 1. oauth_config.resource (admin) — authoritative 2. learned_aud (this user) — authoritative for this user 3. gateway_url origin — advisory (or authoritative when OAUTH_REQUIRE_CONFIGURED_RESOURCE=true) Removed: _persist_learned_audience, _is_well_formed_audience, _OAUTH_CONFIG_RACE_PRONE_KEYS backfill, TestPersistLearnedAudience, TestOAuthConfigRaceProneKeyBackfill. Tests (2429 pass): TestLearnedAudiencePersistence (7), TestLearnedAudPrecedence (6), 3 new per-user enforcement tests. Docs: oauth-design.md, oauth-resource-configuration.md, oauth-troubleshooting.md, oauth.md, admin.html rewritten for per-user model. Signed-off-by: Jonathan Springer <jps@s390x.com> * fix(oauth): address 2nd-review MEDIUM + hardening (5 fixes) Follow-up to the per-user learned audience redesign (commit 45615752d). The 2nd-round review found the same class of RBAC-bypass bug persisted in a different location (the DCR persist path) plus 3 quality items and a migration docstring nit. MEDIUM — DCR path leaked auto-derived resource to shared config: - oauth_router.initiate_oauth_flow (DCR block) previously persisted the whole request-local oauth_config dict (including the auto-derived resource) to gateway.oauth_config via protect_oauth_config_for_storage. The /oauth/authorize/{gateway_id} route enforces gateway access but not gateways.update, so an authenticated caller could pin the shared audience for all users via origin derivation — the exact RBAC-bypass class the callback-path redesign eliminated by moving learned aud to OAuthToken.learned_aud. - Fix: build a persist_dict that strips the request-local 'resource' before writing to shared config. Admin-configured resource is preserved as-is; request-derived origin stays request-local only. - Regression tests (2 new): - test_initiate_oauth_flow_dcr_does_not_persist_request_local_resource: proves 'resource' is NOT in gateway.oauth_config after DCR. - test_initiate_oauth_flow_dcr_preserves_admin_configured_resource: proves an admin's explicit resource survives DCR persistence. LOW — decode-exception log echoed attacker-controlled parser output: - _decode_token_claims_unverified logged 'Unverified JWT decode failed (%s): %s' with type(exc).__name__ and str(exc). Malformed IdP tokens can produce parser exceptions whose string form includes token fragments. - Fix: log only 'Unverified JWT decode failed: %s' with the class name; drop the exception message text. MINOR — _coerce_aud_claim accepted empty aud shapes: - Empty string and empty list passed through _coerce_aud_claim as-is, which combined with the 'if learned_aud is not None' guard in store_tokens could silently overwrite a previously-learned per-user audience with garbage from a malformed IdP response. - Fix: reject empty strings and empty/whitespace-only-list shapes; return None so the store_tokens guard preserves existing state. Mirrors the removed _is_well_formed_audience semantics from the per-user redesign. MINOR — stale comments in oauth_router.py: - 3 sites (docstring of _derive_resource_origin + comments in initiate_oauth_flow and oauth_callback) still described 'the auto-learn callback will overwrite oauth_config.resource on the first successful flow'. That flow was removed; the callback now writes OAuthToken.learned_aud per-user. - Fix: reword to describe the actual behavior — request-local outbound derivation, per-user learned inbound validation via OAuthToken — and add pointers to the DCR strip logic + validator. NITPICK — migration downgrade docstring: - Said 'callback-driven persistence reverts to writing gateway.oauth_config.resource on downgrade' — but the downgrade only drops columns; the code that used to write there was also removed. - Fix: clarify that this is schema rollback only; full behavior reversion requires application-code rollback too. Regression test (Fix 5b) — two-user learned aud isolation: - New test_two_user_isolation_write_and_read in test_token_storage_service.py: * store_tokens for user A does not touch user B's row * get_user_learned_audience for user A returns only A's values * get_user_learned_audience for user B returns only B's values - Replaces the narrative that was implicit in the deleted TestPersistLearnedAudience — the 'first-user vs second-user ownership' scenario is now covered in a focused location. Verification: 899 tests pass, ruff clean. Signed-off-by: Jonathan Springer <jps@s390x.com> * test(oauth): direct empty-shape coverage for _coerce_aud_claim + blank stored resource DCR regression Closes two coverage gaps flagged by the 3rd-round review of commit b582ba25f: test_oauth_manager.py — TestExtractTokenClaims: - Parametrized test_coerce_aud_claim_rejects_empty_shapes covers 8 cases ('', ' ', '\t\n', [], [''], [' ', '\t'], ['valid', ''], ['valid', ' ']) — all must return None so a malformed IdP response cannot bypass the 'if learned_aud is not None' guard in TokenStorageService.store_tokens and silently overwrite a previously-learned per-user audience. - Parametrized test_coerce_aud_claim_passes_valid_shapes verifies string, list-of-strings, and single-element list pass through unchanged. - Parametrized test_coerce_aud_claim_rejects_non_string_types covers None, int, float, dict, mixed lists, and list-of-None — completes the shape-rejection matrix. test_oauth_router.py — TestOAuthRouterAdditionalCoverage: - test_initiate_oauth_flow_dcr_preserves_blank_stored_resource: documents the current DCR-persist semantics for gateway.oauth_config.resource == ''. The strip logic uses 'if stored_resource is None: pop' else preserve, so an empty string (falsy but not None) takes the preserve branch and is persisted as-is. This locks the intent: blank is 'explicit admin state' rather than 'no admin config'. If empty resource is later declared invalid, the fix belongs in gateway config validation, not in this DCR strip. Verification: 58 tests pass in the two affected classes; 4 ruff unused-variable warnings in test_oauth_manager.py:1602-1746 are pre-existing (unrelated to these additions). Signed-off-by: Jonathan Springer <jps@s390x.com> * test(gateway): stub get_user_learned_audience on mocked TokenStorageService fetch_tools_after_oauth() now awaits TokenStorageService.get_user_learned_audience() to resolve per-user learned audiences. Tests that mock TokenStorageService but only stubbed get_user_token raised 'MagicMock can't be used in await expression', which in the audience-mismatch tests surfaced as a pytest.raises regex mismatch rather than the intended GatewayConnectionError. Add AsyncMock(return_value=(None, None)) stubs alongside every get_user_token stub in the two affected test files. Behavior-preserving: (None, None) matches the 'no learned audience recorded yet' path already exercised elsewhere. Fixes 6 failing tests: - test_gateway_service_oauth_comprehensive::test_fetch_tools_after_oauth_success - test_gateway_service_oauth_comprehensive::test_audience_mismatch_blocks_before_connection - test_gateway_service_oauth_comprehensive::test_opaque_token_no_warnings - test_gateway_service_oauth_comprehensive::test_audience_mismatch_raises_before_connection_with_diagnostic - test_gateway_service_oauth_comprehensive::test_streamablehttp_401_includes_diagnostic_info - test_gateway_service_extended::test_fetch_tools_after_oauth_prompt_stale_removal_uses_original_name Signed-off-by: Jonathan Springer <jps@s390x.com> * chore(oauth): surface OAUTH_REQUIRE_CONFIGURED_RESOURCE + pin JS resource-field pre-population Two small items from PR IBM#4476 re-review: 1. .env.example: add OAUTH_REQUIRE_CONFIGURED_RESOURCE=false with an operator- facing comment explaining the advisory-vs-blocking tradeoff and pointing to docs/docs/manage/oauth-resource-configuration.md. Setting was already implemented in config.py and documented; only .env.example missed it, so operators scanning that file for available toggles could not find it. 2. tests/unit/js: pin the oauth-resource-{gw,a2a}-edit pre-population logic (gateways.js:530-535, a2aAgents.js:589-593) with 2 tests per file — one for the RFC 8707 string shape, one for the list shape (comma-space joined into the input value so the operator can edit it in the same form the admin parser expects on submit). Prevents a silent refactor regression on the list-shape branch that had no test coverage. Also refreshes .secrets.baseline line numbers to account for the 10-line .env.example addition (no new detected secrets). Signed-off-by: Jonathan Springer <jps@s390x.com> * fix(oauth): propagate origin-only resource to refresh + validator fallback Address PR IBM#4476 review from Lang-Akshay — two consistency defects where the origin-derivation helper added to oauth_router was not propagated to the sibling code paths that must agree on the auto-derived audience. 1. token_storage_service refresh path (services/token_storage_service.py:410). Previously used normalize_resource(gateway.url), sending the FULL URL (path included) as the RFC 8707 resource param on refresh — while the initial authorization and callback exchanges in oauth_router send only the ORIGIN (scheme + netloc). The mismatch causes IdPs to fail the refresh outright or mint a token whose aud no longer matches the audience the user already validated against on first login. Fix uses the shared _derive_resource_origin helper so all three outbound resource sites agree. 2. token_validation_service fallback (services/token_validation_service.py:208). _validate_audience previously fell back to gateway_url verbatim (full URL) when neither configured resource nor learned_aud was available. Under OAUTH_REQUIRE_CONFIGURED_RESOURCE=true, valid origin-level aud tokens (what real IdPs mint when we request an origin-level resource) get rejected as authoritative mismatches. Fix derives the origin so the fallback matches the value oauth_router sends on the outbound side; falls through to raw gateway_url only when derivation returns None (URN / scheme-less). Structural change: promoted _derive_resource_origin from mcpgateway.routers.oauth_router (previously private to the router module) to mcpgateway.services.token_validation_service, next to _derive_issuer_from_token_url. Services should not import from routers; the helper is now shared cleanly by the OAuth route, the refresh path, and the validator. oauth_router re-exports it so the existing test import continues to work. Test coverage: - test_token_storage_service: * test_refresh_derives_resource_origin_from_path_bearing_gateway_url — new, asserts refresh sends origin-only resource for a path-bearing gateway URL. * test_refresh_skips_resource_when_gateway_url_is_urn — new, URN gateway.url yields no origin, refresh omits resource + logs warning. * test_refresh_derived_gateway_url_normalizes_to_empty_logs_warning — removed. Was patching urllib.parse.urlunparse to force normalize_resource to return empty; that mechanism no longer exists on this path. The URN test above provides equivalent 'derivation returns None → warning' coverage. - test_token_validation_service: * test_audience_falls_back_to_gateway_url renamed to test_audience_falls_back_to_gateway_url_origin and updated to the new contract (token aud = origin, gateway_url = full path → match). * test_audience_fallback_uses_urn_verbatim_when_origin_not_derivable — new, pins the URN corner case where the raw value stays as fallback. * test_strict_mode_accepts_origin_aud_against_path_bearing_gateway_url — new, the exact regression Akshay flagged: OAUTH_REQUIRE_CONFIGURED_RESOURCE=true + path-bearing gateway URL + origin aud must MATCH. Signed-off-by: Jonathan Springer <jps@s390x.com> * fix(db): point learned_aud/iss migration at current main head The migration's down_revision targeted b6c7d8e9f0a1, but main advanced to d21698ae4a19 (RBAC unique-constraints fix, IBM#4636) after this branch was cut, producing two alembic heads. Repoint down_revision at d21698ae4a19; alembic heads now reports a single head (c9f8e7d6a4b3). Signed-off-by: Jonathan Springer <jps@s390x.com> * refactor(oauth): consolidate RFC 8707 resource handling into utils module Review-driven consolidation of everything resource-shaped, previously smeared across four modules: - New mcpgateway/utils/oauth_resource.py owns derive_resource_origin (the auto-derived audience heuristic shared by authorize/callback/refresh/ validator), normalize_resource (RFC 8707 fragment/query policy), and parse_oauth_resource_form. Eliminates the private cross-module imports (mid-file noqa:E402 import in oauth_router, function-level import in the refresh path). - _resolve_refresh_resource extracts the refresh-path resource policy from the 230-line _refresh_access_token, and restores the omit_resource=true early-return (regression vs. the documented contract: resource key must be absent, not merely unsent). - TokenValidationResult.audience_source enum (configured/learned/derived) replaces the post-hoc audience_authoritative flag; the strict-setting rule now lives in one expression in blocking_errors with no mutation on the mismatch path. - Tests: audience_source assertions migrated; two stale urlunparse-patch tests removed/retargeted to the new module; omit_resource health-suite test updated to expect origin derivation. Signed-off-by: Jonathan Springer <jps@s390x.com> * fix(oauth): pin learned audience to configured issuer at learning time Restores the issuer-pinning hardening that was lost in the per-user redesign: when oauth_config['issuer'] is configured, complete_authorization_code_flow now persists learned_aud/learned_iss only if the token's iss claim matches (trailing slashes normalized; a missing iss claim also refuses). A stale or misrouted token from a different AS can no longer inject an audience for the wrong IdP, and the None pass-through leaves any previously-learned per-user value intact. Also deletes the dead single-claim extractors (_extract_token_audience, _extract_token_issuer) — _extract_aud_and_iss is now the only extraction API — and retargets all extractor tests and stale patch.object mocks to it. Adds TestIssuerPinningOnAudienceLearning: 6 tests covering no-issuer, match, trailing-slash, mismatch, missing iss, and opaque-token cases. Signed-off-by: Jonathan Springer <jps@s390x.com> * fix(admin): consolidate OAuth form parsing and drop inert A2A Resource field - _assemble_oauth_config_from_fields replaces the four near-identical ~45-line OAuth form-parsing blocks (gateway create/edit, A2A create/edit) — the duplication that caused the round-1 review miss where the gateway create form silently dropped oauth_resource. encrypt_secret and include_resource parameters capture the only real differences between sites. - Removes the OAuth Resource (Audience) field from the A2A agent forms: review found no A2A code path consumes oauth_config['resource'] (no per-user token storage, refresh, or audience validation on the A2A path), so the field was stored-but-inert. A2A parsing sites pass include_resource=False. Proper A2A audience validation tracked in IBM#5815. Signed-off-by: Jonathan Springer <jps@s390x.com> * test: mock network timeouts in slow health-check and JWKS tests Four pre-existing tests each took exactly 30.01s (the health-check/network timeout) on machines without outbound network: - TestCheckSingleHealthAuthCode (x3) never mocked the HTTP layer, letting a real GET to http://gw.test ride out health_check_timeout. A new _patch_unreachable_http_client helper fails fast with httpx.ConnectError; the failure-handler assertions are unchanged. - test_verify_oauth_access_token_jwt_error patched jwt.PyJWKClient, but production code instantiates vc._NoRedirectPyJWKClient and caches it in _oauth_jwks_client_cache, so the patch was a no-op and a real JWKS fetch rode out the network timeout. Patch the class actually used and clear the shared cache. test_gateway_service.py drops from ~94s to ~4s; the verify_credentials file runs in ~0.3s. Signed-off-by: Jonathan Springer <jps@s390x.com> * chore: refresh detect-secrets baseline Two audited entries (test_oauth_audience.py, test_auth_ratelimiter_redis.py) no longer match current file content and were flagged stale by the detect-secrets pre-commit hook, failing CI. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> --------- Signed-off-by: Jonathan Springer <jps@s390x.com> Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> Co-authored-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
…M#5764) * fix(mcp-apps): honour deprecated flat ui/resourceUri metadata key Servers that emit only the deprecated flat `_meta["ui/resourceUri"]` key lost their tool/UI association when passing through the gateway, because `merge_mcp_protocol_meta` returned early when `_meta.ui` was absent. The flat key is now folded into the nested MCP Apps metadata during the same merge step, mirroring the normalization the reference SDK performs in `registerAppTool`. The canonical nested `_meta.ui.resourceUri` takes precedence when both shapes are present and disagree, matching the consumer-side rule in the SDK's `getToolUiResourceUri`. Only well-formed `ui://` strings are folded in. An invalid legacy value would otherwise fail `validate_extension_metadata` and cause the whole tool to be rejected during gateway sync, so unusable values keep the existing "ignore it" behaviour and are logged instead. Signed-off-by: PhilipAD <9472774+PhilipAD@users.noreply.github.com> * fix(mcp-apps): keep a present canonical resourceUri authoritative Nested-key precedence was decided by truthiness, so a present but falsey canonical value (`""`, `False`, `0`) was treated as absent and replaced by the deprecated flat URI. The reference consumer falls back only when the canonical value is undefined; a malformed canonical value stays authoritative and is rejected downstream by `validate_extension_metadata` rather than silently swapped. Precedence is now decided by key presence. Upstream-controlled URIs are also no longer logged verbatim: userinfo, query and fragment are stripped and the remainder is truncated, so a compromised upstream cannot use the ingest path to write unbounded or credential-bearing values into the gateway logs. Legacy values that are structurally unusable (`ui://` with nothing after the scheme) are now ignored alongside non-`ui://` values, so nothing is folded in that cannot resolve to a real UI resource. Signed-off-by: PhilipAD <9472774+PhilipAD@users.noreply.github.com> * fix(mcp-apps): make legacy URI log redaction order-independent Rebuild _loggable_uri on urlsplit/urlunsplit so the authority is resolved before any component is dropped. The previous implementation stripped the query and fragment first and only then looked for userinfo, so an unencoded delimiter inside the credential ("https://user:pa?ss@host/x") truncated the value mid-credential and leaked the surviving prefix into the log line. Stripping userinfo from the parsed netloc alone does not close this: for that input urlsplit yields a netloc of "user:pa" with no "@" left to strip. When an "@" appears in a component that is dropped -- or the URI is not hierarchical and has no authority at all -- nothing preceding it can be shown to be credential-free, so the authority is redacted instead. A URI that cannot be parsed is reported by length only. Also cover the shared ingest paths: a legacy-only tool now has a test through GatewayService._validate_tools confirming it survives ToolCreate validation, and the resource and resource-template paths are covered through the real connect_to_sse_server chain. Documents the legacy key, its precedence rule and its ingest-only scope in the MCP Apps architecture guide, and switches to str.removeprefix. Part of IBM#2527. Signed-off-by: PhilipAD <9472774+PhilipAD@users.noreply.github.com> --------- Signed-off-by: PhilipAD <9472774+PhilipAD@users.noreply.github.com> Co-authored-by: PhilipAD <9472774+PhilipAD@users.noreply.github.com>
* Update docker-scan to upload SARIF file with finding. Signed-off-by: Jonathan Springer <jps@s390x.com> * Containerfile updates & Try SBOM module Signed-off-by: Jonathan Springer <jps@s390x.com> * fix(ci): address docker-scan SARIF review findings - Grant security-events: write to fedramp-compliance and scan jobs so codeql-action/upload-sarif no longer 403s and ejects merge-queue PRs - Add push: [main] trigger so SARIF uploads land on the stable main ref instead of ephemeral gh-readonly-queue/* refs - Make scan-fedramp report-only (fail-build: false): the FIPS image is built from full ubi9/ubi on floating tags while .grype.yaml ignores were calibrated for the lite image's base - Set upload-artifact: false on both Generate SBOM steps to stop duplicate SBOM artifacts - Guard Display/Upload SARIF steps on a non-empty sarif output so a failed build no longer produces spurious cat/upload errors - Run Upload SBOM with if: always() so the triage artifact survives a grype failure - Refresh stale header and aggregator comments for the four-job layout Signed-off-by: Jonathan Springer <jps@s390x.com> --------- Signed-off-by: Jonathan Springer <jps@s390x.com>
…to testable submodule (IBM#6008) * refactor(middleware): extract auth-context middleware registration into testable submodule Move the CSRF / PasswordChangeEnforcement / AuthContext registration block out of mcpgateway/main.py into mcpgateway/middleware/auth_context_stack.py as register_auth_context_middleware(). Registration order is unchanged; the trio is extracted so its load-bearing ordering (AuthContext must run before CSRF so request.state.user is populated, issue IBM#5739) can be unit-tested against a bare FastAPI app without importing the assembled application singleton. Rewrite test_admin_csrf_binding.py accordingly: delete the five full-stack TestClient tests that depended on the shared mcpgateway.main.app singleton (they failed under the full xdist suite because middleware registration is import-time and settings-dependent, so AuthContextMiddleware was absent when another module imported mcpgateway.main first during collection), convert the enforce_admin_csrf block to direct-call unit tests with a mocked Request, and add hermetic ordering tests against the extracted function. End-to-end CSRF coverage remains with the Playwright security suite. Signed-off-by: Jonathan Springer <jps@s390x.com> * fix: align CSRF cookie name fallbacks with config default The getattr() fallback defaults in csrf_middleware.py and csrf_service.py used "csrf_token" while config.py defaults csrf_cookie_name to "mcpgateway_csrf_token". Align the fallbacks so a missing attribute yields the same cookie name as the configured default. Also patch test_config.py CSRF tests to supply JWT_SECRET_KEY and AUTH_ENCRYPTION_SECRET via patch.dict so Settings(_env_file=None) instantiates in a clean environment. Signed-off-by: Jonathan Springer <jps@s390x.com> --------- Signed-off-by: Jonathan Springer <jps@s390x.com>
* feat(gateway): add prompt and resource counts to GatewayRead Signed-off-by: Gabriel Costa <gabrielcg@proton.me> * Optimise queries + test Signed-off-by: Gabriel Costa <gabrielcg@proton.me> * Address comments Signed-off-by: Gabriel Costa <gabrielcg@proton.me> * Add tests Signed-off-by: Gabriel Costa <gabrielcg@proton.me> * Add helper for capability count Signed-off-by: Gabriel Costa <gabrielcg@proton.me> * Fix documentation Signed-off-by: Gabriel Costa <gabrielcg@proton.me> --------- Signed-off-by: Gabriel Costa <gabrielcg@proton.me>
* fix(security): enforce API token scope-based access control Signed-off-by: Rakhi Dutta <rakhibiswas@yahoo.com> * fix: add A2A API endpoint permission patterns and JWT token scope extraction for scoped token authorization Signed-off-by: Rakhi Dutta <rakhibiswas@yahoo.com> * test: add resource_scopes to API token test mocks Signed-off-by: Rakhi Dutta <rakhibiswas@yahoo.com> * token scipe naming Signed-off-by: Rakhi Dutta <rakhibiswas@yahoo.com> * docs(security): address PR review feedback - Add comment explaining JWT empty list handling behavior - Add token type detection documentation in RBAC middleware - Update TOKEN_SCOPE_NAMING.md references to correct line numbers - Remove reference to non-existent regression test file - Verify A2A route pattern coverage (all endpoints protected) All review issues addressed: 1. JWT empty list handling - Added clarifying comment 2. A2A route patterns - Verified all 19 endpoints have @require_permission 3. Token type detection - Added inline documentation 4. Test file reference - Updated to point to actual test file Refs: IBM#4671 Signed-off-by: Rakhi Dutta <rakhibiswas@yahoo.com> * fix(security): enforce scope checks for JWT tokens with empty permissions list to match database token behavior Signed-off-by: Rakhi Dutta <rakhibiswas@yahoo.com> * docs(security): add JWT malformed token logging and clarify naming convention permanence Signed-off-by: Rakhi Dutta <rakhibiswas@yahoo.com> * fix: prevent JWT empty dict scope bypass (empty dict is falsy in Python) Signed-off-by: Rakhi Dutta <rakhibiswas@yahoo.com> * test: add comprehensive scope enforcement test coverage Signed-off-by: Rakhi Dutta <rakhibiswas@yahoo.com> * remove test file Signed-off-by: Rakhi Dutta <rakhibiswas@yahoo.com> * fix(security): reject malformed JWT scopes and clarify session token design Signed-off-by: Rakhi Dutta <rakhibiswas@yahoo.com> * test(security): add layer 1+2 interaction test, reject malformed JWT scopes Signed-off-by: Rakhi Dutta <rakhibiswas@yahoo.com> * trailing newline Signed-off-by: Rakhi Dutta <rakhibiswas@yahoo.com> * pre-commit Signed-off-by: Rakhi Dutta <rakhibiswas@yahoo.com> * test coverage improve Signed-off-by: Rakhi Dutta <rakhibiswas@yahoo.com> * coverage Signed-off-by: Rakhi Dutta <rakhibiswas@yahoo.com> * fix(security): enforce token scopes and Layer 1 consistency Signed-off-by: Rakhi Dutta <rakhibiswas@yahoo.com> * pre-commit fix Signed-off-by: Rakhi Dutta <rakhibiswas@yahoo.com> * test cases fix Signed-off-by: Rakhi Dutta <rakhibiswas@yahoo.com> * test coverage Signed-off-by: Rakhi Dutta <rakhibiswas@yahoo.com> * fix(security): correct Layer 1 token scope enforcement semantics Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> * fix(security): grant servers.use to tokens with MCP method scopes Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> --------- Signed-off-by: Rakhi Dutta <rakhibiswas@yahoo.com> Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> Co-authored-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com>
* test: add SSRF validator with deny/allow coverage
Adds a stdlib-only, framework-independent URL validator (ssrf.py) that
enforces a scheme allowlist, rejects embedded credentials, applies an
optional hostname allowlist, resolves hostnames via a monkeypatchable
seam, and blocks loopback/RFC1918/link-local/CGNAT/multicast/reserved/
unspecified addresses (plus operator-configured CIDRs) on every
resolved candidate IP. Successful validation pins the result to a
single resolved IP to avoid a DNS-rebinding window between validation
and fetch.
This is step 1 of a 4-task plan; wiring the validator into the actual
HTTP fetch path in server_fastmcp.py is deferred to a follow-up task.
Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
* fix: fail closed on SSRF protection flag and IPv6 localhost escape hatch
MARKDOWN_SSRF_PROTECTION_ENABLED defaults to on, but the shared
_env_bool() helper treated any unrecognized value as false, silently
disabling all per-IP SSRF checks on a typo. It now goes through a
dedicated _env_enabled_by_default() that only an explicit falsey value
can turn off.
_is_blocked_ip() also checked is_reserved before is_loopback; since
IPv6Address('::1').is_reserved is True, MARKDOWN_ALLOW_LOCALHOST was a
no-op for IPv6 loopback even though it worked for 127.0.0.1. Loopback
is now checked first and independently of the other always-blocked
categories.
Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
* fix: pin outbound connections and validate every redirect hop against SSRF guard
fetch_url_content previously made a single unvalidated httpx request and
let httpx auto-follow redirects, so a first-hop SSRF check (if any) could
be bypassed by a public host redirecting to an internal address on a
later hop. get_session() now disables httpx's automatic redirect
following, and fetch_url_content implements its own bounded redirect
loop that re-validates each hop through ssrf.validate_url(), connects to
the validated target's pinned resolved IP (Host header and SNI set to
the original hostname) to close the DNS-rebinding window between
validation and connect, and streams the response body, aborting as soon
as it exceeds MAX_CONTENT_SIZE instead of buffering the full response
first.
_detect_content_type now receives the final, post-redirect URL rather
than the pre-fetch original, since extension sniffing should follow
where the content actually came from.
batch_convert's urls and timeout Field declarations are now bounded
(max_length=50, le=MAX_TIMEOUT) so one call cannot fan out to an
unbounded number of internal targets.
Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
* test: cover redirect re-validation and pinning; log blocked SSRF URLs
Adds a regression test that mocks a 302 redirect to the cloud metadata
address and asserts fetch_url_content blocks it without the second hop
ever connecting - guarding against a future refactor that hoists
validate_url() above the redirect loop and silently restores the SSRF.
Extends the existing fetch_url_content mock test to assert what was
actually passed to session.stream(): the pinned IP in the URL, and the
original hostname in both the Host header and the sni_hostname
extension, so tampering with the pinning wiring is caught by tests
instead of passing silently.
Adds a logger.warning() in the SsrfBlockedError handler so blocked URLs
are visible in server-side logs even though the caller only sees the
generic "URL is not allowed" message.
Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
* docs: correct SSRF security claims and document new env vars
Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
* ci: run url_to_markdown_server tests on PR
Add a dedicated workflow that installs the url_to_markdown_server
package's dev extras and runs its pytest suite (including the SSRF
regression tests) whenever files under
mcp-servers/python/url_to_markdown_server/** change. Previously no
CI workflow executed this package's tests, so a future refactor
could silently reintroduce the SSRF vulnerability with nothing to
catch it.
Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
* fix: IDNA-encode hostnames, log SSRF block reasons, and test streaming/redirect limits
Three fixes to url_to_markdown_server's SSRF guard raised by final review:
- IDNA-encode caller-supplied hostnames (skipping literal IPs) before the
allowlist check, so a Unicode hostname no longer reaches httpx's Host
header encoding as raw UnicodeEncodeError. Malformed IDN labels now
surface as the existing generic SsrfBlockedError instead of propagating
an unhandled exception to the MCP caller.
- Add server-side logging (ssrf.py had none) so operators can diagnose why
a URL was blocked without leaking that detail to the untrusted caller,
whose error message stays the fixed generic string. A malformed CIDR in
MARKDOWN_BLOCKED_NETWORKS is now logged and skipped instead of silently
failing closed on every request - the unconditional deny categories
(private/loopback/link-local/CGNAT/etc.) are unaffected either way.
- Add coverage for the streamed-size-cap abort and redirect-exhaustion
branches in fetch_url_content, both previously unexercised despite the
README documenting the streaming-abort behavior as a security feature.
Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
* fix: drop stray shebang from ssrf.py, refresh secrets baseline
pre-commit's check-shebang-scripts-are-executable flagged ssrf.py: it
carried a #!/usr/bin/env python3 shebang copied from the entry-point
module's header convention despite being a plain library module with
no execute bit. Replaced with the coding-declaration line used by this
package's other non-entrypoint modules.
.secrets.baseline line numbers/generated_at refreshed via
detect-secrets-scan after the upstream rebase shifted file contents;
no new or unaudited findings.
Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
* test: add real-environment e2e verification for the SSRF fix
Adds tests/e2e_verify_ssrf_fix.py, which starts the actual FastMCP
server as a subprocess over real stdio transport and drives it with a
real MCP client (fastmcp.Client) - no mocked DNS/HTTP, unlike
test_ssrf.py and test_server.py. Real local HTTP servers stand in for
"internal" victim services and prove attacker-reachable targets are
never connected to; a real outbound request to https://example.com
proves legitimate fetches still work.
18 cases across 3 configurations: secure defaults (cloud metadata,
loopback, RFC1918, CGNAT, IPv6 loopback, credential-in-URL, disallowed
scheme, non-resolving IDN hostname, legitimate fetch, batch_convert's
50-URL cap, a mixed batch, and two unrelated tools for regression);
MARKDOWN_ALLOW_LOCALHOST=true (redirect-hop re-validation against a
real local redirector, and proof the escape hatch doesn't cross
categories); and the same flag with tightened MARKDOWN_MAX_CONTENT_SIZE
/ MARKDOWN_MAX_REDIRECT_HOPS to exercise the streaming-abort and
redirect-loop-bound branches within a fast run.
Not wired into pytest/CI (real outbound network access and several
local subprocess servers are inappropriate for a fast, isolated CI
run) - run manually per the module docstring.
Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
* fix: address IPv6 site-local/6to4 bypass, cross-host connection reuse, and unbounded DNS in SSRF guard
Addresses review findings on the SSRF fix (GHSA-gv3m-3mvj-w6f2):
- ssrf.py: block IPv6 site-local (fec0::/10) addresses explicitly, and
unwrap 6to4-embedded IPv4 addresses before categorization so an embedded
loopback/link-local target can't hide behind Python's blanket
is_private=True classification for the whole 2002::/16 block.
- server_fastmcp.py: pin the raw TCP destination via a per-request
contextvar and a custom network backend instead of rewriting the request
URL to the resolved IP. Keeping the request origin as the real hostname
keeps HTTPX's connection pool, TLS SNI, and cookie jar scoped per
hostname, so two hostnames that resolve to the same IP can no longer
share a TLS-authenticated connection.
- ssrf.py: bound DNS resolution with the caller's own request timeout on a
dedicated thread pool, so a slow/unresponsive resolver can't silently
outlive the advertised timeout or exhaust the shared default executor.
Adds regression tests for all three findings.
Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
* fix: honor egress proxies and match blocked CIDRs against both address forms
Addresses the second round of review findings on the SSRF fix
(GHSA-gv3m-3mvj-w6f2):
- server_fastmcp.py: passing an explicit transport to httpx.AsyncClient
disables HTTPX's automatic loading of HTTP_PROXY/HTTPS_PROXY/ALL_PROXY
(allow_env_proxies is only enabled when transport is None), so a
deployment with a configured egress proxy silently connected direct
instead - breaking proxy-only deployments and bypassing proxies used for
outbound filtering or auditing. get_session() now detects configured
proxy variables and, when present, lets HTTPX build its own transport so
the proxy is honored. Connect-time IP pinning is skipped on that path
(the socket goes to the proxy, which resolves the destination itself, so
the validated address cannot be enforced end to end) and a warning states
this explicitly rather than degrading silently. All other SSRF controls
are unchanged on both paths.
- ssrf.py: _unwrap_embedded_ipv4() replaced the address before
MARKDOWN_BLOCKED_NETWORKS was evaluated, and unwrapping changes the
address family - so an operator-configured IPv6 CIDR could never match a
6to4 or IPv4-mapped literal inside it. With MARKDOWN_BLOCKED_NETWORKS=
2002::/16, http://[2002:0808:0808::]/ was allowed (the unwrapped 8.8.8.8
is public, so it also cleared every category check, making the bypass
independent of the escape hatches). Operator CIDRs are now matched against
both the address as written and its unwrapped form; category checks
continue to use the unwrapped form. Matching both is required in both
directions: an IPv4 CIDR such as 10.0.0.0/8 can only match the unwrapped
address of a 6to4 literal wrapping that range.
Documents the proxy behavior and the previously undocumented
MARKDOWN_DNS_TIMEOUT / MARKDOWN_DNS_RESOLVER_THREADS variables, and adds
regression tests for both findings.
Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
* fix: keep IP pinning on NO_PROXY and unmatched destinations when an egress proxy is configured
get_session() previously disabled connect-time IP pinning for the whole
session whenever any proxy env var was set, even for destinations that
NO_PROXY excludes from the proxy and therefore connect direct. That
reopened the validate-to-connect DNS-rebinding window IP pinning exists
to close, for traffic the proxy never touches at all.
The pinned AsyncHTTPTransport is now always the client's default
transport. Proxy-matching destinations are mounted onto separate,
unpinned proxy transports built from httpx's own environment-proxy
mount map, so NO_PROXY entries route back onto the pinned default
instead of falling through to a broader proxy mount.
Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
* fix: fail closed on proxy-routed destinations unless explicitly opted in
Any destination that would previously have been routed through a
configured HTTP_PROXY/HTTPS_PROXY/ALL_PROXY got its connect-time IP
pinning silently disabled, since the proxy performs its own DNS
resolution and the validated address cannot be enforced end-to-end on
that path. That reopens the DNS-rebinding window pinning exists to
close for every destination the proxy would have handled, not just the
NO_PROXY-excluded ones the prior fix covered.
get_session() now ignores proxy environment variables entirely - and
keeps every destination pinned - unless
MARKDOWN_ALLOW_UNSAFE_PROXY_PINNING is explicitly set. NO_PROXY and
unmatched destinations were already unaffected either way. Documented
in the README and docs page; regression tests cover the fail-closed
default, the opt-in path, and a live DNS-rebinding attempt against a
proxy-matched destination.
Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
* fix: refuse proxy-matched destinations by default, bound DNS resolver admission
Falling back to a direct (pinned) connection for a proxy-matched
destination closed the DNS-rebinding gap but silently bypassed
whatever egress policy (audit logging, DLP, destination allowlisting)
the operator configured the proxy to enforce - its own kind of control
bypass distinct from the rebinding risk pinning addresses. get_session()
now always mounts a configured proxy, as HTTPX itself would, but
fetch_url_content() refuses to actually send a request through it -
before any connection to either the proxy or the destination - unless
MARKDOWN_ALLOW_UNSAFE_PROXY_PINNING is set. NO_PROXY and unmatched
destinations are unaffected either way.
Also bounds admission to the dedicated DNS resolver thread pool.
ThreadPoolExecutor's own work queue is unbounded, so a burst of
slow-resolving hostnames could queue indefinitely behind the fixed
worker threads; a semaphore sized to the pool now rejects a lookup
that can't get a permit within its timeout instead of queueing it, with
the permit held until the underlying resolver call actually completes
rather than until the caller's own wait gives up.
Regression tests cover both: a proxy-matched request is rejected
without contacting either the proxy or the destination (verified via
DNS rebinding) while the opt-in path still works end to end, and the
resolver admission bound rejects when saturated while releasing only
on genuine completion.
Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
* chore: mark e2e SSRF verifier executable, refresh secrets baseline
check-shebang-scripts-are-executable requires the shebang'd e2e script
to carry the execute bit. Baseline regeneration is timestamp and
line-number drift only, from rebasing onto upstream/main.
Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
* fix: reject non-positive max_concurrent in batch_convert
max_concurrent had an upper bound (le=10) but no lower one, so a
caller-supplied 0 (or negative) reached asyncio.Semaphore() directly.
A zero-valued semaphore never grants its first permit, so every
convert_single_url() task blocks forever at `async with semaphore`
and asyncio.gather() never returns - an MCP caller could hang a
batch_convert call indefinitely with no server-side timeout to
recover it.
Adds ge=1 so FastMCP/Pydantic rejects the value before batch_convert()
starts, plus a regression test covering both 0 and -1.
Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
* chore: exclude e2e SSRF verifier from test-naming hook, refresh secrets baseline
tests/e2e_verify_ssrf_fix.py is an intentionally-manual script (needs real
outbound network access, not wired into CI - see its own docstring), not a
pytest test file, but the name-tests-test hook's exclude list only covers
top-level tests/ paths and missed this package's nested tests/ directory.
Baseline refresh is line-number drift only, from rebasing onto upstream/main.
Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
* chore: refresh secrets baseline after upstream rebase
Line-number drift only, from rebasing onto latest upstream/main.
detect-secrets-scan and detect-secrets-audit both report zero
unaudited/live/real secrets.
Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
* test: close remaining diff-coverage gaps in the SSRF fix
diff-cover against origin/main flagged six untested branches this PR's
own diff introduced: _PinnedNetworkBackend's unix-socket/sleep
passthroughs, _environment_proxy_mounts()'s fail-closed exception
path, the redirect-missing-Location-header guard, the declared
Content-Length pre-check, the httpx.HTTPStatusError handler, and
ssrf.py's admission-release RuntimeError swallow (closed event loop).
None of these are reachable from the existing happy-path/SSRF-deny
tests, so each gets a direct, mocked regression test. Diff coverage
is now 100% (up from 96%).
Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
* style: fix two ruff findings introduced by the SSRF fix
- ssrf.py:1 - drop the unnecessary UTF-8 coding declaration (UP009);
the file is Python 3-only and ruff already flags this as redundant.
- server_fastmcp.py - wrap the fail-closed proxy-config log message
(E501, 167 chars) across two string literals.
No behavior change. Ruff findings on the two touched files: 15 -> 13.
Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
---------
Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
Co-authored-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
Bumps the cryptography floor from >=49.0.0 to >=50.0.0. 50.0.0 was published after this project's rolling 10-day exclude-newer window, so it needs an explicit tool.uv.exclude-newer-package entry (matching the existing pattern for cpex/authlib-style bumps) to let uv's resolver consider it at all. Verified: targeted JWT/crypto/auth unit tests pass; the Grype CVE gate no longer flags this dependency on the built image (only pre-existing, unrelated medium p11-kit RPM findings remain); the FIPS-enabled (ENABLE_FIPS=true, PYTHON_VERSION=3.11) image builds and passes fedramp-validate.sh (22/22), and both images resolve cryptography==50.0.0. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> Co-authored-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
…IBM#5198) * fix(oauth): remove blocking unique constraint for multi-user DCR clients Fixes IBM#5191 Migration 14ac971cee42 added app_user_email column and created unique index on (gateway_id, app_user_email) but failed to drop the original UniqueConstraint 'unique_gateway_user' on (gateway_id, user_id). This prevented multiple ContextForge users from storing OAuth tokens when they shared the same OAuth provider user_id, blocking multi-user OAuth flows for DCR-based gateways. This migration (7ab59991e017) properly removes the old constraint and creates the correct UniqueConstraint 'uq_oauth_gateway_user' on (gateway_id, app_user_email). Uses table recreation for SQLite and direct DDL for PostgreSQL. Tested with direct SQL verification showing successful multi-user token storage after migration. Signed-off-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com> * Changed pure SQL to sqlalchemy Signed-off-by: cafalchio <mcafalchio@gmail.com> * Secrets Signed-off-by: cafalchio <mcafalchio@gmail.com> * Addressed issues, updated head Signed-off-by: cafalchio <mcafalchio@gmail.com> * Addressed comments. Downgrade fails on duplicates Signed-off-by: cafalchio <mcafalchio@gmail.com> * Fixed alembic head Signed-off-by: cafalchio <mcafalchio@gmail.com> * remove index before creating a new one Signed-off-by: cafalchio <mcafalchio@gmail.com> * Fixed docs Signed-off-by: cafalchio <mcafalchio@gmail.com> --------- Signed-off-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com> Signed-off-by: cafalchio <mcafalchio@gmail.com> Co-authored-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com> Co-authored-by: cafalchio <mcafalchio@gmail.com>
Rewrite docs/docs/architecture/roadmap.md around the H2 2026 plan, linking each theme to its tracking epic or issue, and add docs/docs/architecture/releases.md with a release-by-release table of the most impactful changes since 1.0.0. Register the new Release History page in the architecture .pages nav and refresh .secrets.baseline for the shifted line number. Signed-off-by: Jonathan Springer <jps@s390x.com> Co-authored-by: jonpspri <jonpspri@users.noreply.github.com>
* feat: consume Cpex control execution records Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * fix(plugins): suppress pylint unused-import on cpex feature-detection probe Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * style: align inline comments in control_telemetry.py Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * test(plugins): add targeted coverage tests for control-telemetry and cpex-compat - Hoist all imports to top-level (no inline imports inside test methods) - Add TestAggregateExceptionPath: covers aggregate() inner except path (lines 192-193) - Add TestGetMaxResultsExceptionPath: covers _get_max_results() except branch (lines 484-485) - Add TestBuildFlattenedAttributesEdgeCases: error_code branch, per-record except, outer except (lines 553-555, 561-563) - Add TestEmitDbSpansExceptionPaths: empty-attrs skip (line 327), rollback on start_span failure (lines 342-348), close() failure swallowed (lines 353-354) - Add TestEmitOtelSpansActivePath: verifies OTel summary+result spans emitted when otel_tracing_enabled+otel_context_active (lines 379-387) - Add TestRecordControlTelemetryTruncated: truncated attribute present (line 260) - Add TestRecordControlTelemetryFlatten: flatten_results branch (lines 267-268) - Add TestExecutionRecordsSupportedImportError: ImportError branch (lines 51-52) - Add wildcard rule tests in TestWildcardAttributeMapping: key>256 rejection (line 535), wildcard/exact removal compilation (lines 543-551), segment extraction mismatch (line 593), ValueError fallback (lines 657-660) - Add TestConcurrencyIsolation and TestObservabilityServiceAPICompatibility Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * fix(plugins): address review findings from issue IBM#5785 PR IBM#6003 Duration unit (finding IBM#3): - Rename cpex.control.duration -> cpex.control.duration_ns in aggregate(), _per_control_attributes(), and both sinks to follow OTel unit-suffix convention. Rename rules targeting 'cpex.control.duration_ns' are now semantically correct. Missing ControlExecutionRecord fields (finding IBM#2): - Add cpex.control.plugin_id, cpex.control.plugin_kind, cpex.control.matched, cpex.control.applied, cpex.control.payload_modified to per-control result spans. - Add cpex.control.result.requested_allowed (emitted only when requested_allow != None). results_count semantics (finding IBM#6): - Fix aggregate() results_count to reflect min(accumulated, max_results) — the number of records exported after the per-invocation cap, per IBM#5785 spec. invocation_count semantics (finding IBM#6): - Count only controls that actually ran (completed/error/timeout); exclude skipped/disabled/cancelled per IBM#5785 aggregation spec. Timeout post-hook gap (finding IBM#5): - Add ctl_acc parameter to _run_timeout_post_invoke(); all four call sites now pass ctl_acc=_ctl_acc so timeout post-hook execution records feed the per-invocation accumulator and appear in the emitted telemetry. Fail-closed ValueError fallback (finding #8b): - apply_attribute_mapping() now returns attributes unchanged when compile_attribute_policy raises ValueError (e.g. otel.* destination, empty key, key > 256 chars). Previously it constructed partially-validated exact mappings that could bypass the otel.* guard — now fail-closed. CPEX denial path documentation (finding IBM#4): - Add explanatory comment in except (PluginError, PluginViolationError) block documenting the upstream CPEX 0.1.2 framework gap: when violations_as_exceptions=True, CPEX raises before appending the denying plugin's ControlExecutionRecord, so _ctl_acc captures pre-denial records only. The effective_allow=False outcome is preserved via pre_denied flag. Tests updated for all changes above; 741 passed, 33 skipped. Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * fix gaps Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * fix(plugins): account for tier-3 export-cap drops in cpex.control.truncated Records beyond CPEX_CONTROL_TELEMETRY_MAX_RESULTS were silently discarded by islice() in _emit_db_spans() and _emit_otel_spans() without being counted in the cpex.control.truncated summary attribute, leaving downstream operators with an incomplete truncation indicator. Fix: - Add _export_cap_dropped field to ControlTelemetryAccumulator - Add mark_export_cap_dropped(count) method called once per emit in record_control_telemetry() before building the summary span - truncated property now returns _truncated + _export_cap_dropped covering all three tiers: Tier 1: per-hook cap (MAX_RECORDS_PER_HOOK = 64) Tier 2: per-call accumulation cap (MAX_RECORDS_PER_CALL = 128) Tier 3: export cap (CPEX_CONTROL_TELEMETRY_MAX_RESULTS, default 32) Tests: - Add TestExportCapTruncation (5 tests) in test_control_telemetry.py - Update TestRecordControlTelemetryTruncated to assert correct total across all three tiers (2 tier-1/2 drops + 96 tier-3 drops = 98) Relates to IBM#5785 Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * fix linting Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * fix(plugins): address blocking review findings on cpex control telemetry PII / agent.id (blocking): - Gate cpex.control.agent.id behind CPEX_CONTROL_TELEMETRY_EMIT_AGENT_ID (default false). Adds _emit_agent_id_enabled() helper and new config field with explicit PII/GDPR warning in description and .env.example. config_keys amplification (functionally impacting): - Add _MAX_CONFIG_KEY_LEN=128 per-key cap and _MAX_CONFIG_KEYS_JOINED_LEN=4096 total joined-string cap. Applied in both _per_control_attributes() and _build_flattened_attributes() to prevent large/adversarial key names causing telemetry amplification or data leakage. Default-off (functionally impacting): - Change CPEX_CONTROL_TELEMETRY_ENABLED default from true to false. Each traced tool call creates up to 1 summary + MAX_RESULTS result DB spans; operators must explicitly opt in after reviewing storage/cardinality impact. E2E tests set CPEX_CONTROL_TELEMETRY_ENABLED=true explicitly via os.environ. E2E denial-path tests (test coverage): - Add denying_plugin_app fixture: PIIFilter with block_on_detection=true and enforcement_mode=pre_invoke. - Add TestDenialPathTelemetry with two tests: test_pre_invoke_denial_summary_span_allowed_false: verifies result.allowed=false and enforcement_point=pre when a real plugin denies on pre_invoke. test_pre_invoke_denial_tool_never_executed: verifies tool arguments do not leak into control telemetry attributes on the denial path. Both tests skip gracefully when PIIFilter does not detect PII (environment variation). Unit test fixes: - Add _enabled_settings() helper in test_record_control_telemetry.py so all TestRecordControlTelemetryDB tests work correctly with the new default=false. - Patch cfg_mod.settings in all DB-sink tests and TestRecordControlTelemetryTruncated. Relates to IBM#5785, PR IBM#6003 Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * addressed comments Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * fixed secrets Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * fix(plugins): address four follow-up review findings on cpex control telemetry Fix 1 — wildcard insertion order corrupted by cache key sorting: - apply_attribute_mapping() was building its cache key with sorted(mapping.items()), which reordered wildcard rules and changed which match fired first. The documented contract is 'first matching wildcard wins' (compile_attribute_policy evaluates rules in insertion order). Changed to tuple(mapping.items()) to preserve the caller's declared rule order. - Regression test: TestWildcardInsertionOrderPreserved (3 cases) in test_record_control_telemetry.py. Fix 2 — config_keys joined cap was char-based, not byte-aware: - Both projections (_per_control_attributes and _build_flattened_attributes) used joined[:_MAX_CONFIG_KEYS_JOINED_LEN] which counts Unicode code points, not UTF-8 bytes. Multibyte key names (e.g. CJK characters) could therefore exceed the intended byte budget. - Fixed both sites to use _safe_str(joined, _MAX_CONFIG_KEYS_JOINED_LEN) which is already byte-aware (encodes to UTF-8 before slicing). - Regression tests: TestConfigKeysByteCapAware (2 cases) covering multibyte input and ASCII round-trip. Fix 3 — _enforcement_point returned 'none' on denial with empty records: - When CPEX raises PluginViolationError with violations_as_exceptions=True it does so before appending a ControlExecutionRecord, so the accumulator can have no records even though a pre-hook fired. _enforcement_point was only inspecting acc.records and returned 'none' in this case. - Fixed to also check acc.pre_denied / acc.post_denied flags so a first-plugin pre-invoke denial correctly reports enforcement_point='pre'. - E2E denial test tightened: removed 'none' from accepted enforcement_point values; removed pytest.skip on missing summary span (absence is a regression); removed 500 from accepted status codes (indicates unhandled exception). - Unit tests: TestEnforcementPointDenialFlags (3 cases). Fix 4 — PluginError on first plugin emits no telemetry: - When the first plugin in the chain raises PluginError the accumulator has no records and no denial flags, so record_control_telemetry() returned early at the empty-accumulator guard, emitting nothing. Plugin outages were invisible in observability dashboards. - Added _plugin_errored field and mark_plugin_error() method to ControlTelemetryAccumulator. The outer except PluginError handler in tool_service.py now calls mark_plugin_error() before record_control_telemetry(). - Empty-accumulator guard now includes plugin_errored so first-plugin failures always emit a summary span. - aggregate() emits cpex.control.plugin_error=True when the flag is set, making plugin outages distinguishable from both successful allows and policy denials. - effective_allowed remains True on PluginError (intentional — outage != denial). - Unit tests: TestMarkPluginError (6 cases) in test_control_telemetry.py. 155 unit tests pass; ruff and pylint 10.00/10. Relates to IBM#5785, PR IBM#6003 Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * fix(plugins): address blocking and warning findings from ja8zyjits review Blocking 1: CHANGELOG.md CPEX_CONTROL_TELEMETRY_ENABLED default 'true' → 'false' to match mcpgateway/config.py:1987 (default=False). Operators would have expected the feature on by default when it is off. Blocking 2: mark_export_cap_dropped() idempotency — changed from += to a first-write-wins assignment guarded by _export_cap_dropped == 0. Subsequent calls on the same accumulator are now no-ops, preventing cpex.control.truncated inflation on retry paths or test reuse. Two regression tests added (TestExportCapTruncation::test_mark_export_cap_dropped_is_idempotent and ::test_mark_export_cap_dropped_zero_then_positive). Warning 1: Replaced undefined 'S4 principles' reference in control_telemetry.py module docstring with an explicit description: 'bounded cardinality and field-level sanitization ... _safe_str/_safe_num helpers and field-name allowlists' pointing to plugins/utils.py. Warning 2: Extracted the 5 repeated record_control_telemetry() call blocks in tool_service.py::invoke_tool() into a single nested helper _emit_ctl_telemetry() that closes over _ctl_acc, name, app_user_email, user_email, gateway_name, and server_id. Identity mapping rationale previously inline at the normal-return site is now in the helper docstring; exception-path sites are reduced to a single _emit_ctl_telemetry() call with a back-reference comment. 145 plugin unit tests pass; ruff all-clear; pylint 10.00/10. Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * fix(plugins): sanitize config_keys and make plugin_error result.allowed indeterminate config_keys sanitization (vishu-bh finding): - Add _sanitize_config_key() helper that validates each key against _CONFIG_KEY_RE (^[A-Za-z0-9_.-]{1,64}$). Rejects commas (the CSV delimiter), CR/LF/NUL/TAB (log-injection vectors), non-ASCII, and secret-shaped tokens containing '='. Keys that fail are dropped entirely rather than truncated, preventing ambiguous joined output. - Both _per_control_attributes() and _build_flattened_attributes() now use the validator. When every key in a record fails, the attribute is omitted rather than emitting an empty string. - 14 new tests in TestSanitizeConfigKey: comma, CR, LF, NUL, TAB, non-ASCII, secret-marker, empty, oversized, max-length, and three end-to-end tests through _per_control_attributes(). PluginError indeterminate decision (vishu-bh finding): - mark_plugin_error() now accepts hook='pre'|'post'|'' and stores it in _plugin_error_hook for enforcement_point derivation. - aggregate() omits cpex.control.result.allowed entirely when plugin_errored=True and no denial flag is set (decision is indeterminate — the chain did not complete). When both plugin_errored and a denial flag are set, denial takes precedence and result.allowed is emitted as False. - _enforcement_point() gains a third fallback tier: plugin_error_hook, so a first-plugin crash still reports the correct enforcement point rather than 'none'. - tool_service.py: _ctl_last_hook tracks 'pre'/'post' across all invoke sites; outer except PluginError passes hook=_ctl_last_hook. - 9 new tests in TestPluginErrorIndeterminate. - Updated TestMarkPluginError::test_plugin_error_flag_bypasses... to assert result.allowed is ABSENT (was: True) for the first-plugin error path. 849 plugin unit tests pass; ruff all-clear; pylint 10.00/10. Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * fixed ruff format Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * fix: tolerate stale merge-queue ref on SARIF upload When a merge-queue attempt is ejected (e.g. transient Docker cache error) the ephemeral ref refs/heads/gh-readonly-queue/main/pr-* is deleted. Any retried attempt carries the same stale ref baked into the job context, so codeql-action/upload-sarif fails with 'ref not found', failing fedramp-compliance and scan jobs and ejecting the queue again -- workflow_run_attempt 1, 2, and 3 all hit this loop. Add continue-on-error: true to both Upload SARIF to CodeQL steps so a stale-ref rejection from the CodeQL API does not fail the job. The scan result is unaffected; SARIF lands on the push-to-main run that follows a successful merge. Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * update secrets Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * undo .yml change Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> --------- Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>
* Release 1.0.7
Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>
* update stale 1.0.6 -> 1.0.7
Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>
* fix pre-commit
Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>
* fix: patch npm audit vulnerabilities
Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com>
* Fix e2e test wrapper : CI
Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>
* fix: pass AUTH_ENCRYPTION_SECRET to Generate JWT token step
The uv run invocation of create_jwt_token imports mcpgateway.config
at module level, triggering the full settings validator. DYNAMIC_ENC_SECRET
from GITHUB_ENV is not automatically named AUTH_ENCRYPTION_SECRET —
it must be explicitly mapped in each step's env: block that invokes Python.
Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>
* fix CI/pytest-rust
Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>
* add secrets.baseline
Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>
* fix(helm): set runAsUser so non-root containers can start
Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com>
* added change log for release 1.0.7
Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>
* Fix vulnerable packages found using pip-audit
Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>
* migration-fix
Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>
* fixed unit tests
Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>
* fixed patch.env and tests
Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>
* fix: warn at startup when CSRF cookie or header names are overridden
Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com>
* add replace_me to work in dev
Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>
* fix: compare CSRF header name case-insensitively and pin header constants
Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com>
* Fix root test
Signed-off-by: Gabriel Costa <gabrielcg@proton.me>
* chore: improve support bundle redaction and settings secret typing
Derive the set of settings fields excluded from a generated support
bundle from the Settings model itself instead of a hand-maintained list.
A hardcoded list goes stale as settings are added; deriving it means any
field declared with a secret type is covered without a second edit. A
narrow name-based rule acts as a backstop for string-typed fields, with
SecretStr typing remaining the enforced convention.
Type csrf_secret_key and identity_claims_secret as SecretStr. Their
fallback behaviour is unchanged: both still derive from JWT_SECRET_KEY
when left unset. Note that SecretStr("") is truthy, so the emptiness
checks guarding those fallbacks go through get_secret_value(), and the
assignments wrap explicitly because Settings does not enable
validate_assignment.
Strip credentials from every string-valued *_url setting rather than two
named ones, and mask environment variables whose names end in _URL, so
inline credentials in connection strings are handled uniformly.
Describe what each bundle member actually offers in the bundle README,
the module docstring, and the Admin UI blurb: deterministic exclusion for
the settings file, name-based masking for environment variables, and
best-effort pattern matching for logs. Prompt the operator to review a
bundle before sharing it.
Document that CSRF_SECRET_KEY reuses JWT_SECRET_KEY when unset, so the
two can be set separately and rotated independently.
Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
* fix: redact credentials from settings by value rather than field name
Two gaps in the support bundle's settings redaction, both raised in
review.
The userinfo pattern shared by the URL and log sanitizers required at
least one character for the username, so a DSN of the form
scheme://:password@host — accepted by several clients, and the common
Redis shape — did not match and the password reached the bundle
verbatim. The username group now matches zero characters.
Sanitization also keyed off the field name, applying only to keys ending
in _url and only to scalar strings. A field name says nothing about the
shape of the value behind it, so this missed collection-typed settings
entirely: webhook_logging_urls (list[str]), siem_destinations
(list[dict], whose entries hold per-destination tokens), and
otel_exporter_otlp_headers, a key=value blob that conventionally carries
an Authorization header and whose name matches no secret-name rule.
Every settings value is now walked and sanitized on its content. Strings
are matched against the same patterns already applied to log text; lists
and dicts are walked element-wise. Inside a dict, a key whose name looks
like a secret has its value replaced outright, since the value carries no
pattern to match on — that check is deliberately broader than the one
screening our own typed settings, because nested keys come from
operator-authored config where "token" is a credential far more often
than it is part of a benign name like token_expiry.
An empty string now stays empty instead of becoming null, since "set but
blank" and "unset" are different facts to whoever reads the bundle.
Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
* updated changelog
Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>
* Fix secretsbaseline
Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>
* added internal issues to security fixes to changelog
Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>
* removed duplciate entries on changelog
Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>
* updated changelog
Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>
---------
Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>
Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com>
Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com>
Signed-off-by: Gabriel Costa <gabrielcg@proton.me>
Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
Co-authored-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com>
Co-authored-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com>
Co-authored-by: __cafa__ <47299428+cafalchio@users.noreply.github.com>
Co-authored-by: Gabriel Costa <gabrielcg@proton.me>
Co-authored-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
Signed-off-by: Lang-Akshay <akshay.shinde26@ibm.com>
* feat: add v1 plugin discovery API Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> * fix: address plugin discovery review feedback Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> * fix: rebase plugin permission migration Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> * fix: log plugin discovery failures Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> * fix: rebase plugin permission migration Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> --------- Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com>
…#5765) * feat(mcp-apps): support standard MCP messages over the AppBridge The AppBridge accepted only tools/call and rejected everything else with -32601. The MCP Apps specification lets an app send four standard MCP messages, so an app could not read a resource, log to the host, or check connection health through the gateway. resources/read, notifications/message and ping are now routed. Session ownership and the stored server binding are validated before any method is dispatched, so every bridge method inherits the scoping tools/call already had: - resources/read reuses the identity and team scoping captured on the session and is confined to the bound server, so an app cannot reach resources its originating user could not read. The gateway-id header is stripped as it is for tool calls. - notifications/message is recorded for observability and never proxied upstream, matching the lifecycle in the specification. - ping is answered by the gateway without an upstream call. The allowlist is explicit rather than derived from the core MCP method set, so core methods such as tools/list, resources/list, prompts/list and resources/subscribe stay rejected. The advertised bridge capability and the router now share one constant so they cannot drift apart. Signed-off-by: PhilipAD <9472774+PhilipAD@users.noreply.github.com> * feat(mcp-apps): authorize AppBridge resources/read per method The RPC endpoint is guarded by `tools.execute`, which was the whole surface when `tools/call` was the only bridge method. Now that it also dispatches `resources/read`, the endpoint decorator no longer proves the caller may read resources: token and team scoping decide what a caller can see, RBAC decides whether they may read at all, and the two are separate layers. A caller who keeps `tools.execute` after `resources.read` is revoked could otherwise keep reading through an existing AppBridge session. `resources/read` now runs the same `_ensure_rpc_permission` check that normal MCP RPC routing applies, with an endpoint-level regression test that drives the real route (rather than `__wrapped__`) so the route decorator and the method check are both exercised. `notifications/message` no longer returns a JSON-RPC result. A notification carries no id and must not receive a response, so it is acknowledged at the transport level with 202 Accepted and an empty body, matching the Streamable HTTP rule for notification-only input and this gateway's own internal notification endpoint. Also drops the duplicate local `mcp_method_registry` import in `_handle_rpc_authenticated()` that shadowed the new module-level binding (pylint W0621/W0404), extracts the tools/call branch into a helper to keep the endpoint within the return-statement limit, and makes the architecture doc's opening summary agree with the supported-method table. Signed-off-by: PhilipAD <9472774+PhilipAD@users.noreply.github.com> * feat(mcp-apps): reject an id-bearing AppBridge notifications/message JSON-RPC distinguishes a notification from a request by the presence of the id member, not by its value, so "id": null is still a request that requires a response. The handler treated every notifications/message as a notification and answered with an empty 202, which would silently discard a response an id-bearing caller is waiting on. The request form is now rejected with -32600 rather than acknowledged, since notifications/message is defined as terminating at the host with no reply. The success test omits the id member entirely, and a parametrized regression test covers the id-bearing form across null, string and numeric ids. Part of IBM#2527. Signed-off-by: PhilipAD <9472774+PhilipAD@users.noreply.github.com> * feat(mcp-apps): share AppBridge header stripping and cap notification log fields Extract the gateway routing header strip into _app_bridge_request_headers so resources/read and tools/call share one source of truth, cap the level and logger fields of an AppBridge log notification the same way the data field is capped, and cover the plugin error mapping of resources/read. Signed-off-by: PhilipAD <9472774+PhilipAD@users.noreply.github.com> --------- Signed-off-by: PhilipAD <9472774+PhilipAD@users.noreply.github.com> Co-authored-by: PhilipAD <9472774+PhilipAD@users.noreply.github.com>
* fix: reject narrowed admin tokens on OAuth DCR management endpoints Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * fix(tests): pass mock_admin_request to DCR management router calls after rebase Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> --------- Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>
Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>
* fix(teams): scope admin team listings by token teams Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com> * test(teams): cover public-only admin team listings Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com> --------- Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com>
Signed-off-by: cafalchio <mcafalchio@gmail.com>
* Update Removed for mcpgateway/wrapper.py Signed-off-by: Jonathan Springer <jps@s390x.com> (cherry picked from commit 927c1a5) Signed-off-by: Jitesh Nair <jiteshnair@ibm.com> * fix: restore missing test alias help edits Restore the Makefile help edits lost from commit e04a3d2. Remove deprecated markers from test-mcp-cli and test-mcp-rbac. Both targets remain plain aliases for test-e2e. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * reverted deprecation removal Signed-off-by: Jitesh Nair <jiteshnair@ibm.com> --------- Signed-off-by: Jonathan Springer <jps@s390x.com> Signed-off-by: Jitesh Nair <jiteshnair@ibm.com> Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> Co-authored-by: Jonathan Springer <jps@s390x.com> Co-authored-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
* test: add HTTP-stack tests for candidate-only handshake and fix field descriptions Signed-off-by: Rakhi Dutta <rakhibiswas@yahoo.com> (cherry picked from commit 9dd5764) Signed-off-by: Jitesh Nair <jiteshnair@ibm.com> * test: add live-gateway e2e tests for candidate-only credential handshake Signed-off-by: Rakhi Dutta <rakhibiswas@yahoo.com> (cherry picked from commit 4bd2ce0) Signed-off-by: Jitesh Nair <jiteshnair@ibm.com> * test: fix handler state isolation and defensive id extraction Signed-off-by: Rakhi Dutta <rakhibiswas@yahoo.com> (cherry picked from commit 817b209) Signed-off-by: Jitesh Nair <jiteshnair@ibm.com> * test: strengthen candidate-only handshake e2e coverage Signed-off-by: Rakhi Dutta <rakhibiswas@yahoo.com> (cherry picked from commit dad9c48) Signed-off-by: Jitesh Nair <jiteshnair@ibm.com> * test: adapt restored candidate handshake fixtures Restore permission decorators for HTTP-stack checks. Add type hints to recovered live tests. Correct the temporary database description. Refs IBM#6877 Signed-off-by: Jitesh Nair <jiteshnair@ibm.com> --------- Signed-off-by: Rakhi Dutta <rakhibiswas@yahoo.com> Signed-off-by: Jitesh Nair <jiteshnair@ibm.com> Co-authored-by: Rakhi Dutta <rakhibiswas@yahoo.com>
…6682) The Gateway model in mcpgateway/common/models.py annotated last_seen twice. An annotation without assignment does not remove the earlier = None default; the field stays optional with default None on both revisions (verified with Pydantic 2.13). Treat the removal as duplicate-annotation cleanup, not a behavior fix. Add contract tests in tests/unit/mcpgateway/test_models.py that pin last_seen's None default and datetime round-trip. Closes IBM#6681 Signed-off-by: Jonathan Springer <jps@s390x.com>
…BM#7181) * feat: emit ttlMs and cacheScope on modern list, read and discover results Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> * fix: address review on ttlMs and cacheScope emission Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> * fix: make the live cache-directive test deterministic Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> * chore: refresh secrets baseline after rebase Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> --------- Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> Co-authored-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
…eseries (IBM#7138) * feat(observability): add status counts to timeseries Signed-off-by: Clare Byrne <clare.byrne1@ibm.com> * test(observability): verify aligned status response contract Signed-off-by: Clare Byrne <clare.byrne1@ibm.com> * test(observability): add PostgreSQL parity coverage Signed-off-by: Clare Byrne <clare.byrne1@ibm.com> * docs(observability): document non-terminal trace counts Signed-off-by: Clare Byrne <clare.byrne1@ibm.com> * test(observability): remove gated PostgreSQL test Signed-off-by: Clare Byrne <clare.byrne1@ibm.com> * test(observability): verify viewer status counts Add a live regression that records successful and denied MCP requests. Verify exact timeseries deltas with a platform_viewer token. Signed-off-by: Clare Byrne <clare.byrne1@ibm.com> * chore: refresh secrets baseline Signed-off-by: Clare Byrne <clare.byrne1@ibm.com> * test(observability): isolate live status count regression Run the status-count regression against a dedicated gateway and temporary SQLite database. Preserve exact platform_viewer assertions without shared traffic. Signed-off-by: Clare Byrne <clare.byrne1@ibm.com> --------- Signed-off-by: Clare Byrne <clare.byrne1@ibm.com> Co-authored-by: Clare Byrne <clare.byrne1@ibm.com>
Scoped tokens received 403 on /a2a-agents/plugin-bindings because the path never matched ^/a2a(?:$|/). Align middleware permissions with the router decorators (tools.read / tools.manage_plugins) and map tool plugin_bindings POST/DELETE the same way instead of tools.update/delete. Closes IBM#6701 Signed-off-by: vyrnsynx <153433026+vyrnsynx@users.noreply.github.com>
Give the scoped-token scenario a distinct test name. Use TeamManagementService in its fixture. Assert ownership and membership checks. Refs IBM#6004 Signed-off-by: Jitesh Nair <jiteshnair@ibm.com>
…BM#7190) * fix(oauth): restore private_key_jwt follow-ups dropped from IBM#6991 The pre-merge rebase of IBM#6991 dropped the follow-up commits that had been pushed to its branch, so the squash on main lacks save-time key validation, shared method normalization, and per-retry assertion re-signing that the PR description documents. This restores them on top of current main. Signed-off-by: Mohan Lakshmaiah <mohan.economist@gmail.com> * test(oauth): scope live tools and verify retry assertions Verify fresh client assertions across all six token flows after retry. Signed-off-by: Jitesh Nair <jiteshnair@ibm.com> * rebased and fixed precommit Signed-off-by: Jitesh Nair <jiteshnair@ibm.com> --------- Signed-off-by: Mohan Lakshmaiah <mohan.economist@gmail.com> Signed-off-by: Jitesh Nair <jiteshnair@ibm.com> Co-authored-by: Jitesh Nair <jiteshnair@ibm.com>
…7183) * test: validate pre-provisioned SSO users with IdP tokens Validate API-provisioned users with HTTPS Keycloak bearer tokens. Cover DB permissions, team changes, inactive accounts, provider mismatch, and the disabled provisioning API. Restore shared provider settings and remove fixture data. Closes IBM#6583 Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com> * docs: document SSO user provisioning workflow Document provider IDs, external-token setup, directory sync, and provisioning records. Closes IBM#6584 Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com> * test: cover PKCE login and upstream tool execution Verify public-client PKCE login and a real MCP echo call. Reject wrong verifiers, callback-state mismatches, and execution without permission. Refs IBM#6583 Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com> * test: verify provider binding after MCP mismatch attempts Check saved provider binding after REST and MCP mismatch attempts. Record MCP authentication separately; retain REST denial and valid DB permissions. Refs IBM#6583 Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com> * test: cover MCP denial for missing and inactive SSO users Check missing and inactive accounts through MCP. Document issuer trust limits for provider binding. Refs IBM#6583 Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com> --------- Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com>
* fix: let a real upstream JSON-RPC error through the error-response hook Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> * fix: preserve the upstream JSON-RPC error code in tool error results Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> * fix: match the SDK's error-body rules and cover direct proxy and live tool errors Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> * fix: label only upstream-sent JSON-RPC errors and carry result meta through coercion Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> * fix: report the tool error code in _meta without claiming its origin Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> * fix: remove the unused error-body helper Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> --------- Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com>
* feat: answer prompt failures with -32602 and relay upstream prompt errors Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> * fix: keep prompt errors transport-neutral and sanitise relayed error data Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> --------- Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com>
Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com>
…serverInfo stamping (IBM#6566) (IBM#7145) * feat: configure streamable HTTP server with app name and version for serverInfo stamping (IBM#6566) Configure ContextForgeMCPServer with settings.app_name and __version__ so the MCP SDK stamps the correct serverInfo metadata on modern responses. Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * test: add live gateway regression for serverInfo stamping (IBM#6566) Add black-box tests for the /mcp endpoint. Assert the modern serverInfo stamp carries APP_NAME and the package version. Assert a legacy initialize succeeds without modern metadata. Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> --------- Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>
IBM#6820) * feat(api): add named response models for A2A invoke and jsonrpc routes The three A2A invocation routes declared response_model=Dict[str, Any], so the generated OpenAPI schema described an arbitrary object and generated clients produced `unknown`. They now declare A2AInvokeResponse and A2AJsonRpcResponse, which share the JSON-RPC envelope fields. The envelope is intentionally permissive (Any leaves plus extra="allow"): these routes forward whatever the target agent returned, so a stricter type would turn agent drift into a gateway 500 instead of the agent's response. response_model_exclude_unset=True keeps that contract by serializing only the keys the agent actually returned. Refs IBM#6699 Signed-off-by: fei <204683769+feiiiiii5@users.noreply.github.com> * docs: add CHANGELOG entry for A2A named response models (IBM#6820) Refs IBM#6699 Signed-off-by: fei <204683769+feiiiiii5@users.noreply.github.com> * docs: correct A2A response routes and schema notes Signed-off-by: fei <204683769+feiiiiii5@users.noreply.github.com> * chore: refresh secrets baseline line numbers Signed-off-by: fei <204683769+feiiiiii5@users.noreply.github.com> --------- Signed-off-by: fei <204683769+feiiiiii5@users.noreply.github.com> Co-authored-by: Marek Dano <marekdano@users.noreply.github.com>
Register provisioning when the admin API and SSO are enabled. Retain the startup gate and existing authorization checks. Update configuration, documentation, and flag-specific tests. Verify external IdP authentication with the admin API disabled. Use the current MCP error-hook method in the live suite. Refs IBM#7218 Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com> Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
* added escapeHtml to the result body to neutralize HTML injection vector Signed-off-by: Lang-Akshay <akshay.shinde26@ibm.com> * fix(rust): upgrade rustls to 0.23.46 Raise the rustls minimum and update the workspace lockfile to resolve RUSTSEC-2026-0346 through RUSTSEC-2026-0349. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * fix(ci): complete rustls audit and fixture license metadata Record an automated delta audit for rustls 0.23.46. Declare the completion test server license as Apache-2.0. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * chore: remove optional rustls audit reviewer field Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * chore: add rustls audit reviewer details Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> --------- Signed-off-by: Lang-Akshay <akshay.shinde26@ibm.com> Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> Co-authored-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
* fix(otel): suppress tenantless background spans Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> * fix(tests): add required file header Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> * fix(otel): redact baggage and validate request trace policy Sanitize baggage keys before attribute prefixing. Preserve optional tracing imports and clarify processor sampling dependencies. Add live OTLP acceptance tests, health metric assertions, and observability CI coverage. Document metric semantics and upgrade defaults. Validation: 23830 unit tests and 1225 doctests pass. Both live OTLP cases pass. Changed-line coverage reaches 95%. Container E2E validation remains pending an isolated Docker engine. Full documentation builds and strict mypy retain existing blockers. Closes IBM#7085 Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> * fix(ci): include observability coverage and wait for trace export Append SDK-dependent test coverage before diff-cover. Wait for invocation spans and their exported parents in the live tracing test. Validation: 77 focused tests pass, including both live gateway cases. Observability changed-line coverage reaches 93.5%. Ruff and whitespace checks pass. Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> * chore: refresh secrets baseline line numbers Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> --------- Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com>
* fix(oauth): isolate exchanged tokens by subject credential Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * test(oauth): verify token exchange credential isolation Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * docs: clarify token exchange cache identity Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * chore: refresh secrets baseline after rebase Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> --------- Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> Co-authored-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
) * feat: allow any JSON value as structuredContent for 2026-era callers Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> * fix: keep structured results intact on internal dispatch and validate explicit null Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> * fix: compare JSON values strictly when checking for a text copy Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> --------- Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com>
* fix: enforce ownership on configuration import updates POST /v1/import with conflict_strategy=update overwrote tools, resources, prompts and gateways owned by other teams. Bulk registration looked up public rows by name with no tenant predicate and never checked ownership. The import call to update_gateway() omitted user_email, which skipped its ownership guard. Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * fix: close the tool import race and add ownership regression tests register_tools_bulk loaded conflicts twice. A tool published between the two queries was updated without an ownership check. Load conflicts once per chunk and reuse the result. Reject any name that appears after the load. Add database-backed ownership tests for tools, resources and prompts. Add a live-gateway test for the two-tenant import case. Refs: PSIRT 4084216 Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * chore: refresh secrets baseline after rebase Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> --------- Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> Co-authored-by: Brian Hussey <brian.hussey@ie.ibm.com>
…BM#7250) mcp-server-filesystem is being removed from contextforge-examples due to the security risks of exposing arbitrary filesystem paths. Replace all three doc files that referenced it: - docs/docs/using/reverse-proxy.md: replace filesystem-server entries in the CLI options example, Docker Compose service, and multi-server YAML with mcp-server-fetch equivalents (no volume mounts required) - docs/docs/using/mcpgateway-translate.md: replace the two mcp-server-filesystem usages with mcp-server-fetch; correct the accompanying JS snippet method from resources/list to tools/list - license-policy.toml: remove the filesystem-server allowlist entry and its comment (the crate is no longer present) mcp-server-fetch is the official Python MCP server from the modelcontextprotocol/servers repository, published on PyPI, and requires no arguments, credentials, or volume mounts. Signed-off-by: Jonathan Springer <jps@s390x.com>
* feat: adopt per-user MCP server rate limiting Adopt cpex-rate-limiter 0.1.11 and expose opt-in configuration. Normalize gateway-less plugin context identity. Add Redis acceptance coverage for both transport modes and gateway replicas. Closes IBM#6230 Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> * fix: skip plugin integration gate on draft PRs Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> * fix: declare completion test server license Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> * fix: address per-server rate limiter review feedback Require cpex-rate-limiter 0.1.12 for corrected prompt hooks. Cover scoped users, tenant isolation, wildcard bindings, and REST/MCP prompts. Run acceptance coverage through both enforcement paths. PyPI publication remains pending. Regenerate uv.lock after publication. Validation uses the merged upstream correction: 802 unit tests, 56 existing integration tests, and 24 live gateway cases pass. Static hooks pass with UV_NO_SYNC=1 while publication is pending. Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> * fix: lock published rate limiter 0.1.12 Resolve the published prompt-hook correction from PyPI. Update only cpex-rate-limiter and its lockfile metadata. Validation: 802 unit tests, 56 existing integration tests, and 24 live gateway cases pass against the published wheel. Normal pre-commit hooks and uv lock --check pass. Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> --------- Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com>
* chore: remove Rust MCP config surface (issue IBM#5819) Remove the experimental_rust_mcp_* family of settings, the RUST_MCP_MODE translation entrypoint, and all Rust MCP env vars from .env.example and docker-compose.yml. Changes: - mcpgateway/config.py: drop experimental_rust_mcp_runtime_enabled, experimental_rust_mcp_runtime_url, experimental_rust_mcp_runtime_uds, experimental_rust_mcp_runtime_timeout_seconds, experimental_rust_mcp_session_core_enabled, experimental_rust_mcp_event_store_enabled, experimental_rust_mcp_resume_core_enabled, experimental_rust_mcp_live_stream_core_enabled, experimental_rust_mcp_affinity_core_enabled, experimental_rust_mcp_session_auth_reuse_enabled, mcp_rust_ingress, mcp_rust_public_proxy_upstream, and the _validate_experimental_rust_runtime_uds field validator - docker-entrypoint.sh: remove apply_rust_mcp_mode_defaults(), print_mcp_runtime_mode(), start_managed_rust_mcp_runtime(), all RUST_MCP_*/EXPERIMENTAL_RUST_MCP_*/MCP_RUST_* variable declarations, and the conditional managed-sidecar startup branch - .env.example: remove Rust MCP (simple) block and residual inline references in the auth cache section - docker-compose.yml: remove ENABLE_RUST/ENABLE_RUST_MCP_RMCP build args and all Rust MCP environment entries from the gateway service; add '# pragma: allowlist secret' to 16 pre-existing false-positive vault-radar findings (env var references and placeholder values) - tests/unit/mcpgateway/test_config.py: remove four tests asserting on deleted fields - tests/unit/test_docker_entrypoint.py: remove test_print_mcp_runtime_mode_warns_when_rust_enabled All settings defaulted to false/disabled and served a runtime past its sunset date (2026-07-07). Hard removal is appropriate. Closes IBM#5819 Signed-off-by: Jonathan Springer <jps@s390x.com> * chore: suppress vault-radar false positives in test_token_exchange_credential_isolation.py Add '# pragma: allowlist secret' to 6 lines flagged as password_assignment findings by Vault Radar. All reference test-only variables (_PASSWORD, admin_password) or pass them as JSON payload fields in live gateway tests, not real credentials. Suppressed lines: 200, 236, 244, 261, 284, 287. Signed-off-by: Jonathan Springer <jps@s390x.com> --------- Signed-off-by: Jonathan Springer <jps@s390x.com>
* fix: paginate downstream MCP catalog responses Add bounded catalog pages and scope-bound cursors for Python and Rust runtimes. Store direct-proxy continuation snapshots in Redis. Closes IBM#6990 Signed-off-by: Vishu <vishu@dhcp-9-224-185-57.iea-ie.ibm.com> * fix: decode catalog visibility for native Rust pagination Accept the Python authorization response field and test the backend contract. This enables native PostgreSQL catalog queries instead of fallback. Refs IBM#6990 Signed-off-by: Vishu <vishu@dhcp-9-224-185-57.iea-ie.ibm.com> * test: reconcile pagination coverage with upstream changes Adapt the header-validation fixture to the paginated catalog adapter. Refresh the secret baseline from main and preserve audited synthetic cursor vectors. Refs IBM#6990 Signed-off-by: Vishu <vishu@dhcp-9-224-185-57.iea-ie.ibm.com> * chore: vet cursor cryptography dependencies Extend existing RustCrypto publisher trust to seven cursor cryptography crates. Verify locked release publishers and refresh imported audit metadata. Refs IBM#6990 Signed-off-by: Vishu <vishu@dhcp-9-224-185-57.iea-ie.ibm.com> * fix: close pagination parity and authorization coverage gaps Preserve resource size in Python listings and verify native PostgreSQL visibility across pages. Check virtual server existence during Rust authentication. Return MCP tool errors for invocation failures. Add PostgreSQL CI execution and resolve both reported E2E failures. Signed-off-by: Vishu <vishu@Vishus-MacBook-Pro.local> * refactor: delegate MCP catalog pagination to Python Remove native Rust catalog pagination and its cursor dependencies. Delegate catalog requests to Python with cursors, metadata, and authorization intact. Add delegation and visibility regression tests. Fix a shared service mock that leaked between tests. Refs IBM#6990 Signed-off-by: vishu bhatnagar <vishu.bhatnagar@ibm.com> * fix: preserve MCP routing across session affinity Preserve gateway selection across local and forwarded requests. Advertise MCP Apps from authenticated request scopes. Preserve expanded resource URIs and expected tool failures. Isolate asynchronous E2E tests from synchronous Playwright fixtures. Correct correlation-reference fixtures and document Bandit false positives. Enforce forward-listener initialization without removable assertions. Refs IBM#6990 Signed-off-by: vishu bhatnagar <vishu.bhatnagar@ibm.com> * test: isolate session registry dependency imports Import missing-dependency cases into separate modules. Preserve the shared registry class and availability flags across tests. Signed-off-by: vishu bhatnagar <vishu.bhatnagar@ibm.com> * fix: address Python MCP pagination review findings Normalize protocol and internal session headers for cursor binding. Exclude tracing headers while preserving authorization and tenant bindings. Bound Redis snapshot bytes, snapshot count, and concurrent collectors. Serialize prompts without gateway relationship queries and log sanitized failures. Remove Rust changes and unrelated tool-error, resource-URI, and Apps changes. Document Python-only scope, compatibility, Redis limits, and rollout requirements. Preserve public complete-list helpers for external compatibility. Validation: 1004 post-rebase regression tests pass, with one skipped. The earlier full unit suite passes 23989 tests. Eight production pagination tests pass. Full E2E retains five failures outside pagination. The merge gate remains incomplete. Refs IBM#6990 Signed-off-by: vishu bhatnagar <vishu.bhatnagar@ibm.com> Signed-off-by: Vishu <vishu@MAC-C6CA73.station> * test: fix pagination CI regressions after rebase Update the untrusted server-header test to mock list_catalog_page(). Preserve the assertion that the request body controls server selection. Refresh audited secrets baseline line numbers after the rebase. Validation: 607 RPC module tests pass. All pre-commit hooks pass. Refs IBM#6990 Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> * test: preserve cache directives with paginated adapters Update cache directive mocks to use the paginated catalog handler. Assert that list adapters retain cache directives and nextCursor. Refresh audited baseline line numbers after the rebase. Validation: pagination and transport tests pass with one legacy skip. Relevant pre-commit hooks pass. Refs IBM#6990 Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> * fix: address remaining Python pagination review comments Preserve prompt titles and SDK descriptors in paginated adapters. Separate collector slots from retained snapshot bytes and permit 32 collectors. Keep publication quotas atomic and make Redis memory information optional. Document gateway-update cursor invalidation and remove unrelated changes. Validation: 24279 unit tests, 155 pagination tests, and eight Redis tests pass. Seven production-image pagination and authorization tests pass. Ruff, interrogate, pylint, pre-commit, coverage, and secret checks pass. Changed-line coverage reaches 98 percent. The production image builds. Full E2E retains fixture and event-loop failures in the isolated setup. Bandit reports existing findings after requested scope cleanup. Refs IBM#6990 Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> * fix: resolve downstream pagination review concerns Reuse scoped proxy snapshots and release completed traversals. Prevent collection after snapshot reuse expires. Correct internal error statuses and preserve complete tools lists for bodyless bridges. Authenticate proxy scope fingerprints with a separate HMAC key. Reuse serialized descriptors and log sanitized cursor rejection categories. Align Compose collector defaults and document compatibility changes. Remove unrelated annotations and test changes. Add regression coverage for snapshot capacity, ordering, visibility, bridge compatibility, and error mapping. Validation: 24,379 unit tests, 10 Redis tests, and seven live tests pass. Changed-line coverage reaches 99 percent. Ruff, interrogate, Pylint, and changed-file pre-commit checks pass. Native Rust and the full three-replica gate remain unverified. Refs IBM#6990 Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> * fix: preserve shared catalog snapshots until expiry Keep shared snapshots available for overlapping traversals and final-page retries. Preserve fixed expiration and enforce retained byte and snapshot limits. Add service, Redis, and live MCP regression coverage. Update snapshot lifetime guidance. Validation: 905 tests pass; one test skips. Type checks pass. Ruff reports five existing blank-line errors in the live-test file. Refs IBM#6990 Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> * chore: refresh secrets baseline Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> --------- Signed-off-by: Vishu <vishu@dhcp-9-224-185-57.iea-ie.ibm.com> Signed-off-by: Vishu <vishu@Vishus-MacBook-Pro.local> Signed-off-by: vishu bhatnagar <vishu.bhatnagar@ibm.com> Signed-off-by: Vishu <vishu@MAC-C6CA73.station> Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> Co-authored-by: Vishu <vishu@dhcp-9-224-185-57.iea-ie.ibm.com> Co-authored-by: Vishu <vishu@Vishus-MacBook-Pro.local> Co-authored-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
* update dependencies 10-08 Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * test(a2a): tolerate fastapi.endpoint intermediate span in trace ancestry check fastapi 0.142.2 introduced built-in OTel telemetry that emits fastapi.dependencies, fastapi.endpoint, and fastapi.serialization child spans inside route handlers when a global TracerProvider is configured. The test gateway sets OTEL_TRACES_EXPORTER=console, which activates the global provider, so FastAPI's telemetry now inserts fastapi.endpoint between the request span and a2a.invoke. The previous assertion that a2a.invoke is a direct child of the request span therefore fails. Add _is_descendant() which walks the parent chain from a child span up to any ancestor using all exported spans. Replace the direct-parent check with a descendant check so the test tolerates any number of intermediate framework spans. All correctness invariants are preserved: same trace ID end-to-end, inbound traceparent adopted, a2a.invoke span ID injected into the downstream request. Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * test(transport): use session.call_tool() to test Mcp-Param rejection directly mcp 2.3.0 added automatic HEADER_MISMATCH recovery to Client.call_tool(): on a -32020 rejection it relists tools and retries once, so the error never surfaces to the caller when the retry succeeds. Use client.session.call_tool() (ClientSession) instead. ClientSession has no retry logic, so the server-side -32020 rejection surfaces as MCPError directly. The server behaviour is unchanged: it still rejects mismatched and absent Mcp-Param-* headers before dispatch. Only the test entry point changes to bypass the SDK recovery layer. Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> --------- Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>
* chore: Replace stylelint with biome Signed-off-by: Gabriel Costa <gabrielcg@proton.me> * feat:secrets updated Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> --------- Signed-off-by: Gabriel Costa <gabrielcg@proton.me> Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> Co-authored-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com>
IBM#7259) * fix: reject oversize _meta with -32602 and keep upstream resource error codes Validate request _meta on tools/call, resources/read and prompts/get before any service call, and answer a violation with -32602 instead of -32603. Exempt the mandatory io.modelcontextprotocol envelope keys from the key-count and depth limits so modern clients that declare nested capabilities pass. Relay the upstream JSON-RPC error of a failed resources/read through ResourceUpstreamError. A modern caller sees -32602 in place of the retired -32002 on resources/read and prompts/get. Refs IBM#6568 Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * addressed comments Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * chore: update secrets baseline line numbers Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> --------- Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>
* fixed: config for MCP inbound protocol mode in docker to auto Signed-off-by: Lang-Akshay <akshay.shinde26@ibm.com> * fixup: make pre-commit Signed-off-by: Lang-Akshay <akshay.shinde26@ibm.com> --------- Signed-off-by: Lang-Akshay <akshay.shinde26@ibm.com>
Verify modern MCP elicitation through a live gateway for URL and form modes. Check accept, decline, cancel, opaque state, and URL access. Closes IBM#6643 Signed-off-by: yashwant <ranayash2022@gmail.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Related issue
Closes #6643
Summary
Add six live-gateway cases for modern MCP elicitation: URL and form modes, each with accept, decline, and cancel.
The controlled upstream records retry payloads and returns a distinct result for each action. Assertions cover unchanged elicitation parameters, preserved request state, and action-only URL responses without content.
An HTTP request recorder detects access to the elicitation URL before and after the retry. A positive control verifies the recorder before federation.
Verification
Python 3.12.15, locked dependencies, source gateway at
a211ab7d8b13e8ea1aabad20ba18699af7629056, isolated SQLite database.MCP_INBOUND_PROTOCOL_MODE=auto MCP_CLIENT_CONNECT_MODE=auto RUST_MCP_MODE=off MCP_ELICITATION_UPSTREAM_HOST=127.0.0.1 LIVE_GATEWAY_REQUIRED=1 pytest -p playwright tests/live_gateway/e2e/test_e2e.py -k TestElicitationRelay -v --tb=short --no-covmake ruffmake interrogatemake pylint TARGET=mcpgateway/make ruff isort CHECK=1 TARGET=tests/live_gateway/e2e/test_e2e.pymake detect-secrets-scangit diff --checkPYTEST_XDIST_AUTO_NUM_WORKERS=2 make testpython -m pytest tests/unit/test_tls_compose.py -q --tb=shortdocker composeto official standalone Docker Compose v5.6.0The trailing slash on the Pylint target selects
.pylintrcdirectly. Windows checked out the.pylintrc.mcpgatewaysymlink as plain text.Remaining validation
The full suite completed. Its only failure was an unavailable Docker command, and the affected module passed after the local tooling correction.
The full suite was not repeated after that correction. This pull request remains a draft pending production-stack validation.
The Docker production-stack rebuild and full
make test-e2ewere not run because Docker integration is unavailable in WSL.The six new cases ran against a real gateway process with modern inbound and outbound MCP enabled.
make coverage diff-coverwas not run. This change adds test coverage without changing production code.