Repository navigation
Conversation
…M#5997) * fix: remove Python comment from JavaScript code in OAuth callback The OAuth callback endpoint was embedding a Python pragma comment (# pragma: allowlist secret) inside JavaScript code at line 838. This invalid syntax prevented the JavaScript fetch() call from executing, causing the 'Fetch Tools from MCP Server' button to silently fail after OAuth authorization. Removed the Python comment to fix JavaScript parsing. The button should now properly trigger the POST request to /oauth/fetch-tools. Signed-off-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com> * update .secrets.baseline Signed-off-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com> --------- Signed-off-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com> Co-authored-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com>
…5924) * feat(dataplane): publish per-backend add_headers/remove_headers The dataplane consumer (contextforge-gateway-rs IBM#4544) now accepts per-backend `add_headers` (map, overrides passthrough) and `remove_headers` (list, applied last) alongside the existing `passthrough_headers`. Thread both fields through the publisher so the published BackendConfig matches the consumer's contract. Sourced via getattr with the consumer-side defaults ({} / []) since the Gateway model has no columns for these yet; the publisher lights up automatically once the columns are added, with no further change here. Signed-off-by: Lang-Akshay <akshay.shinde26@ibm.com> * feat: added imgration and updated tests Signed-off-by: Lang-Akshay <akshay.shinde26@ibm.com> --------- Signed-off-by: Lang-Akshay <akshay.shinde26@ibm.com>
Align domain blocklist evaluation with the host used for the connection. Canonicalize both the parsed host and the configured entries through one helper so equivalent spellings compare equal: percent-encoding, case, IPv6 brackets, trailing dots, uncompressed address literals, and Unicode versus punycode domains. Add regression coverage. Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com>
Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com>
…BM#5631) * feat(observability): improve upstream MCP session error diagnostics Enhance error handling in upstream_session_registry to provide actionable, specific error messages when upstream MCP session creation fails. Replaces generic 'unhandled errors in a TaskGroup' message with categorized errors. Key improvements: - Unwrap ExceptionGroup before string conversion to preserve root cause - Categorize errors into 13 distinct types (connection_refused, timeout, ssl_tls, auth_unauthorized, auth_forbidden, dns_resolution, etc.) - Add structured logging with error_category, exception_type, and metadata for correlation across log aggregation systems - Include full traceback via exc_info for deep diagnosis - Surface error category in RuntimeError message for user visibility This enables operators to quickly identify whether failures are due to: - Network issues (connection refused/reset, DNS, timeouts) - Authentication problems (401/403) - SSL/TLS certificate issues - Upstream server errors (5xx) Error messages now display as: [connection_refused] ConnectionRefusedError: Connection refused [auth_unauthorized] HTTPStatusError: 401 Unauthorized [ssl_tls] SSLError: certificate verify failed [timeout] TimeoutError: Session initialization timeout Benefits: - Faster MTTR for production incidents - Actionable error messages without requiring verbose logging - Consistent error handling across MCP session modes - Better alerting and monitoring capabilities Testing: - All 68 existing upstream_session_registry tests pass - 10 new tests validate error categorization for all failure modes - Backward compatible (still raises RuntimeError) Closes IBM#5608 Signed-off-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com> * test(observability): improve structured logger exception handling test coverage Enhance test_structured_logger_exception_handling to ensure lines 462-465 are covered by properly mocking get_structured_logger at the module level where it's imported. The test now verifies that: - The structured logger is actually invoked (confirming except block is hit) - Primary error message is preserved when structured logging fails - Structured logger failures don't disrupt the main error flow Also fix f-string formatting in upstream_session_registry.py per ruff format. This brings coverage of the structured logging exception handler to 100%. Signed-off-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com> * update .secrets.baseline Signed-off-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com> * update .secrets.baseline Signed-off-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com> * feat(observability): address code review findings for upstream session error diagnostics Address all blocking issues and warnings from code review: 1. **Type Safety Enhancement** (Issue IBM#5) - Improve type: ignore comment with detailed rationale - Explain why type checker cannot infer concrete Exception after unwrapping - Lines: upstream_session_registry.py:387-391 2. **Logging Diagnostics Test Coverage** (Issue IBM#2) - Add test_logger_error_call_with_exc_info: verify logger.error called with exc_info - Add test_structured_logger_metadata_payload: verify structured logger metadata - Tests validate both standard logging (with traceback) and structured logging paths - Lines: test_upstream_session_error_categories.py:459-543 3. **Cross-Layer Consistency Regression Test** (Issue IBM#3) - Add test_cross_layer_error_message_consistency - Verify registry RuntimeError surfaces categorized text through consuming layer - Ensures fix remains effective across error propagation chain - Lines: test_upstream_session_error_categories.py:547-599 4. **Feature Documentation** (Issue IBM#1 - Blocking) - Add comprehensive "Upstream Session Error Diagnostics" section to observability-otel.md - Document all 13 error categories with descriptions and common causes - Include error message format examples - Provide structured logging metadata schema - Add monitoring/alerting examples (Prometheus, Datadog, Splunk) - Explain ExceptionGroup unwrapping behavior - Lines: observability-otel.md:651-756 All tests pass (18/18). Code review findings fully addressed. Related: IBM#5608 Signed-off-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com> * fix(observability): address code review findings for upstream session error diagnostics This commit implements all feedback from code review, addressing both blocking issues and suggested improvements to the upstream MCP session error diagnostics enhancement. ## Blocking Issues Fixed ### 1. Credential Sanitization (Security) - **Problem**: Exception messages with URLs containing secrets (API keys, tokens) were flowing unsanitized to client-facing RuntimeError messages, logs, and structured logging sinks - **Fix**: Use `sanitize_exception_message()` in `_categorize_upstream_error()` to redact sensitive query params before returning sanitized message - **Coverage**: Added regression tests for API key and Bearer token redaction ### 2. httpx Exception Type Categorization - **Problem**: Real httpx timeout/connection exceptions (httpx.ConnectTimeout, httpx.ReadTimeout, httpx.ConnectError) fell through to 'unknown' category - **Fix**: Check `isinstance(root_cause, httpx.TimeoutException)` to catch all httpx timeout types; handle httpx.ConnectError with message inspection for 'refused' vs generic connection errors - **Coverage**: Added regression tests for httpx.ConnectTimeout, httpx.ReadTimeout, and httpx.ConnectError with/without "refused" message ## Improvements Implemented ### Refactoring - Extracted error categorization logic into pure function `_categorize_upstream_error()` - Returns: (error_category, exception_type, sanitized_message, exception_count) - Makes taxonomy testable as pure function (no async task/transport mocking needed) - Enables future code reuse by tool_service.py for consistency ### Structured Logging Enhancements - Added `correlation_id` from request context for cross-layer correlation - Changed `metadata={...}` to `error_details={...}` for consistency with tool_service.py - `error_details` maps to dedicated column; `metadata` contains non-error context - Added debug logging for structured logger failures (was silent `except: pass`) ### Error Category Additions - Added `mcp_protocol_error` category for McpError (failed session.initialize()) - Tightened `ssl.SSLError` check to use isinstance() before string matching - Fixed httpx.ConnectError to categorize as 'connection_refused' when message contains "refused" ### Log Level Handling - Downgraded post-ready errors (teardown races) to WARNING level - Pre-ready failures remain ERROR (blocks session creation) - Only ERROR-level failures trigger structured logging (reduces alert noise) ### Exception Group Metadata - Track and log `exception_count` when BaseExceptionGroup contains multiple exceptions - Append " (N exceptions in group)" to log messages when count > 1 - Include `exception_count` in structured logging `error_details` ### Documentation Updates - Removed obsolete `MCP_SESSION_POOL_ENABLED` references from observability-otel.md - Updated to describe actual trigger: Mcp-Session-Id header presence - Added `mcp_protocol_error` to error categories table ## Test Coverage ### New Regression Tests (9 added) 1. `test_httpx_connect_timeout_category` - httpx.ConnectTimeout → timeout 2. `test_httpx_read_timeout_category` - httpx.ReadTimeout → timeout 3. `test_httpx_connect_error_with_refused_message` - "refused" → connection_refused 4. `test_httpx_connect_error_generic` - no "refused" → connection_error 5. `test_credential_sanitization_in_http_error` - API key redaction 6. `test_credential_sanitization_with_bearer_token` - Bearer token redaction 7. `test_mcp_protocol_error_category` - McpError → mcp_protocol_error 8. `test_ssl_error_category_with_isinstance_check` - ssl.SSLError via isinstance 9. `test_exception_group_with_multiple_exceptions_logged` - exception_count > 1 ### Modified Tests (2 updated) - `test_structured_logger_metadata_payload` - validates error_details structure - `test_post_ready_error_is_warning_level` - documents WARNING downgrade behavior ### Test Results - All 28 tests in test_upstream_session_error_categories.py pass - All 68 tests in test_upstream_session_registry.py pass (no regressions) - Total: 96 tests passing ## Security Impact **CRITICAL**: This fix prevents credential disclosure that was introduced by the original PR. Before this fix, URLs with secrets in query params (e.g., `?apiKey=secret123`) would leak to client-facing error messages. The sanitization now redacts all sensitive query params using static fallback patterns (api_key, token, password, etc.) and supports gateway-specific param names when available. ## Implementation Notes ### auth_query_params Threading Decision The review suggested threading gateway `auth_query_params_decrypted` through SessionCreateRequest for full credential redaction. However: - Static fallback in `sanitize_exception_message()` already covers common cases (api_key, token, password, Bearer tokens, etc.) - Threading through would require larger changes (SessionCreateRequest fields, all call sites, decryption at registry level) - Current implementation documents this trade-off in code comments ### Story 2 Consistency (tool_service.py) The refactored `_categorize_upstream_error()` function is now available for tool_service.py to call in a future PR, which would deliver full consistency across both session paths. This PR focuses on the registry path where the issue was reported. Closes feedback items from code review on IBM#5608 Signed-off-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com> * fix linter issues Signed-off-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com> * fix(upstream): handle real-world httpx exception chains and timeout paths Fixes three blocking issues in upstream session error categorization identified during real-socket e2e verification: 1. **connection_refused** detection: httpx.ConnectError wraps ConnectionRefusedError deep in __context__/__cause__ chain. Added _find_in_chain() helper to unwrap exception chains so real refused connections produce correct category instead of generic connection_error. 2. **ssl_tls** detection: Moved ssl.SSLError check to unwrap exception chains (httpx.ConnectError wrapping SSLError) and added fallback chain walk at end of categorization logic. 3. **timeout** at asyncio.wait_for call site: Owner task receives CancelledError (BaseException, excluded from except Exception), so asyncio.wait_for timeout never hit the owner's exception handler. Added explicit except asyncio.TimeoutError block at call site with categorization, sanitization, structured logging, and RuntimeError wrapping to match owner-task error path. 4. **Credential leak via exc_info**: Removed exc_info=exc from all logger.error/warning calls. Python's traceback formatter renders the raw exception __str__, bypassing sanitized message strings and reintroducing credential disclosure for HTTPStatusError. Added end-to-end integration tests (test_upstream_session_error_e2e.py) that validate fixes against real TCP sockets, blackhole listeners, and HTTP servers with no mocking. Updated test expectations: - test_logger_error_call_with_exc_info renamed to test_logger_error_call_without_exc_info and flipped assertion - test_default_session_factory_cancelled_path_runs_on_ready_timeout now expects RuntimeError wrapping TimeoutError with categorization All unit tests (86) and e2e tests (4) passing. Closes IBM#5608 Signed-off-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com> * fix(tests): filter caplog records by ERROR level in test_remove_root_generic_exception The test was failing after rebase because it captured all mcpgateway logs, including DEBUG level logs from get_db and user request logging. Updated the assertion to filter caplog records to only include ERROR level logs, ensuring only the expected "Failed to remove root" error message is validated. Signed-off-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com> * fix linter issues Signed-off-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com> * test: add coverage for uncovered lines in upstream_session_registry.py Add tests to cover previously uncovered lines: - Line 311: _find_in_chain return current path - Line 329: connection_refused fallback message check - Lines 375-377: SSL error detection via _find_in_chain - Line 542: WARNING level log post-ready - Lines 665-666: structured logging exception during timeout Coverage improved from 91.7% to higher coverage for upstream session registry. Tests added: - test_categorize_upstream_error_ssl_error_in_exception_chain - test_categorize_upstream_error_connection_refused_message_fallback - test_categorize_upstream_error_find_in_chain_returns_current - test_default_session_factory_logs_warning_on_post_ready_failure - test_default_session_factory_timeout_structured_logging_failure Signed-off-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com> * fixed pre-commit Signed-off-by: Jitesh Nair <jiteshnair@ibm.com> * fixed pre-commit Signed-off-by: Jitesh Nair <jiteshnair@ibm.com> --------- Signed-off-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com> Signed-off-by: Jitesh Nair <jiteshnair@ibm.com> Co-authored-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com> Co-authored-by: Jitesh Nair <jiteshnair@ibm.com>
…o-derived validation (IBM#4476) * fix(oauth): harden audience learning with issuer pinning and shape checks Address follow-up review feedback for the audience-learning fix to make the trust model defensible end-to-end and the data path defensive against unexpected IdP responses. Production code changes: - Factor the unverified JWT decode into _decode_token_claims_unverified() with a docstring explicitly documenting the trust model: signatures and exp/iss/aud are NOT verified; this path is for non-authoritative metadata extraction only; authoritative validation happens upstream when the token is presented to the protected resource. - Add _extract_token_issuer() paralleling _extract_token_audience(); both now reuse the shared decoder. complete_authorization_code_flow returns token_iss alongside token_aud. - _persist_learned_audience() now runs two defensive checks before writing. Shape validation: token_aud must be a non-empty string or non-empty list of non-empty strings; numbers, empty containers, mixed lists, and dicts are silently dropped so a malformed IdP response cannot pollute persisted state. Issuer pinning: when oauth_config has an 'issuer' configured, the token's iss must match it -- prevents a stale or misrouted token from a different AS injecting an audience for the wrong IdP. When no issuer is configured, behavior is unchanged. - Tighten _extract_token_audience return type from Any to Optional[Union[str, List[str]]]; reject claim values whose type doesn't match (numbers, mixed-type lists, dicts). Test changes: - TestExtractTokenClaims (test_oauth_manager.py): 13 unit tests covering the audience and issuer extractors -- string, list, missing claim, unexpected types, mixed list, opaque token, malformed JWT, empty input. - TestPersistLearnedAudience (test_oauth_router.py): parametrized shape-validation test plus 4 issuer-pinning tests (no issuer configured, matching iss, mismatching iss, missing iss). - test_token_validation_service.py: 2 round-trip tests confirming an opaque learned aud (string and list) successfully validates against the persisted opaque resource (the flagship PR scenario). - test_oauth_manager.py: existing assertion on complete_authorization_code_flow result dict updated to expect the new token_iss field. Signed-off-by: Jonathan Springer <jps@s390x.com> * fix(oauth): add diagnostic logs and multi-user regression test for audience learning Final follow-up bundle from review feedback: - Diagnosability (S4): emit DEBUG breadcrumbs on each silent skip path so operators chasing 'audience never learned' reports can distinguish between the causes: * _decode_token_claims_unverified logs when an exception is caught -- distinguishes 'token was opaque' (normal) from 'JWT library raised something unexpected' (worth investigating). DEBUG level keeps it out of production logs by default but available when troubleshooting. * _persist_learned_audience logs when token_aud is malformed/absent and when the resource is already set (first-write skip). The issuer-mismatch breadcrumb already existed. - Docstring (M5): add explicit Returns: section to _persist_learned_audience documenting the side-effect-only contract (oauth_config mutation + db.flush). - Multi-user regression test (M2): explicit guard for first-write-only semantics under the multi-user scenario. Two users complete the callback against the same shared gateway with different audience values; the second user's callback must observe the already-set resource and skip its write entirely. This keeps the protection visible even if the short-circuit logic is later refactored. - Test coverage for the new DEBUG logs: parametrized shape-validation test now asserts the breadcrumb is emitted; new test confirms decode failure logs at DEBUG; complement test confirms empty-input short-circuit does NOT log (would be noise). Signed-off-by: Jonathan Springer <jps@s390x.com> * feat(oauth): expose Resource field in UI, derive origin fallback, soften auto-derived validation Implement the maintainer-recommended Option-2 hybrid approach for issue IBM#4384 on top of the auto-learn fix: UI: OAuth Resource (Audience) field - Add 'Resource (Audience)' input next to Issuer URL in all four OAuth configuration forms (gateway create + edit, A2A agent create + edit) in templates/admin.html. - Wire up form parsing in admin.py to extract oauth_resource and merge it into oauth_config['resource']; supports comma-separated multi-value input per RFC 8707, stored as str when one value or list when multiple. - Pre-populate the field on edit dialogs from gateway.oauthConfig.resource in admin_ui/gateways.js and admin_ui/a2aAgents.js (handles both string and list shapes). Origin-extraction fallback (replaces full-URL fallback) - Add _derive_resource_origin() in oauth_router.py: returns scheme+netloc only, dropping path/query/fragment. Returns None for URN / scheme-less inputs so the caller falls back to auto-learn. - /oauth/authorize and /oauth/callback now use this helper instead of the full-path _normalize_resource_url(gateway.url) when no resource is configured. This matches real OAuth provider behavior (Salesforce, Azure AD, Okta typically issue origin-level audiences). Softened validation for auto-derived resources - TokenValidationResult gains audience_authoritative: bool = True. When False, audience mismatch is dropped from blocking_errors so the token still forwards (the upstream MCP server is the authoritative validator). - _validate_audience() sets audience_authoritative = bool(configured_resource) on mismatch. Configured (admin-set or learned) -> blocking; gateway-URL fallback -> advisory. Tests - Origin-extraction parametrized test (5 hierarchical URLs + 6 non-URL inputs) covers the new helper. - Validator authoritative/advisory split: 4 new tests on TokenValidationResult.blocking_errors and validate_oauth_token_claims covering configured-resource (blocking) and no-resource fallback (advisory) paths. Closes the remaining IBM#4384 acceptance criteria gaps: - 'OAuth Resource field is visible in Admin UI' - 'Auto-derives origin if not set' - 'Auto-derived resource logs warnings but doesn't block' - 'Existing gateways without resource field continue working' Signed-off-by: Jonathan Springer <jps@s390x.com> * fix(oauth): address PR IBM#4476 review findings from msureshkumar88 Blocking findings: - admin.py: Add Gateway form Resource field was a no-op — extract shared _parse_oauth_resource() helper (URI-aware, comma-in-URI safe) and wire it into _parse_gateway_data_from_request plus the three edit sites, eliminating a 9-line duplicated block copy-pasted 3x. - templates/admin.html: A2A edit dialog Discover button lost status feedback after the Resource-field restructuring — move the oauth-discover-status div back into the Issuer wrapper. Hardening (deep review): - config.py + token_validation_service.py: Add OAUTH_REQUIRE_CONFIGURED_RESOURCE setting (default false). - encryption_service.py: Add _OAUTH_CONFIG_RACE_PRONE_KEYS backfill so a stale UI save that omits resource does not silently wipe a callback-learned audience. Cleanup: - oauth_router.py: Remove dead _normalize_resource_url() (zero production call sites) plus its urlunparse import; delete both associated test classes. - oauth_manager.py: Consolidate JWT decode into _extract_aud_and_iss() single-decode helper. Tests: - test_gateway_service.py: TestFetchToolsAfterOAuthEnforcementPoint (4). - test_admin.py: TestParseOAuthResource + TestParseGatewayDataOAuthResource (20). - test_encryption_service.py: TestOAuthConfigRaceProneKeyBackfill (7). - test_token_validation_service.py: TestOAuthRequireConfiguredResourceSetting (4). Documentation: - templates/admin.html: Rewrite Resource-field help text to reflect the race-safe UI behavior. Signed-off-by: Jonathan Springer <jps@s390x.com> * docs(oauth): audience handling, resource configuration, and troubleshooting Absorb PR IBM#4477's docs onto this branch, with one accuracy correction to reflect the race-safe UI behavior added in the preceding commit. New file: - docs/docs/manage/oauth-resource-configuration.md — end-user guide to the RFC 8707 resource / audience field with provider patterns, multi- resource configs, and the API-based re-learn workflow. architecture/oauth-design.md: - Add 'Audience handling and RFC 8707' section documenting three-stage resolution, first-write-only + shape validation + issuer pinning, authoritative vs. advisory split, OAUTH_REQUIRE_CONFIGURED_RESOURCE escape hatch, race-safe backfill, and implementation references. - Remove obsolete 'Known Gap: No local audience/scope pre-validation' admonition. - Update 'Forwarded as-is' bullet to describe actual pre-validation. manage/oauth-troubleshooting.md: - Add 'Token audience mismatch' section: symptom, diagnosis, common causes and fixes, debug logs, gateway vs. upstream responsibility. - Cross-link to oauth-resource-configuration.md. manage/oauth.md: - Add Resource field to Admin UI field list with cross-link. - Add tip admonition explaining zero-config default. - Add 'See also' pointer in existing Resource Parameter section. manage/.pages: - Wire oauth-resource-configuration.md into nav. Accuracy correction vs IBM#4477: 'Forcing a Re-learn' via UI-clear no longer applies after the encryption backfill. Re-learn now requires a PUT with explicit resource: null. Applied consistently across all three touched files. Function/log/setting/error-string references verified against code. Signed-off-by: Jonathan Springer <jps@s390x.com> * fix(oauth): per-user learned audience (MEDIUM security fix) Reviewer's MEDIUM finding: the OAuth callback path enforces gateway access but not gateways.update, so allowing every authenticated callback to write shared gateway.oauth_config.resource let any user with gateway access mutate global state on behalf of all users, AND created a cross-tenant DoS in multi-tenant IdP setups where whichever user authenticated first pinned the audience for everyone else. Redesign: move callback-driven audience persistence from shared gateway.oauth_config.resource to per-user OAuthToken.learned_aud / learned_iss columns. Each user's OAuth callback only writes to their own row, and each user's tool-fetch validation only reads their own learned value. Schema: - Add learned_aud (JSON) and learned_iss (String) to OAuthToken - Alembic migration c9f8e7d6a4b3 (idempotent) Write path: - TokenStorageService.store_tokens accepts learned_aud/learned_iss; None-valued args preserve existing values. - OAuthManager.complete_authorization_code_flow forwards _extract_aud_and_iss output to store_tokens. - oauth_router.oauth_callback no longer calls _persist_learned_audience. Read path: - TokenStorageService.get_user_learned_audience returns tuple. - gateway_service.fetch_tools_after_oauth passes learned_aud to validator. - token_validation_service._validate_audience precedence: 1. oauth_config.resource (admin) — authoritative 2. learned_aud (this user) — authoritative for this user 3. gateway_url origin — advisory (or authoritative when OAUTH_REQUIRE_CONFIGURED_RESOURCE=true) Removed: _persist_learned_audience, _is_well_formed_audience, _OAUTH_CONFIG_RACE_PRONE_KEYS backfill, TestPersistLearnedAudience, TestOAuthConfigRaceProneKeyBackfill. Tests (2429 pass): TestLearnedAudiencePersistence (7), TestLearnedAudPrecedence (6), 3 new per-user enforcement tests. Docs: oauth-design.md, oauth-resource-configuration.md, oauth-troubleshooting.md, oauth.md, admin.html rewritten for per-user model. Signed-off-by: Jonathan Springer <jps@s390x.com> * fix(oauth): address 2nd-review MEDIUM + hardening (5 fixes) Follow-up to the per-user learned audience redesign (commit 45615752d). The 2nd-round review found the same class of RBAC-bypass bug persisted in a different location (the DCR persist path) plus 3 quality items and a migration docstring nit. MEDIUM — DCR path leaked auto-derived resource to shared config: - oauth_router.initiate_oauth_flow (DCR block) previously persisted the whole request-local oauth_config dict (including the auto-derived resource) to gateway.oauth_config via protect_oauth_config_for_storage. The /oauth/authorize/{gateway_id} route enforces gateway access but not gateways.update, so an authenticated caller could pin the shared audience for all users via origin derivation — the exact RBAC-bypass class the callback-path redesign eliminated by moving learned aud to OAuthToken.learned_aud. - Fix: build a persist_dict that strips the request-local 'resource' before writing to shared config. Admin-configured resource is preserved as-is; request-derived origin stays request-local only. - Regression tests (2 new): - test_initiate_oauth_flow_dcr_does_not_persist_request_local_resource: proves 'resource' is NOT in gateway.oauth_config after DCR. - test_initiate_oauth_flow_dcr_preserves_admin_configured_resource: proves an admin's explicit resource survives DCR persistence. LOW — decode-exception log echoed attacker-controlled parser output: - _decode_token_claims_unverified logged 'Unverified JWT decode failed (%s): %s' with type(exc).__name__ and str(exc). Malformed IdP tokens can produce parser exceptions whose string form includes token fragments. - Fix: log only 'Unverified JWT decode failed: %s' with the class name; drop the exception message text. MINOR — _coerce_aud_claim accepted empty aud shapes: - Empty string and empty list passed through _coerce_aud_claim as-is, which combined with the 'if learned_aud is not None' guard in store_tokens could silently overwrite a previously-learned per-user audience with garbage from a malformed IdP response. - Fix: reject empty strings and empty/whitespace-only-list shapes; return None so the store_tokens guard preserves existing state. Mirrors the removed _is_well_formed_audience semantics from the per-user redesign. MINOR — stale comments in oauth_router.py: - 3 sites (docstring of _derive_resource_origin + comments in initiate_oauth_flow and oauth_callback) still described 'the auto-learn callback will overwrite oauth_config.resource on the first successful flow'. That flow was removed; the callback now writes OAuthToken.learned_aud per-user. - Fix: reword to describe the actual behavior — request-local outbound derivation, per-user learned inbound validation via OAuthToken — and add pointers to the DCR strip logic + validator. NITPICK — migration downgrade docstring: - Said 'callback-driven persistence reverts to writing gateway.oauth_config.resource on downgrade' — but the downgrade only drops columns; the code that used to write there was also removed. - Fix: clarify that this is schema rollback only; full behavior reversion requires application-code rollback too. Regression test (Fix 5b) — two-user learned aud isolation: - New test_two_user_isolation_write_and_read in test_token_storage_service.py: * store_tokens for user A does not touch user B's row * get_user_learned_audience for user A returns only A's values * get_user_learned_audience for user B returns only B's values - Replaces the narrative that was implicit in the deleted TestPersistLearnedAudience — the 'first-user vs second-user ownership' scenario is now covered in a focused location. Verification: 899 tests pass, ruff clean. Signed-off-by: Jonathan Springer <jps@s390x.com> * test(oauth): direct empty-shape coverage for _coerce_aud_claim + blank stored resource DCR regression Closes two coverage gaps flagged by the 3rd-round review of commit b582ba25f: test_oauth_manager.py — TestExtractTokenClaims: - Parametrized test_coerce_aud_claim_rejects_empty_shapes covers 8 cases ('', ' ', '\t\n', [], [''], [' ', '\t'], ['valid', ''], ['valid', ' ']) — all must return None so a malformed IdP response cannot bypass the 'if learned_aud is not None' guard in TokenStorageService.store_tokens and silently overwrite a previously-learned per-user audience. - Parametrized test_coerce_aud_claim_passes_valid_shapes verifies string, list-of-strings, and single-element list pass through unchanged. - Parametrized test_coerce_aud_claim_rejects_non_string_types covers None, int, float, dict, mixed lists, and list-of-None — completes the shape-rejection matrix. test_oauth_router.py — TestOAuthRouterAdditionalCoverage: - test_initiate_oauth_flow_dcr_preserves_blank_stored_resource: documents the current DCR-persist semantics for gateway.oauth_config.resource == ''. The strip logic uses 'if stored_resource is None: pop' else preserve, so an empty string (falsy but not None) takes the preserve branch and is persisted as-is. This locks the intent: blank is 'explicit admin state' rather than 'no admin config'. If empty resource is later declared invalid, the fix belongs in gateway config validation, not in this DCR strip. Verification: 58 tests pass in the two affected classes; 4 ruff unused-variable warnings in test_oauth_manager.py:1602-1746 are pre-existing (unrelated to these additions). Signed-off-by: Jonathan Springer <jps@s390x.com> * test(gateway): stub get_user_learned_audience on mocked TokenStorageService fetch_tools_after_oauth() now awaits TokenStorageService.get_user_learned_audience() to resolve per-user learned audiences. Tests that mock TokenStorageService but only stubbed get_user_token raised 'MagicMock can't be used in await expression', which in the audience-mismatch tests surfaced as a pytest.raises regex mismatch rather than the intended GatewayConnectionError. Add AsyncMock(return_value=(None, None)) stubs alongside every get_user_token stub in the two affected test files. Behavior-preserving: (None, None) matches the 'no learned audience recorded yet' path already exercised elsewhere. Fixes 6 failing tests: - test_gateway_service_oauth_comprehensive::test_fetch_tools_after_oauth_success - test_gateway_service_oauth_comprehensive::test_audience_mismatch_blocks_before_connection - test_gateway_service_oauth_comprehensive::test_opaque_token_no_warnings - test_gateway_service_oauth_comprehensive::test_audience_mismatch_raises_before_connection_with_diagnostic - test_gateway_service_oauth_comprehensive::test_streamablehttp_401_includes_diagnostic_info - test_gateway_service_extended::test_fetch_tools_after_oauth_prompt_stale_removal_uses_original_name Signed-off-by: Jonathan Springer <jps@s390x.com> * chore(oauth): surface OAUTH_REQUIRE_CONFIGURED_RESOURCE + pin JS resource-field pre-population Two small items from PR IBM#4476 re-review: 1. .env.example: add OAUTH_REQUIRE_CONFIGURED_RESOURCE=false with an operator- facing comment explaining the advisory-vs-blocking tradeoff and pointing to docs/docs/manage/oauth-resource-configuration.md. Setting was already implemented in config.py and documented; only .env.example missed it, so operators scanning that file for available toggles could not find it. 2. tests/unit/js: pin the oauth-resource-{gw,a2a}-edit pre-population logic (gateways.js:530-535, a2aAgents.js:589-593) with 2 tests per file — one for the RFC 8707 string shape, one for the list shape (comma-space joined into the input value so the operator can edit it in the same form the admin parser expects on submit). Prevents a silent refactor regression on the list-shape branch that had no test coverage. Also refreshes .secrets.baseline line numbers to account for the 10-line .env.example addition (no new detected secrets). Signed-off-by: Jonathan Springer <jps@s390x.com> * fix(oauth): propagate origin-only resource to refresh + validator fallback Address PR IBM#4476 review from Lang-Akshay — two consistency defects where the origin-derivation helper added to oauth_router was not propagated to the sibling code paths that must agree on the auto-derived audience. 1. token_storage_service refresh path (services/token_storage_service.py:410). Previously used normalize_resource(gateway.url), sending the FULL URL (path included) as the RFC 8707 resource param on refresh — while the initial authorization and callback exchanges in oauth_router send only the ORIGIN (scheme + netloc). The mismatch causes IdPs to fail the refresh outright or mint a token whose aud no longer matches the audience the user already validated against on first login. Fix uses the shared _derive_resource_origin helper so all three outbound resource sites agree. 2. token_validation_service fallback (services/token_validation_service.py:208). _validate_audience previously fell back to gateway_url verbatim (full URL) when neither configured resource nor learned_aud was available. Under OAUTH_REQUIRE_CONFIGURED_RESOURCE=true, valid origin-level aud tokens (what real IdPs mint when we request an origin-level resource) get rejected as authoritative mismatches. Fix derives the origin so the fallback matches the value oauth_router sends on the outbound side; falls through to raw gateway_url only when derivation returns None (URN / scheme-less). Structural change: promoted _derive_resource_origin from mcpgateway.routers.oauth_router (previously private to the router module) to mcpgateway.services.token_validation_service, next to _derive_issuer_from_token_url. Services should not import from routers; the helper is now shared cleanly by the OAuth route, the refresh path, and the validator. oauth_router re-exports it so the existing test import continues to work. Test coverage: - test_token_storage_service: * test_refresh_derives_resource_origin_from_path_bearing_gateway_url — new, asserts refresh sends origin-only resource for a path-bearing gateway URL. * test_refresh_skips_resource_when_gateway_url_is_urn — new, URN gateway.url yields no origin, refresh omits resource + logs warning. * test_refresh_derived_gateway_url_normalizes_to_empty_logs_warning — removed. Was patching urllib.parse.urlunparse to force normalize_resource to return empty; that mechanism no longer exists on this path. The URN test above provides equivalent 'derivation returns None → warning' coverage. - test_token_validation_service: * test_audience_falls_back_to_gateway_url renamed to test_audience_falls_back_to_gateway_url_origin and updated to the new contract (token aud = origin, gateway_url = full path → match). * test_audience_fallback_uses_urn_verbatim_when_origin_not_derivable — new, pins the URN corner case where the raw value stays as fallback. * test_strict_mode_accepts_origin_aud_against_path_bearing_gateway_url — new, the exact regression Akshay flagged: OAUTH_REQUIRE_CONFIGURED_RESOURCE=true + path-bearing gateway URL + origin aud must MATCH. Signed-off-by: Jonathan Springer <jps@s390x.com> * fix(db): point learned_aud/iss migration at current main head The migration's down_revision targeted b6c7d8e9f0a1, but main advanced to d21698ae4a19 (RBAC unique-constraints fix, IBM#4636) after this branch was cut, producing two alembic heads. Repoint down_revision at d21698ae4a19; alembic heads now reports a single head (c9f8e7d6a4b3). Signed-off-by: Jonathan Springer <jps@s390x.com> * refactor(oauth): consolidate RFC 8707 resource handling into utils module Review-driven consolidation of everything resource-shaped, previously smeared across four modules: - New mcpgateway/utils/oauth_resource.py owns derive_resource_origin (the auto-derived audience heuristic shared by authorize/callback/refresh/ validator), normalize_resource (RFC 8707 fragment/query policy), and parse_oauth_resource_form. Eliminates the private cross-module imports (mid-file noqa:E402 import in oauth_router, function-level import in the refresh path). - _resolve_refresh_resource extracts the refresh-path resource policy from the 230-line _refresh_access_token, and restores the omit_resource=true early-return (regression vs. the documented contract: resource key must be absent, not merely unsent). - TokenValidationResult.audience_source enum (configured/learned/derived) replaces the post-hoc audience_authoritative flag; the strict-setting rule now lives in one expression in blocking_errors with no mutation on the mismatch path. - Tests: audience_source assertions migrated; two stale urlunparse-patch tests removed/retargeted to the new module; omit_resource health-suite test updated to expect origin derivation. Signed-off-by: Jonathan Springer <jps@s390x.com> * fix(oauth): pin learned audience to configured issuer at learning time Restores the issuer-pinning hardening that was lost in the per-user redesign: when oauth_config['issuer'] is configured, complete_authorization_code_flow now persists learned_aud/learned_iss only if the token's iss claim matches (trailing slashes normalized; a missing iss claim also refuses). A stale or misrouted token from a different AS can no longer inject an audience for the wrong IdP, and the None pass-through leaves any previously-learned per-user value intact. Also deletes the dead single-claim extractors (_extract_token_audience, _extract_token_issuer) — _extract_aud_and_iss is now the only extraction API — and retargets all extractor tests and stale patch.object mocks to it. Adds TestIssuerPinningOnAudienceLearning: 6 tests covering no-issuer, match, trailing-slash, mismatch, missing iss, and opaque-token cases. Signed-off-by: Jonathan Springer <jps@s390x.com> * fix(admin): consolidate OAuth form parsing and drop inert A2A Resource field - _assemble_oauth_config_from_fields replaces the four near-identical ~45-line OAuth form-parsing blocks (gateway create/edit, A2A create/edit) — the duplication that caused the round-1 review miss where the gateway create form silently dropped oauth_resource. encrypt_secret and include_resource parameters capture the only real differences between sites. - Removes the OAuth Resource (Audience) field from the A2A agent forms: review found no A2A code path consumes oauth_config['resource'] (no per-user token storage, refresh, or audience validation on the A2A path), so the field was stored-but-inert. A2A parsing sites pass include_resource=False. Proper A2A audience validation tracked in IBM#5815. Signed-off-by: Jonathan Springer <jps@s390x.com> * test: mock network timeouts in slow health-check and JWKS tests Four pre-existing tests each took exactly 30.01s (the health-check/network timeout) on machines without outbound network: - TestCheckSingleHealthAuthCode (x3) never mocked the HTTP layer, letting a real GET to http://gw.test ride out health_check_timeout. A new _patch_unreachable_http_client helper fails fast with httpx.ConnectError; the failure-handler assertions are unchanged. - test_verify_oauth_access_token_jwt_error patched jwt.PyJWKClient, but production code instantiates vc._NoRedirectPyJWKClient and caches it in _oauth_jwks_client_cache, so the patch was a no-op and a real JWKS fetch rode out the network timeout. Patch the class actually used and clear the shared cache. test_gateway_service.py drops from ~94s to ~4s; the verify_credentials file runs in ~0.3s. Signed-off-by: Jonathan Springer <jps@s390x.com> * chore: refresh detect-secrets baseline Two audited entries (test_oauth_audience.py, test_auth_ratelimiter_redis.py) no longer match current file content and were flagged stale by the detect-secrets pre-commit hook, failing CI. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> --------- Signed-off-by: Jonathan Springer <jps@s390x.com> Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> Co-authored-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
…M#5764) * fix(mcp-apps): honour deprecated flat ui/resourceUri metadata key Servers that emit only the deprecated flat `_meta["ui/resourceUri"]` key lost their tool/UI association when passing through the gateway, because `merge_mcp_protocol_meta` returned early when `_meta.ui` was absent. The flat key is now folded into the nested MCP Apps metadata during the same merge step, mirroring the normalization the reference SDK performs in `registerAppTool`. The canonical nested `_meta.ui.resourceUri` takes precedence when both shapes are present and disagree, matching the consumer-side rule in the SDK's `getToolUiResourceUri`. Only well-formed `ui://` strings are folded in. An invalid legacy value would otherwise fail `validate_extension_metadata` and cause the whole tool to be rejected during gateway sync, so unusable values keep the existing "ignore it" behaviour and are logged instead. Signed-off-by: PhilipAD <9472774+PhilipAD@users.noreply.github.com> * fix(mcp-apps): keep a present canonical resourceUri authoritative Nested-key precedence was decided by truthiness, so a present but falsey canonical value (`""`, `False`, `0`) was treated as absent and replaced by the deprecated flat URI. The reference consumer falls back only when the canonical value is undefined; a malformed canonical value stays authoritative and is rejected downstream by `validate_extension_metadata` rather than silently swapped. Precedence is now decided by key presence. Upstream-controlled URIs are also no longer logged verbatim: userinfo, query and fragment are stripped and the remainder is truncated, so a compromised upstream cannot use the ingest path to write unbounded or credential-bearing values into the gateway logs. Legacy values that are structurally unusable (`ui://` with nothing after the scheme) are now ignored alongside non-`ui://` values, so nothing is folded in that cannot resolve to a real UI resource. Signed-off-by: PhilipAD <9472774+PhilipAD@users.noreply.github.com> * fix(mcp-apps): make legacy URI log redaction order-independent Rebuild _loggable_uri on urlsplit/urlunsplit so the authority is resolved before any component is dropped. The previous implementation stripped the query and fragment first and only then looked for userinfo, so an unencoded delimiter inside the credential ("https://user:pa?ss@host/x") truncated the value mid-credential and leaked the surviving prefix into the log line. Stripping userinfo from the parsed netloc alone does not close this: for that input urlsplit yields a netloc of "user:pa" with no "@" left to strip. When an "@" appears in a component that is dropped -- or the URI is not hierarchical and has no authority at all -- nothing preceding it can be shown to be credential-free, so the authority is redacted instead. A URI that cannot be parsed is reported by length only. Also cover the shared ingest paths: a legacy-only tool now has a test through GatewayService._validate_tools confirming it survives ToolCreate validation, and the resource and resource-template paths are covered through the real connect_to_sse_server chain. Documents the legacy key, its precedence rule and its ingest-only scope in the MCP Apps architecture guide, and switches to str.removeprefix. Part of IBM#2527. Signed-off-by: PhilipAD <9472774+PhilipAD@users.noreply.github.com> --------- Signed-off-by: PhilipAD <9472774+PhilipAD@users.noreply.github.com> Co-authored-by: PhilipAD <9472774+PhilipAD@users.noreply.github.com>
* Update docker-scan to upload SARIF file with finding. Signed-off-by: Jonathan Springer <jps@s390x.com> * Containerfile updates & Try SBOM module Signed-off-by: Jonathan Springer <jps@s390x.com> * fix(ci): address docker-scan SARIF review findings - Grant security-events: write to fedramp-compliance and scan jobs so codeql-action/upload-sarif no longer 403s and ejects merge-queue PRs - Add push: [main] trigger so SARIF uploads land on the stable main ref instead of ephemeral gh-readonly-queue/* refs - Make scan-fedramp report-only (fail-build: false): the FIPS image is built from full ubi9/ubi on floating tags while .grype.yaml ignores were calibrated for the lite image's base - Set upload-artifact: false on both Generate SBOM steps to stop duplicate SBOM artifacts - Guard Display/Upload SARIF steps on a non-empty sarif output so a failed build no longer produces spurious cat/upload errors - Run Upload SBOM with if: always() so the triage artifact survives a grype failure - Refresh stale header and aggregator comments for the four-job layout Signed-off-by: Jonathan Springer <jps@s390x.com> --------- Signed-off-by: Jonathan Springer <jps@s390x.com>
…to testable submodule (IBM#6008) * refactor(middleware): extract auth-context middleware registration into testable submodule Move the CSRF / PasswordChangeEnforcement / AuthContext registration block out of mcpgateway/main.py into mcpgateway/middleware/auth_context_stack.py as register_auth_context_middleware(). Registration order is unchanged; the trio is extracted so its load-bearing ordering (AuthContext must run before CSRF so request.state.user is populated, issue IBM#5739) can be unit-tested against a bare FastAPI app without importing the assembled application singleton. Rewrite test_admin_csrf_binding.py accordingly: delete the five full-stack TestClient tests that depended on the shared mcpgateway.main.app singleton (they failed under the full xdist suite because middleware registration is import-time and settings-dependent, so AuthContextMiddleware was absent when another module imported mcpgateway.main first during collection), convert the enforce_admin_csrf block to direct-call unit tests with a mocked Request, and add hermetic ordering tests against the extracted function. End-to-end CSRF coverage remains with the Playwright security suite. Signed-off-by: Jonathan Springer <jps@s390x.com> * fix: align CSRF cookie name fallbacks with config default The getattr() fallback defaults in csrf_middleware.py and csrf_service.py used "csrf_token" while config.py defaults csrf_cookie_name to "mcpgateway_csrf_token". Align the fallbacks so a missing attribute yields the same cookie name as the configured default. Also patch test_config.py CSRF tests to supply JWT_SECRET_KEY and AUTH_ENCRYPTION_SECRET via patch.dict so Settings(_env_file=None) instantiates in a clean environment. Signed-off-by: Jonathan Springer <jps@s390x.com> --------- Signed-off-by: Jonathan Springer <jps@s390x.com>
* feat(gateway): add prompt and resource counts to GatewayRead Signed-off-by: Gabriel Costa <gabrielcg@proton.me> * Optimise queries + test Signed-off-by: Gabriel Costa <gabrielcg@proton.me> * Address comments Signed-off-by: Gabriel Costa <gabrielcg@proton.me> * Add tests Signed-off-by: Gabriel Costa <gabrielcg@proton.me> * Add helper for capability count Signed-off-by: Gabriel Costa <gabrielcg@proton.me> * Fix documentation Signed-off-by: Gabriel Costa <gabrielcg@proton.me> --------- Signed-off-by: Gabriel Costa <gabrielcg@proton.me>
* fix(security): enforce API token scope-based access control Signed-off-by: Rakhi Dutta <rakhibiswas@yahoo.com> * fix: add A2A API endpoint permission patterns and JWT token scope extraction for scoped token authorization Signed-off-by: Rakhi Dutta <rakhibiswas@yahoo.com> * test: add resource_scopes to API token test mocks Signed-off-by: Rakhi Dutta <rakhibiswas@yahoo.com> * token scipe naming Signed-off-by: Rakhi Dutta <rakhibiswas@yahoo.com> * docs(security): address PR review feedback - Add comment explaining JWT empty list handling behavior - Add token type detection documentation in RBAC middleware - Update TOKEN_SCOPE_NAMING.md references to correct line numbers - Remove reference to non-existent regression test file - Verify A2A route pattern coverage (all endpoints protected) All review issues addressed: 1. JWT empty list handling - Added clarifying comment 2. A2A route patterns - Verified all 19 endpoints have @require_permission 3. Token type detection - Added inline documentation 4. Test file reference - Updated to point to actual test file Refs: IBM#4671 Signed-off-by: Rakhi Dutta <rakhibiswas@yahoo.com> * fix(security): enforce scope checks for JWT tokens with empty permissions list to match database token behavior Signed-off-by: Rakhi Dutta <rakhibiswas@yahoo.com> * docs(security): add JWT malformed token logging and clarify naming convention permanence Signed-off-by: Rakhi Dutta <rakhibiswas@yahoo.com> * fix: prevent JWT empty dict scope bypass (empty dict is falsy in Python) Signed-off-by: Rakhi Dutta <rakhibiswas@yahoo.com> * test: add comprehensive scope enforcement test coverage Signed-off-by: Rakhi Dutta <rakhibiswas@yahoo.com> * remove test file Signed-off-by: Rakhi Dutta <rakhibiswas@yahoo.com> * fix(security): reject malformed JWT scopes and clarify session token design Signed-off-by: Rakhi Dutta <rakhibiswas@yahoo.com> * test(security): add layer 1+2 interaction test, reject malformed JWT scopes Signed-off-by: Rakhi Dutta <rakhibiswas@yahoo.com> * trailing newline Signed-off-by: Rakhi Dutta <rakhibiswas@yahoo.com> * pre-commit Signed-off-by: Rakhi Dutta <rakhibiswas@yahoo.com> * test coverage improve Signed-off-by: Rakhi Dutta <rakhibiswas@yahoo.com> * coverage Signed-off-by: Rakhi Dutta <rakhibiswas@yahoo.com> * fix(security): enforce token scopes and Layer 1 consistency Signed-off-by: Rakhi Dutta <rakhibiswas@yahoo.com> * pre-commit fix Signed-off-by: Rakhi Dutta <rakhibiswas@yahoo.com> * test cases fix Signed-off-by: Rakhi Dutta <rakhibiswas@yahoo.com> * test coverage Signed-off-by: Rakhi Dutta <rakhibiswas@yahoo.com> * fix(security): correct Layer 1 token scope enforcement semantics Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> * fix(security): grant servers.use to tokens with MCP method scopes Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> --------- Signed-off-by: Rakhi Dutta <rakhibiswas@yahoo.com> Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> Co-authored-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com>
* test: add SSRF validator with deny/allow coverage
Adds a stdlib-only, framework-independent URL validator (ssrf.py) that
enforces a scheme allowlist, rejects embedded credentials, applies an
optional hostname allowlist, resolves hostnames via a monkeypatchable
seam, and blocks loopback/RFC1918/link-local/CGNAT/multicast/reserved/
unspecified addresses (plus operator-configured CIDRs) on every
resolved candidate IP. Successful validation pins the result to a
single resolved IP to avoid a DNS-rebinding window between validation
and fetch.
This is step 1 of a 4-task plan; wiring the validator into the actual
HTTP fetch path in server_fastmcp.py is deferred to a follow-up task.
Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
* fix: fail closed on SSRF protection flag and IPv6 localhost escape hatch
MARKDOWN_SSRF_PROTECTION_ENABLED defaults to on, but the shared
_env_bool() helper treated any unrecognized value as false, silently
disabling all per-IP SSRF checks on a typo. It now goes through a
dedicated _env_enabled_by_default() that only an explicit falsey value
can turn off.
_is_blocked_ip() also checked is_reserved before is_loopback; since
IPv6Address('::1').is_reserved is True, MARKDOWN_ALLOW_LOCALHOST was a
no-op for IPv6 loopback even though it worked for 127.0.0.1. Loopback
is now checked first and independently of the other always-blocked
categories.
Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
* fix: pin outbound connections and validate every redirect hop against SSRF guard
fetch_url_content previously made a single unvalidated httpx request and
let httpx auto-follow redirects, so a first-hop SSRF check (if any) could
be bypassed by a public host redirecting to an internal address on a
later hop. get_session() now disables httpx's automatic redirect
following, and fetch_url_content implements its own bounded redirect
loop that re-validates each hop through ssrf.validate_url(), connects to
the validated target's pinned resolved IP (Host header and SNI set to
the original hostname) to close the DNS-rebinding window between
validation and connect, and streams the response body, aborting as soon
as it exceeds MAX_CONTENT_SIZE instead of buffering the full response
first.
_detect_content_type now receives the final, post-redirect URL rather
than the pre-fetch original, since extension sniffing should follow
where the content actually came from.
batch_convert's urls and timeout Field declarations are now bounded
(max_length=50, le=MAX_TIMEOUT) so one call cannot fan out to an
unbounded number of internal targets.
Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
* test: cover redirect re-validation and pinning; log blocked SSRF URLs
Adds a regression test that mocks a 302 redirect to the cloud metadata
address and asserts fetch_url_content blocks it without the second hop
ever connecting - guarding against a future refactor that hoists
validate_url() above the redirect loop and silently restores the SSRF.
Extends the existing fetch_url_content mock test to assert what was
actually passed to session.stream(): the pinned IP in the URL, and the
original hostname in both the Host header and the sni_hostname
extension, so tampering with the pinning wiring is caught by tests
instead of passing silently.
Adds a logger.warning() in the SsrfBlockedError handler so blocked URLs
are visible in server-side logs even though the caller only sees the
generic "URL is not allowed" message.
Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
* docs: correct SSRF security claims and document new env vars
Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
* ci: run url_to_markdown_server tests on PR
Add a dedicated workflow that installs the url_to_markdown_server
package's dev extras and runs its pytest suite (including the SSRF
regression tests) whenever files under
mcp-servers/python/url_to_markdown_server/** change. Previously no
CI workflow executed this package's tests, so a future refactor
could silently reintroduce the SSRF vulnerability with nothing to
catch it.
Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
* fix: IDNA-encode hostnames, log SSRF block reasons, and test streaming/redirect limits
Three fixes to url_to_markdown_server's SSRF guard raised by final review:
- IDNA-encode caller-supplied hostnames (skipping literal IPs) before the
allowlist check, so a Unicode hostname no longer reaches httpx's Host
header encoding as raw UnicodeEncodeError. Malformed IDN labels now
surface as the existing generic SsrfBlockedError instead of propagating
an unhandled exception to the MCP caller.
- Add server-side logging (ssrf.py had none) so operators can diagnose why
a URL was blocked without leaking that detail to the untrusted caller,
whose error message stays the fixed generic string. A malformed CIDR in
MARKDOWN_BLOCKED_NETWORKS is now logged and skipped instead of silently
failing closed on every request - the unconditional deny categories
(private/loopback/link-local/CGNAT/etc.) are unaffected either way.
- Add coverage for the streamed-size-cap abort and redirect-exhaustion
branches in fetch_url_content, both previously unexercised despite the
README documenting the streaming-abort behavior as a security feature.
Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
* fix: drop stray shebang from ssrf.py, refresh secrets baseline
pre-commit's check-shebang-scripts-are-executable flagged ssrf.py: it
carried a #!/usr/bin/env python3 shebang copied from the entry-point
module's header convention despite being a plain library module with
no execute bit. Replaced with the coding-declaration line used by this
package's other non-entrypoint modules.
.secrets.baseline line numbers/generated_at refreshed via
detect-secrets-scan after the upstream rebase shifted file contents;
no new or unaudited findings.
Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
* test: add real-environment e2e verification for the SSRF fix
Adds tests/e2e_verify_ssrf_fix.py, which starts the actual FastMCP
server as a subprocess over real stdio transport and drives it with a
real MCP client (fastmcp.Client) - no mocked DNS/HTTP, unlike
test_ssrf.py and test_server.py. Real local HTTP servers stand in for
"internal" victim services and prove attacker-reachable targets are
never connected to; a real outbound request to https://example.com
proves legitimate fetches still work.
18 cases across 3 configurations: secure defaults (cloud metadata,
loopback, RFC1918, CGNAT, IPv6 loopback, credential-in-URL, disallowed
scheme, non-resolving IDN hostname, legitimate fetch, batch_convert's
50-URL cap, a mixed batch, and two unrelated tools for regression);
MARKDOWN_ALLOW_LOCALHOST=true (redirect-hop re-validation against a
real local redirector, and proof the escape hatch doesn't cross
categories); and the same flag with tightened MARKDOWN_MAX_CONTENT_SIZE
/ MARKDOWN_MAX_REDIRECT_HOPS to exercise the streaming-abort and
redirect-loop-bound branches within a fast run.
Not wired into pytest/CI (real outbound network access and several
local subprocess servers are inappropriate for a fast, isolated CI
run) - run manually per the module docstring.
Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
* fix: address IPv6 site-local/6to4 bypass, cross-host connection reuse, and unbounded DNS in SSRF guard
Addresses review findings on the SSRF fix (GHSA-gv3m-3mvj-w6f2):
- ssrf.py: block IPv6 site-local (fec0::/10) addresses explicitly, and
unwrap 6to4-embedded IPv4 addresses before categorization so an embedded
loopback/link-local target can't hide behind Python's blanket
is_private=True classification for the whole 2002::/16 block.
- server_fastmcp.py: pin the raw TCP destination via a per-request
contextvar and a custom network backend instead of rewriting the request
URL to the resolved IP. Keeping the request origin as the real hostname
keeps HTTPX's connection pool, TLS SNI, and cookie jar scoped per
hostname, so two hostnames that resolve to the same IP can no longer
share a TLS-authenticated connection.
- ssrf.py: bound DNS resolution with the caller's own request timeout on a
dedicated thread pool, so a slow/unresponsive resolver can't silently
outlive the advertised timeout or exhaust the shared default executor.
Adds regression tests for all three findings.
Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
* fix: honor egress proxies and match blocked CIDRs against both address forms
Addresses the second round of review findings on the SSRF fix
(GHSA-gv3m-3mvj-w6f2):
- server_fastmcp.py: passing an explicit transport to httpx.AsyncClient
disables HTTPX's automatic loading of HTTP_PROXY/HTTPS_PROXY/ALL_PROXY
(allow_env_proxies is only enabled when transport is None), so a
deployment with a configured egress proxy silently connected direct
instead - breaking proxy-only deployments and bypassing proxies used for
outbound filtering or auditing. get_session() now detects configured
proxy variables and, when present, lets HTTPX build its own transport so
the proxy is honored. Connect-time IP pinning is skipped on that path
(the socket goes to the proxy, which resolves the destination itself, so
the validated address cannot be enforced end to end) and a warning states
this explicitly rather than degrading silently. All other SSRF controls
are unchanged on both paths.
- ssrf.py: _unwrap_embedded_ipv4() replaced the address before
MARKDOWN_BLOCKED_NETWORKS was evaluated, and unwrapping changes the
address family - so an operator-configured IPv6 CIDR could never match a
6to4 or IPv4-mapped literal inside it. With MARKDOWN_BLOCKED_NETWORKS=
2002::/16, http://[2002:0808:0808::]/ was allowed (the unwrapped 8.8.8.8
is public, so it also cleared every category check, making the bypass
independent of the escape hatches). Operator CIDRs are now matched against
both the address as written and its unwrapped form; category checks
continue to use the unwrapped form. Matching both is required in both
directions: an IPv4 CIDR such as 10.0.0.0/8 can only match the unwrapped
address of a 6to4 literal wrapping that range.
Documents the proxy behavior and the previously undocumented
MARKDOWN_DNS_TIMEOUT / MARKDOWN_DNS_RESOLVER_THREADS variables, and adds
regression tests for both findings.
Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
* fix: keep IP pinning on NO_PROXY and unmatched destinations when an egress proxy is configured
get_session() previously disabled connect-time IP pinning for the whole
session whenever any proxy env var was set, even for destinations that
NO_PROXY excludes from the proxy and therefore connect direct. That
reopened the validate-to-connect DNS-rebinding window IP pinning exists
to close, for traffic the proxy never touches at all.
The pinned AsyncHTTPTransport is now always the client's default
transport. Proxy-matching destinations are mounted onto separate,
unpinned proxy transports built from httpx's own environment-proxy
mount map, so NO_PROXY entries route back onto the pinned default
instead of falling through to a broader proxy mount.
Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
* fix: fail closed on proxy-routed destinations unless explicitly opted in
Any destination that would previously have been routed through a
configured HTTP_PROXY/HTTPS_PROXY/ALL_PROXY got its connect-time IP
pinning silently disabled, since the proxy performs its own DNS
resolution and the validated address cannot be enforced end-to-end on
that path. That reopens the DNS-rebinding window pinning exists to
close for every destination the proxy would have handled, not just the
NO_PROXY-excluded ones the prior fix covered.
get_session() now ignores proxy environment variables entirely - and
keeps every destination pinned - unless
MARKDOWN_ALLOW_UNSAFE_PROXY_PINNING is explicitly set. NO_PROXY and
unmatched destinations were already unaffected either way. Documented
in the README and docs page; regression tests cover the fail-closed
default, the opt-in path, and a live DNS-rebinding attempt against a
proxy-matched destination.
Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
* fix: refuse proxy-matched destinations by default, bound DNS resolver admission
Falling back to a direct (pinned) connection for a proxy-matched
destination closed the DNS-rebinding gap but silently bypassed
whatever egress policy (audit logging, DLP, destination allowlisting)
the operator configured the proxy to enforce - its own kind of control
bypass distinct from the rebinding risk pinning addresses. get_session()
now always mounts a configured proxy, as HTTPX itself would, but
fetch_url_content() refuses to actually send a request through it -
before any connection to either the proxy or the destination - unless
MARKDOWN_ALLOW_UNSAFE_PROXY_PINNING is set. NO_PROXY and unmatched
destinations are unaffected either way.
Also bounds admission to the dedicated DNS resolver thread pool.
ThreadPoolExecutor's own work queue is unbounded, so a burst of
slow-resolving hostnames could queue indefinitely behind the fixed
worker threads; a semaphore sized to the pool now rejects a lookup
that can't get a permit within its timeout instead of queueing it, with
the permit held until the underlying resolver call actually completes
rather than until the caller's own wait gives up.
Regression tests cover both: a proxy-matched request is rejected
without contacting either the proxy or the destination (verified via
DNS rebinding) while the opt-in path still works end to end, and the
resolver admission bound rejects when saturated while releasing only
on genuine completion.
Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
* chore: mark e2e SSRF verifier executable, refresh secrets baseline
check-shebang-scripts-are-executable requires the shebang'd e2e script
to carry the execute bit. Baseline regeneration is timestamp and
line-number drift only, from rebasing onto upstream/main.
Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
* fix: reject non-positive max_concurrent in batch_convert
max_concurrent had an upper bound (le=10) but no lower one, so a
caller-supplied 0 (or negative) reached asyncio.Semaphore() directly.
A zero-valued semaphore never grants its first permit, so every
convert_single_url() task blocks forever at `async with semaphore`
and asyncio.gather() never returns - an MCP caller could hang a
batch_convert call indefinitely with no server-side timeout to
recover it.
Adds ge=1 so FastMCP/Pydantic rejects the value before batch_convert()
starts, plus a regression test covering both 0 and -1.
Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
* chore: exclude e2e SSRF verifier from test-naming hook, refresh secrets baseline
tests/e2e_verify_ssrf_fix.py is an intentionally-manual script (needs real
outbound network access, not wired into CI - see its own docstring), not a
pytest test file, but the name-tests-test hook's exclude list only covers
top-level tests/ paths and missed this package's nested tests/ directory.
Baseline refresh is line-number drift only, from rebasing onto upstream/main.
Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
* chore: refresh secrets baseline after upstream rebase
Line-number drift only, from rebasing onto latest upstream/main.
detect-secrets-scan and detect-secrets-audit both report zero
unaudited/live/real secrets.
Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
* test: close remaining diff-coverage gaps in the SSRF fix
diff-cover against origin/main flagged six untested branches this PR's
own diff introduced: _PinnedNetworkBackend's unix-socket/sleep
passthroughs, _environment_proxy_mounts()'s fail-closed exception
path, the redirect-missing-Location-header guard, the declared
Content-Length pre-check, the httpx.HTTPStatusError handler, and
ssrf.py's admission-release RuntimeError swallow (closed event loop).
None of these are reachable from the existing happy-path/SSRF-deny
tests, so each gets a direct, mocked regression test. Diff coverage
is now 100% (up from 96%).
Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
* style: fix two ruff findings introduced by the SSRF fix
- ssrf.py:1 - drop the unnecessary UTF-8 coding declaration (UP009);
the file is Python 3-only and ruff already flags this as redundant.
- server_fastmcp.py - wrap the fail-closed proxy-config log message
(E501, 167 chars) across two string literals.
No behavior change. Ruff findings on the two touched files: 15 -> 13.
Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
---------
Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
Co-authored-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
Bumps the cryptography floor from >=49.0.0 to >=50.0.0. 50.0.0 was published after this project's rolling 10-day exclude-newer window, so it needs an explicit tool.uv.exclude-newer-package entry (matching the existing pattern for cpex/authlib-style bumps) to let uv's resolver consider it at all. Verified: targeted JWT/crypto/auth unit tests pass; the Grype CVE gate no longer flags this dependency on the built image (only pre-existing, unrelated medium p11-kit RPM findings remain); the FIPS-enabled (ENABLE_FIPS=true, PYTHON_VERSION=3.11) image builds and passes fedramp-validate.sh (22/22), and both images resolve cryptography==50.0.0. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> Co-authored-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
…IBM#5198) * fix(oauth): remove blocking unique constraint for multi-user DCR clients Fixes IBM#5191 Migration 14ac971cee42 added app_user_email column and created unique index on (gateway_id, app_user_email) but failed to drop the original UniqueConstraint 'unique_gateway_user' on (gateway_id, user_id). This prevented multiple ContextForge users from storing OAuth tokens when they shared the same OAuth provider user_id, blocking multi-user OAuth flows for DCR-based gateways. This migration (7ab59991e017) properly removes the old constraint and creates the correct UniqueConstraint 'uq_oauth_gateway_user' on (gateway_id, app_user_email). Uses table recreation for SQLite and direct DDL for PostgreSQL. Tested with direct SQL verification showing successful multi-user token storage after migration. Signed-off-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com> * Changed pure SQL to sqlalchemy Signed-off-by: cafalchio <mcafalchio@gmail.com> * Secrets Signed-off-by: cafalchio <mcafalchio@gmail.com> * Addressed issues, updated head Signed-off-by: cafalchio <mcafalchio@gmail.com> * Addressed comments. Downgrade fails on duplicates Signed-off-by: cafalchio <mcafalchio@gmail.com> * Fixed alembic head Signed-off-by: cafalchio <mcafalchio@gmail.com> * remove index before creating a new one Signed-off-by: cafalchio <mcafalchio@gmail.com> * Fixed docs Signed-off-by: cafalchio <mcafalchio@gmail.com> --------- Signed-off-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com> Signed-off-by: cafalchio <mcafalchio@gmail.com> Co-authored-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com> Co-authored-by: cafalchio <mcafalchio@gmail.com>
Rewrite docs/docs/architecture/roadmap.md around the H2 2026 plan, linking each theme to its tracking epic or issue, and add docs/docs/architecture/releases.md with a release-by-release table of the most impactful changes since 1.0.0. Register the new Release History page in the architecture .pages nav and refresh .secrets.baseline for the shifted line number. Signed-off-by: Jonathan Springer <jps@s390x.com> Co-authored-by: jonpspri <jonpspri@users.noreply.github.com>
* feat: consume Cpex control execution records Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * fix(plugins): suppress pylint unused-import on cpex feature-detection probe Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * style: align inline comments in control_telemetry.py Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * test(plugins): add targeted coverage tests for control-telemetry and cpex-compat - Hoist all imports to top-level (no inline imports inside test methods) - Add TestAggregateExceptionPath: covers aggregate() inner except path (lines 192-193) - Add TestGetMaxResultsExceptionPath: covers _get_max_results() except branch (lines 484-485) - Add TestBuildFlattenedAttributesEdgeCases: error_code branch, per-record except, outer except (lines 553-555, 561-563) - Add TestEmitDbSpansExceptionPaths: empty-attrs skip (line 327), rollback on start_span failure (lines 342-348), close() failure swallowed (lines 353-354) - Add TestEmitOtelSpansActivePath: verifies OTel summary+result spans emitted when otel_tracing_enabled+otel_context_active (lines 379-387) - Add TestRecordControlTelemetryTruncated: truncated attribute present (line 260) - Add TestRecordControlTelemetryFlatten: flatten_results branch (lines 267-268) - Add TestExecutionRecordsSupportedImportError: ImportError branch (lines 51-52) - Add wildcard rule tests in TestWildcardAttributeMapping: key>256 rejection (line 535), wildcard/exact removal compilation (lines 543-551), segment extraction mismatch (line 593), ValueError fallback (lines 657-660) - Add TestConcurrencyIsolation and TestObservabilityServiceAPICompatibility Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * fix(plugins): address review findings from issue IBM#5785 PR IBM#6003 Duration unit (finding IBM#3): - Rename cpex.control.duration -> cpex.control.duration_ns in aggregate(), _per_control_attributes(), and both sinks to follow OTel unit-suffix convention. Rename rules targeting 'cpex.control.duration_ns' are now semantically correct. Missing ControlExecutionRecord fields (finding IBM#2): - Add cpex.control.plugin_id, cpex.control.plugin_kind, cpex.control.matched, cpex.control.applied, cpex.control.payload_modified to per-control result spans. - Add cpex.control.result.requested_allowed (emitted only when requested_allow != None). results_count semantics (finding IBM#6): - Fix aggregate() results_count to reflect min(accumulated, max_results) — the number of records exported after the per-invocation cap, per IBM#5785 spec. invocation_count semantics (finding IBM#6): - Count only controls that actually ran (completed/error/timeout); exclude skipped/disabled/cancelled per IBM#5785 aggregation spec. Timeout post-hook gap (finding IBM#5): - Add ctl_acc parameter to _run_timeout_post_invoke(); all four call sites now pass ctl_acc=_ctl_acc so timeout post-hook execution records feed the per-invocation accumulator and appear in the emitted telemetry. Fail-closed ValueError fallback (finding #8b): - apply_attribute_mapping() now returns attributes unchanged when compile_attribute_policy raises ValueError (e.g. otel.* destination, empty key, key > 256 chars). Previously it constructed partially-validated exact mappings that could bypass the otel.* guard — now fail-closed. CPEX denial path documentation (finding IBM#4): - Add explanatory comment in except (PluginError, PluginViolationError) block documenting the upstream CPEX 0.1.2 framework gap: when violations_as_exceptions=True, CPEX raises before appending the denying plugin's ControlExecutionRecord, so _ctl_acc captures pre-denial records only. The effective_allow=False outcome is preserved via pre_denied flag. Tests updated for all changes above; 741 passed, 33 skipped. Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * fix gaps Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * fix(plugins): account for tier-3 export-cap drops in cpex.control.truncated Records beyond CPEX_CONTROL_TELEMETRY_MAX_RESULTS were silently discarded by islice() in _emit_db_spans() and _emit_otel_spans() without being counted in the cpex.control.truncated summary attribute, leaving downstream operators with an incomplete truncation indicator. Fix: - Add _export_cap_dropped field to ControlTelemetryAccumulator - Add mark_export_cap_dropped(count) method called once per emit in record_control_telemetry() before building the summary span - truncated property now returns _truncated + _export_cap_dropped covering all three tiers: Tier 1: per-hook cap (MAX_RECORDS_PER_HOOK = 64) Tier 2: per-call accumulation cap (MAX_RECORDS_PER_CALL = 128) Tier 3: export cap (CPEX_CONTROL_TELEMETRY_MAX_RESULTS, default 32) Tests: - Add TestExportCapTruncation (5 tests) in test_control_telemetry.py - Update TestRecordControlTelemetryTruncated to assert correct total across all three tiers (2 tier-1/2 drops + 96 tier-3 drops = 98) Relates to IBM#5785 Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * fix linting Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * fix(plugins): address blocking review findings on cpex control telemetry PII / agent.id (blocking): - Gate cpex.control.agent.id behind CPEX_CONTROL_TELEMETRY_EMIT_AGENT_ID (default false). Adds _emit_agent_id_enabled() helper and new config field with explicit PII/GDPR warning in description and .env.example. config_keys amplification (functionally impacting): - Add _MAX_CONFIG_KEY_LEN=128 per-key cap and _MAX_CONFIG_KEYS_JOINED_LEN=4096 total joined-string cap. Applied in both _per_control_attributes() and _build_flattened_attributes() to prevent large/adversarial key names causing telemetry amplification or data leakage. Default-off (functionally impacting): - Change CPEX_CONTROL_TELEMETRY_ENABLED default from true to false. Each traced tool call creates up to 1 summary + MAX_RESULTS result DB spans; operators must explicitly opt in after reviewing storage/cardinality impact. E2E tests set CPEX_CONTROL_TELEMETRY_ENABLED=true explicitly via os.environ. E2E denial-path tests (test coverage): - Add denying_plugin_app fixture: PIIFilter with block_on_detection=true and enforcement_mode=pre_invoke. - Add TestDenialPathTelemetry with two tests: test_pre_invoke_denial_summary_span_allowed_false: verifies result.allowed=false and enforcement_point=pre when a real plugin denies on pre_invoke. test_pre_invoke_denial_tool_never_executed: verifies tool arguments do not leak into control telemetry attributes on the denial path. Both tests skip gracefully when PIIFilter does not detect PII (environment variation). Unit test fixes: - Add _enabled_settings() helper in test_record_control_telemetry.py so all TestRecordControlTelemetryDB tests work correctly with the new default=false. - Patch cfg_mod.settings in all DB-sink tests and TestRecordControlTelemetryTruncated. Relates to IBM#5785, PR IBM#6003 Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * addressed comments Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * fixed secrets Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * fix(plugins): address four follow-up review findings on cpex control telemetry Fix 1 — wildcard insertion order corrupted by cache key sorting: - apply_attribute_mapping() was building its cache key with sorted(mapping.items()), which reordered wildcard rules and changed which match fired first. The documented contract is 'first matching wildcard wins' (compile_attribute_policy evaluates rules in insertion order). Changed to tuple(mapping.items()) to preserve the caller's declared rule order. - Regression test: TestWildcardInsertionOrderPreserved (3 cases) in test_record_control_telemetry.py. Fix 2 — config_keys joined cap was char-based, not byte-aware: - Both projections (_per_control_attributes and _build_flattened_attributes) used joined[:_MAX_CONFIG_KEYS_JOINED_LEN] which counts Unicode code points, not UTF-8 bytes. Multibyte key names (e.g. CJK characters) could therefore exceed the intended byte budget. - Fixed both sites to use _safe_str(joined, _MAX_CONFIG_KEYS_JOINED_LEN) which is already byte-aware (encodes to UTF-8 before slicing). - Regression tests: TestConfigKeysByteCapAware (2 cases) covering multibyte input and ASCII round-trip. Fix 3 — _enforcement_point returned 'none' on denial with empty records: - When CPEX raises PluginViolationError with violations_as_exceptions=True it does so before appending a ControlExecutionRecord, so the accumulator can have no records even though a pre-hook fired. _enforcement_point was only inspecting acc.records and returned 'none' in this case. - Fixed to also check acc.pre_denied / acc.post_denied flags so a first-plugin pre-invoke denial correctly reports enforcement_point='pre'. - E2E denial test tightened: removed 'none' from accepted enforcement_point values; removed pytest.skip on missing summary span (absence is a regression); removed 500 from accepted status codes (indicates unhandled exception). - Unit tests: TestEnforcementPointDenialFlags (3 cases). Fix 4 — PluginError on first plugin emits no telemetry: - When the first plugin in the chain raises PluginError the accumulator has no records and no denial flags, so record_control_telemetry() returned early at the empty-accumulator guard, emitting nothing. Plugin outages were invisible in observability dashboards. - Added _plugin_errored field and mark_plugin_error() method to ControlTelemetryAccumulator. The outer except PluginError handler in tool_service.py now calls mark_plugin_error() before record_control_telemetry(). - Empty-accumulator guard now includes plugin_errored so first-plugin failures always emit a summary span. - aggregate() emits cpex.control.plugin_error=True when the flag is set, making plugin outages distinguishable from both successful allows and policy denials. - effective_allowed remains True on PluginError (intentional — outage != denial). - Unit tests: TestMarkPluginError (6 cases) in test_control_telemetry.py. 155 unit tests pass; ruff and pylint 10.00/10. Relates to IBM#5785, PR IBM#6003 Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * fix(plugins): address blocking and warning findings from ja8zyjits review Blocking 1: CHANGELOG.md CPEX_CONTROL_TELEMETRY_ENABLED default 'true' → 'false' to match mcpgateway/config.py:1987 (default=False). Operators would have expected the feature on by default when it is off. Blocking 2: mark_export_cap_dropped() idempotency — changed from += to a first-write-wins assignment guarded by _export_cap_dropped == 0. Subsequent calls on the same accumulator are now no-ops, preventing cpex.control.truncated inflation on retry paths or test reuse. Two regression tests added (TestExportCapTruncation::test_mark_export_cap_dropped_is_idempotent and ::test_mark_export_cap_dropped_zero_then_positive). Warning 1: Replaced undefined 'S4 principles' reference in control_telemetry.py module docstring with an explicit description: 'bounded cardinality and field-level sanitization ... _safe_str/_safe_num helpers and field-name allowlists' pointing to plugins/utils.py. Warning 2: Extracted the 5 repeated record_control_telemetry() call blocks in tool_service.py::invoke_tool() into a single nested helper _emit_ctl_telemetry() that closes over _ctl_acc, name, app_user_email, user_email, gateway_name, and server_id. Identity mapping rationale previously inline at the normal-return site is now in the helper docstring; exception-path sites are reduced to a single _emit_ctl_telemetry() call with a back-reference comment. 145 plugin unit tests pass; ruff all-clear; pylint 10.00/10. Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * fix(plugins): sanitize config_keys and make plugin_error result.allowed indeterminate config_keys sanitization (vishu-bh finding): - Add _sanitize_config_key() helper that validates each key against _CONFIG_KEY_RE (^[A-Za-z0-9_.-]{1,64}$). Rejects commas (the CSV delimiter), CR/LF/NUL/TAB (log-injection vectors), non-ASCII, and secret-shaped tokens containing '='. Keys that fail are dropped entirely rather than truncated, preventing ambiguous joined output. - Both _per_control_attributes() and _build_flattened_attributes() now use the validator. When every key in a record fails, the attribute is omitted rather than emitting an empty string. - 14 new tests in TestSanitizeConfigKey: comma, CR, LF, NUL, TAB, non-ASCII, secret-marker, empty, oversized, max-length, and three end-to-end tests through _per_control_attributes(). PluginError indeterminate decision (vishu-bh finding): - mark_plugin_error() now accepts hook='pre'|'post'|'' and stores it in _plugin_error_hook for enforcement_point derivation. - aggregate() omits cpex.control.result.allowed entirely when plugin_errored=True and no denial flag is set (decision is indeterminate — the chain did not complete). When both plugin_errored and a denial flag are set, denial takes precedence and result.allowed is emitted as False. - _enforcement_point() gains a third fallback tier: plugin_error_hook, so a first-plugin crash still reports the correct enforcement point rather than 'none'. - tool_service.py: _ctl_last_hook tracks 'pre'/'post' across all invoke sites; outer except PluginError passes hook=_ctl_last_hook. - 9 new tests in TestPluginErrorIndeterminate. - Updated TestMarkPluginError::test_plugin_error_flag_bypasses... to assert result.allowed is ABSENT (was: True) for the first-plugin error path. 849 plugin unit tests pass; ruff all-clear; pylint 10.00/10. Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * fixed ruff format Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * fix: tolerate stale merge-queue ref on SARIF upload When a merge-queue attempt is ejected (e.g. transient Docker cache error) the ephemeral ref refs/heads/gh-readonly-queue/main/pr-* is deleted. Any retried attempt carries the same stale ref baked into the job context, so codeql-action/upload-sarif fails with 'ref not found', failing fedramp-compliance and scan jobs and ejecting the queue again -- workflow_run_attempt 1, 2, and 3 all hit this loop. Add continue-on-error: true to both Upload SARIF to CodeQL steps so a stale-ref rejection from the CodeQL API does not fail the job. The scan result is unaffected; SARIF lands on the push-to-main run that follows a successful merge. Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * update secrets Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * undo .yml change Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> --------- Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>
* Release 1.0.7
Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>
* update stale 1.0.6 -> 1.0.7
Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>
* fix pre-commit
Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>
* fix: patch npm audit vulnerabilities
Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com>
* Fix e2e test wrapper : CI
Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>
* fix: pass AUTH_ENCRYPTION_SECRET to Generate JWT token step
The uv run invocation of create_jwt_token imports mcpgateway.config
at module level, triggering the full settings validator. DYNAMIC_ENC_SECRET
from GITHUB_ENV is not automatically named AUTH_ENCRYPTION_SECRET —
it must be explicitly mapped in each step's env: block that invokes Python.
Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>
* fix CI/pytest-rust
Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>
* add secrets.baseline
Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>
* fix(helm): set runAsUser so non-root containers can start
Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com>
* added change log for release 1.0.7
Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>
* Fix vulnerable packages found using pip-audit
Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>
* migration-fix
Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>
* fixed unit tests
Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>
* fixed patch.env and tests
Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>
* fix: warn at startup when CSRF cookie or header names are overridden
Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com>
* add replace_me to work in dev
Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>
* fix: compare CSRF header name case-insensitively and pin header constants
Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com>
* Fix root test
Signed-off-by: Gabriel Costa <gabrielcg@proton.me>
* chore: improve support bundle redaction and settings secret typing
Derive the set of settings fields excluded from a generated support
bundle from the Settings model itself instead of a hand-maintained list.
A hardcoded list goes stale as settings are added; deriving it means any
field declared with a secret type is covered without a second edit. A
narrow name-based rule acts as a backstop for string-typed fields, with
SecretStr typing remaining the enforced convention.
Type csrf_secret_key and identity_claims_secret as SecretStr. Their
fallback behaviour is unchanged: both still derive from JWT_SECRET_KEY
when left unset. Note that SecretStr("") is truthy, so the emptiness
checks guarding those fallbacks go through get_secret_value(), and the
assignments wrap explicitly because Settings does not enable
validate_assignment.
Strip credentials from every string-valued *_url setting rather than two
named ones, and mask environment variables whose names end in _URL, so
inline credentials in connection strings are handled uniformly.
Describe what each bundle member actually offers in the bundle README,
the module docstring, and the Admin UI blurb: deterministic exclusion for
the settings file, name-based masking for environment variables, and
best-effort pattern matching for logs. Prompt the operator to review a
bundle before sharing it.
Document that CSRF_SECRET_KEY reuses JWT_SECRET_KEY when unset, so the
two can be set separately and rotated independently.
Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
* fix: redact credentials from settings by value rather than field name
Two gaps in the support bundle's settings redaction, both raised in
review.
The userinfo pattern shared by the URL and log sanitizers required at
least one character for the username, so a DSN of the form
scheme://:password@host — accepted by several clients, and the common
Redis shape — did not match and the password reached the bundle
verbatim. The username group now matches zero characters.
Sanitization also keyed off the field name, applying only to keys ending
in _url and only to scalar strings. A field name says nothing about the
shape of the value behind it, so this missed collection-typed settings
entirely: webhook_logging_urls (list[str]), siem_destinations
(list[dict], whose entries hold per-destination tokens), and
otel_exporter_otlp_headers, a key=value blob that conventionally carries
an Authorization header and whose name matches no secret-name rule.
Every settings value is now walked and sanitized on its content. Strings
are matched against the same patterns already applied to log text; lists
and dicts are walked element-wise. Inside a dict, a key whose name looks
like a secret has its value replaced outright, since the value carries no
pattern to match on — that check is deliberately broader than the one
screening our own typed settings, because nested keys come from
operator-authored config where "token" is a credential far more often
than it is part of a benign name like token_expiry.
An empty string now stays empty instead of becoming null, since "set but
blank" and "unset" are different facts to whoever reads the bundle.
Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
* updated changelog
Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>
* Fix secretsbaseline
Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>
* added internal issues to security fixes to changelog
Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>
* removed duplciate entries on changelog
Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>
* updated changelog
Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>
---------
Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>
Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com>
Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com>
Signed-off-by: Gabriel Costa <gabrielcg@proton.me>
Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
Co-authored-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com>
Co-authored-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com>
Co-authored-by: __cafa__ <47299428+cafalchio@users.noreply.github.com>
Co-authored-by: Gabriel Costa <gabrielcg@proton.me>
Co-authored-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
Signed-off-by: Lang-Akshay <akshay.shinde26@ibm.com>
* feat: add v1 plugin discovery API Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> * fix: address plugin discovery review feedback Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> * fix: rebase plugin permission migration Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> * fix: log plugin discovery failures Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> * fix: rebase plugin permission migration Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> --------- Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com>
…#5765) * feat(mcp-apps): support standard MCP messages over the AppBridge The AppBridge accepted only tools/call and rejected everything else with -32601. The MCP Apps specification lets an app send four standard MCP messages, so an app could not read a resource, log to the host, or check connection health through the gateway. resources/read, notifications/message and ping are now routed. Session ownership and the stored server binding are validated before any method is dispatched, so every bridge method inherits the scoping tools/call already had: - resources/read reuses the identity and team scoping captured on the session and is confined to the bound server, so an app cannot reach resources its originating user could not read. The gateway-id header is stripped as it is for tool calls. - notifications/message is recorded for observability and never proxied upstream, matching the lifecycle in the specification. - ping is answered by the gateway without an upstream call. The allowlist is explicit rather than derived from the core MCP method set, so core methods such as tools/list, resources/list, prompts/list and resources/subscribe stay rejected. The advertised bridge capability and the router now share one constant so they cannot drift apart. Signed-off-by: PhilipAD <9472774+PhilipAD@users.noreply.github.com> * feat(mcp-apps): authorize AppBridge resources/read per method The RPC endpoint is guarded by `tools.execute`, which was the whole surface when `tools/call` was the only bridge method. Now that it also dispatches `resources/read`, the endpoint decorator no longer proves the caller may read resources: token and team scoping decide what a caller can see, RBAC decides whether they may read at all, and the two are separate layers. A caller who keeps `tools.execute` after `resources.read` is revoked could otherwise keep reading through an existing AppBridge session. `resources/read` now runs the same `_ensure_rpc_permission` check that normal MCP RPC routing applies, with an endpoint-level regression test that drives the real route (rather than `__wrapped__`) so the route decorator and the method check are both exercised. `notifications/message` no longer returns a JSON-RPC result. A notification carries no id and must not receive a response, so it is acknowledged at the transport level with 202 Accepted and an empty body, matching the Streamable HTTP rule for notification-only input and this gateway's own internal notification endpoint. Also drops the duplicate local `mcp_method_registry` import in `_handle_rpc_authenticated()` that shadowed the new module-level binding (pylint W0621/W0404), extracts the tools/call branch into a helper to keep the endpoint within the return-statement limit, and makes the architecture doc's opening summary agree with the supported-method table. Signed-off-by: PhilipAD <9472774+PhilipAD@users.noreply.github.com> * feat(mcp-apps): reject an id-bearing AppBridge notifications/message JSON-RPC distinguishes a notification from a request by the presence of the id member, not by its value, so "id": null is still a request that requires a response. The handler treated every notifications/message as a notification and answered with an empty 202, which would silently discard a response an id-bearing caller is waiting on. The request form is now rejected with -32600 rather than acknowledged, since notifications/message is defined as terminating at the host with no reply. The success test omits the id member entirely, and a parametrized regression test covers the id-bearing form across null, string and numeric ids. Part of IBM#2527. Signed-off-by: PhilipAD <9472774+PhilipAD@users.noreply.github.com> * feat(mcp-apps): share AppBridge header stripping and cap notification log fields Extract the gateway routing header strip into _app_bridge_request_headers so resources/read and tools/call share one source of truth, cap the level and logger fields of an AppBridge log notification the same way the data field is capped, and cover the plugin error mapping of resources/read. Signed-off-by: PhilipAD <9472774+PhilipAD@users.noreply.github.com> --------- Signed-off-by: PhilipAD <9472774+PhilipAD@users.noreply.github.com> Co-authored-by: PhilipAD <9472774+PhilipAD@users.noreply.github.com>
* fix: reject narrowed admin tokens on OAuth DCR management endpoints Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * fix(tests): pass mock_admin_request to DCR management router calls after rebase Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> --------- Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>
Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>
* fix(teams): scope admin team listings by token teams Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com> * test(teams): cover public-only admin team listings Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com> --------- Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com>
Signed-off-by: cafalchio <mcafalchio@gmail.com>
IBM#4654) * refactor: centralize Layer-1 visibility filter across REST endpoints Replace 22 inline Layer-1 derivations with centralized get_scoped_resource_access_context() calls for consistency and maintainability. Changes: - Refactor 18 endpoints to use centralized helper - Preserve 4 exceptions (A2A federation, custom contracts, audit capture) - Update 13 test mocks from 3-tuple to 2-tuple pattern - Add regression tests for basic-auth admin behavior Bug fix: Basic-auth/dev-mode admins now correctly receive admin bypass (None, None) instead of public-only scope. The inline pattern missed non-JWT admin contexts; the centralized helper handles them correctly. Closes IBM#4451 Signed-off-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com> * fix: complete Layer-1 visibility centralization in main Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> * chore: tighten Layer-1 exception comments and helper tests Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> * fix: memoize Layer-1 derivation and add owner-visibility integration tests Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> * test: retarget stale Layer-1 mocks and extend owner-visibility coverage Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> * test: finish Layer-1 mock retargeting and add memoization coverage Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> * docs: surface admin visibility change under Breaking Changes Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> * fix(mcp-apps): keep admin email for owner matching on AppBridge session creation Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> * chore: refresh secrets baseline line offsets Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> --------- Signed-off-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com> Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> Co-authored-by: Bogdan-Marius-Catanus <bogdan-marius.catanus@ibm.com> Co-authored-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com>
* patch timeouts for additionals span Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * fix(tests): clarify outbound mock as temporary workaround in otel e2e The _mock_outbound_rest_request helper and its two call sites now carry an explicit 'TEMPORARY WORKAROUND' comment that explains: - Why mocking the transport layer is architecturally wrong for an E2E suite (the correct fix is a real local stub HTTP server) - Why this intermediate patch is still necessary (the SSRF pinning path builds a fresh client via _build_pinned_rest_http_client() which bypasses any direct _http_client patch, causing the real asyncio.wait_for 60-second timeout to fire) - That the proper solution is tracked separately No functional change — the mock behaviour and test assertions are identical to the previous commit. Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> --------- Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>
* feat(api): non-admin v1 catalog register endpoint
Adds POST /v1/catalog/{catalog_id}/register so non-admin API and UI
callers can register a catalog server without going through the admin
router. Guarded by RBAC servers.create, token-scoped via a new
_PERMISSION_PATTERNS entry, and 404s when the catalog feature is
disabled. Delegates to catalog_service.register_catalog_server and
returns its envelope unchanged, so business failures stay HTTP 200 with
success=false.
The request body is a new minimal CatalogServerRegisterBody (optional
name, optional api_key); the catalog id comes from the path and OAuth
configuration remains out of scope.
Also maps GET /catalog to servers.read in the same middleware block.
_check_permission_restrictions is default-deny for unmapped paths, so
permission-scoped tokens carrying servers.read were rejected before
reaching the list endpoint's own RBAC check. It ships here rather than
alongside the catalog UI because epic/ui-rewrite carries React changes
only, and splitting the two entries across branches would conflict on
adjacent lines.
Closes IBM#5969
Signed-off-by: Ahmad Al Tamimi <altamimi.dev@gmail.com>
* fix(security): gate v1 catalog register on gateways.create without admin bypass
Catalog registration bottoms out in gateway_service.register_gateway, so the
v1 route now requires both servers.create and gateways.create with
allow_admin_bypass=False on each, matching the admin twin and the canonical
POST /v1/gateways boundary.
Also maps business failures to real status codes on the v1 route: unknown
catalog id -> 404, already registered -> 409, and blanks the envelope's error
field on remaining failures (the service captures raw exception text there).
The register body now validates the name override via SecurityValidator and
caps api_key length. The admin registration endpoint keeps its unchanged
HTTP 200 envelope contract.
Signed-off-by: Ahmad Al Tamimi <altamimi.dev@gmail.com>
* style: fix ruff format wrap flagged by pre-commit CI
Wraps the over-long early-return in catalog_service.py per ruff format, uses an
env-var placeholder for the api_key example in the catalog docs so detect-secrets
does not flag it, and refreshes the catalog.md baseline line number shifted by
the new docs section.
Signed-off-by: Ahmad Al Tamimi <altamimi.dev@gmail.com>
* fix(api): stamp caller identity on v1 catalog server registration
The v1 POST /catalog/{id}/register route bound the authenticated user only
for RBAC and passed no identity to the service, so catalog-registered
gateways were created with no created_by/owner_email/team_id. The creator
could not later update or delete their own gateway and audit metadata
lacked attribution.
Thread created_by/owner_email/team_id from the route through
CatalogService.register_catalog_server into both gateway-creation paths
(gateway_service.register_gateway and the OAuth skip-initialization
DbGateway insert). Team is derived like the canonical /gateways route:
public-only tokens get team_id=None. Catalog visibility stays public.
Signed-off-by: Ahmad Al Tamimi <altamimi.dev@gmail.com>
* chore: refresh .secrets.baseline line numbers after catalog doc addition
Signed-off-by: Ahmad Al Tamimi <altamimi.dev@gmail.com>
---------
Signed-off-by: Ahmad Al Tamimi <altamimi.dev@gmail.com>
….12 (IBM#6158) * update python dependencies/update min supported version to py3.12 Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * comment cleanup Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * fix(fips): bump PYTHON_VERSION 3.11 → 3.12 for FedRAMP/FIPS builds pyproject.toml requires Python >=3.12,<3.14 since the dep-update commit (f40dd64), but the FIPS build path still injected PYTHON_VERSION=3.11, causing uv to fail with an unsatisfiable constraint. Update both sites that hardcoded the old version: - Makefile: ENABLE_FIPS_BUILD container-build args - .github/workflows/docker-scan.yml: FedRAMP image build step Python 3.12 is available in UBI 9 (added in RHEL 9.4) so no base image change is needed. Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> --------- Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>
…e verify (IBM#6093) * fixed make dist and make verify Signed-off-by: Lang-Akshay <akshay.shinde26@ibm.com> * fixup: secret baseline Signed-off-by: Lang-Akshay <akshay.shinde26@ibm.com> * fixup: secret baseline Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> --------- Signed-off-by: Lang-Akshay <akshay.shinde26@ibm.com> Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> Co-authored-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
* chore: routine npm/rust dependency updates
- bump npm dependencies (biome 2.5.6, rollup 4.62.3, postcss 8.5.25, dompurify 3.4.13, nanoid 3.3.18, js-yaml 4.3.1, acorn 8.18.0 and others)
- Update package-lock.json to latest compatible versions within existing ranges
- Add min-release-age-exclude for dompurify, js-yaml, nanoid to .npmrc
- Rebuild Admin UI bundle with updated dependencies
- all cargo crates up to date no changes required
Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>
* fix: enforce uv>=0.6.9 and pin greenlet to 3.5.4 to fix CI
The relative exclude-newer duration syntax ("10 days") requires uv>=0.6.9.
On older uv, pyproject.toml settings-discovery fails silently, which causes
uv to treat the lock's sentinel cutoff (0001-01-01T00:00:00Z) as a mismatch
and discard the lockfile entirely. The subsequent fresh resolution then picks
greenlet 3.5.5, which only has macos/win wheels and fails to install on the
manylinux_2_39 Linux CI runner.
Fix:
- Add required-version = ">=0.6.9" to [tool.uv] so old uv fails loudly
rather than silently mangling the lockfile
- Add greenlet = "2026-07-22T13:00:00Z" to [tool.uv.exclude-newer-package]
to cap resolution at 3.5.4 (the last version with manylinux wheels)
until a manylinux wheel ships for 3.5.5+
Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>
* docs: add changelog entries for Python 3.12 min and uv>=0.6.9 requirement
Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>
* Add secrets
Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>
---------
Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>
…7183) * test: validate pre-provisioned SSO users with IdP tokens Validate API-provisioned users with HTTPS Keycloak bearer tokens. Cover DB permissions, team changes, inactive accounts, provider mismatch, and the disabled provisioning API. Restore shared provider settings and remove fixture data. Closes IBM#6583 Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com> * docs: document SSO user provisioning workflow Document provider IDs, external-token setup, directory sync, and provisioning records. Closes IBM#6584 Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com> * test: cover PKCE login and upstream tool execution Verify public-client PKCE login and a real MCP echo call. Reject wrong verifiers, callback-state mismatches, and execution without permission. Refs IBM#6583 Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com> * test: verify provider binding after MCP mismatch attempts Check saved provider binding after REST and MCP mismatch attempts. Record MCP authentication separately; retain REST denial and valid DB permissions. Refs IBM#6583 Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com> * test: cover MCP denial for missing and inactive SSO users Check missing and inactive accounts through MCP. Document issuer trust limits for provider binding. Refs IBM#6583 Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com> --------- Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com>
* fix: let a real upstream JSON-RPC error through the error-response hook Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> * fix: preserve the upstream JSON-RPC error code in tool error results Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> * fix: match the SDK's error-body rules and cover direct proxy and live tool errors Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> * fix: label only upstream-sent JSON-RPC errors and carry result meta through coercion Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> * fix: report the tool error code in _meta without claiming its origin Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> * fix: remove the unused error-body helper Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> --------- Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com>
* feat: answer prompt failures with -32602 and relay upstream prompt errors Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> * fix: keep prompt errors transport-neutral and sanitise relayed error data Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> --------- Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com>
Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com>
…serverInfo stamping (IBM#6566) (IBM#7145) * feat: configure streamable HTTP server with app name and version for serverInfo stamping (IBM#6566) Configure ContextForgeMCPServer with settings.app_name and __version__ so the MCP SDK stamps the correct serverInfo metadata on modern responses. Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * test: add live gateway regression for serverInfo stamping (IBM#6566) Add black-box tests for the /mcp endpoint. Assert the modern serverInfo stamp carries APP_NAME and the package version. Assert a legacy initialize succeeds without modern metadata. Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> --------- Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>
IBM#6820) * feat(api): add named response models for A2A invoke and jsonrpc routes The three A2A invocation routes declared response_model=Dict[str, Any], so the generated OpenAPI schema described an arbitrary object and generated clients produced `unknown`. They now declare A2AInvokeResponse and A2AJsonRpcResponse, which share the JSON-RPC envelope fields. The envelope is intentionally permissive (Any leaves plus extra="allow"): these routes forward whatever the target agent returned, so a stricter type would turn agent drift into a gateway 500 instead of the agent's response. response_model_exclude_unset=True keeps that contract by serializing only the keys the agent actually returned. Refs IBM#6699 Signed-off-by: fei <204683769+feiiiiii5@users.noreply.github.com> * docs: add CHANGELOG entry for A2A named response models (IBM#6820) Refs IBM#6699 Signed-off-by: fei <204683769+feiiiiii5@users.noreply.github.com> * docs: correct A2A response routes and schema notes Signed-off-by: fei <204683769+feiiiiii5@users.noreply.github.com> * chore: refresh secrets baseline line numbers Signed-off-by: fei <204683769+feiiiiii5@users.noreply.github.com> --------- Signed-off-by: fei <204683769+feiiiiii5@users.noreply.github.com> Co-authored-by: Marek Dano <marekdano@users.noreply.github.com>
Register provisioning when the admin API and SSO are enabled. Retain the startup gate and existing authorization checks. Update configuration, documentation, and flag-specific tests. Verify external IdP authentication with the admin API disabled. Use the current MCP error-hook method in the live suite. Refs IBM#7218 Signed-off-by: Pratik Gandhi <gandhipratik203@gmail.com> Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
* added escapeHtml to the result body to neutralize HTML injection vector Signed-off-by: Lang-Akshay <akshay.shinde26@ibm.com> * fix(rust): upgrade rustls to 0.23.46 Raise the rustls minimum and update the workspace lockfile to resolve RUSTSEC-2026-0346 through RUSTSEC-2026-0349. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * fix(ci): complete rustls audit and fixture license metadata Record an automated delta audit for rustls 0.23.46. Declare the completion test server license as Apache-2.0. Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * chore: remove optional rustls audit reviewer field Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * chore: add rustls audit reviewer details Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> --------- Signed-off-by: Lang-Akshay <akshay.shinde26@ibm.com> Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> Co-authored-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
* fix(otel): suppress tenantless background spans Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> * fix(tests): add required file header Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> * fix(otel): redact baggage and validate request trace policy Sanitize baggage keys before attribute prefixing. Preserve optional tracing imports and clarify processor sampling dependencies. Add live OTLP acceptance tests, health metric assertions, and observability CI coverage. Document metric semantics and upgrade defaults. Validation: 23830 unit tests and 1225 doctests pass. Both live OTLP cases pass. Changed-line coverage reaches 95%. Container E2E validation remains pending an isolated Docker engine. Full documentation builds and strict mypy retain existing blockers. Closes IBM#7085 Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> * fix(ci): include observability coverage and wait for trace export Append SDK-dependent test coverage before diff-cover. Wait for invocation spans and their exported parents in the live tracing test. Validation: 77 focused tests pass, including both live gateway cases. Observability changed-line coverage reaches 93.5%. Ruff and whitespace checks pass. Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> * chore: refresh secrets baseline line numbers Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> --------- Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com>
* fix(oauth): isolate exchanged tokens by subject credential Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * test(oauth): verify token exchange credential isolation Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * docs: clarify token exchange cache identity Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> * chore: refresh secrets baseline after rebase Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> --------- Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> Co-authored-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
) * feat: allow any JSON value as structuredContent for 2026-era callers Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> * fix: keep structured results intact on internal dispatch and validate explicit null Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> * fix: compare JSON values strictly when checking for a text copy Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com> --------- Signed-off-by: Madhu Mohan Jaishankar <madhu.mohan.jaishankar@ibm.com>
* fix: enforce ownership on configuration import updates POST /v1/import with conflict_strategy=update overwrote tools, resources, prompts and gateways owned by other teams. Bulk registration looked up public rows by name with no tenant predicate and never checked ownership. The import call to update_gateway() omitted user_email, which skipped its ownership guard. Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * fix: close the tool import race and add ownership regression tests register_tools_bulk loaded conflicts twice. A tool published between the two queries was updated without an ownership check. Load conflicts once per chunk and reuse the result. Reject any name that appears after the load. Add database-backed ownership tests for tools, resources and prompts. Add a live-gateway test for the two-tenant import case. Refs: PSIRT 4084216 Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * chore: refresh secrets baseline after rebase Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> --------- Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> Co-authored-by: Brian Hussey <brian.hussey@ie.ibm.com>
…BM#7250) mcp-server-filesystem is being removed from contextforge-examples due to the security risks of exposing arbitrary filesystem paths. Replace all three doc files that referenced it: - docs/docs/using/reverse-proxy.md: replace filesystem-server entries in the CLI options example, Docker Compose service, and multi-server YAML with mcp-server-fetch equivalents (no volume mounts required) - docs/docs/using/mcpgateway-translate.md: replace the two mcp-server-filesystem usages with mcp-server-fetch; correct the accompanying JS snippet method from resources/list to tools/list - license-policy.toml: remove the filesystem-server allowlist entry and its comment (the crate is no longer present) mcp-server-fetch is the official Python MCP server from the modelcontextprotocol/servers repository, published on PyPI, and requires no arguments, credentials, or volume mounts. Signed-off-by: Jonathan Springer <jps@s390x.com>
* feat: adopt per-user MCP server rate limiting Adopt cpex-rate-limiter 0.1.11 and expose opt-in configuration. Normalize gateway-less plugin context identity. Add Redis acceptance coverage for both transport modes and gateway replicas. Closes IBM#6230 Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> * fix: skip plugin integration gate on draft PRs Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> * fix: declare completion test server license Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> * fix: address per-server rate limiter review feedback Require cpex-rate-limiter 0.1.12 for corrected prompt hooks. Cover scoped users, tenant isolation, wildcard bindings, and REST/MCP prompts. Run acceptance coverage through both enforcement paths. PyPI publication remains pending. Regenerate uv.lock after publication. Validation uses the merged upstream correction: 802 unit tests, 56 existing integration tests, and 24 live gateway cases pass. Static hooks pass with UV_NO_SYNC=1 while publication is pending. Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> * fix: lock published rate limiter 0.1.12 Resolve the published prompt-hook correction from PyPI. Update only cpex-rate-limiter and its lockfile metadata. Validation: 802 unit tests, 56 existing integration tests, and 24 live gateway cases pass against the published wheel. Normal pre-commit hooks and uv lock --check pass. Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> --------- Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com>
* chore: remove Rust MCP config surface (issue IBM#5819) Remove the experimental_rust_mcp_* family of settings, the RUST_MCP_MODE translation entrypoint, and all Rust MCP env vars from .env.example and docker-compose.yml. Changes: - mcpgateway/config.py: drop experimental_rust_mcp_runtime_enabled, experimental_rust_mcp_runtime_url, experimental_rust_mcp_runtime_uds, experimental_rust_mcp_runtime_timeout_seconds, experimental_rust_mcp_session_core_enabled, experimental_rust_mcp_event_store_enabled, experimental_rust_mcp_resume_core_enabled, experimental_rust_mcp_live_stream_core_enabled, experimental_rust_mcp_affinity_core_enabled, experimental_rust_mcp_session_auth_reuse_enabled, mcp_rust_ingress, mcp_rust_public_proxy_upstream, and the _validate_experimental_rust_runtime_uds field validator - docker-entrypoint.sh: remove apply_rust_mcp_mode_defaults(), print_mcp_runtime_mode(), start_managed_rust_mcp_runtime(), all RUST_MCP_*/EXPERIMENTAL_RUST_MCP_*/MCP_RUST_* variable declarations, and the conditional managed-sidecar startup branch - .env.example: remove Rust MCP (simple) block and residual inline references in the auth cache section - docker-compose.yml: remove ENABLE_RUST/ENABLE_RUST_MCP_RMCP build args and all Rust MCP environment entries from the gateway service; add '# pragma: allowlist secret' to 16 pre-existing false-positive vault-radar findings (env var references and placeholder values) - tests/unit/mcpgateway/test_config.py: remove four tests asserting on deleted fields - tests/unit/test_docker_entrypoint.py: remove test_print_mcp_runtime_mode_warns_when_rust_enabled All settings defaulted to false/disabled and served a runtime past its sunset date (2026-07-07). Hard removal is appropriate. Closes IBM#5819 Signed-off-by: Jonathan Springer <jps@s390x.com> * chore: suppress vault-radar false positives in test_token_exchange_credential_isolation.py Add '# pragma: allowlist secret' to 6 lines flagged as password_assignment findings by Vault Radar. All reference test-only variables (_PASSWORD, admin_password) or pass them as JSON payload fields in live gateway tests, not real credentials. Suppressed lines: 200, 236, 244, 261, 284, 287. Signed-off-by: Jonathan Springer <jps@s390x.com> --------- Signed-off-by: Jonathan Springer <jps@s390x.com>
* fix: paginate downstream MCP catalog responses Add bounded catalog pages and scope-bound cursors for Python and Rust runtimes. Store direct-proxy continuation snapshots in Redis. Closes IBM#6990 Signed-off-by: Vishu <vishu@dhcp-9-224-185-57.iea-ie.ibm.com> * fix: decode catalog visibility for native Rust pagination Accept the Python authorization response field and test the backend contract. This enables native PostgreSQL catalog queries instead of fallback. Refs IBM#6990 Signed-off-by: Vishu <vishu@dhcp-9-224-185-57.iea-ie.ibm.com> * test: reconcile pagination coverage with upstream changes Adapt the header-validation fixture to the paginated catalog adapter. Refresh the secret baseline from main and preserve audited synthetic cursor vectors. Refs IBM#6990 Signed-off-by: Vishu <vishu@dhcp-9-224-185-57.iea-ie.ibm.com> * chore: vet cursor cryptography dependencies Extend existing RustCrypto publisher trust to seven cursor cryptography crates. Verify locked release publishers and refresh imported audit metadata. Refs IBM#6990 Signed-off-by: Vishu <vishu@dhcp-9-224-185-57.iea-ie.ibm.com> * fix: close pagination parity and authorization coverage gaps Preserve resource size in Python listings and verify native PostgreSQL visibility across pages. Check virtual server existence during Rust authentication. Return MCP tool errors for invocation failures. Add PostgreSQL CI execution and resolve both reported E2E failures. Signed-off-by: Vishu <vishu@Vishus-MacBook-Pro.local> * refactor: delegate MCP catalog pagination to Python Remove native Rust catalog pagination and its cursor dependencies. Delegate catalog requests to Python with cursors, metadata, and authorization intact. Add delegation and visibility regression tests. Fix a shared service mock that leaked between tests. Refs IBM#6990 Signed-off-by: vishu bhatnagar <vishu.bhatnagar@ibm.com> * fix: preserve MCP routing across session affinity Preserve gateway selection across local and forwarded requests. Advertise MCP Apps from authenticated request scopes. Preserve expanded resource URIs and expected tool failures. Isolate asynchronous E2E tests from synchronous Playwright fixtures. Correct correlation-reference fixtures and document Bandit false positives. Enforce forward-listener initialization without removable assertions. Refs IBM#6990 Signed-off-by: vishu bhatnagar <vishu.bhatnagar@ibm.com> * test: isolate session registry dependency imports Import missing-dependency cases into separate modules. Preserve the shared registry class and availability flags across tests. Signed-off-by: vishu bhatnagar <vishu.bhatnagar@ibm.com> * fix: address Python MCP pagination review findings Normalize protocol and internal session headers for cursor binding. Exclude tracing headers while preserving authorization and tenant bindings. Bound Redis snapshot bytes, snapshot count, and concurrent collectors. Serialize prompts without gateway relationship queries and log sanitized failures. Remove Rust changes and unrelated tool-error, resource-URI, and Apps changes. Document Python-only scope, compatibility, Redis limits, and rollout requirements. Preserve public complete-list helpers for external compatibility. Validation: 1004 post-rebase regression tests pass, with one skipped. The earlier full unit suite passes 23989 tests. Eight production pagination tests pass. Full E2E retains five failures outside pagination. The merge gate remains incomplete. Refs IBM#6990 Signed-off-by: vishu bhatnagar <vishu.bhatnagar@ibm.com> Signed-off-by: Vishu <vishu@MAC-C6CA73.station> * test: fix pagination CI regressions after rebase Update the untrusted server-header test to mock list_catalog_page(). Preserve the assertion that the request body controls server selection. Refresh audited secrets baseline line numbers after the rebase. Validation: 607 RPC module tests pass. All pre-commit hooks pass. Refs IBM#6990 Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> * test: preserve cache directives with paginated adapters Update cache directive mocks to use the paginated catalog handler. Assert that list adapters retain cache directives and nextCursor. Refresh audited baseline line numbers after the rebase. Validation: pagination and transport tests pass with one legacy skip. Relevant pre-commit hooks pass. Refs IBM#6990 Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> * fix: address remaining Python pagination review comments Preserve prompt titles and SDK descriptors in paginated adapters. Separate collector slots from retained snapshot bytes and permit 32 collectors. Keep publication quotas atomic and make Redis memory information optional. Document gateway-update cursor invalidation and remove unrelated changes. Validation: 24279 unit tests, 155 pagination tests, and eight Redis tests pass. Seven production-image pagination and authorization tests pass. Ruff, interrogate, pylint, pre-commit, coverage, and secret checks pass. Changed-line coverage reaches 98 percent. The production image builds. Full E2E retains fixture and event-loop failures in the isolated setup. Bandit reports existing findings after requested scope cleanup. Refs IBM#6990 Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> * fix: resolve downstream pagination review concerns Reuse scoped proxy snapshots and release completed traversals. Prevent collection after snapshot reuse expires. Correct internal error statuses and preserve complete tools lists for bodyless bridges. Authenticate proxy scope fingerprints with a separate HMAC key. Reuse serialized descriptors and log sanitized cursor rejection categories. Align Compose collector defaults and document compatibility changes. Remove unrelated annotations and test changes. Add regression coverage for snapshot capacity, ordering, visibility, bridge compatibility, and error mapping. Validation: 24,379 unit tests, 10 Redis tests, and seven live tests pass. Changed-line coverage reaches 99 percent. Ruff, interrogate, Pylint, and changed-file pre-commit checks pass. Native Rust and the full three-replica gate remain unverified. Refs IBM#6990 Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> * fix: preserve shared catalog snapshots until expiry Keep shared snapshots available for overlapping traversals and final-page retries. Preserve fixed expiration and enforce retained byte and snapshot limits. Add service, Redis, and live MCP regression coverage. Update snapshot lifetime guidance. Validation: 905 tests pass; one test skips. Type checks pass. Ruff reports five existing blank-line errors in the live-test file. Refs IBM#6990 Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> * chore: refresh secrets baseline Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> --------- Signed-off-by: Vishu <vishu@dhcp-9-224-185-57.iea-ie.ibm.com> Signed-off-by: Vishu <vishu@Vishus-MacBook-Pro.local> Signed-off-by: vishu bhatnagar <vishu.bhatnagar@ibm.com> Signed-off-by: Vishu <vishu@MAC-C6CA73.station> Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> Signed-off-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com> Co-authored-by: Vishu <vishu@dhcp-9-224-185-57.iea-ie.ibm.com> Co-authored-by: Vishu <vishu@Vishus-MacBook-Pro.local> Co-authored-by: Suresh Kumar Moharajan <suresh.kumar.m@ibm.com>
* update dependencies 10-08 Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * test(a2a): tolerate fastapi.endpoint intermediate span in trace ancestry check fastapi 0.142.2 introduced built-in OTel telemetry that emits fastapi.dependencies, fastapi.endpoint, and fastapi.serialization child spans inside route handlers when a global TracerProvider is configured. The test gateway sets OTEL_TRACES_EXPORTER=console, which activates the global provider, so FastAPI's telemetry now inserts fastapi.endpoint between the request span and a2a.invoke. The previous assertion that a2a.invoke is a direct child of the request span therefore fails. Add _is_descendant() which walks the parent chain from a child span up to any ancestor using all exported spans. Replace the direct-parent check with a descendant check so the test tolerates any number of intermediate framework spans. All correctness invariants are preserved: same trace ID end-to-end, inbound traceparent adopted, a2a.invoke span ID injected into the downstream request. Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * test(transport): use session.call_tool() to test Mcp-Param rejection directly mcp 2.3.0 added automatic HEADER_MISMATCH recovery to Client.call_tool(): on a -32020 rejection it relists tools and retries once, so the error never surfaces to the caller when the retry succeeds. Use client.session.call_tool() (ClientSession) instead. ClientSession has no retry logic, so the server-side -32020 rejection surfaces as MCPError directly. The server behaviour is unchanged: it still rejects mismatched and absent Mcp-Param-* headers before dispatch. Only the test entry point changes to bypass the SDK recovery layer. Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> --------- Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>
* chore: Replace stylelint with biome Signed-off-by: Gabriel Costa <gabrielcg@proton.me> * feat:secrets updated Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> --------- Signed-off-by: Gabriel Costa <gabrielcg@proton.me> Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> Co-authored-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com>
IBM#7259) * fix: reject oversize _meta with -32602 and keep upstream resource error codes Validate request _meta on tools/call, resources/read and prompts/get before any service call, and answer a violation with -32602 instead of -32603. Exempt the mandatory io.modelcontextprotocol envelope keys from the key-count and depth limits so modern clients that declare nested capabilities pass. Relay the upstream JSON-RPC error of a failed resources/read through ResourceUpstreamError. A modern caller sees -32602 in place of the retired -32002 on resources/read and prompts/get. Refs IBM#6568 Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * addressed comments Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * chore: update secrets baseline line numbers Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> --------- Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>
* fixed: config for MCP inbound protocol mode in docker to auto Signed-off-by: Lang-Akshay <akshay.shinde26@ibm.com> * fixup: make pre-commit Signed-off-by: Lang-Akshay <akshay.shinde26@ibm.com> --------- Signed-off-by: Lang-Akshay <akshay.shinde26@ibm.com>
The tool conflict checks compared raw names. The set_custom_name_and_slug listener stores the slug of the custom name in DbTool.name, with the gateway prefix for a gateway tool. A name that differs from its own slug never matched its stored row. The checks accepted duplicates in one visibility scope. register_tool() also had no private-scope check. Compute the stored name the same way as the listener, with slugify() or build_gateway_tool_invocation_name(). Check it against public tools, team tools of the same team, and private tools of the same owner. _tool_name_scope() is the one definition of these scopes. - register_tool() uses LIMIT 1, so legacy duplicate rows raise ToolNameConflictError (HTTP 409) and not MultipleResultsFound (HTTP 400). - update_tool() checks the final stored name on a rename, a custom_name change, or a visibility change. One helper covers tools with and without a gateway. - _process_tool_chunk() matches each incoming tool by its position. It applies the conflict strategy to existing rows and to earlier tools of the same chunk. - _load_authorized_tool_conflicts() and _existing_tools_query() look up stored names. The import ownership check and the late-collision check therefore cover slug-equivalent names. PR IBM#6534 fixes the federation gap of the issue. A database-level unique constraint per visibility scope needs a migration and a maintainer decision. This change does not include it. gRPC reflection sync does not use these checks. Refs IBM#6189 Signed-off-by: Waleed Khalid <wak327@gmail.com>
wak327
force-pushed
the
fix/6189-tool-name-conflict-normalized
branch
from
October 20, 2026 14:51
32b4b20 to
2ba04cf
Compare
* chore: bump version to 1.1.0 Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * chore: npm update — 12 packages updated, 0 vulnerabilities Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * chore: bump NODEJS_IMAGE pin to 10.2-1792386062 Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * chore: update uv.lock Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * chore: update Cargo.lock for v1.1.0 version bump Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * chore: normalise .bumpversion.cfg indentation Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * update secrets Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * chore: bump version to 1.1.0 Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * chore: npm update — 12 packages updated, 0 vulnerabilities Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * chore: bump NODEJS_IMAGE pin to 10.2-1792386062 Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * chore: update uv.lock Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * chore: update Cargo.lock for v1.1.0 version bump Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * chore: normalise .bumpversion.cfg indentation Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * update secrets Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * test: fix live-gateway e2e suite ordering, error shapes and cleanup Run async e2e tests on a worker thread. The session-scoped sync Playwright fixture leaves an event loop running on the main thread, and pytest-asyncio then fails with "Runner.run() cannot be called from a running event loop". Accept both error shapes where the stateful-session /rpc path and the SDK handler differ: unknown tool, unknown prompt, ReDoS rejection, array structuredContent for legacy callers, and template read URIs. Delete the private tenant gateways with the tenant context that created them. The admin context got 404, so the gateways leaked and blocked the fast_time registration on the next stack start. Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * Correctly ignore checkov finding on this line Signed-off-by: Brian Hussey <brian.hussey@ie.ibm.com> * docs: set 1.1.0 changelog release date to 2026-10-19 Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * fix: map completion route to tools.read in token scoping The completion endpoint requires tools.read, but _PERMISSION_PATTERNS had no entry for POST /protocol/completion/complete. Default-deny then rejected tokens scoped to tools.read. Add the pattern and allow and deny tests. Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * chore: remove Rust MCP leftovers from Makefile, docs, and comments Remove the testing-up-rust and testing-rebuild-rust targets and the RUST_MCP_MODE and RUST_MCP_LOG variables. The entrypoint no longer reads them. Drop the Rust sections from the testing, compose, and performance docs. Remove stale Rust ingress text from the Origin/Host comments and .env.example. Note shipped pagination and completions in the roadmap. Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * fix: move checkov skip off the nginx FROM line Docker parses text after the image reference on a FROM line as arguments. Place the CKV_DOCKER_7 skip on its own line above FROM. Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * fix(admin-ui): keep table rows in innerHTML guard; use delegated handlers in edit modals (IBM#7271) The innerHTML guard added in IBM#7076 runs every write through DOMPurify, which parses in a <body> context. The HTML parser drops bare <tr>/<td> there, so tbody.innerHTML = "<tr>..." rendered cell text with no rows (log viewer). Sanitize writes to TBODY/THEAD/TFOOT/TR inside a table wrapper and strip it afterwards. Replace inline onclick/onchange in the team and user edit forms with the delegated data-action-click/data-action-change handlers, since inline handlers are blocked by the guard and CSP. Add Admin.toggleUseDefaultMaxMembers for the "Use global default" checkbox. Playwright: test_scroll_reset_on_tab_navigation used Page.wait_for_function, which strict CSP rejects (EvalError); use wait_for_js_condition() like the other admin tests. Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * release v1.1.0 performance results Signed-off-by: Lang-Akshay <akshay.shinde26@ibm.com> * fixup: release v1.1.0 performance results Signed-off-by: Lang-Akshay <akshay.shinde26@ibm.com> * feat:added release 1.1.0 changelog Signed-off-by: Lang-Akshay <akshay.shinde26@ibm.com> * feat: added missing mentions of main merged PR Signed-off-by: Lang-Akshay <akshay.shinde26@ibm.com> * feat: added missing main PRs Signed-off-by: Lang-Akshay <akshay.shinde26@ibm.com> * Update secrets baseline Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> * chore: update web UI to v0.2.4 and document startup Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> * docs: move web UI section after operations and tooling Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> * docs: match web UI changelog entry formatting Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> * docs: replace gateway demo with new web UI GIF Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> * docs: address web UI release review feedback Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> * docs: use smaller web UI demo GIF Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> * docs: add optional web UI container quick start Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> * update secrets Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> --------- Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> Signed-off-by: Brian Hussey <brian.hussey@ie.ibm.com> Signed-off-by: Lang-Akshay <akshay.shinde26@ibm.com> Signed-off-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com> Co-authored-by: Brian Hussey <brian.hussey@ie.ibm.com> Co-authored-by: Lang-Akshay <akshay.shinde26@ibm.com> Co-authored-by: Vishu Bhatnagar <vishu.bhatnagar@ibm.com>
IBM#7290) * Required updates to secrets.baseline, README.md and SECURITY.md and CODEOWNERS Signed-off-by: Brian Hussey <brian.hussey@ie.ibm.com> * Updates to containers to pass container security check Signed-off-by: Brian Hussey <brian.hussey@ie.ibm.com> * Update rust deps to pass rust-deny Signed-off-by: Brian Hussey <brian.hussey@ie.ibm.com> * Vetted the rustls-webpki change Signed-off-by: Brian Hussey <brian.hussey@ie.ibm.com> * feat: upgraded transiet depency certifi and bypassed 10 days dependency upgrade rule Signed-off-by: Lang-Akshay <255891396+Lang-Akshay@users.noreply.github.com> --------- Signed-off-by: Brian Hussey <brian.hussey@ie.ibm.com> Signed-off-by: Lang-Akshay <255891396+Lang-Akshay@users.noreply.github.com> Co-authored-by: Lang-Akshay <255891396+Lang-Akshay@users.noreply.github.com>
…x/s390x and linux/ppc64le (IBM#7295) Signed-off-by: Lang-Akshay <akshay.shinde26@ibm.com>
- Update CPEX and package exclude-newer dates in pyproject.toml - Bump Pillow and uv minimum bounds in pyproject.toml - Regenerate lockfile with uv lock --upgrade - Update docs/requirements.txt Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com>
…BM#7300) The ca-certificates RPM in UBI10 ships emSign Root CA - G1/C1 and emSign ECC Root CA - G3/C3 in a monolithic p11-kit bundle. Running microdnf upgrade alone does not remove them when the RPM version has not changed. Add an explicit revocation step to the runtime stage that splits the consolidated bundle, copies the emSign/eMudhra PEM certs into the p11-kit blocklist directory, and re-runs update-ca-trust. A build-time assertion aborts the image build if any root survives. Pin certifi>=2026.10.19 so no resolver can select an older release, and add a per-package exclude-newer-package cutoff so the rolling exclude-newer window does not block the updated release. Refresh uv.lock to certifi 2026.10.19. The emSign roots remain in the certifi 2026.10.19 bundle (the upstream main branch is already clean but no new PyPI release has shipped yet). Add tests/security/test_certifi_ca_bundle.py with parametrised checks for each removed root; roots not yet dropped by a certifi PyPI release skip with an advisory message and will self-activate once the release lands, guiding the next version floor bump. OS trust store path (ssl.create_default_context): fixed. certifi path (httpx verify=True): floor set, awaiting next PyPI release. Signed-off-by: prakhar-singh1928 <prakhar.singh1928@ibm.com> Co-authored-by: Brian Hussey <brian.hussey@ie.ibm.com>
Signed-off-by: Waleed Khalid <wak327@gmail.com> # Conflicts: # CHANGELOG.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🐛 Bug-fix PR
📌 Summary
Tool conflict checks compared raw tool names with the stored
DbTool.namevalue.DbTool.nameholds the slug of the custom name, with the gateway prefix for a gateway tool. Names that differ only in case or separator characters passed the check. One visibility scope then held two tools with the same invocation name, andtools/callcannot tell them apart.This PR makes tool creation, tool update, and configuration import compare the stored name in the target visibility scope.
Refs #6189
🔁 Reproduction Steps
Issue #6189 lists the gaps. These steps use the default
GATEWAY_TOOL_NAME_SEPARATOR(-) onmain:POST /toolswith a public tool namedmy-tool. Then sendPOST /toolswith a public tool namedMy_Tool. Both requests succeed, and two rows storemy-tool.POST /toolstwice with a public tool namedget_weather. Both requests succeed, because the check comparesget_weatherwith the storedget-weather.register_tool()has no private-scope branch.PUT /tools/{id}with{"custom_name": "My_Tool"}for a second public tool. The rename succeeds, and two rows storemy-tool.My_Toolandmy-tool. Both rows are created.🐞 Root Cause
All paths are in
mcpgateway/services/tool_service.py. Theset_custom_name_and_sluglistener inmcpgateway/db.pycomputesDbTool.nameat flush time.register_tool()compared the rawtool.namewithDbTool.name. It had no branch forprivatevisibility.update_tool()compared the rawcustom_namewithDbTool.custom_namefor a tool without a gateway. A rename to a slug-equivalent name did not match. A change ofcustom_namealone ran no check for such a tool._process_tool_chunk()compared raw names withDbTool.name, and only with existing rows. Two slug-equivalent names in one import batch were both added.💡 Fix Description
All source changes are in
mcpgateway/services/tool_service.py._tool_name_scope()returns the filter for one namespace: all public tools, the team tools of one team, or the private tools of one owner. It returnsNonewhen the team or the owner is missing. Every conflict check uses it._stored_tool_name()computes the stored name of a new tool the same way as the listener. It usesslugify()for a tool without a gateway andbuild_gateway_tool_invocation_name()for a gateway tool._check_new_tool_name_conflict()runs inregister_tool(). It usesLIMIT 1, so legacy duplicate rows raiseToolNameConflictError(HTTP 409) and notMultipleResultsFound(HTTP 400). The private scope usesowner_email, orcreated_bywhenowner_emailis empty. The new row uses the same owner.update_tool()computes the final custom name and the final stored name. It runs_check_tool_name_conflict()on a rename, acustom_namechange, or a visibility change. That helper now comparesDbTool.nameand replaces_check_gateway_tool_invocation_name_conflict(). It keeps theget_for_update()row lock and excludes the tool itself._process_tool_chunk(),_process_single_tool_for_bulk()and_create_tool_object()now annotate the team ID asOptional[str].register_tools_bulk()already takesteam_id: Optional[str], and team IDs are strings._process_tool_chunk()computes one stored name for each incoming tool and matches by position. A tool that the chunk added earlier counts as an existing tool, so the conflict strategy applies to it. Withupdate, the later entry updates the pending row._process_single_tool_for_bulk()now receives oneexisting_tooland not a name map._load_authorized_tool_conflicts()and_existing_tools_query()(added onmainby fix: enforce ownership on configuration import updates #7243) look up stored names in the scope from_tool_name_scope(). The import ownership check and the late-collision check of fix: enforce ownership on configuration import updates #7243 therefore cover slug-equivalent names. A non-owner who importsShared_Toolwithconflict_strategy=updategetsPermission deniedfor the storedshared-toolrow.Tests:
tests/unit/mcpgateway/services/test_tool_service.py::TestStoredToolNameConflictuses the real test database. It covers the public, team, and private scopes, thecreated_byowner fallback, legacy duplicate rows, and gateway-prefixed names. It also covers renames throughnameandcustom_name, a visibility change, and a rename that keeps the stored name of the tool itself. For bulk import, it covers existing rows, two names in one batch, and one name on two gateways. Expected names come fromslugify()andbuild_gateway_tool_invocation_name(), so the tests pass for any separator.tests/live_gateway/e2e/test_e2e.py::TestToolNameConflictsendsPOST /tools(HTTP 409) andPUT /tools/{id}(HTTP 400) to a live gateway. It readsGATEWAY_TOOL_NAME_SEPARATORto build the expected stored name._process_single_tool_for_bulk()tests passexisting_tool. Theupdate_tool()doctest setsname,custom_name, andvisibilitytoNone, so the mock does not reachslugify().Behavior changes
POST /tools, the Admin UI add form, and configuration import reject a name whose stored name a tool in the same scope already uses.PUT /tools/{id}and the Admin UI edit form reject a rename or a visibility change to a stored name in use.PUT /tools/{id}keeps its existing HTTP 400 for an update conflict. The Admin UI edit form keeps HTTP 409.visibility="team"and noteam_idruns no conflict query. Before, it queried the private tools of the owner, which is the wrong scope.register_tool()already skipped this case.Deviations from the issue
custom_namevalues. This PR comparesDbTool.name, the stored invocation name.tools/callresolves tools by that value. Two gateways can each exposesearch. Their tools store<gateway-a>-searchand<gateway-b>-search, so they do not collide. Acustom_namecomparison rejects that valid case.mainby Fix for Internal 453 #6534. This PR reuses its gateway-prefixed name helper.Known residual gaps
mcpgateway/services/grpc_service.pyaddsDbToolrows without a scope check. The feature is off by default (MCPGATEWAY_GRPC_ENABLED=false), and the issue does not list it.update_tool()reads the conflict row throughget_for_update(), which expects one row. If two legacy duplicate rows already use the target stored name, the update fails withMultipleResultsFound(HTTP 400) and not withToolNameConflictError.renamestrategy does not check the generated<name>_imported_<timestamp>name again. This behavior is unchanged.📏 Reviewability
triage🧪 Verification
All commands ran from the repository root.
pythonis the project virtual environment interpreter.uv tool run ruff==0.15.20 check mcpgateway/services/tool_service.py tests/unit/mcpgateway/services/test_tool_service.py tests/unit/mcpgateway/services/test_tool_service_coverage.py tests/live_gateway/e2e/test_e2e.pytool_service.pyclean. 4 findings on lines this PR does not change: 3useless-import-aliasintest_tool_service_coverage.py, 1 blank-line rule attest_e2e.py:3955uv tool run ruff==0.15.20 format --checkon the same filestool_service.pyformatted. The 3 test files have format diffs only in hunks this PR does not changemake pylint TARGET=mcpgateway/services/tool_service.pymake pylint TARGET="tests/unit/mcpgateway/services/test_tool_service.py tests/unit/mcpgateway/services/test_tool_service_coverage.py tests/live_gateway/e2e/test_e2e.py"W0621 redefined-outer-namefor pytest fixtures, as in the rest of these filesuv tool run interrogate==1.7.0 -v mcpgateway/services/tool_service.pypython -m mypy mcpgateway/services/tool_service.py --follow-imports=silentmain(same local mypy version). Three messages are new. Each is on an existing assignment in the bulkupdatestrategy, for exampleexisting_tool.input_schema = tool.input_schema. mypy checks these lines now becauseexisting_toolhas theDbTooltype, notAnypython -m pytest tests/unit/mcpgateway/services/test_tool_service.py tests/unit/mcpgateway/services/test_tool_service_coverage.py tests/unit/mcpgateway/services/test_row_level_locking.py tests/unit/mcpgateway/services/test_tool_audit_no_db.py tests/unit/mcpgateway/test_email_extraction_consistency.py -n 8GATEWAY_TOOL_NAME_SEPARATOR=-- python -m pytest tests/unit/mcpgateway/services/test_tool_service.py -k TestStoredToolNameConflictmain-k TestStoredToolNameConflictrun withtool_service.pyfrommainpython -m pytest --doctest-modules mcpgateway/services/tool_service.py mcpgateway/utils/create_slug.py --no-cov, with test-only values forJWT_SECRET_KEY,AUTH_ENCRYPTION_SECRET,PLATFORM_ADMIN_PASSWORD,DEFAULT_USER_PASSWORD, andBASIC_AUTH_PASSWORDDATABASE_URL='sqlite:///:memory:' TEST_DATABASE_URL='sqlite:///:memory:' ARGON2ID_TIME_COST=1 ARGON2ID_MEMORY_COST=1024 python -m pytest -n auto --maxfail=0 --ignore=tests/fuzz --ignore=tests/manual --ignore=test.py --ignore=tests/live_gateway(themake testenvironment and ignore list)--cov=mcpgateway --cov-report=xml:<file>, thendiff-cover <file> --compare-branch=main --fail-under=90tool_service.py: 55 changed lines, 100% covereduv tool run --from 'git+https://github.com/ibm/detect-secrets.git@076672a9a01abdfc7ecee2e7d14f08cdccb73976' detect-secrets scan --use-all-pluginson the 5 changed filesa211ab7dtests/unit/mcpgateway/services/test_import_service.pytest_register_tools_bulk_update_ownership_matches_stored_namewithtool_service.pyfrommainTool name conflictand notPermission deniedpython -m pytest --doctest-modules mcpgateway/services/tool_service.pywith test-only secretspython -m pytest -p playwright --collect-only tests/live_gateway/e2e/test_e2e.py -k TestToolNameConflictChecks not run:
make docker-nuke docker-prod-rust testing-up RUST_MCP_MODE=andmake test-e2e: no gateway stack runs on the test host.make docker-nukeremoves local Docker state, and the host runs unrelated containers. The new live-gateway tests are collected but not executed.make coverage: the diff coverage above comes from the affected unit tests, not from the full suite.make detect-secrets-scan: this target rewrites.secrets.baseline. The scan above reads only the changed files.📐 MCP Compliance (if relevant)
✅ Checklist
make black isort pre-commit)