[BUG] Notification channel dropdown/config fails to load in trigger action editor when using Workspaces with a single local Data Source

[snussbaumermpreis]

What is the bug?

When creating or editing a Monitor's Trigger inside a Workspace, the notification channel selector in the Action editor cannot load any channels, even though valid Notifications plugin channels, senders, and recipient groups exist and are correctly returned by the Notifications REST API directly.

Opening the Trigger's Action panel triggers two requests that both fail with 400 Bad Request:

GET /w/<workspaceId>/api/notifications/get_configs?from_index=0&max_items=5000&sort_field=name&sort_order=asc
→ 400 { "message": "[request query.config_type]: expected at least one defined value but got [undefined]" }

GET /w/<workspaceId>/api/notifications/features
→ 400 { "message": "[request query.dataSourceId]: expected value of type [string] but got [undefined]" }

The second error is the key signal: the frontend is not passing dataSourceId on these requests when the workspace's only configured Data Source is the local/default cluster ("localcluster"). Either the local data source is represented as an empty string or omitted entirely, but either way the request fails validation before reaching the Notifications plugin.

As a direct consequence, any existing trigger action referencing a valid destination_id (a Notifications config_id) cannot be edited or re-saved through the UI — the channel field shows as empty/unresolved, and attempting to save risks the action being cleared, because the UI cannot resolve the referenced config_id against a channel list it failed to load.

How can one reproduce the bug?

1. Enable Workspaces (workspace.enabled: true) together with data_source.enabled: true, with only a single local Data Source available (no additional registered remote Data Sources).
2. Create a Notifications channel (config_type: email) via Notifications → Channels, with a valid SMTP sender and recipient group. Confirm it's retrievable via GET _plugins/_notifications/configs.
3. Inside a Workspace, create or open a Monitor and go to a Trigger's Action section.
4. Open the browser Network tab. Observe GET /w/<workspaceId>/api/notifications/get_configs and GET /w/<workspaceId>/api/notifications/features both return 400.
5. The channel dropdown in the Action editor remains empty despite channels existing.
6. As a workaround, add the action directly via PUT _plugins/_alerting/monitors/{monitor_id} referencing the channel's config_id as destination_id. The monitor saves and executes correctly (verified via _execute), confirming the channel/action itself is valid — the failure is isolated to the Workspace-scoped Notifications API calls made by the Alerting Dashboards UI.
7. Reopening the same trigger in the UI afterward still shows the channel as unresolved/empty, and the Action cannot be edited/saved through the UI without losing the reference.

Confirmed workaround

Setting data_source.enabled: false in opensearch_dashboards.yml (with a single local cluster and no remote data sources needed) and restarting OpenSearch Dashboards resolves the issue immediately: the channel dropdown loads correctly and existing actions resolve as expected. This strongly confirms the root cause is isolated to the dataSourceId parameter handling described below, and is not related to Workspaces, Notifications configuration, permissions, or the Alerting destination.allow_list.

What is the expected behavior?

• The Alerting Dashboards trigger/action editor should successfully load Notifications channels when operating inside a Workspace with data_source.enabled: true and only a local Data Source configured.
• Existing actions referencing valid destination_id/config_id values should resolve and display correctly in the UI, and remain editable without risk of the reference being silently dropped on save.

What is your host/environment?

• OpenSearch: 3.6 (self-managed)
• OpenSearch Dashboards: 3.6
• Plugins: alerting-dashboards-plugin, dashboards-notifications, Security plugin enabled
• Workspaces enabled (workspace.enabled: true)
• Bug reproduces with data_source.enabled: true; confirmed resolved with data_source.enabled: false
• Single local Data Source ("localcluster") — no additional remote Data Sources registered
• Notifications plugin: filter_by_backend_roles: false, standard config, 26+ existing configs (channels/senders/groups) confirmed valid via direct API queries

Root cause hypothesis

This appears to violate the documented design for dataSourceId handling. Per the official developer guide for multi-data-source plugin integration (https://opensearch.org/blog/develop-guideline-multiple-data-source-in-opensearch-and-plugins/):

"To register server-side APIs, set up the API route with dataSourceId as an optional query parameter... If dataSourceId is provided, use the multiple data sources client to query the remote cluster based on the ID. Otherwise, use the OpenSearch client to query the local cluster."

The Notifications API routes (get_configs, features) as called by the Alerting Dashboards frontend in a Workspace context appear to enforce dataSourceId as a required string parameter instead, rejecting the request with 400 when it is undefined/empty (i.e., when the local cluster is the active data source). This looks like either (a) a validation schema bug in the Notifications plugin's route handlers, treating an optional parameter as required, or (b) a bug in how the Alerting Dashboards frontend workspace/data-source context passes (or fails to pass) the local cluster's identifier to these specific API calls.

Given the confirmed workaround (disabling data_source.enabled entirely fixes it), this is very likely (b): a frontend issue specific to how the Alerting Dashboards plugin queries Notifications when the Data Source feature is active but the local cluster (rather than a registered remote data source) is selected.

Additional context

Direct Dev Tools queries against the same cluster work correctly and return all expected data, e.g.:

GET _plugins/_notifications/configs
GET _plugins/_notifications/features
GET _cluster/settings?include_defaults=true&filter_path=**.plugins.alerting

All confirm channels, senders, and the allow_list (which includes email) are correctly configured. Combined with the confirmed data_source.enabled: false workaround, this isolates the issue specifically to how the Alerting Dashboards frontend (or the Notifications plugin route handlers it calls) constructs/requires the dataSourceId query parameter when the Data Source feature is enabled but only the local cluster is in use.

[dpereiraegoi]

Same error here with v3.7.0

[pitch-d]

We also got affected by this bug when upgrading from 3.2.2 to 3.7.0.
We can't edit old monitors nor create new ones