Repository navigation
ROB-1291 Send the Prometheus tenant from the namespace label #2213
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Closed
Closed
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,20 @@ | ||
| import logging | ||
| from typing import Optional | ||
|
|
||
| from robusta.integrations.kubernetes.api_client_utils import get_namespace_labels | ||
|
|
||
|
|
||
| def resolve_tenant(key: str, namespace: Optional[str]) -> Optional[str]: | ||
| """ | ||
| Returns the value of the `key` label of a namespace, read from the cached namespace data (NAMESPACE_DATA_TTL). | ||
| Returns None when there is no namespace or no such label, so the caller keeps its configured default tenant. | ||
| """ | ||
| if not namespace: | ||
| return None | ||
| try: | ||
| tenant = (get_namespace_labels(namespace) or {}).get(key) | ||
| except KeyError: # a namespace created after the last reload, or deleted | ||
| tenant = None | ||
| if not tenant: | ||
| logging.warning(f"No '{key}' label on namespace {namespace}, using the default Prometheus tenant") | ||
| return tenant |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,125 @@ | ||
| from types import SimpleNamespace | ||
| from unittest.mock import patch | ||
|
|
||
| from robusta.core.model import base_params | ||
| from robusta.core.model.base_params import PrometheusParams, PrometheusQueryParams | ||
| from robusta.core.reporting.base import FindingSubject | ||
| from robusta.core.reporting.consts import FindingSubjectType | ||
| from robusta.integrations.prometheus.tenant import resolve_tenant | ||
|
|
||
| NAMESPACE_LABELS = "robusta.integrations.prometheus.tenant.get_namespace_labels" | ||
|
|
||
|
|
||
| def _namespaces(labels_by_namespace): | ||
| return lambda namespace: labels_by_namespace[namespace] | ||
|
|
||
|
|
||
| def test_tenant_from_namespace_label(): | ||
| with patch(NAMESPACE_LABELS, side_effect=_namespaces({"shop": {"team": "corporate"}})): | ||
| assert resolve_tenant("team", "shop") == "corporate" | ||
|
|
||
|
|
||
| def test_no_namespace_reads_nothing(): | ||
| with patch(NAMESPACE_LABELS) as labels: | ||
| assert resolve_tenant("team", None) is None | ||
| labels.assert_not_called() | ||
|
|
||
|
|
||
| def test_missing_label_returns_none(): | ||
| with patch(NAMESPACE_LABELS, side_effect=_namespaces({"shop": {"app": "web"}})): | ||
| assert resolve_tenant("team", "shop") is None | ||
|
|
||
|
|
||
| def test_namespace_with_no_labels_returns_none(): | ||
| with patch(NAMESPACE_LABELS, side_effect=_namespaces({"shop": None})): | ||
| assert resolve_tenant("team", "shop") is None | ||
|
|
||
|
|
||
| def test_unknown_namespace_returns_none(): | ||
| with patch(NAMESPACE_LABELS, side_effect=KeyError("shop")): | ||
| assert resolve_tenant("team", "shop") is None | ||
|
|
||
|
|
||
| def _event(subject: FindingSubject): | ||
| return SimpleNamespace(get_subject=lambda: subject) | ||
|
|
||
|
|
||
| # The pod and alert labels carry another value on purpose: only the namespace label counts | ||
| POD_SUBJECT = FindingSubject( | ||
| name="web", subject_type=FindingSubjectType.TYPE_POD, namespace="shop", labels={"team": "from-pod"} | ||
| ) | ||
| NAMESPACE_ONLY_SUBJECT = FindingSubject(name="Unresolved", subject_type=FindingSubjectType.TYPE_NONE, namespace="shop") | ||
| UNRESOLVED_SUBJECT = FindingSubject(name="Unresolved", subject_type=FindingSubjectType.TYPE_NONE) | ||
| NAMESPACES = {"shop": {"team": "corporate"}, "billing": {"team": "retail"}} | ||
|
|
||
|
|
||
| def test_no_tenant_label_configured_changes_nothing(): | ||
| params = PrometheusParams(prometheus_additional_headers={"X-Scope-OrgID": "sent"}) | ||
| with patch.object(base_params, "PROMETHEUS_TENANT_LABEL", None), patch(NAMESPACE_LABELS) as labels: | ||
| params.post_event_initialization(_event(POD_SUBJECT)) | ||
| assert params.prometheus_additional_headers == {"X-Scope-OrgID": "sent"} | ||
| labels.assert_not_called() | ||
|
|
||
|
|
||
| def test_tenant_from_subject_namespace_ignores_resource_labels(): | ||
| params = PrometheusParams() | ||
| with patch.object(base_params, "PROMETHEUS_TENANT_LABEL", "team"), patch( | ||
| NAMESPACE_LABELS, side_effect=_namespaces(NAMESPACES) | ||
| ): | ||
| params.post_event_initialization(_event(POD_SUBJECT)) | ||
| assert params.prometheus_additional_headers == {"X-Scope-OrgID": "corporate"} | ||
|
|
||
|
|
||
| def test_namespace_only_alert_gets_tenant(): | ||
| params = PrometheusParams() | ||
| with patch.object(base_params, "PROMETHEUS_TENANT_LABEL", "team"), patch( | ||
| NAMESPACE_LABELS, side_effect=_namespaces(NAMESPACES) | ||
| ): | ||
| params.post_event_initialization(_event(NAMESPACE_ONLY_SUBJECT)) | ||
| assert params.prometheus_additional_headers == {"X-Scope-OrgID": "corporate"} | ||
|
|
||
|
|
||
| def test_tenant_header_replaces_sent_one_and_keeps_others(): | ||
| params = PrometheusParams(prometheus_additional_headers={"X-Scope-OrgID": "forged", "X-Other": "1"}) | ||
| with patch.object(base_params, "PROMETHEUS_TENANT_LABEL", "team"), patch( | ||
| NAMESPACE_LABELS, side_effect=_namespaces(NAMESPACES) | ||
| ): | ||
| params.post_event_initialization(_event(POD_SUBJECT)) | ||
| assert params.prometheus_additional_headers == {"X-Scope-OrgID": "corporate", "X-Other": "1"} | ||
|
|
||
|
|
||
| def test_no_namespace_keeps_default_tenant(): | ||
| params = PrometheusParams(prometheus_additional_headers={"X-Scope-OrgID": "default"}) | ||
| with patch.object(base_params, "PROMETHEUS_TENANT_LABEL", "team"), patch(NAMESPACE_LABELS) as labels: | ||
| params.post_event_initialization(_event(UNRESOLVED_SUBJECT)) | ||
| assert params.prometheus_additional_headers == {"X-Scope-OrgID": "default"} | ||
| labels.assert_not_called() | ||
|
|
||
|
|
||
| def test_query_params_namespace_wins_over_event_subject(): | ||
| params = PrometheusQueryParams(promql_query="up", duration={"duration_minutes": 5}, namespace="billing") | ||
| with patch.object(base_params, "PROMETHEUS_TENANT_LABEL", "team"), patch( | ||
| NAMESPACE_LABELS, side_effect=_namespaces(NAMESPACES) | ||
| ): | ||
| params.post_event_initialization(_event(POD_SUBJECT)) | ||
| assert params.prometheus_additional_headers == {"X-Scope-OrgID": "retail"} | ||
|
|
||
|
|
||
| def test_requests_do_not_share_headers(): | ||
| # Two requests built from the same global config, as merge_global_params does, for different tenants | ||
| global_config = {"prometheus_additional_headers": {"X-Scope-OrgID": "default"}} | ||
| with patch.object(base_params, "PROMETHEUS_TENANT_LABEL", "team"), patch( | ||
| NAMESPACE_LABELS, side_effect=_namespaces(NAMESPACES) | ||
| ): | ||
| first, second = [ | ||
| PrometheusQueryParams( | ||
| **global_config.copy(), promql_query="up", duration={"duration_minutes": 5}, namespace=namespace | ||
| ) | ||
| for namespace in ("shop", "billing") | ||
| ] | ||
| first.post_event_initialization(_event(UNRESOLVED_SUBJECT)) | ||
| second.post_event_initialization(_event(UNRESOLVED_SUBJECT)) | ||
|
|
||
| assert first.prometheus_additional_headers == {"X-Scope-OrgID": "corporate"} | ||
| assert second.prometheus_additional_headers == {"X-Scope-OrgID": "retail"} | ||
| assert global_config == {"prometheus_additional_headers": {"X-Scope-OrgID": "default"}} |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Script executed:
Repository: robusta-dev/robusta
Length of output: 41600
Script executed:
Repository: robusta-dev/robusta
Length of output: 29817
Script executed:
Repository: robusta-dev/robusta
Length of output: 27858
Script executed:
Repository: robusta-dev/robusta
Length of output: 29879
Script executed:
Repository: robusta-dev/robusta
Length of output: 14280
Authorize the selected namespace before resolving the tenant.
PrometheusQueryParams.namespaceoverrides the event subject namespace. An authenticated action request can pass this field through toprometheus_enricher, which sends the query with the selected namespace's tenant header. Add namespace-level authorization at the UI or action entrypoint before accepting a namespace that differs from the event subject.