Skip to content

Mask URL passwords in the SQL query log, fix #1630 - #1631

Merged
Slach merged 2 commits into
Altinity:masterfrom
bt-macole:fix/redact-url-credentials-in-sql-log
Oct 21, 2026
Merged

Slach merged 2 commits into
Altinity:masterfrom
bt-macole:fix/redact-url-credentials-in-sql-log

Conversation

@bt-macole

@bt-macole bt-macole commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

Fix #1630

Problem

With api.create_integration_tables: true and api.password set, CreateIntegrationTables builds the system.backup_actions, system.backup_list and system.backup_version DDL with the API credentials in the URL() engine argument (...?pass=<password>&user=<user>). LogQuery writes every query as is, at info level with the default clickhouse.log_sql_queries: true, so the API password reaches the logs on every server start.

Fix

LogQuery passes the message through a new maskURLCredentials helper before logging:

  • pass= / password= query string values become [MASKED] (the integration tables use pass=; password= covers user URL() tables whose DDL is logged during restore)
  • the password in scheme://user:password@host becomes [MASKED]

The query returned by LogQuery, which is the one sent to ClickHouse, is unchanged. The [MASKED] marker matches the one pkg/config already uses for sensitive env overrides. The existing helpers (maskSensitiveEnvValue, MaskEnvOverrideCommand, restoreCloudRedact) work on env names, command lines and known secret values, none of which fits query text inside pkg/clickhouse.

ChangeLog: entry added under v2.8.2 BUG FIXES.

Tests

  • New TestLogQueryMasksURLCredentials in pkg/clickhouse/log_query_test.go captures the zerolog output of LogQuery and checks the integration-table DDL built the same way as CreateIntegrationTables (url.Values.Encode()), a password= parameter, userinfo, a query with args, and a query without credentials that must stay unchanged. It also asserts that LogQuery returns the original query.
  • Mutation check: reverting the LogQuery change fails 4 of 5 subtests; disabling only the query-parameter regex fails 3; disabling only the userinfo regex fails 1; narrowing the regex to pass= fails the password= subtest.
  • gofmt -l ./pkg ./cmd printed nothing, go vet ./... passed, go test ./pkg/... ./cmd/... passed. The docker-based integration tests (test/integration, build tag integration) were not run locally.

🤖 Generated with Claude Code

With api.create_integration_tables and api.password set, every server
start logged the system.backup_* DDL with the API password in the URL()
engine argument. LogQuery now masks pass= / password= query string
values and user:password@ userinfo in the logged message; the query
sent to ClickHouse is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Slach Slach added this to the 2.8.3 milestone Oct 20, 2026
@Slach

Slach commented Oct 20, 2026

Copy link
Copy Markdown
Collaborator

could you ask your LLM how your fix affect performance?

@Slach Slach modified the milestones: 2.8.3, 2.8.2 Oct 20, 2026
LogQuery built and masked the message even when the event level was
disabled (log_sql_queries: false without debug logging), which became
measurably more expensive once the URL credential regexps were added.
Return early when the event is disabled, and run the regexps only when
the message contains "://", since both masked forms are URL parts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@coveralls

Copy link
Copy Markdown

Coverage Report for CI Build 39458900994

Warning

Build has drifted: This PR's base is out of sync with its target branch, so coverage data may include unrelated changes.
Quick fix: rebase this PR. Learn more →

Coverage decreased (-2.5%) to 67.829%

Details

  • Coverage decreased (-2.5%) from the base build.
  • Patch coverage: 2 uncovered changes across 1 file (11 of 13 lines covered, 84.62%).
  • 828 coverage regressions across 21 files.

Uncovered Changes

File Changed Covered %
pkg/clickhouse/clickhouse.go 13 11 84.62%

Coverage Regressions

828 previously-covered lines in 21 files lost coverage.

Top 10 Files by Coverage Loss Lines Losing Coverage Coverage
pkg/backup/restore_cloud.go 240 33.84%
pkg/storage/cos.go 161 3.56%
pkg/server/server.go 114 56.79%
pkg/storage/gcs.go 86 44.11%
pkg/clickhouse/clickhouse.go 81 80.45%
pkg/storage/s3.go 31 61.27%
pkg/storage/general.go 28 67.41%
cmd/clickhouse-backup/main.go 22 77.73%
pkg/backup/backuper.go 19 74.61%
pkg/backup/restore_remote.go 12 56.73%

Coverage Stats

Coverage Status
Relevant Lines: 29446
Covered Lines: 19973
Line Coverage: 67.83%
Coverage Strength: 26708.6 hits per line

💛 - Coveralls

@Slach
Slach merged commit 04940b5 into Altinity:master Oct 21, 2026
30 checks passed
@Slach

Slach commented Oct 21, 2026

Copy link
Copy Markdown
Collaborator

@bt-macole thanks for contribution

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

API password is logged in plain text when api.create_integration_tables is enabled

3 participants