Allowlist remote database engine connection errors in upgrade check#108560
Conversation
|
cc @maxknv — could you review this? It allowlists two background-cleaner |
The upgrade check asserts that the server log contains no <Error> lines.
Test 04210_show_remote_databases_in_system_tables creates two remote database
engines pointing at the deliberately unreachable host 192.0.2.1 (RFC5737
TEST-NET-1):
- ENGINE = PostgreSQL('192.0.2.1:5432', ...)
- ENGINE = MySQL('192.0.2.1:3306', ...)
PostgreSQL: DatabasePostgreSQL::loadStoredObjects activates the
removeOutdatedTables cleaner task, whose periodic pool->get() times out against
the unreachable host and logs <Error> for each retry, including right after the
upgrade restart reloads the persisted engine.
MySQL: the engine probes the server while loading the persisted object after the
upgrade restart, logging <Error> from mysqlxx::Pool, the PoolWithFailover retry
logger, and the DatabaseMySQL ALL_CONNECTION_TRIES_FAILED exception.
The upgrade-test environment has no real PostgreSQL or MySQL server, so these
connection errors are fixture noise, not a compatibility regression (real
PostgreSQL/MySQL regressions are covered by integration tests with a live server).
Allowlist them in upgrade_runner.sh by requiring the component AND the
connection-failure symptom together, so genuine DatabasePostgreSQL/DatabaseMySQL
errors (logic, parsing, a different error code) still surface.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
7ea15b6 to
fe57785
Compare
|
Updated the fix to also cover the MySQL engine that test Pre-PR validation gate (click to expand)
Session id: cron:clickhouse-worker-slot-0:20260626-014500 |
|
Workflow [PR], commit [d6c123a] Summary: ✅ AI ReviewSummaryThis PR's remaining net change against Findings
Final VerdictStatus: Minimum required actions:
|
CI finish ledger — fe57785Every failure below has an owner: a fixing PR (ours or external), or a full-effort fix task whose fixing-PR link will be posted here when it opens. Only
Session id: cron:our-pr-ci-monitor:20260626-160000 |
|
@alexey-milovidov this logical error is not caused by this PR (it only changes Root cause: Fix in #108730: run the final task inline if scheduling fails, so its future always carries the real result/exception. Reproduced locally (MinIO + fault injection) and added a unit test that fails without the fix. |
…eck-postgresql-cleaner-noise
|
Continued via Merged fresh The merge also pulls in The two prior CI reds are unrelated to this change (which only adds MySQL/PostgreSQL connection-failure matchers):
Review: the |
|
Continued via The branch is already fully merged with Action this run: the gated CI workflow run for the merge commit Expected on the fresh run: the
The Blocked only on fresh CI completing + maintainer merge. |
|
Continued via Fresh CI on
The only two reds are unrelated flakes — impossible for a CI-only change to
No code change needed: APPROVED, AI Review ✅ Approve, 0 unresolved threads, |
…eck-postgresql-cleaner-noise # Conflicts: # tests/docker_scripts/upgrade_runner.sh
|
Continued via The branch was re-merged with fresh One scope change worth noting:
Still approved, 0 unresolved review threads. The only prior red was the unrelated |

Changelog category (leave one):
Changelog entry (a user-readable short description of the changes that goes into CHANGELOG.md):
...
Description
The
Upgrade checkasserts the server log contains no<Error>lines after the upgrade restart. It started failing with five lines from a single test, originating from two remote database engines:Root cause. Test
04210_show_remote_databases_in_system_tablescreates two remote database engines pointing at the deliberately unreachable host192.0.2.1(RFC5737 TEST-NET-1, a documentation address):CREATE DATABASE ... ENGINE = PostgreSQL('192.0.2.1:5432', ...)ATTACH DATABASE ... ENGINE = MySQL('192.0.2.1:3306', ...) SETTINGS connect_timeout = 1, connection_max_tries = 1For PostgreSQL,
DatabasePostgreSQL::loadStoredObjectsactivates theremoveOutdatedTablescleaner task, whose periodicpool->get()connects unconditionally and times out against the unreachable host, logging<Error>for each retry, including right after the upgrade restart reloads the persisted engine.For MySQL, the engine probes the server while loading the persisted object after the upgrade restart, logging
<Error>frommysqlxx::Pool, thePoolWithFailoverretry logger, and theDatabaseMySQLALL_CONNECTION_TRIES_FAILEDexception.The upgrade-test environment runs the stateless suite and has no real PostgreSQL or MySQL server, so any connection error against this fixture host is noise rather than a compatibility regression. Genuine PostgreSQL/MySQL regressions are covered by integration tests with a live server.
Fix. Allowlist these lines in
tests/docker_scripts/upgrade_runner.sh. Each matcher requires the component AND the connection-failure symptom together, so otherDatabasePostgreSQL/DatabaseMySQLerrors (logic errors, parsing, a different error code) still surface. This follows the established pattern for the same class of test-fixture-against-unreachable-resource noise already in the allowlist (for example the librdkafka andStorageKafka2broker-transport entries).The errors themselves are correct engine behavior and are intentionally not changed.
Validation. Running the exact
upgrade_runner.shfilter pipeline against the real upgrade-checkclickhouse-server.upgrade.logfrom the failing run (CI report: https://s3.amazonaws.com/clickhouse-test-reports/json.html?PR=107927&sha=fd3cb89f6afe8c19648b2cab9a32bb74ed8dc9c3&name_0=PR&name_1=Upgrade%20check%20%28amd_release%29 ):<Error>lines (3 MySQL + 2 PostgreSQL) leak through -> check FAILs.DatabaseMySQL/mysqlxx::Poollogic, parse, and different-code errors, plus a non-mysqlApplication: Connection to ...line, all still surface.Version info
26.7.1.407