Cherry pick #104416 to 26.5: Hide MySQL and PostgreSQL databases from system.tables by default by robot-ch-test-poll · Pull Request #106037 · ClickHouse/ClickHouse · GitHub
Skip to content

Cherry pick #104416 to 26.5: Hide MySQL and PostgreSQL databases from system.tables by default#106037

Merged
robot-ch-test-poll merged 31 commits into
backport/26.5/104416from
cherrypick/26.5/104416
May 28, 2026
Merged

Cherry pick #104416 to 26.5: Hide MySQL and PostgreSQL databases from system.tables by default#106037
robot-ch-test-poll merged 31 commits into
backport/26.5/104416from
cherrypick/26.5/104416

Conversation

@robot-ch-test-poll

Copy link
Copy Markdown
Contributor

Original pull-request #104416

Do not merge this PR manually

This pull-request is a first step of an automated backporting.
It contains changes similar to calling git cherry-pick locally.
If you intend to continue backporting the changes, then resolve all conflicts if any.
Otherwise, if you do not want to backport them, then just close this pull-request.

The check results does not matter at this step - you can safely ignore them.

Troubleshooting

If the conflicts were resolved in a wrong way

If this cherry-pick PR is completely screwed by a wrong conflicts resolution, and you want to recreate it:

  • delete the pr-cherrypick label from the PR
  • delete this branch from the repository

You also need to check the Original pull-request for pr-backports-created label, and delete if it's presented there

The PR source

The PR is created in the CI job

pamarcos and others added 30 commits May 8, 2026 18:39
…bases_in_system_tables

Broaden the setting to also hide MySQL and PostgreSQL databases from
system.tables / system.columns / system.completions when disabled, since
they too require remote calls to enumerate tables. The old name is
preserved as an alias.

Internal renames: GetDatabasesOptions::with_datalake_catalogs ->
with_external_databases; DatabaseCatalog::isDatalakeCatalog ->
isExternalDatabase; hasDatalakeCatalogs -> hasExternalDatabases;
databases_without_datalake_catalogs -> databases_without_external.
Each assertion in the test now prints a short marker describing what
it checks, so a failure in the reference diff is self-explanatory.
The test only exercised the PostgreSQL engine path. MySQL takes the
same code path in DatabaseCatalog::isExternalDatabase, but a regression
that special-cases PostgreSQL would have slipped through. Add a MySQL
case using ATTACH (CREATE DATABASE ... ENGINE = MySQL connects eagerly,
ATTACH does not) with a short connect_timeout so the tolerated failure
is fast.
- DatabaseCatalog.h: fix grammar in the renamed comment ("are implement"
  -> "are implemented", "protect ourself" -> "protect ourselves").
- DatabaseCatalog.cpp:2244: update outdated TableNameHints comment that
  still said "Skip datalake catalogs" to reflect the broader scope.
- 04210: add an assertion that system.completions also does not list
  external databases by default.
The new stateless test runs in about 5 seconds and remains stable across repeated runs, so it should be eligible for fasttest to improve CI coverage.\n\nRef: #104416
Per review feedback from @nikitamikhaylov on PR #104416: replace the
free helper with engine-name string compare in DatabaseCatalog.cpp by
a virtual method on IDatabase. DataLake / MySQL / PostgreSQL override
to return true; the default is false, so other engines (Atomic,
Replicated, MaterializedPostgreSQL, ...) keep the right behavior
without needing to be aware of the new classification.

The naming is intentionally distinct from the existing isExternal()
("engine does not support ClickHouse internal tables") and
isDatalakeCatalog() (narrower). See the doc comment in IDatabase.h.
Fast test does not build libpq or libmysql, so CREATE DATABASE ENGINE =
PostgreSQL/MySQL silently does nothing in that environment, and the
test's reference no longer matches the actual output.

This is the same reason 01114_mysql_database_engine_segfault.sql and
03790_materialized_postgresql_nullptr_dereference.sql carry the tag.

Reverts removal in 527e58e. CI fast-test failure:
https://s3.amazonaws.com/clickhouse-test-reports/json.html?PR=104416&sha=527e58ed25888cbc3061c7c1de91e18777c227ae&name_0=PR
Co-authored-by: Kseniia Sumarokova <sumarokovakseniia@gmail.com>
Rename the database-level helper to avoid confusion with IDatabase::isExternal
and include MaterializedPostgreSQL in remote database filtering.
Pass  through the test client API for integration tests that intentionally inspect MySQL and PostgreSQL metadata in  and .

PR: #104416

CI: https://s3.amazonaws.com/clickhouse-test-reports/json.html?PR=104416&sha=158f2327e19aebbbfd2e588f15754e44210183a7&name_0=PR
Rename the new setting and `DatabaseCatalog` filtering option to use remote database terminology consistently, while keeping `show_data_lake_catalogs_in_system_tables` as the compatibility alias.

Convert the stateless coverage from `.sh` to `.sql` now that the SQL test runner can provide unique database names.

PR: #104416
Use remote database terminology consistently in comments changed by the PR and avoid the old external database wording near `isRemoteDatabase`.

PR: #104416
Explain `isExternal` in terms of database engines that do not own ClickHouse table metadata instead of using vague wording.

PR: #104416
The merged master documentation uses `installable`; add it to the aspell ignore dictionary so the style check passes.

PR: #104416
Use `currentDatabase` after switching to each remote database so the stateless test validates filtering with the same database predicate required by the style check.

PR: #104416
Do not classify `MaterializedPostgreSQL` as a remote database. Its table
enumeration uses local `DatabaseAtomic` metadata, unlike the `PostgreSQL`
database engine remote catalog enumeration.

PR: #104416
Apply the remote database setting override to `SHOW COLUMNS` and `SHOW INDEXES`.

This keeps explicit SHOW entrypoints working for `MySQL` and `PostgreSQL` databases.

PR: #104416
Replace the `HACK` wording with a direct explanation that explicit `SHOW TABLES` should include the requested remote database.

PR: #104416
# Conflicts:
#	ci/jobs/scripts/check_style/aspell-ignore/en/aspell-dict.txt
# Conflicts:
#	ci/jobs/scripts/check_style/aspell-ignore/en/aspell-dict.txt
The `02995_new_settings_history` check now compares against the `26.5.1` baseline, so this PR-only setting must be registered in a later history block.

CI report: https://s3.amazonaws.com/clickhouse-test-reports/json.html?PR=104416&sha=c06e440163ccc3ed3f05c2b97b575b08c586f0a4&name_0=PR&name_1=Fast%20test

PR: #104416
…es-setting

Hide MySQL and PostgreSQL databases from system.tables by default
@robot-ch-test-poll robot-ch-test-poll added pr-cherrypick Cherry-pick of merge-commit before backporting. Do not use manually - automated use only! do not test disable testing on pull request labels May 28, 2026
@robot-ch-test-poll robot-ch-test-poll merged commit 663d211 into backport/26.5/104416 May 28, 2026
@robot-ch-test-poll robot-ch-test-poll deleted the cherrypick/26.5/104416 branch May 28, 2026 11:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

do not test disable testing on pull request pr-cherrypick Cherry-pick of merge-commit before backporting. Do not use manually - automated use only!

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants