{{ message }}
Repository navigation
Tags: Layerbase-LLC/pgsqlite
Tags
fix(metadata): stop quoted identifiers from bricking databases on wake
A client that quotes its DDL (Laravel, Doctrine, Django and friends emit
CREATE TABLE "themes" ("created_at" datetime ...)) made pgsqlite copy the
raw SQL token straight into __pgsqlite_schema, so the metadata held
'"created_at"' with the quotes embedded in the TEXT value while SQLite - and
PRAGMA table_info - held 'created_at'. Runtime lookups query by the clean name
and silently missed, falling back to decltype inference, so nothing looked
wrong until the next wake: the startup drift validator compares the two sides
as raw strings, reported every column of every table as both missing from
SQLite and missing from metadata, and hard-exited. The database could never be
opened again.
Two further gaps compounded it. ALTER TABLE never wrote metadata at all, so a
column added after creation was permanent one-directional drift. DROP TABLE
never deleted a table's metadata rows and ALTER TABLE ... RENAME TO never moved
them, so the ORM table-rebuild pattern (CREATE "__temp__users"; INSERT SELECT;
DROP "users"; RENAME "__temp__users" TO "users") left rows for a table that no
longer exists plus stale pre-rebuild rows. The DROP TABLE regex also matched
bare words only, so quoted drops skipped enum-usage cleanup and cache
invalidation entirely.
Changes:
- utils/identifier.rs: one shared normalize_identifier (plus is_quoted /
quote helpers) handling the doubled-quote escape.
- create_table_translator: normalize the column token before it becomes a
type-mapping key, an enum column, or an array column, while the emitted SQL
keeps the original quoting. Also normalize SERIAL/PRIMARY KEY matching.
This is the root fix; every metadata write path derives its names from here.
- Normalize again at each __pgsqlite_schema write site (executor, extended,
three db_handler paths, TypeMetadata::store_type_mappings) and in both
extract_table_name_from_create implementations, so no future path can
reintroduce a quoted row.
- ddl/schema_maintenance.rs: new post-DDL hook wired into every db_handler
execute path. ALTER TABLE ADD COLUMN now records metadata (pg type from the
CREATE TABLE translator, SQLite type read back from PRAGMA table_info so it
can never disagree), DROP COLUMN and DROP TABLE delete it, RENAME TO and
RENAME COLUMN move it. Best-effort: bookkeeping never fails a statement the
user already saw succeed.
- schema_drift: normalize identifiers on both sides and match the legacy
quoted spelling, so poisoned metadata cannot brick a database even if a
write path were missed.
- migration/identifier_repair.rs: idempotent startup self-heal that unquotes
identifier values in the __pgsqlite_* tables, running just before the drift
check. Databases already poisoned by an older binary boot and repair
themselves with no manual surgery; on a primary-key clash the clean row
wins and the poisoned duplicate is dropped.
- tests/metadata_identifier_quoting_test.rs: 16 regression tests covering
each class. All 16 fail against v0.0.22-layerbase-6 and pass here.
fix(drift): stop false-positive on SERIAL/AUTOINCREMENT columns The startup schema-drift check compared the metadata sqlite_type stored in __pgsqlite_schema against the type reported by PRAGMA table_info. For a SERIAL/BIGSERIAL/IDENTITY column pgsqlite records the constraint-bearing string "INTEGER PRIMARY KEY AUTOINCREMENT", but PRAGMA table_info only returns the bare declared type token "INTEGER". normalize_sqlite_type did not strip the constraint suffix, so the two never compared equal and any existing database with a SERIAL column refused to open with a spurious "expected SQLite type 'INTEGER PRIMARY KEY AUTOINCREMENT' but found 'INTEGER'" drift error. Strip trailing column constraints (PRIMARY KEY, AUTOINCREMENT, NOT NULL, etc.) before comparing types. The drift check only compares column type, not constraints, so dropping the suffix is safe and still catches real type drift. Add a regression test that reproduces the SERIAL case.
Layerbase fork build v0.0.22-layerbase-4 sqlite_master internal-table filter: splice the filter onto the original query text instead of re-serializing the parsed AST, which corrupted complex predicates (large NOT IN lists, ESCAPE clauses) and broke client list-tables queries.
Also hide materialized pg_catalog/information_schema views from sqlit… …e_master Extends the client sqlite_master/sqlite_schema filter to exclude the 24 pg_catalog and information_schema compatibility views pgsqlite materializes as real SQLite views (source of truth: migration/registry.rs) in addition to the __pgsqlite_* tables, the four materialized pg_catalog tables, and their internal indexes. A raw type='view' scan or an unqualified SELECT * now shows only the user's own schema. The exclusion is an exact-name set, never a pg_% / information_schema_% wildcard, so a user-owned view or table such as pg_my_report is still returned. Filter applies to both name and tbl_name for symmetry, keeps the same alias/JOIN awareness, and fails open on any parse failure.
PreviousNext