Activity

1. tuyndoan commented on Aug 21, 2026

   

   tuyndoan

   on Aug 21, 2026

   Contributor

   More actions

   Opened a fix in #21220 — use bigquery.NullInt64 for OrdinalPosition so NULL from BigQuery no longer fails schema sync.
2. d-bytebase commented on Aug 21, 2026

   

   d-bytebase

   on Aug 21, 2026

   Collaborator

   More actions

   Thanks for the detailed report — the error is reproducible and I've traced it to how we decode ordinal_position from INFORMATION_SCHEMA.COLUMNS. BigQuery returns NULL there for pseudo columns, and we assume a non-null integer.

   Before we settle on a fix, could you run these two queries against the dataset that fails? They only read metadata — no table data — and the first is aggregated so it won't reveal your table names.

   1. Which columns have a NULL ordinal position, and what kind of object are they on:

   SELECT
     t.table_type,
     c.column_name,
     c.data_type,
     c.is_hidden,
     c.is_system_defined,
     c.is_partitioning_column,
     COUNT(*) AS column_count
   FROM `PROJECT_ID`.DATASET_ID.INFORMATION_SCHEMA.COLUMNS AS c
   LEFT JOIN `PROJECT_ID`.DATASET_ID.INFORMATION_SCHEMA.TABLES AS t
     ON  c.table_catalog = t.table_catalog
     AND c.table_schema  = t.table_schema
     AND c.table_name    = t.table_name
   WHERE c.ordinal_position IS NULL
   GROUP BY 1, 2, 3, 4, 5, 6
   ORDER BY column_count DESC;

   2. What object types the dataset contains:

   SELECT table_type, COUNT(*) AS table_count
   FROM `PROJECT_ID`.DATASET_ID.INFORMATION_SCHEMA.TABLES
   GROUP BY table_type
   ORDER BY table_count DESC;

   Replace PROJECT_ID and DATASET_ID with the dataset that fails to sync. The first query's output is what we need most — it tells us whether this is coming from ingestion-time partitioning (_PARTITIONTIME / _PARTITIONDATE) or from external tables (_FILE_NAME), which points at different fixes.
3. mangareader commented on Aug 22, 2026

   

   mangareader

   on Aug 22, 2026

   Last edited by mangareader

   Author

   More actions

   could you run these two queries against the dataset that fails? They only read metadata — no table data — and the first is aggregated so it won't reveal your table names.

   table_type | column_name | data_type | is_hidden | is_system_defined | is_partitioning_column | column_count
   BASE TABLE | _PARTITIONTIME | TIMESTAMP | YES | YES | YES | 2
   

   
   BASE TABLE | 70
   VIEW | 6
   EXTERNAL | 1
   
4. tuyndoan commented on Aug 24, 2026

   

   tuyndoan

   on Aug 24, 2026

   Contributor

   More actions

   Thanks for running those queries — that confirms this is ingestion-time partitioning, not external-table _FILE_NAME.

   _PARTITIONTIME is a hidden/system-defined pseudo column (is_hidden = YES, is_system_defined = YES, is_partitioning_column = YES) with ordinal_position IS NULL. That's the case #21220 already handles: schema sync now excludes hidden columns (WHERE is_hidden = 'NO') and skips any remaining NULL ordinals, so these two pseudo columns no longer abort the whole database sync.