Fix VARIANT cast reading wrong rows under a filter by mlafeldt · Pull Request #23031 · duckdb/duckdb · GitHub
Skip to content

Fix VARIANT cast reading wrong rows under a filter - #23031

Merged
Mytherin merged 2 commits into
duckdb:v1.5-variegatafrom
mlafeldt:fix/variant-to-variant-selection
Jun 4, 2026
Merged

Mytherin merged 2 commits into
duckdb:v1.5-variegatafrom
mlafeldt:fix/variant-to-variant-selection

Conversation

@mlafeldt

@mlafeldt mlafeldt commented Jun 3, 2026 •

Copy link
Copy Markdown
Member

ConvertVariantToVariant indexed the source by the raw loop counter, ignoring the incoming selection vector, so nested VARIANT values were read from the wrong rows under a filter. Map the index through GetMappedIndex like the other converters.

Before:

❯ duckdb
-- Loading resources from /Users/mathias/.duckdbrc
DuckDB v1.5.3 (Variegata)
Enter ".help" for usage hints.
🟡◗ CREATE TABLE t AS SELECT i, MAP{'k': i::VARIANT} AS m FROM range(2) t(i);
🟡◗ SELECT m::VARIANT FROM t WHERE i = 1;
┌──────────────────────────┐
│   CAST(m AS "VARIANT")   │
│         variant          │
├──────────────────────────┤
│ [{'key': k, 'value': 0}] │
└──────────────────────────┘
🟡◗ SELECT m::VARCHAR FROM t WHERE i = 1;
┌────────────────────┐
│ CAST(m AS VARCHAR) │
│      varchar       │
├────────────────────┤
│ {k=1}              │
└────────────────────┘

('value': 0 is wrong)

After:

❯ ./build/release/duckdb
-- Loading resources from /Users/mathias/.duckdbrc
DuckDB v1.5.4-dev84 (Development Version, 4e6c9e90a7)
Enter ".help" for usage hints.
🟡◗ CREATE TABLE t AS SELECT i, MAP{'k': i::VARIANT} AS m FROM range(2) t(i);
🟡◗ SELECT m::VARIANT FROM t WHERE i = 1;
┌──────────────────────────┐
│   CAST(m AS "VARIANT")   │
│         variant          │
├──────────────────────────┤
│ [{'key': k, 'value': 1}] │
└──────────────────────────┘
🟡◗ SELECT m::VARCHAR FROM t WHERE i = 1;
┌────────────────────┐
│ CAST(m AS VARCHAR) │
│      varchar       │
├────────────────────┤
│ {k=1}              │
└────────────────────┘

Reported on Discord

@mlafeldt mlafeldt added the Member label Jun 3, 2026
@mlafeldt
mlafeldt force-pushed the fix/variant-to-variant-selection branch from 4e6c9e9 to 4d71e9a Compare June 3, 2026 12:13
@mlafeldt
mlafeldt marked this pull request as draft June 3, 2026 12:13
@mlafeldt
mlafeldt marked this pull request as ready for review June 3, 2026 12:13
@mlafeldt
mlafeldt marked this pull request as draft June 3, 2026 19:58
@mlafeldt
mlafeldt marked this pull request as ready for review June 3, 2026 19:58
@mlafeldt
mlafeldt marked this pull request as draft June 3, 2026 20:23
@mlafeldt
mlafeldt marked this pull request as ready for review June 3, 2026 20:23
mlafeldt added 2 commits June 4, 2026 09:22
ConvertVariantToVariant indexed the source by the raw loop counter, ignoring the
incoming selection vector, so nested VARIANT values were read from the wrong rows
under a filter. Map the index through GetMappedIndex like the other converters.
@mlafeldt
mlafeldt force-pushed the fix/variant-to-variant-selection branch from 8b232f2 to 216fe98 Compare June 4, 2026 07:23
@mlafeldt

mlafeldt commented Jun 4, 2026

Copy link
Copy Markdown
Member Author

@mlafeldt
mlafeldt marked this pull request as draft June 4, 2026 07:24
@mlafeldt
mlafeldt marked this pull request as ready for review June 4, 2026 07:24
@Mytherin
Mytherin merged commit be167b0 into duckdb:v1.5-variegata Jun 4, 2026
111 checks passed
@Mytherin

Mytherin commented Jun 4, 2026

Copy link
Copy Markdown
Member

Thanks!

@mlafeldt
mlafeldt deleted the fix/variant-to-variant-selection branch June 4, 2026 12:26
@adhamsalama

Copy link
Copy Markdown

krlmlr pushed a commit to krlmlr/duckdb that referenced this pull request Jun 21, 2026
…ilter; ADBC 1.1.0 rich errors)

Conflict-free bundle of the clean commits between resolutions: VARIANT cast reading wrong rows under a filter, and the ADBC 1.1.0 Rich Error Metadata API. No conflicts with main.
krlmlr pushed a commit to krlmlr/duckdb that referenced this pull request Jun 21, 2026
`ConvertVariantToVariant` indexed the source by the raw loop counter,
ignoring the incoming selection vector, so nested `VARIANT` values were
read from the wrong rows under a filter. Map the index through
`GetMappedIndex` like the other converters.

Before:

```
❯ duckdb
-- Loading resources from /Users/mathias/.duckdbrc
DuckDB v1.5.3 (Variegata)
Enter ".help" for usage hints.
🟡◗ CREATE TABLE t AS SELECT i, MAP{'k': i::VARIANT} AS m FROM range(2) t(i);
🟡◗ SELECT m::VARIANT FROM t WHERE i = 1;
┌──────────────────────────┐
│   CAST(m AS "VARIANT")   │
│         variant          │
├──────────────────────────┤
│ [{'key': k, 'value': 0}] │
└──────────────────────────┘
🟡◗ SELECT m::VARCHAR FROM t WHERE i = 1;
┌────────────────────┐
│ CAST(m AS VARCHAR) │
│      varchar       │
├────────────────────┤
│ {k=1}              │
└────────────────────┘
```

(`'value': 0` is wrong)

After:

```
❯ ./build/release/duckdb
-- Loading resources from /Users/mathias/.duckdbrc
DuckDB v1.5.4-dev84 (Development Version, 4e6c9e9)
Enter ".help" for usage hints.
🟡◗ CREATE TABLE t AS SELECT i, MAP{'k': i::VARIANT} AS m FROM range(2) t(i);
🟡◗ SELECT m::VARIANT FROM t WHERE i = 1;
┌──────────────────────────┐
│   CAST(m AS "VARIANT")   │
│         variant          │
├──────────────────────────┤
│ [{'key': k, 'value': 1}] │
└──────────────────────────┘
🟡◗ SELECT m::VARCHAR FROM t WHERE i = 1;
┌────────────────────┐
│ CAST(m AS VARCHAR) │
│      varchar       │
├────────────────────┤
│ {k=1}              │
└────────────────────┘
```

Reported on
[Discord](https://discord.com/channels/909674491309850675/1125676736311480340/1511698549354664020)
krlmlr pushed a commit to krlmlr/duckdb that referenced this pull request Jun 21, 2026
…ilter; ADBC 1.1.0 rich errors)

Conflict-free bundle of the clean commits between resolutions: VARIANT cast reading wrong rows under a filter, and the ADBC 1.1.0 Rich Error Metadata API. No conflicts with main.
krlmlr pushed a commit to krlmlr/duckdb that referenced this pull request Jun 21, 2026
Brings these v1.5-variegata commits; they do not overlap with the main/v2.0
line, so they merge with no conflicts:

- Fix VARIANT cast reading wrong rows under a filter (duckdb#23031)
- fix(adbc): implement ADBC 1.1.0 Rich Error Metadata API (duckdb#23073)
krlmlr pushed a commit to krlmlr/duckdb that referenced this pull request Jun 21, 2026
Brings these v1.5-variegata commits; they do not overlap with the main/v2.0
line, so they merge with no conflicts:

- Fix VARIANT cast reading wrong rows under a filter (duckdb#23031)
- fix(adbc): implement ADBC 1.1.0 Rich Error Metadata API (duckdb#23073)
kaka11chen pushed a commit to kaka11chen/astrovela_vane that referenced this pull request Aug 14, 2026
`ConvertVariantToVariant` indexed the source by the raw loop counter,
ignoring the incoming selection vector, so nested `VARIANT` values were
read from the wrong rows under a filter. Map the index through
`GetMappedIndex` like the other converters.

Before:

```
❯ duckdb
-- Loading resources from /Users/mathias/.duckdbrc
DuckDB v1.5.3 (Variegata)
Enter ".help" for usage hints.
🟡◗ CREATE TABLE t AS SELECT i, MAP{'k': i::VARIANT} AS m FROM range(2) t(i);
🟡◗ SELECT m::VARIANT FROM t WHERE i = 1;
┌──────────────────────────┐
│   CAST(m AS "VARIANT")   │
│         variant          │
├──────────────────────────┤
│ [{'key': k, 'value': 0}] │
└──────────────────────────┘
🟡◗ SELECT m::VARCHAR FROM t WHERE i = 1;
┌────────────────────┐
│ CAST(m AS VARCHAR) │
│      varchar       │
├────────────────────┤
│ {k=1}              │
└────────────────────┘
```

(`'value': 0` is wrong)

After:

```
❯ ./build/release/duckdb
-- Loading resources from /Users/mathias/.duckdbrc
DuckDB v1.5.4-dev84 (Development Version, 4e6c9e90a7)
Enter ".help" for usage hints.
🟡◗ CREATE TABLE t AS SELECT i, MAP{'k': i::VARIANT} AS m FROM range(2) t(i);
🟡◗ SELECT m::VARIANT FROM t WHERE i = 1;
┌──────────────────────────┐
│   CAST(m AS "VARIANT")   │
│         variant          │
├──────────────────────────┤
│ [{'key': k, 'value': 1}] │
└──────────────────────────┘
🟡◗ SELECT m::VARCHAR FROM t WHERE i = 1;
┌────────────────────┐
│ CAST(m AS VARCHAR) │
│      varchar       │
├────────────────────┤
│ {k=1}              │
└────────────────────┘
```

Reported on
[Discord](https://discord.com/channels/909674491309850675/1125676736311480340/1511698549354664020)

Upstream-Commit: duckdb/duckdb@be167b0
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants