Repository navigation
Allow empty STRUCT types - #23273
Allow empty STRUCT types#23273
Conversation
|
Nice! To motivation for this PR is to eventually add (named) parameter-packs for functions (python style *args/**kwargs), so we can fold all varargs into two structs, one unnamed struct for extra positional arguments and one named struct containing all the extra arguments passed by name. But if there are none of either it's a lot cleaner if we can still pass a struct in the expected argument slot, just have it be empty instead. |
This comment was marked as spam.
This comment was marked as spam.
Sorry, something went wrong.
This comment was marked as spam.
This comment was marked as spam.
Sorry, something went wrong.
This comment was marked as spam.
This comment was marked as spam.
Sorry, something went wrong.
This comment was marked as spam.
This comment was marked as spam.
Sorry, something went wrong.
This comment was marked as spam.
This comment was marked as spam.
Sorry, something went wrong.
This comment was marked as spam.
This comment was marked as spam.
Sorry, something went wrong.
This comment was marked as spam.
This comment was marked as spam.
Sorry, something went wrong.
This comment was marked as spam.
This comment was marked as spam.
Sorry, something went wrong.
This comment was marked as spam.
This comment was marked as spam.
Sorry, something went wrong.
This comment was marked as spam.
This comment was marked as spam.
Mytherin
left a comment
There was a problem hiding this comment.
Thanks! I think allowing this is a good idea. Can we add this new type to test_all_types so that we can verify all clients deal with this correctly?
|
Is there any chance this PR also finds it important to be able to make partially unnamed structs ? |
|
I've actually been thinking about spinning out "unnamed structs" entirely from structs, naming them I think having partially unnamed structs seems kind of odd. If we want to allow the use case that you describe I think we should just allow empty identifiers (which I suppose is how this happens, e.g. parsing a JSON like |
|
Sorry, didn't see you responded to Thijs. Will add this to |
|
Yes agreed, it's separate from this PR |
|
All green @Mytherin |
|
Thanks! |
This PR adds a new `LogicalType::TUPLE` which replaces the previous
"unnamed" special case of struct-types with a dedicated type.
### Syntax Changes
- `{}` now returns a valid empty `STRUCT` (follow up to #23273)
- `()` now returns a valid empty `TUPLE`
- `(x,)` (with a trailing comma) is parsed as a single-element tuple
(python style)
- `(a, b, c,)` - trailing commas are ok for tuple literals in general.
This also round-trips in string conversions, e.g. `(1,)::VARCHAR` is
printed with a trailing comma.
### JSON/Parquet/Arrow/`VARIANT` conversion
This PR also adds support for json/parquet/arrow conversion of tuples.
- For JSON and `VARIANT`, we follow the python convention and turn
tuples into arrays.
- For parquet and arrow, we turn the tuples into structs with dummy
names, `element1, element2, ... elementN`, following the same convention
as when `UNNEST`ing a tuple.
### Storage Compatibility
- We preserves backwards (and forwards) compatibility by transparently
serializing tuples as empty structs when targeting old database
versions, and deserializing empty structs as tuples when reading them.
- This also means that we can now store and persist `TUPLE`s in columns,
which was not supported for unnamed structs.
- Additionally, tuples and structs are trivially implicitly castable to
each other, so any (future) function signature changes should not affect
user code.
### C-API Compatibility
Similarly to how we deal with tuples in storage, we do not add a new
`DUCKDB_LOGICAL_TYPE_TUPLE`, but transparently represent tuples as
unnamed structs, and vice-versa. So any existing C-API code should just
work and not have to make any changes.
### Code Changes
I've made most of the internal struct type helper (e.g.
`StructType::GetChildren`) also work on tuples so that most of the
execution engine and extension code will remain oblivious to this
change. However, I've had to keep the `StructType::IsUnnamed` as we
somehow create (partially?) empty/unnamed structs in various places when
dealing with `VARIANT`s. This should be investigated and resolved in a
follow-up PR - ideally we should enforce the invariant that structs
cannot be unnamed all over the system and make the need for this
function redundant. Or maybe we decide that an empty string is a valid
struct member, given that structs can't have duplicate members anyway.
Either way we need to work it out.
There are also a bunch of `struct_` functions that could be simplified
and/or overloaded for TUPLEs. I've tried to keep the changes minimal in
this PR, but we should probably make another pass over these in the
future as well.
Covers duckdb-web #6921 (duckdb/duckdb#23273). Notes that STRUCT types may be empty, with a New in 2.0 callout, and links it from the What's New in the Docs for 2.0 page.

This PR enables STRUCT types to be empty, e.g.
struct_pack()now returns{}.Syntax-wise, this means that
NULL:STRUCT, andCREATE TABLE t1(s STRUCT)is now valid -STRUCTrepresents an empty struct, butSTRUCT()is still a syntax-error, as type parameters lists can't be empty, but maybe thats something we want to support in the future.The existing struct-functions should all work with empty structs, except
UNNEST(andstruct_extract) which will throw appropriate errors.Attempting to persist an empty struct in an older database format will throw an error as well.