rdblue commented on code in PR #520: URL: https://github.com/apache/parquet-format/pull/520#discussion_r2305531958
########## VariantShredding.md: ########## @@ -42,7 +42,31 @@ Variant values are stored in Parquet fields named `value`. Each `value` field may have an associated shredded field named `typed_value` that stores the value when it matches a specific type. When `typed_value` is present, readers **must** reconstruct shredded values according to this specification. -For example, a Variant field, `measurement` may be shredded as long values by adding `typed_value` with type `int64`: +The Parquet columns used to store variant metadata and values must be accessed by name, not by position. + +In order to avoid ambiguity, this specification always uses the term "`null`" to mean the variant +null value (binary encoding: `00`). The phrase "missing" or "NULL" (all caps) always refers to an +`optional` value that is not present (= SQL NULL). Review Comment: I don't think it's valuable to add a SQL null concept. It's probably helpful to call out an encoded Variant null, but null is already a thing in Parquet so it would be best to use the Parquet concept instead of adding another null that comes with a lot of baggage (3-value comparison). It's also fine to try to visually disambiguate between them, but this introduces a lot of changes just from moving from null to NULL, so I would prefer doing that in some other way. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
