hi folks, As discussed on the recent GitHub PR [1], as a means of reconciling the long-standing cross-implementation incompatibilities with Union types, it's been proposed to remove the top-level validity bitmap from the Union data layout and let validity be determined exclusively by the child arrays of the union. So the only additional data needed to form a union are the type ids (and for the dense union, the offsets).
I do not think this change meaningfully alters the semantics of Union types and I think it also simplifies their construction, so I would be in favor of making it for 1.0.0. I can create a PR with the relevant alterations but wanted to raise the issue now so if there is consensus about doing this, that we can act quickly to implement it. Thanks, Wes [1]: https://github.com/apache/arrow/pull/7290
