This does seem like an odd choice, I suspect this was just a matter of convenience of implementation since the javadoc makes no claims about field order.
In general schema transforms don't take care to maintain a particular field order and I'd recommend against relying on it. Instead fields should be addressed by name, either with Row.getValue(Sring), or by mapping to a user type. Is there a reason you want to rely on a particular field order? Maybe when writing to certain IOs field order could be important. On Tue, Jan 19, 2021 at 1:36 PM Tao Li <t...@zillow.com> wrote: > Hi community, > > > > I have been experimenting with Select.Flattened transform and noticed that > the field order in the flattened schema is not consistent with the order of > the top level fields from the original schema. For example, in the original > schema, we have field “foo” as the first field and it has a nested field > “bar”. After applying the flattening transform, the new field “foo.bar” > becomes the last field in the flattened schema. Seems like the order of the > new fields are not that deterministic in the flattened schema. Is this an > expected behavior? Don’t we guarantee any ordering of the flattened fields > (e.g. being consistent with the original order)? Thanks! >
