Hi Brian, Thanks for your quick response. I totally agree that the we should not rely on any assumption on the field order and we can always specify the order of the flattened fields as we want. There is no blocker issue for me with the current behavior, but I am just wondering if may be convenient in some use cases if we can just keep the order (roughly) consistent with the order of the parent fields from the original schema.
From: Brian Hulette <bhule...@google.com> Reply-To: "user@beam.apache.org" <user@beam.apache.org> Date: Wednesday, January 20, 2021 at 9:42 AM To: user <user@beam.apache.org> Subject: Re: Regarding the field ordering after Select.Flattened transform This does seem like an odd choice, I suspect this was just a matter of convenience of implementation since the javadoc makes no claims about field order. In general schema transforms don't take care to maintain a particular field order and I'd recommend against relying on it. Instead fields should be addressed by name, either with Row.getValue(Sring), or by mapping to a user type. Is there a reason you want to rely on a particular field order? Maybe when writing to certain IOs field order could be important. On Tue, Jan 19, 2021 at 1:36 PM Tao Li <t...@zillow.com<mailto:t...@zillow.com>> wrote: Hi community, I have been experimenting with Select.Flattened transform and noticed that the field order in the flattened schema is not consistent with the order of the top level fields from the original schema. For example, in the original schema, we have field “foo” as the first field and it has a nested field “bar”. After applying the flattening transform, the new field “foo.bar” becomes the last field in the flattened schema. Seems like the order of the new fields are not that deterministic in the flattened schema. Is this an expected behavior? Don’t we guarantee any ordering of the flattened fields (e.g. being consistent with the original order)? Thanks!
