Calcite has improved to reconstruct ROW back in the output. See [1]. Beam need to update Calcite dependency to > 1.21 to adopt that.
[1]: https://jira.apache.org/jira/browse/CALCITE-3138 -Rui On Thu, Feb 13, 2020 at 9:05 PM Talat Uyarer <[email protected]> wrote: > Hi, > > I am trying to Beam SQL. But something is wrong. I have nested row > records. I read them as Pcollection<Row> and apply Select * query and > compare with initial rows. Looks like nested rows are flatten by calcite. > How do you have any idea how can I avoid this? > > I added a same testcase for my issue: > > Schema nestedSchema = > Schema.builder() > .addInt32Field("f_nestedInt") > .addStringField("f_nestedString") > .addInt32Field("f_nestedIntPlusOne") > .build(); > Schema inputType = > Schema.builder().addInt32Field("f_int").addRowField("f_row", > nestedSchema).build(); > > PCollection<Row> input = > pipeline.apply( > Create.of( > Row.withSchema(inputType) > .addValues( > 1, Row.withSchema(nestedSchema).addValues(312, "CC", > 313).build()) > .build()) > .withRowSchema(inputType)) > .setRowSchema(inputType); > > PCollection<Row> result = > input > .apply( > SqlTransform.query( > "SELECT * FROM PCOLLECTION")); > > PAssert.that(result) > .containsInAnyOrder(Row.withSchema(inputType) > .addValues( > 1, Row.withSchema(nestedSchema).addValues(312, "CC", 313).build()) > .build()); > > > Thank you so much in advance. > >
