Hi Theo, The most logical reason is that nested attributes were added later than watermarks were :) I agree that it's something that would be worthwhile to improve. If you can and want to make a contribution on this, that would be great.
Best regards, Martijn On Wed, Dec 14, 2022 at 9:24 AM Theodor Wübker <theo.wueb...@inside-m2m.de> wrote: > Actually, this behaviour is documented > <https://nightlies.apache.org/flink/flink-docs-master/docs/dev/table/sql/create/#create-table> > (See > the Watermarks section, where it is stated that the column must be a > “top-level” column). So I suppose, there is a reason. Nevertheless it is > quite a limiting factor, since it makes me unable to use Window queries > with the desired timestamp as watermark … I suppose one workaround could be > to transform the table so the attribute is at top level and then use it as > a watermark. But in my case this would be quite the effort. > > My question remains, what is the reason for this behaviour? Also, are > there any good workarounds for this? > > Thanks, > > -Theo > > On 14. Dec 2022, at 08:13, Theodor Wübker <theo.wueb...@inside-m2m.de> > wrote: > > Hey everyone, > > I have encountered a problem with my Table API Program. I am trying to use > a nested attribute as a watermark. The structure of my schema is a row, > which itself has 3 rows as attributes and they again have some attributes, > especially the Timestamp that I want to use as a watermark. Flink does not > let me reference it using the dot operator, sadly. I checked the sourcecode > and tracked it down to this (part of a) method in the DefaultSchemaResolver: > > private Column validateTimeColumn(String columnName, List<Column> columns) { > final Optional<Column> timeColumn = > columns.stream().filter(c -> > c.getName().equals(columnName)).findFirst(); > if (!timeColumn.isPresent()) { > throw new ValidationException( > String.format( > "Invalid column name '%s' for rowtime attribute in > watermark declaration. Available columns are: %s", > columnName, > > columns.stream().map(Column::getName).collect(Collectors.toList()))); > } ... > > The list of available columns is just the 3 rows and none of the nested > attributes. Is there a reason for nested columns being unavailable for > watermark declaration? Or am I overseeing something / doing something wrong? > > -Theo > > >
