Actually, this behaviour is documented <https://nightlies.apache.org/flink/flink-docs-master/docs/dev/table/sql/create/#create-table> (See the Watermarks section, where it is stated that the column must be a “top-level” column). So I suppose, there is a reason. Nevertheless it is quite a limiting factor, since it makes me unable to use Window queries with the desired timestamp as watermark … I suppose one workaround could be to transform the table so the attribute is at top level and then use it as a watermark. But in my case this would be quite the effort.
My question remains, what is the reason for this behaviour? Also, are there any
good workarounds for this?
Thanks,
-Theo
> On 14. Dec 2022, at 08:13, Theodor Wübker <theo.wueb...@inside-m2m.de> wrote:
>
> Hey everyone,
>
> I have encountered a problem with my Table API Program. I am trying to use a
> nested attribute as a watermark. The structure of my schema is a row, which
> itself has 3 rows as attributes and they again have some attributes,
> especially the Timestamp that I want to use as a watermark. Flink does not
> let me reference it using the dot operator, sadly. I checked the sourcecode
> and tracked it down to this (part of a) method in the DefaultSchemaResolver:
>
> private Column validateTimeColumn(String columnName, List<Column> columns) {
> final Optional<Column> timeColumn =
> columns.stream().filter(c ->
> c.getName().equals(columnName)).findFirst();
> if (!timeColumn.isPresent()) {
> throw new ValidationException(
> String.format(
> "Invalid column name '%s' for rowtime attribute in
> watermark declaration. Available columns are: %s",
> columnName,
>
> columns.stream().map(Column::getName).collect(Collectors.toList())));
> } ...
> The list of available columns is just the 3 rows and none of the nested
> attributes. Is there a reason for nested columns being unavailable for
> watermark declaration? Or am I overseeing something / doing something wrong?
>
> -Theo
smime.p7s
Description: S/MIME cryptographic signature
