Hi Gyula,
some disclaimer: the type system rework is still ongoing and there a a
couple of known issues and missing end-to-end tests around this topic.
I would therefore recommend to declare the sink as `STRING NULL` for now.
Can you open an issue for your concrete use case with some example
source/query/sink SQL and I'm happy to look into this.
Actually, `NULLIF()` should do the trick in the query but unfortunately
the current Calcite behavior is not what one would expect.
Thanks,
Timo
On 09.04.20 15:53, Gyula Fóra wrote:
Hi All!
We ran into a problem while trying to insert data read from kafka into a
table sink where some of the columns are not nullable.
The problem is that from Kafka we can only read nullable columns in JSON
format otherwise you get the following error:
org.apache.flink.table.api.ValidationException: Type STRING NOT NULL of
table field 'first' does not match with the physical type STRING of the
'first' field of the TableSource return type.
On the other hand no matter what we do with the nullable column (things
like select where ... is not null) the type will always STRING and will
be incompatible with the sink leading to the following error:
Query schema: [first: STRING, ...]
Sink schema: [first: STRING NOT NULL, ...]
Any idea on how to resolve this type mismatch between nullable and
non-nullable data? I feel that a query like (select x from y where x is
not null ) should change the type to not null.
Thanks
Gyula
