Hi,
I have source json data like:
{"ts": "2020-11-09T20:26:10.368123Z", "user_id": "user1", "action": "click"}
...
my sql is:
create table t (
user_id string,
action string,
ts timestamp,
watermark for ts as ts - interval '5' second
) with (
'connector' = 'kafka',
'topic' = 'test',
'json.timestamp-format.standard' = 'ISO-8601'
...
)
this does not work since ISO-8601 does not expect `Z` at the end of the
timestamp.
It only works for "2020-11-09T20:26:10.368". However, I'm not able to
change the format.
I checked a few threads and somebody suggested to use udf for unsupported
timestamp format. what would the create table statement look like? I also
need watermark working.
I'm thinking about this:
create table t (
user_id string,
action string,
ts string,
transform_ts_format(ts) as new_ts,
watermark for new_ts as new_ts - interval '5' second
) with (
...
q:
1. how does watermark know new_ts is a valid timestamp?
2. is it possible to reuse ts without introducing a new column?
Thanks,
Fanbin
