Re:Re: Could you please give me a hand about json object in flink sql

Hi, Thanks so much for your support! But sorry to say I'm still confused about it. No matter what the udf looks like, the first thing I need confirm is the type of 'content' in TableSink, what's the type of it should be, should I use type Row, like this? CREATE TABLE TableSink (

Re: Unbalanced distribution of keyed stream to downstream parallel operators

Hi Isidoros, Two assumptions in my mind: 1. Records are not evenly distributed across different keys, e.g. some accountId just has more events than others. If the record distribution is predicable, you can try to combine other fields or include more information into the key field to help bala

Re: [External] Re: Potential Bug with Date Serialization for Table Stream

Hi Martijn, Thank you for following up on this. We ended up changing two parts: When creating the DataType we instead used new AtomicDataType(new DateType(false), java.sql.Date.class); So we could override the conversion class for the constructor

Re: Flink SQL and data shuffling (keyBy)

Hi! I don't think there is a way to achieve that without resorting to DataStream API. I don't know if using the PARTITIONED BY clause in the create statement of the table can help to "balance" the data, see https://nightlies.apache.org/flink/flink-docs-master/docs/dev/table/sql/create/#partitioned

Re: DBT-flink profile?

Hi Georg, I'm not aware of anyone currently working on a DBT/Flink integration. It would be great if someone would want to investigate or experiment with those who have knowledge and time on DBT and Flink. Best regards, Martijn Visser https://twitter.com/MartijnVisser82 https://github.com/Martij

Re: [External] Re: Potential Bug with Date Serialization for Table Stream

Hi Tom, Sorry for the late reply, I missed this. In the upcoming Flink 1.15 a number of improvements on CAST will be included [1] Would you be able to test this with the current RC0 of Flink 1.15? [2] Best regards, Martijn Visser https://twitter.com/MartijnVisser82 https://github.com/MartijnViss

Re: Question about community collaboration options

Hi Ted, This is a great question. There are usually bi-weekly sync meetings to discuss plans and progress for the next Flink release. For example, there was a regular one for the Flink 1.15 release [1] I do see some things that we could improve on as a Flink community. For example, there are quit

Unbalanced distribution of keyed stream to downstream parallel operators

Hello, we ran a flink application version 1.13.2 that consists of a kafka source with one partition so far then we filter the data based on some conditions, mapped to POJOS and we transform to a KeyedStream based on an accountId long property from the POJO. The downstream operators are 10 CEP oper

Re: Could you please give me a hand about json object in flink sql

Hi, I’m afraid you have to use a UDTF to parse the content and construct the final json string manually. The key problem is that the field “content” is actually a STRING, although it looks like a json object. Currently the json format provided by Flink could not handle this kind of field defin

Re: [DataStream API] Watermarks not closing TumblingEventTimeWindow with Reduce function

hi~ Can you send your full code ？ Ryan van Huuksloot 于2022年3月31日周四 22:58写道： > Hello! > > *Problem:* > I am connecting to a Kafka Source with the Watermark Strategy below. > > val watermarkStrategy = WatermarkStrategy > .forBoundedOutOfOrderness(Duration.of(2, ChronoUnit.HOURS)) > .withTimest