Re: Converting from table to stream, following Avro schema

2022-05-13 Thread Dhavan Vaidya
Hi Dian, This is great news, really! Hope to see this soon in stable :D Thanks a lot! On Fri, 13 May 2022 at 15:25, Dian Fu wrote: > Hi Dhavan, > > This should be a known issue and has already been fixed in > https://issues.apache.org/jira/browse/FLINK-27282. However, this ticket > was fixed

Re: Converting from table to stream, following Avro schema

2022-05-13 Thread Dian Fu
Hi Dhavan, This should be a known issue and has already been fixed in https://issues.apache.org/jira/browse/FLINK-27282. However, this ticket was fixed recently and so it's still not released. Regards, Dian On Thu, May 12, 2022 at 7:26 PM Dhavan Vaidya wrote: > Hey Dian, > > Though my HTTP ca

Re: Converting from table to stream, following Avro schema

2022-05-12 Thread Dhavan Vaidya
Hey Dian, Though my HTTP call's response is indeed JSON, I needed to serialize data into Avro for Kafka sink. Since you said it supports Row inside Row, I investigated deeper and found out that since Row class *sorts by key names*, I had to strictly follow the order in output_type parameter in `p

Re: Converting from table to stream, following Avro schema

2022-05-10 Thread Dian Fu
Hi Dhavan, The type of the `ds` appearing in `t_env.from_data_stream(ds) should be known. Otherwise, it's impossible to infer the schema of the converted table, as a result, `raw` type will be used which makes the schema of the resulting table not expected. You could either declare the type of th

Converting from table to stream, following Avro schema

2022-05-10 Thread Dhavan Vaidya
Hello, I am consuming Kafka messages with Table API connector. I cannot use DataStream API because it does not support Confluent Avro. After consuming the messages, I am converting to DataStream API and using ProcessFunction. The ProcessFunction makes async http calls and emits results with a com