Re: Not able to run with two task manager in native flink in kubernetes with high availability mode

2021-04-11 Thread Yang Wang
Hi Priyanka Manickam, If you are using the native Kubernetes integration, the TaskManagers will be started/stopped dynamically on demands. It also means you could not control the number of running TaskManager pods. Refer to here[1] for more information. In your case, numOfTaskSlots has been confi

Re: Flink 1.13 and CSV (batch) writing

2021-04-11 Thread Kurt Young
The Flink community has a plan to delete the DataSet API in the future, the requirements will be fulfilled by both Table & DataStream API. It would be helpful to let us know what kind of functionality is missing in these two APIs. If you have further information you want to share, please let us kno

how to convert DataStream to Table

2021-04-11 Thread vtygoss
Hi All, there is a scenario where I need to process OGG Log data in kafka using Flink Sql. I can convert the OGG Log Stream to DataStream and each event has RowKind, but i have trouble converting DataStream to a Table. For test, i tried StreamTableEnvironment#fromDataStream and createTemporary

Re: Flink 1.13 and CSV (batch) writing

2021-04-11 Thread Flavio Pompermaier
Thanks for the suggestions Kurt. Actually I could use Table Api I think, it's just that most of our Flink code use DataSet Api. Il dom 11 apr 2021, 13:44 Kurt Young ha scritto: > Thanks for the suggestions Flavio. Join without window & left outer join > already worked in Table API & SQL. > And f

Re: Flink 1.13 and CSV (batch) writing

2021-04-11 Thread Kurt Young
Thanks for the suggestions Flavio. Join without window & left outer join already worked in Table API & SQL. And for reduceGroup, you can try either user defined aggregate function or use table aggregate which is available in Table API now. I'm wondering whether these can meet your requirement, or y