Re: Kafka Source Recovery Behavior

2021-11-18 Thread Steven Wu
Qingsheng, For the scenario described by Mason in the original email, I think it is safe to remove split/topic upson recovery without worrying about data loss, since it is a conscious choice by the user to switch to a different set of topics. I thought the problem is that KafkaSourceReader just r

Re: [Statefun] Questions on recovery

2021-11-18 Thread Hady Januar Willi
Hi Igal, Thank you for your response, understood the strategies. Best, Hady On Wed, Nov 3, 2021 at 9:06 PM Igal Shilman wrote: > Hello Hady, > Glad to see that you are testing StateFun! > > Regarding that exception, I think that this is not the root cause. The > root cause is as you wrote that

Re: SqlSnapshot throws NullPointerException when used in conjunction with CTE

2021-11-18 Thread Caizhi Weng
Hi! Thanks for raising this issue. This is a known issue [1] that is caused by WITH clause + lookup joins. A workaround is to use CREATE TEMPORARY VIEW ... AS ... instead of WITH to create a view. I've closed your issue as it is a duplicate. Let's continue our discussion in [1]. [1] https://issu

Re: Connector for TCP/IP messages

2021-11-18 Thread Caizhi Weng
Hi! We have a SocketSourceFunction in the flink-examples-table module. Note that this is only an example and cannot be used in production. See its java doc for more information. If you want a production-ready socket source you can modify that and implement your own. Robert Cullen 于2021年11月19日周五

SqlSnapshot throws NullPointerException when used in conjunction with CTE

2021-11-18 Thread Yuval Itzchakov
Hi, Flink 1.14.0 Scala 2.12 Java 11 Given the following program: package com.flink.bug import org.apache.flink.api.scala.createTypeInformation import org.apache.flink.streaming.api.scala.StreamExecutionEnvironment import org.apache.flink.table.api.DataTypes import org.apache.flink.table.api.Sche

Connector for TCP/IP messages

2021-11-18 Thread Robert Cullen
Does flink have a connector for TCP/IP messages? -- Robert Cullen 240-475-4490

Re: Could not retrieve submitted JobGraph from state handle

2021-11-18 Thread Alexey Trenikhun
Thank you Yang and Alexander. It happened while testing restore from k8s cluster backup, likely job was upgraded after “backup”, but before simulation of disaster. Thanks, Alexey From: Yang Wang Sent: Tuesday, November 16, 2021 8:00:13 PM To: Alexander Preuß Cc:

Table to DataStream conversion and field ordering

2021-11-18 Thread Oliver Moser
Hi! I'm running into a problem when converting back and forth from a streaming table to a data stream. Given the following table DDL: create table masterdata ( facility text, manufacturer text, serial integer, latitude double precision, longitude double precis

Re: [DISCUSS] Shall casting functions return null or throw exceptions for invalid input

2021-11-18 Thread Kurt Young
Sorry I forgot to add user ML. I also would like to gather some users feedback on this thing. Since I didn't get any feedback on this topic before from users. Best, Kurt On Thu, Nov 18, 2021 at 6:33 PM Kurt Young wrote: > (added user ML to this thread) > > HI all, > > I would like to raise a d