Re: Better way to clean up state when connect

2018-05-12 Thread Xingcan Cui
Hi Chengzhi, you said the Stream B which comes from a file will be updated unpredictably. I wonder if you could share more about how to judge an item (from Stream A I suppose) is not in the file and what watermark generation strategy did you choose? Best, Xingcan > On May 12, 2018, at 12:48 A

Re: A strange exception while consumption using a multi topic Kafka Connector

2018-05-12 Thread Ted Yu
I took a look at ./flink-connectors/flink-connector-kafka-0.9/src/main/java/org/apache/flink/streaming/connectors/kafka/internal/Kafka09Fetcher.java It seems the List subscribedPartitionStates was being modified when runFetchLoop iterated the List. This can happen if, e.g., FlinkKafkaConsumer run

Re: A strange exception while consumption using a multi topic Kafka Connector

2018-05-12 Thread Vishal Santoshi
?? On Fri, May 11, 2018, 8:15 PM Vishal Santoshi wrote: > java.util.ConcurrentModificationException > at > java.util.LinkedList$ListItr.checkForComodification(LinkedList.java:966) > at java.util.LinkedList$ListItr.next(LinkedList.java:888) > at > org.apache.flink.streaming.co

Re:Re: Join two data streams on a given key and diffrent common window size.

2018-05-12 Thread lehaxing
thanks a lot ,I'm sorry to recover so late,I will pay attention to this. At 2018-04-23 19:03:11, "Fabian Hueske" wrote: Hi, The semantics of the joins offered by the DataStream API in Flink 1.4 and before as well as the upcoming 1.5 version are a bit messed up, IMO. Since Flink 1.4, Flink