Re: A strange exception while consumption using a multi topic Kafka Connector

2018-05-14 Thread Vishal Santoshi
Yep, Thanks. We have a set up where topics are constantly being added on a kakfa-to-hdfs pipeline. On Sun, May 13, 2018 at 11:58 AM, Ted Yu wrote: > FLINK-9349 was logged. > > FYI > > On Sat, May 12, 2018 at 7:52 AM, Ted Yu wrote: > >> I took a look at ./flink-connectors/flink-co >> nnector-kaf

Re: A strange exception while consumption using a multi topic Kafka Connector

2018-05-13 Thread Ted Yu
FLINK-9349 was logged. FYI On Sat, May 12, 2018 at 7:52 AM, Ted Yu wrote: > I took a look at ./flink-connectors/flink-connector-kafka-0.9/src/main/ > java/org/apache/flink/streaming/connectors/kafka/ > internal/Kafka09Fetcher.java > > It seems the List subscribedPartitionStates was being modifi

Re: A strange exception while consumption using a multi topic Kafka Connector

2018-05-12 Thread Ted Yu
I took a look at ./flink-connectors/flink-connector-kafka-0.9/src/main/java/org/apache/flink/streaming/connectors/kafka/internal/Kafka09Fetcher.java It seems the List subscribedPartitionStates was being modified when runFetchLoop iterated the List. This can happen if, e.g., FlinkKafkaConsumer run

Re: A strange exception while consumption using a multi topic Kafka Connector

2018-05-12 Thread Vishal Santoshi
?? On Fri, May 11, 2018, 8:15 PM Vishal Santoshi wrote: > java.util.ConcurrentModificationException > at > java.util.LinkedList$ListItr.checkForComodification(LinkedList.java:966) > at java.util.LinkedList$ListItr.next(LinkedList.java:888) > at > org.apache.flink.streaming.co

A strange exception while consumption using a multi topic Kafka Connector

2018-05-11 Thread Vishal Santoshi
java.util.ConcurrentModificationException at java.util.LinkedList$ListItr.checkForComodification(LinkedList.java:966) at java.util.LinkedList$ListItr.next(LinkedList.java:888) at org.apache.flink.streaming.connectors.kafka.internal.Kafka09Fetcher.runFetchLoop(Kafka09Fetche