Availability for the ElasticSearch 2 streaming connector

2016-02-16 Thread Vieru, Mihail
Hi, in reference to this ticket https://issues.apache.org/jira/browse/FLINK-3115 when do you think that an ElasticSearch 2 streaming connector will become available? Will it make it for the 1.0 release? That would be great, as we are planning to use that particular version of ElasticSearch in the

Re: NPE with Flink Streaming from Kafka

2015-12-02 Thread Vieru, Mihail
ur use case >> but if the state size grows indefinitely you might want to give it a try. >> >> I will write a detailed guide in some days but if you want to get started >> check this one out: >> >> https://docs.google.com/document/d/1xx3J88ZR_kuYYHAil3HD3YSBLqBTKV4Pu1SLwhiRiGs/edit?usp=sharing

Re: NPE with Flink Streaming from Kafka

2015-12-02 Thread Vieru, Mihail
ink.streaming.api.operators.StreamSource$NonTimestampContext.collect(StreamSource.java:92) > > at > org.apache.flink.streaming.connectors.kafka.internals.LegacyFetcher$SimpleConsumerThread.run(LegacyFetcher.java:450) > > > > > > > > > > 2015-12-02 10:45 G

Re: NPE with Flink Streaming from Kafka

2015-12-02 Thread Vieru, Mihail
On Wed, Dec 2, 2015 at 10:18 AM, Vieru, Mihail > wrote: > >> Thank you, Robert! The issue with Kafka is now solved with the >> 0.10-SNAPSHOT dependency. >> >> We have run into an OutOfMemory exception though, which appears to be >> related to the state. As my colleag

Re: NPE with Flink Streaming from Kafka

2015-12-02 Thread Vieru, Mihail
gt; >>> > >> >>> the issue is actually a bug in Kafka. We have a JIRA in Flink for > this > >> >>> as > >> >>> well: https://issues.apache.org/jira/browse/FLINK-3067 > >> >>> > >> >>> Sadly, we hav

NPE with Flink Streaming from Kafka

2015-12-01 Thread Vieru, Mihail
Hi, we get the following NullPointerException after ~50 minutes when running a streaming job with windowing and state that reads data from Kafka and writes the result to local FS. There are around 170 million messages to be processed, Flink 0.10.1 stops at ~8 million. Flink runs locally, started w