Re: Flink - Iteration and Backpressure

Hi Robert/Team, Is there any recommended solution or any other insight on how I must be doing it? Thanks and Regards, Mahesh On Thu, Jun 1, 2017 at 10:32 AM, MAHESH KUMAR wrote: > Hi Robert, > > The Message Auditor System must monitor all the 4 kafka queue and gather > infor

Flink 1.3 - Checkpointing failing

Hi Team, We have some test cases written using StreamingMultipleProgramsTestBase It was working fine in version 1.2, we get the following error in version 1.3 It seems like CheckpointCoordinator fails after this error and Checkpointing no longer occurs. I came across this bug: https://issues.apac

Re: Flink - Iteration and Backpressure

> > > > On Fri, May 26, 2017 at 7:21 PM, MAHESH KUMAR < > r.mahesh.kumar@gmail.com> wrote: > >> Hi Team, >> >> I am trying to build an audit like system where I read messages from "n" >> Kafka queues, key by a unique key and then redu

Flink - Iteration and Backpressure

Hi Team, I am trying to build an audit like system where I read messages from "n" Kafka queues, key by a unique key and then reduce them to a single message, if it has passed through all the "n" Kafka queues in a window time of "m" hours/days, the message has succeeded else it has expired. I can

Flink - Writing Test Case for the Datastream

After def closeServices() = { // Stop Zookeeper and Kafka } } Thanks and Regards, Mahesh -- Mahesh Kumar Ravindranathan Data Streaming Engineer Oracle Marketing Cloud - Social Platform Contact No:+1(720)492-4445

Re: Issue Faced: Not able to get the consumer offsets from Kafka when using Flink with Flink-Kafka Connector

st offset > for the group from the __offsets topic. > > On Tue, Feb 7, 2017 at 6:24 PM, MAHESH KUMAR > wrote: > >> Hi Team, >> >> Kindly let me know if I am doing something wrong. >> >> Kafka Version - kafka_2.11-0.10.1.1 >> Flink Version - flink

Issue Faced: Not able to get the consumer offsets from Kafka when using Flink with Flink-Kafka Connector

or - Group test committed offset 17438 for partition testIn1-6 2017-02-07 09:53:38,738 DEBUG org.apache.kafka.clients.consumer.internals.ConsumerCoordinator - Group test committed offset 0 for partition testIn1-2 2017-02-07 09:53:43,524 DEBUG org.apache.flink.runtime.taskmanager.TaskManager - Sending heartbeat to JobManager 2017-02-07 09:53:43,730 INFO org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Triggering checkpoint 13 @ 1486486423730 -- Mahesh Kumar Ravindranathan Data Streaming Engineer Oracle Marketing Cloud - Social Platform Contact No:+1(720)492-4445