Thanks Robert appreciate your help.
On Fri, May 6, 2016 at 3:07 PM, Robert Metzger <rmetz...@apache.org> wrote:
> Hi,
>
> yes, you can use Kafka's configuration setting for that. Its called
> "auto.offset.reset". Setting it to "latest" will change the restart
> behavior to the current offset ("earliest" is the opposite).
>
> How heavy is the processing you are doing? 4500 events/second sounds not
> like a lot of throughput.
>
> On Wed, May 4, 2016 at 8:24 AM, Balaji Rajagopalan <
> balaji.rajagopa...@olacabs.com> wrote:
>
>> I am using the flink connector to read from a kafka stream, I ran into
>> the problem where the flink job went down due to some application error, it
>> was down for sometime, meanwhile the kafka queue was growing as expected no
>> consumer to consume from the given group , and when I started the flink it
>> started consuming the messages no problem so far, but consumer lag was huge
>> since producer is a fast producer about 4500 events/sec. My question is
>> there any flink connector configuration which can force it read from the
>> latest offset when the flink application starts since in my application
>> logic I do not care about older events.
>>
>> balaji
>>
>
>
