Hello Dev Team, Pls let me know if any option to read data from Kafka (all
partition ) using timestamp . Also can we set custom offset value to
messages ?
Cheers,
Senthil
On Wed, May 24, 2017 at 7:33 PM, SenthilKumar K <senthilec...@gmail.com>
wrote:
> Hi All , We have been using Kafka for our Use Case which helps in
> delivering real time raw logs.. I have a requirement to fetch data from
> Kafka by using offset ..
>
> DataSet Example :
> {"access_date":"2017-05-24 13:57:45.044","format":"json",
> "start":"1490296463.031"}
> {"access_date":"2017-05-24 13:57:46.044","format":"json",
> "start":"1490296463.031"}
> {"access_date":"2017-05-24 13:57:47.044","format":"json",
> "start":"1490296463.031"}
> {"access_date":"2017-05-24 13:58:02.042","format":"json",
> "start":"1490296463.031"}
>
> Above JSON data will be stored in Kafka..
>
> Key --> acces_date in epoch format
> Value --> whole JSON.
>
> Data Access Pattern:
> 1) Get me last 2 minz data ?
> 2) Get me records between 2017-05-24 13:57:42:00 to 2017-05-24
> 13:57:44:00 ?
>
> How to achieve this in Kafka ?
>
> I tried using SimpleConsumer , but it expects partition and not sure
> SimpleConsumer would match our requirement...
>
> Appreciate you help !
>
> Cheers,
> Senthil
>
