Thanks a lot Hans.. By using KafkaConsumer API
https://gist.github.com/senthilec566/16e8e28b32834666fea132afc3a4e2f9 i can
query the data using timestamp ..
It worked !
Now another question to achieve Parallelism on reading data ..
Example : topic : test
partitions : 4
Kafk
The timeindex was added in 0.10 so I think you need to use the new Consumer API
to access this functionality. Specifically you should call offsetsForTimes()
https://kafka.apache.org/0102/javadoc/org/apache/kafka/clients/consumer/Consumer.html#offsetsForTimes(java.util.Map)
-hans
> On May 25, 20
I did an experiment on searching messages using timestamps ..
Step 1: Used Producer with Create Time ( CT )
Step 2 : Verify whether it reflects in Kafka or not
.index .log
.timeindex
These three files in disk
Thanks a lot Mayuresh. I will look into SearchMessageByTimestamp feature in
Kafka ..
Cheers,
Senthil
On Thu, May 25, 2017 at 1:12 PM, Mayuresh Gharat wrote:
> Hi Senthil,
>
> Kafka does allow search message by timestamp after KIP-33 :
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> 3
Hi Senthil,
Kafka does allow search message by timestamp after KIP-33 :
https://cwiki.apache.org/confluence/display/KAFKA/KIP-33+-+Add+a+time+based+log+index#KIP-33-Addatimebasedlogindex-Searchmessagebytimestamp
The new consumer does provide you a way to get offsets by timestamp. You
can use thes
Hello Dev Team, Pls let me know if any option to read data from Kafka (all
partition ) using timestamp . Also can we set custom offset value to
messages ?
Cheers,
Senthil
On Wed, May 24, 2017 at 7:33 PM, SenthilKumar K
wrote:
> Hi All , We have been using Kafka for our Use Case which helps in
Hi All , We have been using Kafka for our Use Case which helps in
delivering real time raw logs.. I have a requirement to fetch data from
Kafka by using offset ..
DataSet Example :
{"access_date":"2017-05-24
13:57:45.044","format":"json","start":"1490296463.031"}
{"access_date":"2017-05-24
13:57:
