We faced similar problem and ended up with implementing variant of golden section search, that reads message using simple consumer and checks the timestamp (timestamps are appended by our producer though, they do not come from any Kafka metadata) till it finds message closest to given date.
Adam 2015-06-30 21:52 GMT+02:00 Shushant Arora <shushantaror...@gmail.com>: > Is it possible using low level consumer to get kafka messages based on > timestamp, say I want to get all messages from last 5 minutes. I don't know > what were offsets of partitions 5 minutes back. > > In low level consumer : when I gave epoch for whichTime , it failed. > requestInfo.put(topicAndPartition, new > PartitionOffsetRequestInfo(whichTime, > 1)); > > Is only latest and earliest supported in timestamp,Is there any way to > filter messages based on timestamp? > > Thanks > Shushant >
