Hi,
you can get the offsets (current and committed offsets) in Flink 1.1 using
the Flink metrics.
In Flink 1.0, we expose the Kafka internal metrics via the accumulator
system (so you can access them from the web interface as well). IIRC, Kafka
exposes a metric for the lag as well.
On Mon, Aug 8,
>From the code in Kafka09Fetcher.java
// if checkpointing is enabled, we are not automatically committing to
Kafka.
kafkaProperties.setProperty(ConsumerConfig.ENABLE_AUTO_COMMIT_CONFIG,
Boolean.toString(!runtimeContext.isCheckpointingEnabled()));
If flink checkpointing
Hi Gordon,
I am consuming the messages from kafka with *FlinkKafkaConsumer09 *and I
have also specified the group.id.
I have *enabled checkpointing*, and below configs
*auto.commit.enable=true
auto.offset.reset=earliest.*
>From your post I could understand that group.id is not much useful as far
Hi,
1. What Flink Kafka connector version are you using?
2. How is your non-Flink consumer fetching data from the topic? Is it using
the old SimpleConsumer, old High-Level Consumer, or the new consumer API?
3. If you are using the new consumer API, are you using
"consumer.assign(…)" or "consumer.s
