How could I use Kafka for offset management with the 0.8.2 version of Kafka`
On Tue, Feb 9, 2016 at 10:54 PM, Yifan Ying wrote:
> Please check out consumer configs.
> http://kafka.apache.org/082/documentation.html#consumerconfigs
>
> On Tue, Feb 9, 2016 at 1:16 PM, Joe San wrote:
>
> > Can I do
Hi John,
The core jar can be built with either Scala 2.10 or Scala 2.11 and the
Scala version is included as part of the artifact. The read me page
explains this in more detail. Search for "all scala versions" to find out
how to run commands for all Scala versions or "particular version of Scala"
Folks
I am trying to work with the latest from the 0.9.0 branch (0.9.0.1-SNAPSHOT)
The build appears to have changed the Maven artifact which it produces, at
least for the "core" part of the project
When I built about a week ago, the Artifact was
org.apache.kafka:kafka_2.11:0.9.0.1-SNAPSHOT
Bu
Hi Matvey,
I have a question about the config value offsets.retention.minutes in kafka
> 0.9.0.0.
> Is this the timeout for when offsets get compacted in the topic or actually
> deleted (as it appears from a cursory reading of
>
> https://github.com/apache/kafka/blob/0.9.0/core/src/main/scala/kaf
One thing to add, the old consumer(0.8) does provide per topic metrics and
they are under the group, 'ConsumerTopicMetrics'.
On Tue, Feb 9, 2016 at 3:20 PM, Yifan Ying wrote:
> Hi all,
>
> We implemented a MetricsReporter to collect metrics from the new producer
> and the new consumer. For the p
Hi all,
We implemented a MetricsReporter to collect metrics from the new producer
and the new consumer. For the producer, we did see a bunch of topic
specific metrics under "producer-topic-metrics" group. But the consumer
seems not providing any per topic metrics. All I saw are consumer-metrics,
c
Thanks Ewen.
Sent from my iPhone
> On Feb 9, 2016, at 12:59 PM, Ewen Cheslack-Postava wrote:
>
> Yeah, you definitely want one that's cheaper than /topics. The root
> resource / is effectively a nop you can check for liveness of the service
> (but not for, e.g., more general health like connec
-- Forwarded message --
From: "Dana Powers"
Date: Feb 9, 2016 7:59 PM
Subject: Re: [kafka-clients] java clients problem
To: "iman teimoornegad"
Cc:
You should email users@kafka.apache.org . this mailing list is used for
development of third party client libraries.
-Dana
On Feb 9
Auto commit is enabled by default (auto.commit.enable) and Zookeeper is the
default storage for compatibility reasons (offsets.storage), but using
Kafka instead is recommended.
See http://kafka.apache.org/090/documentation.html#oldconsumerconfigs for
details on configuring the old consumer.
-Ewen
Please check out consumer configs.
http://kafka.apache.org/082/documentation.html#consumerconfigs
On Tue, Feb 9, 2016 at 1:16 PM, Joe San wrote:
> Can I do automatic offset commit using the highlevel consumer? If so, where
> is the offset being comitted?
>
> On Tue, Feb 9, 2016 at 10:13 PM, Ewen
Can I do automatic offset commit using the highlevel consumer? If so, where
is the offset being comitted?
On Tue, Feb 9, 2016 at 10:13 PM, Ewen Cheslack-Postava
wrote:
> If you are working against 0.8.2 brokers and cannot update, you'll have to
> use the old consumer API for now. There are two p
If you are working against 0.8.2 brokers and cannot update, you'll have to
use the old consumer API for now. There are two parts to it, but you
probably want the high level consumer, documented here:
http://kafka.apache.org/documentation.html#highlevelconsumerapi
-Ewen
On Tue, Feb 9, 2016 at 12:3
Could anyone point me to an older version of the consumer client that I
could use to run against the 0.8.2 version of Kafka?
On Tue, Feb 9, 2016 at 6:57 PM, Ewen Cheslack-Postava
wrote:
> The new consumer wasn't implemented until 0.9.0.0. The API was sketched
> out, but no implementation had bee
I ran into kind of a similar discrepancy, but about UnderReplicatedPartitions.
kafka-topics.sh and zookeeper were saying that we had underreplicated
partitions.
But JMX said that there were none. I took one of the partitions that ZK was
saying was under-replicated and I ran DumpLogSegments on t
Hello -
I am seeing that the node in our Kafka cluster currently elected as
controller is reporting 1 offline partition via JMX
(kafka.controller:type=KafkaController,name=OfflinePartitionsCount).
However, when I use kaka-topics to find the offline partition
(bin/kafka-topics.sh
--zookeeper localh
Yeah, you definitely want one that's cheaper than /topics. The root
resource / is effectively a nop you can check for liveness of the service
(but not for, e.g., more general health like connectivity to ZK or Kafka).
The most it will ever do is return a list of available subresources, so it
should
The new consumer wasn't implemented until 0.9.0.0. The API was sketched
out, but no implementation had been included yet.
-Ewen
On Tue, Feb 9, 2016 at 8:00 AM, Joe San wrote:
> Is this intentioal in the Kafka 0.8.2.0 version,
> org.apache.kafka.clients.consumer.KafkaConsumer, the method:
>
> pu
I am looking to monitor my kakfa rest proxy servers and was wondering if there
is an endpoint that can be checked to see if the server is running. I am
currently checking if endpoint /topics returns a 200. Is that a reasonable
endpoint to run the test against. I would prefer to have one that is
I have a question about the config value offsets.retention.minutes in kafka
0.9.0.0.
Is this the timeout for when offsets get compacted in the topic or actually
deleted (as it appears from a cursory reading of
https://github.com/apache/kafka/blob/0.9.0/core/src/main/scala/kafka/coordinator/GroupMet
Is this intentioal in the Kafka 0.8.2.0 version,
org.apache.kafka.clients.consumer.KafkaConsumer, the method:
public Map> poll(long timeout) {
return null;
}
Which version of 0.8.x.x should I use so that I could do a
consumer.poll(2000)?
Could anyone point me to some sample code where I could get some insights
on how to write a consumer to do the following:
1. poll at regular intervals
2. get a set of messages during each poll operation
3. process the messages, comit the offset
Thanks,
Joe
21 matches
Mail list logo
