Hi All,
Once gone through code found that, While Producer starts it does three
things:
1. Sends Meta-data request
2. Send message to broker(fetching broker list)
3. If number of message to be produce is grater than 0 then again tries to
refresh metadata for outstanding produce requests.
Each of
Hi list,
I thought I'd also mention that the next release of Apache Cloudstack
adds the ability to publish all events happening on throughout the
environment to kafka. Events are published as JSON.
http://cloudstack-administration.readthedocs.org/en/latest/events.html#kafka-configuration
Cheers,
Hello,
I'm using Kafka 0.8.2.1 (in a Scala/Java project) and trying to find out
how to atomically write n messages (message batch) to a single topic
partition. Is there any client API that gives such a guarantee? I
couldn't find a clear answer reading the documentation, API docs (of the
old a
Hi list!
I just wanted to mention a small article I put together to describe an
approach to leverage log compaction when you have compound types and
messages are operations on that compound type with an example use-case:
http://spootnik.org/entries/2015/04/23_hands-on-kafka-dynamic-dns.html
Alwa
Thanks Gwen.
I'm specifically looking for the consumer rewrite API (
org.apache.kafka.clients.consumer.KafkaConsumer). Based on the wiki, this
feature is available only in 0.9.
The specific use case is that, I wanted to use the high level consumer but
with the ability to rollback the offset in ca
We don't normally plan dates for releases, when we are done with
features we want in the next release and happy with quality, we
release. Many Apache communities are like that.
If you need firmer roadmaps and specific release dates, there are few
vendors selling Kafka distributions and support.
A
Since it is a runtime error, Maven dependency is less relevant than
what you have in your class path (unless you built a shaded uber-jar).
You'll need Scala runtime and zkclient jar in the classpath, can you
check that you have those around?
On Thu, Apr 23, 2015 at 6:15 AM, abdul hameed pathan
w
Perhaps this will help:
https://cwiki.apache.org/confluence/display/KAFKA/FAQ#FAQ-Whycan'tmyconsumers/producersconnecttothebrokers?
On Thu, Apr 23, 2015 at 3:24 PM, madhavan kumar
wrote:
> Dear all,
> I am trying to connect my python consumer to a remote kafka server. But
> in kafka/conn.py#r
Hey Sriharsha,
Great, thanks!
For 4:
Yeah the use case for init and close is making use of any kind of metadata.
An example of this would be if you are trying to do range partitioning you
need to map lexicographic ranges to numeric partitions. You might do this
by adding a new property to the co
Dear all,
I am trying to connect my python consumer to a remote kafka server. But
in kafka/conn.py#reinit, the socket call socket.create_connection throws
"connection refused" error.
to confirm, it is not a firewall issue, i tried connecting to other servers
(in the same machine as kafka server
Hi Jay,
Sorry about the KIP formatting . I fixed those in the KIP.
2. We certainly need to add both the serialized and unserialized form for
the key as both are useful.
I added those to the interface.
3. Do we need to add the value? I suspect people will have uses for
computing someth
Hi,
we are getting NoClassDefFoundError at our producer end. We are using only
syn producer. we have 3 data producer systems whose using same
kafka.javaapi.producer.Producer instance.Bellow is the maven dependency
org.apache.kafka
Hi,
Sorry this isn't directly a kafka question, but I was wondering if there are
andy Kafka user groups in (or in near driving range of) Los Angeles. Looking
through meetup.com and the usual web search engines hasn't brought me much
outside of the LA Hadoop user group and I was hoping for somet
Hi,
I'm looking for the 0.9 client release plan.
https://cwiki.apache.org/confluence/display/KAFKA/Kafka+0.9+Consumer+Rewrite+Design
Is there a planned date for this release?
Thanks,
Bharath
For Kafka 0.8.x[.x], refer to
https://cwiki.apache.org/confluence/display/KAFKA/Replication+tools#Replicationtools-6.ReassignPartitionsTool
.
2015-04-23 23:20 GMT+03:00 Corey Nolet :
> I have a cluster of 3 nodes and I've created a topic with some number of
> partitions and some number of replica
I have a cluster of 3 nodes and I've created a topic with some number of
partitions and some number of replicas, let's say 10 and 2, respectively.
Later, after I've got my 3 nodes fairly consumed with data in the 10
partitions, I want to add 2 more nodes to the mix to help balance out the
partition
This is still occurring for us. In addition, it has started occurring on
one of the six nodes in the "healthy" cluster, for no reason we have been
able to determine.
We're willing to put in some serious time to help debug/solve this, but we
need *some* hint as to where to start. I understand that
Thank you, Raja!
2015-04-23 0:34 GMT+02:00 Rajasekar Elango :
> Yes, you pass any consumer property including group.id by having them in
> property file and passing path to it using --consumer.config of consumer
> consumer.
>
> Thanks,
> Raja.
>
> On Wed, Apr 22, 2015 at 1:45 AM, Lukáš Havrlant
Hey Harsha,
A few comments:
Can you finish up the KIP there are some unfinished sentences and odd
whitespace things going on.
Here are the questions I think we should consider:
1. Do we need this at all given that we have the partition argument in
ProducerRecord which gives full control? I think
Hi Kafka users,
I'd like to announce a new open source project, called "Bottled Water", for
getting data from PostgreSQL into Kafka:
http://blog.confluent.io/2015/04/23/bottled-water-real-time-integration-of-postgresql-and-kafka/
https://github.com/confluentinc/bottledwater-pg/
Bottled Water com
Hi,
Here is the KIP for adding a partitioner interface for producer.
https://cwiki.apache.org/confluence/display/KAFKA/KIP-+22+-+Expose+a+Partitioner+interface+in+the+new+producer
There is one open question about how interface should look like. Please take a
look and let me know if you pre
21 matches
Mail list logo
