Could we please publish Chocolatey packages for ZooKeeper and Kafka, to
make it easier for newbies to get started?
https://chocolatey.org/
--
Cheers,
Andrew
We did update the default log dirs. So shouldn't be an issue.
On Thu, Jul 28, 2016 at 2:56 PM Samuel Taylor
wrote:
> I think the default puts the log directory in /tmp/kafka-logs. It may be
> worth configuring your settings to put your logs somewhere else and seeing
> what happens.
>
> On Thu, J
Hi guys,
Should I use SSD for my brokers or not?
What are the pros and cons?
Thank you!
Hi ,
I have a situation .
i have 3 broker(id) ==> 58,57,56,
some how broker 57 went invalid
hduser@node01:~$ kafka-topics --zookeeper node01:2181,node02:2181,rac1:2181
--describe
Topic:clouder PartitionCount:1 ReplicationFactor:2 Configs:
Topic: clouder Partition: 0 Leader: 56 Replicas: 56,5
The replication factor is 4.
> On Jul 28, 2016, at 5:55 PM, David Garcia wrote:
>
> What is your replication for these topics?
>
> On 7/28/16, 3:03 PM, "Kessiler Rodrigues" wrote:
>
>Hey guys,
>
>I have > 5k topics with 5 partitions each in my cluster today.
>
>My actual clu
What is your replication for these topics?
On 7/28/16, 3:03 PM, "Kessiler Rodrigues" wrote:
Hey guys,
I have > 5k topics with 5 partitions each in my cluster today.
My actual cluster configuration is:
6 brokers - 16 vCPUs, 14.4 GB
Nowadays, I’m having so
Hey guys,
I have > 5k topics with 5 partitions each in my cluster today.
My actual cluster configuration is:
6 brokers - 16 vCPUs, 14.4 GB
Nowadays, I’m having some lags when the consumer tries to do rebalance and
sometimes even kafka doesn’t get up.
What would be a best cluster configurati
I think the default puts the log directory in /tmp/kafka-logs. It may be
worth configuring your settings to put your logs somewhere else and seeing
what happens.
On Thu, Jul 28, 2016 at 10:16 AM, David Yu wrote:
> Those are all with default value. Will continue investigation.
>
> On Wed, Jul 27,
Also, reading java docs for KafkaConsumer#poll(timeout) states:
@param timeout The time, in milliseconds, spent waiting in poll if data is not
available. If 0, returns
*immediately with any records that are available now. Must not
be negative.
Yet even setting it to 0 brings no
So I have KafkaConsumer that is deliberately set with server properties
pointing to non-running broker.
Doing KafkaConsumer.poll(100) blocks infinitely even though ‘fetch.max.wait.ms’
is set to 1 millisecond.
Basically I am trying to fail when connection is not possible.
Any idea how to accompli
Hi all,
I have an application where I need to join events from two different
topics. Every event is identified by an id, which is used as the key for
the topic partition. After doing some experiment, I observed that events
will go into different partitions even if the number of partitions for both
Our team is evaluating KStreams and Reactive Kafka (version 0.11-M4) on a
confluent 3.0 cluster. Our testing is very simple (pulling from one topic,
doing a simple transform) and then writing out to another topic.
The performance for the two systems is night and day. Both applications were
ru
Hello,
I'm receiving an error while publishing messages to a kafka topic, the steps I
took were:1. Starting zookeeper server2. Starting kafka server3. Sending
messages to kafka topic with a Kafka Producer
8105 [kafka-producer-network-thread | producer-2] DEBUG
org.apache.kafka.clients.NetworkCli
Hi,
We are having a issue with our kafka cluster when we try to reassign
partitions. Imagine the following topic layout file:
Kafka Version: 0.8.2
{
"partitions": [
{
"partition": 0,
"replicas": [
1,
2,
3
many delete topic functionality related issues got fixed in latest
versions. It highly recommend to move to latest version.
https://issues.apache.org/jira/browse/KAFKA-1757 fixes similar issue on
windows platform.
On Thu, Jul 28, 2016 at 3:40 PM, Ghosh, Prabal Kumar <
prabal.kumar.gh...@sap.com> w
Hi Kafka Users,
We are using kafka 0.8.2.1.We are not able to delete to any topic .
We are using AdminUtils to create and delete topic.
The topics get created successfully with correct Leader and isr for each topic
partitions.
But when we try to delete topic using AdminUtils.deleteTopic(), it fa
Hi, Kafka,
We have a 2 broker staging cluster and a 6 broker prod cluster (all running
0.8.2.1). I'm trying to make sense of the producer performances on both.
Here are some perf test results:
-
Prod:
kafka-producer-perf-test.sh --broker-list
kafkaProd-1:9092,kafkaProd-2:9092,k
Hello guys,
I have a java consumer with these properties:
Properties props = new Properties();
props.put("zookeeper.connect", this.settings.getZookeeperUrl() );
props.put("group.id", this.settings.getGroupId());
props.put("client.id", infoCompania.getCompania());
props.put("zookeeper.
Those are all with default value. Will continue investigation.
On Wed, Jul 27, 2016 at 11:38 PM Manikumar Reddy
wrote:
> also check if any value set for log.retention.bytes broker config
>
> On Wed, Jul 27, 2016 at 8:03 PM, Samuel Taylor
> wrote:
>
> > Is it possible that your log directory is
Hi,
Including IP address in the payload of the messages is possible.
But if my producer is in a private network, how will the producer know the
Gateway IP, from where message is being sent?
Thanks
Gourab
On Thu, Jul 28, 2016 at 3:29 PM, Sivakumaran S wrote:
> +1 Vote. Embody all metadata abou
We're also seeing lots of failures as the TopicPartitionWriter tries
to close WAL files in HDFS [1].
[1] http://pastebin.com/6ipUndZv
On Wed, Jul 27, 2016 at 5:01 PM, Kristoffer Sjögren wrote:
> The workers seems happier when reducing number of partitions for each
> worker. And when adding more
+1 Vote. Embody all metadata about the source at the application layer itself,
if possible within the 1500 bytes limit of TCP packet.
Regards,
Sivakumaran
> On 28-Jul-2016, at 10:48 AM, Michael Noll wrote:
>
> One option is to implement the producer applications in such a way that
> each produ
+1 Vote. Embody all metadata about the source at the application layer itself,
if possible within the 1500 bytes limit of TCP packet.
Regards,
Sivakumaran
> On 28-Jul-2016, at 10:48 AM, Michael Noll wrote:
>
> One option is to implement the producer applications in such a way that
> each produ
One option is to implement the producer applications in such a way that
each producer includes its own IP address in the payload of the messages it
produces.
-Michael
On Tue, Jul 26, 2016 at 10:15 AM, Gourab Chowdhury
wrote:
> Hi,
>
> I am working on a project that includes Apache Kafka. My co
Ok except that I messed up my subscription on dev and didn't receive
anything, sorry, I will check in the archive to see the answer.
Thanks,
On Thu, Jul 28, 2016 at 9:23 AM, Manikumar Reddy
wrote:
> You already got reply from Guozhang on dev mailing list.
>
> On Thu, Jul 28, 2016 at 7:09 AM, Pi
You already got reply from Guozhang on dev mailing list.
On Thu, Jul 28, 2016 at 7:09 AM, Pierre Coquentin <
pierre.coquen...@gmail.com> wrote:
> Hi,
>
> I've a simple technical question about kafka streams.
> In class org.apache.kafka.streams.processor.internals.StreamTask, the
> method "process
Hi,
I've a simple technical question about kafka streams.
In class org.apache.kafka.streams.processor.internals.StreamTask, the
method "process" uses a synchronized block but I don't see why, the method
doesn't seem to be called in a multi-threaded environnement as it's created
and only accessed b
27 matches
Mail list logo
