Apache Spark broadcast error: Error sending message as driverActor is null [message = UpdateBlockInfo(BlockManagerId

HiI'm using Apache Spark 1.1.0 and I'm currently having issue with broadcast method. So when I call broadcast function on a small dataset to a 5 nodes cluster, I experiencing the "Error sending message as driverActor is null" after broadcast the variables several times (apps running under jboss)

Re: Apache Spark broadcast error: Error sending message as driverActor is null [message = UpdateBlockInfo(BlockManagerId

Yep, sorry, had a rough day.. On Thursday, January 22, 2015 2:25 PM, Guozhang Wang wrote: Hi Zijing, Sounds like you sent to the wrong mailing list :P Guozhang On Thu, Jan 22, 2015 at 11:12 AM, Zijing Guo wrote: > HiI'm using Apache Spark 1.1.0 and I'm currently

kafka consumers parallel consuming at consumer level or thread level?

Hi,I have some question regarding how kafka consumers achieve parallel consuming for one topic. Say I have 2 partitions for topic1 and I have a consumer Group A, now:1: If no consumer under consumer Group A subscribe topic1, then no message will be delivery to this consumer group.2: If there is

Re: kafka consumers parallel consuming at consumer level or thread level?

lient (org.apache.kafka.clients.consumer.KafkaConsumer) that is going to be out in the next release, each consumer instance is single-threaded, i.e. it will only have one thread for fetching data. On Wed, Feb 4, 2015 at 3:43 AM, Zijing Guo wrote: Hi,I have some question regarding how kafka consumers achieve par

Can't send a keyedMessage to brokers with partitioner.class=kafka.producer.DefaultPartitioner

Hi,I have a 2 nodes kafka cluster with 2 instances of brokers and zookeepers. And then I create a topic kafka-test with 2 partitions and replication-factor =2. My producer config is:                      {"partitioner.class" "kafka.producer.DefaultPartitioner"                      "metadata.brok

Re: Can't send a keyedMessage to brokers with partitioner.class=kafka.producer.DefaultPartitioner

rokers and zookeepers. And then I create a topic kafka-test with 2 partitions and replication-factor =2. My prod... | | | | View on stackoverflow.com | Preview by Yahoo | | | |   |   On Sunday, February 8, 2015 4:18 AM, Zijing Guo wrote: Hi,I have a 2 nodes kafka cluster with 2 in

Re: What ports does a Kafka Producer/Consumer use other than 2180's and 9090's?

The kafka's port can be configured through server.properties, and if you use kafka's zk, then you can modify the zookeeper.properties file, for clientPort. On Tuesday, February 10, 2015 2:11 PM, Su She wrote: I was looking at the documentation and I see that the broker/server/consum

Re: understanding partition key

Partition key is on producer level, that if you have multiple partitions for a single topic, then you can pass in a key for the KeyedMessage object, and base on different partition.class, it will return a partition number for the producer, and producer will find the leader for that partition.I d

Kafka DefaultPartitioner is not behaved as expected.

Hi community,I have a 2 nodes test cluster with 2 zk instance and 2 broker instance running and I'm experimenting kafka producer in a cluster environment. So I create a topic "foo" with 2 partitions and replication 1.I create a async Producer without defining partition.class (so the partitioner

Re: Kafka DefaultPartitioner is not behaved as expected.

And I'm using kafka version 0.8.2.0 On Thursday, March 5, 2015 11:51 AM, Zijing Guo wrote: Hi community,I have a 2 nodes test cluster with 2 zk instance and 2 broker instance running and I'm experimenting kafka producer in a cluster environment. So I create a topic &q

Re: Kafka DefaultPartitioner is not behaved as expected.

sage), so in your case key is "test message" + e. Thanks, Mayuresh On Thu, Mar 5, 2015 at 9:25 AM, Zijing Guo wrote: > And I'm using kafka version 0.8.2.0 > >      On Thursday, March 5, 2015 11:51 AM, Zijing Guo > wrote: > > >  Hi community,I have a 2 nodes

Re: Kafka DefaultPartitioner is not behaved as expected.

Hi Guozhang,I'm using kafka 0.8.2.0  Thanks On Thursday, March 5, 2015 12:57 PM, Guozhang Wang wrote: Zijing, Which version of Kafka client are you using? On Thu, Mar 5, 2015 at 8:50 AM, Zijing Guo wrote: > Hi community,I have a 2 nodes test cluster with 2 zk instan

Re: Kafka DefaultPartitioner is not behaved as expected.

etBytes, "test message".getBytes)). When I convert the data and key to bytes, the data is not going to the proper partitions. Thanks On Thursday, March 5, 2015 12:59 PM, Zijing Guo wrote: Hi Guozhang,I'm using kafka 0.8.2.0  Thanks     On Thursday, March 5, 2015

Re: Kafka DefaultPartitioner is not behaved as expected.

erializer.class. This StringEncoder takes string as input and as you know string always returns same hash code if the value is the same. On Fri, Mar 6, 2015 at 2:23 AM, Zijing Guo wrote: > And also there something that I think worth mentioning,when I > call prod.send(KeyedMessage("foo&qu

Re: Kafka 0.9 consumer API

Hi all,The document is very beautiful and the Kafka release version for this will be? and what is the timeline? ThanksEdwin On Friday, March 20, 2015 4:20 PM, Rajiv Kurian wrote: Awesome - can't wait for this version to be out! On Fri, Mar 20, 2015 at 12:22 PM, Jay Kreps wrote: