Hi all
Producing more data into Kafka is not effective in my situation,
because the speed of reading Kafka is consistent. I will adopt Saiai's
suggestion to add more receivers.
Kyle
2015-04-30 14:49 GMT+08:00 Saisai Shao :
> From the chart you pasted, I guess you only have one receiver with
>From the chart you pasted, I guess you only have one receiver with storage
level two copies, so mostly your taks are located on two executors. You
could use repartition to redistribute the data more evenly across the
executors. Also add more receiver is another solution.
2015-04-30 14:38 GMT+08:0
t; My environment info]Kyle Lin ---2015/04/30 14:39:32---Hi all My
> environment info
>
> From: Kyle Lin
> To: "user@spark.apache.org"
> Date: 2015/04/30 14:39
> Subject: The Processing loading of Spark streaming on YARN is not in
> balance
> -
/sets
From: Kyle Lin
To: "user@spark.apache.org"
Date: 2015/04/30 14:39
Subject: The Processing loading of Spark streaming on YARN is not in
balance
Hi all
My environment info
Hadoop release version: HDP 2.1
Kakfa: 0.8.1.2.1.4.0
Spark: 1.1.0
My question:
Hi all
My environment info
Hadoop release version: HDP 2.1
Kakfa: 0.8.1.2.1.4.0
Spark: 1.1.0
My question:
I ran Spark streaming program on YARN. My Spark streaming program will
read data from Kafka and doing some processing. But, I found there is
always only ONE executor under processing. As
