Hi all,
I am running a Spark streaming application with ReliableKafkaReceiver (Spark
1.2.0). Constantly I was getting the following exception:
15/01/12 19:07:06 ERROR receiver.BlockGenerator: Error in block pushing thread
java.util.concurrent.TimeoutException: Futures timed out after [30 seconds
Hi All,
I run a Spark streaming application (Spark 1.2.0) on YARN (Hadoop 2.5.2) with
Spark event log enabled. I set the checkpoint dir in the streaming context and
run the app. It started in YARN with application id 'app_id_1' and created the
Spark event log dir /spark/applicationHistory/app_i
, January 06, 2015 1:12 PM
To: Max Xu
Cc: user@spark.apache.org
Subject: Re: Saving data to Hbase hung in Spark streaming application with
Spark 1.2.0
I doubt anyone would deploy hbase 0.98.x on hadoop-1
Looks like hadoop2 profile should be made the default.
Cheers
On Tue, Jan 6, 2015 at 9:49 AM
0.98.7-hadoop1
From: Ted Yu [mailto:yuzhih...@gmail.com]
Sent: Tuesday, January 06, 2015 12:39 PM
To: Max Xu
Cc: user@spark.apache.org
Subject: Re: Saving data to Hbase hung in Spark streaming application with
Spark 1.2.0
Default profile is hbase-hadoop1 so you need to specify
use
spark-examples-1.1.1-hadoop2.5.2.jar from Spark 1.1.1 build (build with hbase
0.98.8-hadoop2 ), the problem went away. I’ll try to run the app again after
rebuild Spark 1.2.0 with 0.98.8-hadoop2.
From: Ted Yu [mailto:yuzhih...@gmail.com]
Sent: Tuesday, January 06, 2015 11:56 AM
To: Max Xu
Cc
Hi all,
I have a Spark streaming application that ingests data from a Kafka topic and
persists received data to Hbase. It works fine with Spark 1.1.1 in YARN cluster
mode. Basically, I use the following code to persist each partition of each RDD
to Hbase:
@Override
void call(It
