Hi Sparklers,
I was wondering if some else has also encountered this... (Actually I am not
even sure if this is an issue)...
I have a spark job that reads data from Hbase does a bunch of transformation
sparkContext.newAPIHadoopRDD -> flatMapToPair -> groupByKey -> mapValues
After this I do a t
I am also not familiar with Scala but I believe the concept is similar to the
concept of String in Java.
accum point to a “Accumulator”. You can change what it points to, but not that
which it points to.
From: Vikram Kalabi [mailto:vikram.apache@gmail.com]
Sent: Monday, September 22, 2014
Nice write-up... very helpful!
-Original Message-
From: Tim Smith [mailto:secs...@gmail.com]
Sent: Wednesday, September 17, 2014 1:11 PM
Cc: spark users
Subject: Re: Stable spark streaming app
I don't have anything in production yet but I now at least have a stable
(running for more th
Thanks Sean,
Makes total sense. I guess I was so caught up with RDD's and all the wonderful
transformations it can do, that I did not think about pain old Java
Collections.sort(list, comparator).
Thanks,
__
Abraham
-Original Message-
From: Sean Owen [mailto:so...
Hi Group,
I am quite fresh in the spark world. There is a particular use case that I just
cannot understand how to accomplish in spark. I am using Cloudera
CDH5/YARN/Java 7.
I have a dataset that has the following characteristics -
A JavaPairRDD that represents the following -
Key => {int ID}
Yes that was very helpful… ☺
Here are a few more I found on my quest to get HBase working with Spark –
This one details about Hbase dependencies and spark classpaths
http://www.abcn.net/2014/07/lighting-spark-with-hbase-full-edition.html
This one has a code overview –
http://www.abcn.net/2014/
Hi,
I had a similar situation in which I needed to read data from HBase and work
with the data inside of a spark context. After much ggling, I finally got
mine to work. There are a bunch of steps that you need to do get this working -
The problem is that the spark context does not know anyt
