thanks for you reply and patience
Best regards
--
qiaou
已使用 Sparrow (http://www.sparrowmailapp.com/?sig)
在 2014年11月12日 星期三，下午3:45，Shixiong Zhu 写道：
> The `conf` object will be sent to other nodes via Broadcast.
>
> Here is the scaladoc of Broadcast:
> http://spark.apache.org/docs/lates
The `conf` object will be sent to other nodes via Broadcast.
Here is the scaladoc of Broadcast:
http://spark.apache.org/docs/latest/api/scala/index.html#org.apache.spark.broadcast.Broadcast
In addition, the object v should not be modified after it is broadcast in
order to ensure that all nodes ge
this work!
but can you explain why should use like this?
--
qiaou
已使用 Sparrow (http://www.sparrowmailapp.com/?sig)
在 2014年11月12日 星期三，下午3:18，Shixiong Zhu 写道：
> You need to create a new configuration for each RDD. Therefore, "val
> hbaseConf = HBaseConfigUtil.getHBaseConfiguration" should be
You need to create a new configuration for each RDD. Therefore, "val
hbaseConf = HBaseConfigUtil.getHBaseConfiguration" should be changed to "val
hbaseConf = new Configuration(HBaseConfigUtil.getHBaseConfiguration)"
Best Regards,
Shixiong Zhu
2014-11-12 14:53 GMT+08:00 qiaou :
> ok here is the
ok here is the code
def hbaseQuery:(String)=>RDD[Result] = {
val generateRdd = (area:String)=>{
val startRowKey = s"$area${RowKeyUtils.convertToHex(startId,
10)}"
val stopRowKey = s"$area${RowKeyUtils.convertToHex(endId,
10)}"
println
Could you provide the code of hbaseQuery? It maybe doesn't support to
execute in parallel.
Best Regards,
Shixiong Zhu
2014-11-12 14:32 GMT+08:00 qiaou :
> Hi：
> I got a problem with using the union method of RDD
> things like this
> I get a function like
> def hbaseQuery(area:string):RD
Hi：
I got a problem with using the union method of RDD
things like this
I get a function like
def hbaseQuery(area:string):RDD[Result]= ???
when i use hbaseQuery('aa').union(hbaseQuery(‘bb’)).count() it returns 0
however when use like this sc.parallize(hbaseQuery('aa’).collect.toL
Hi：
I got a problem with using the union method of RDD
things like this
I get a function like
def hbaseQuery(area:string):RDD[Result]= ???
when i use hbaseQuery('aa').union(hbaseQuery(‘bb’)).count() it returns 0
however when use like this sc.parallize(hbaseQuery('aa’).collect.toL
