Re: spark 1.2 three times slower than spark 1.1, why?

2015-01-21 Thread JaeBoo Jung
Title: Samsung Enterprise Portal mySingle I was recently faced with a similar issue, but unfortunately I could not find out why it happened. Here's jira ticket https://issues.apache.org/jira/browse/SPARK-5081 of my previous post. Please check your shuffle I/O differences between the two in sp

use spark-shell in the source

2014-06-11 Thread JaeBoo Jung
run this line in spark shell style and get outputs. run(line); } Thanks _ JaeBoo, Jung Assistant Engineer / BDA Lab / Samsung SDS

Re: Re: spark table to hive table

2014-05-27 Thread JaeBoo Jung
Title: Samsung Enterprise Portal mySingle I already tried HiveContext as well as SqlContext. But it seems that Spark's HiveContext is not completely same as Apache Hive. For example, SQL like 'SELECT RANK() OVER(ORDER BY VAL1 ASC) FROM TEST LIMIT 10' works fine in Apache Hive, but Spark's Hi