hive table with large column data size

Hi , I want to store binary data (such as images) into hive table but the binary data column might be much larger than other columns per row. I'm worried about the query performance. One way I can think of is to separate binary data storage from other columns by creating 2 hive tables and run 2 s

how to run unit test for specific component only

Hi, I am wondering how to run unit test for specific spark component only. mvn test -DwildcardSuites="org.apache.spark.sql.*" -Dtest=none The above command doesn't seem to work. I'm using spark 1.5. Thanks, Weide

Re: get host from rdd map

: > Can you outline your use case a bit more ? > > Do you want to know all the hosts which would run the map ? > > Cheers > > On Fri, Oct 23, 2015 at 5:16 PM, weoccc wrote: > >> in rdd map function, is there a way i can know the list of host names >> where the

get host from rdd map

in rdd map function, is there a way i can know the list of host names where the map runs ? any code sample would be appreciated ? thx, Weide

spark sql lateral view unresolved attribute exception

Hi, It seems LATERVAL VIEW explode column named 'some_col' can't be resolved if expressed in subquery. Any idea why ? SELECT `fc_clickq`.`some_col` FROM ( SELECT * FROM fc_clickq LATERAL VIEW explode(`overlap`) ltr_table_3 AS `some_col`) fc_clickq ; org.apache.spark.sql.catalyst.errors.package$