Re: spark thrift server as hive on spark running on kubernetes, and more.

what's the difference between DataRoaster and Dask? https://scalingpythonml.com/2020/11/03/a-first-look-at-dask-on-arm-on-k8s.html Thanks. On 2021/12/15 8:42, Kidong Lee wrote: Recently I have written a spark operator to deploy spark applications onto Kubernetes using custom resources. See Data

Re: spark thrift server as hive on spark running on kubernetes, and more.

/dataroaster) to provide data platforms > running on kubernetes with ease. > In particular, with DataRoaster, you can deploy spark thrift server on > kubernetes easily, which is originated from my blog of > https://itnext.io/hive-on-spark-in-kubernetes-115c8e9fa5c1. > In addition to spa

Re: spark thrift server as hive on spark running on kubernetes, and more.

netes with ease. > In particular, with DataRoaster, you can deploy spark thrift server on > kubernetes easily, which is originated from my blog of > https://itnext.io/hive-on-spark-in-kubernetes-115c8e9fa5c1. > In addition to spark thrift server as hive on spark, there are several > c

RE: spark thrift server as hive on spark running on kubernetes, and more.

Hi, thx. Great work. Will test it 😊 Best, Meikel Bode From: Kidong Lee Sent: Freitag, 10. September 2021 01:39 To: user@spark.apache.org Subject: spark thrift server as hive on spark running on kubernetes, and more. Hi, Recently, I have open-sourced a tool called DataRoaster(https

spark thrift server as hive on spark running on kubernetes, and more.

://itnext.io/hive-on-spark-in-kubernetes-115c8e9fa5c1. In addition to spark thrift server as hive on spark, there are several components provided by DataRoaster, for instance, hive metastore, trino, redash, jupyterhub, kafka. To use DataRoaster, - visit https://github.com/cloudcheflabs/dataroaster

Re: Hive on Spark vs Spark on Hive(HiveContext)

in no case be liable for any monetary damages arising from such loss, damage or destruction. On Thu, 1 Jul 2021 at 11:50, Pralabh Kumar wrote: > Hi mich > > Thx for replying.your answer really helps. The comparison was done in > 2016. I would like to know the latest comparison

Re: Hive on Spark vs Spark on Hive(HiveContext)

Hi mich Thx for replying.your answer really helps. The comparison was done in 2016. I would like to know the latest comparison with spark 3.0 Also what you are suggesting is to migrate queries to Spark ,which is hivecontxt or hive on spark, which is what Facebook also did . Is that understanding

Re: Hive on Spark vs Spark on Hive(HiveContext)

> approaches > > >1. One is Hive on Spark , which is similar to changing the execution >engine in hive queries like TEZ. >2. Another one is migrating hive queries to Hivecontext/sparksql , an >approach used by Facebook and presented in Spark conference. >

Hive on Spark vs Spark on Hive(HiveContext)

Hi Dev I am having thousands of legacy hive queries . As a plan to move to Spark , we are planning to migrate Hive queries on Spark . Now there are two approaches 1. One is Hive on Spark , which is similar to changing the execution engine in hive queries like TEZ. 2. Another one is

Re: Hive on Spark in Kubernetes.

Thank you very much! Отправлено с iPhone > 7 окт. 2020 г., в 17:38, mykidong написал(а): > > Hi all, > > I have recently written a blog about hive on spark in kubernetes > environment: > - https://itnext.io/hive-on-spark-in-kubernetes-115c8e9fa5c1 > > In this bl

Hive on Spark in Kubernetes.

Hi all, I have recently written a blog about hive on spark in kubernetes environment: - https://itnext.io/hive-on-spark-in-kubernetes-115c8e9fa5c1 In this blog, you can find how to run hive on kubernetes using spark thrift server compatible with hive server2. Cheers, - Kidong. -- Sent from

Re: [SPARK SQL] Difference between 'Hive on spark' and Spark SQL

want to use additional features and it could become part of your strategy justifying the investment: * hive on Spark - I don’t think it is as much used as the above combination. I am also not sure if it supports recent Spark versions and all Hive features. It would also not really allow you to

[SPARK SQL] Difference between 'Hive on spark' and Spark SQL

daily. We want to improve the performance of these queries and have two options at hand: a. Turn on 'Hive on spark' feature and run HQLs and b. Run those query statements with spark SQL What the difference between these options? Another question is: There is a hive setting 'hi

Re: ERROR: Hive on Spark

Change you table name in query to spam.spamdataset instead of spamdataset. On Sun, Apr 15, 2018 at 2:12 PM Rishikesh Gawade wrote: > Hello there. I am a newbie in the world of Spark. I have been working on a > Spark Project using Java. > I have configured Hive and Spark to run on Hadoop. > As of

ERROR: Hive on Spark

Hello there. I am a newbie in the world of Spark. I have been working on a Spark Project using Java. I have configured Hive and Spark to run on Hadoop. As of now i have created a Hive (derby) database on Hadoop HDFS at the given location(warehouse location): */user/hive/warehouse *and database name

Re: Hive on Spark is not populating correct records

After lots of expermiments, I have figured out that it was a potential bug in cloudera with Hive on Spark. Hive on Spark does not populate consistent output on aggregate functions. Hopefully, it will be fixed in next relaese. -- View this message in context: http://apache-spark-user-list

Hive on Spark Job Monitoring

Hi Team, I wanted to understand how Hive on Spark actually maps to Spark jobs underneath triggered by Hive. AFAIK each Hive query would trigger a new Spark job. But this was contradicted by someone and wanted to confirm what is the real design implementation. Please let me know if there is

Re: What is the difference between hive on spark and spark on hive?

Hive on Spark is Hive which takes sql statements in and creates Spark jobs for processing instead of Mapreduce or Tez. There is no such thing as "Spark on Hive", but there is SparkSQL. SparkSQL can accept both programmatic statements or it can parse SQL statements to produce a na

What is the difference between hive on spark and spark on hive?

What is the difference between hive on spark and spark on hive?

Hive on Spark is not populating correct records

Hi, Not sure whether it is right place to discuss this issue. I am running following Hive query multiple times with execution engine as Hive on Spark and Hive on MapReduce. With Hive on Spark: Result (count) were different of every execution. With Hive on MapReduce: Result (count) were same of

Re: RDD Partitions on HDFS file in Hive on Spark Query

f I could do all I can - Art of War >>> >>> On Mon, Nov 21, 2016 at 4:59 PM, Jörn Franke >>> wrote: >>> >>>> Use as a format orc, parquet or avro because they support any >>>> compression type with parallel processing. Alternatively split you

Re: RDD Partitions on HDFS file in Hive on Spark Query

t;> compression type with parallel processing. Alternatively split your file in >>>> several smaller ones. Another alternative would be bzip2 (but slower in >>>> general) or Lzo (usually it is not included by default in many >>>> distributions). >>>>

Re: RDD Partitions on HDFS file in Hive on Spark Query

le in >>> several smaller ones. Another alternative would be bzip2 (but slower in >>> general) or Lzo (usually it is not included by default in many >>> distributions). >>> >>> On 21 Nov 2016, at 23:17, yeshwanth kumar wrote: >>> >>> Hi

Re: RDD Partitions on HDFS file in Hive on Spark Query

e in several >> smaller ones. Another alternative would be bzip2 (but slower in general) or >> Lzo (usually it is not included by default in many distributions). >> >> On 21 Nov 2016, at 23:17, yeshwanth kumar wrote: >> >> Hi, >> >> we are running

Re: RDD Partitions on HDFS file in Hive on Spark Query

. Another alternative would be bzip2 (but slower in general) or > Lzo (usually it is not included by default in many distributions). > > On 21 Nov 2016, at 23:17, yeshwanth kumar wrote: > > Hi, > > we are running Hive on Spark, we have an external table over snappy > co

Re: RDD Partitions on HDFS file in Hive on Spark Query

On 21 Nov 2016, at 23:17, yeshwanth kumar wrote: > > Hi, > > we are running Hive on Spark, we have an external table over snappy > compressed csv file of size 917.4 M > HDFS block size is set to 256 MB > > as per my Understanding, if i run a query over that external ta

Re: RDD Partitions on HDFS file in Hive on Spark Query

Try changing compression to bzip2 or lzo. For reference - http://comphadoop.weebly.com Thanks, Aniket On Mon, Nov 21, 2016, 10:18 PM yeshwanth kumar wrote: > Hi, > > we are running Hive on Spark, we have an external table over snappy > compressed csv file of size 917.4 M > HDF

RDD Partitions on HDFS file in Hive on Spark Query

Hi, we are running Hive on Spark, we have an external table over snappy compressed csv file of size 917.4 M HDFS block size is set to 256 MB as per my Understanding, if i run a query over that external table , it should launch 4 tasks. one for each block. but i am seeing one executor and one

Re: Presentation in London: Running Spark on Hive or Hive on Spark

ve or Hive on Spark Dr Mich  do you have any slides or videos available for the presentation you did @Canary Wharf?kindest regards marco On Wed, Jul 6, 2016 at 10:37 PM, Mich Talebzadeh wrote: Dear forum members I will be presenting on the topic of "Running Spark on Hive or Hive on Spark, yo

Re: Presentation in London: Running Spark on Hive or Hive on Spark

.@gmail.com] > *Sent:* 15 July 2016 11:04 > *To:* Mich Talebzadeh > *Cc:* user @spark ; user > *Subject:* Re: Presentation in London: Running Spark on Hive or Hive on > Spark > > > > Dr Mich > > do you have any slides or videos available for the presentation you

RE: Presentation in London: Running Spark on Hive or Hive on Spark

It is on the 20th (Wednesday) next week. From: Marco Mistroni [mailto:mmistr...@gmail.com] Sent: 15 July 2016 11:04 To: Mich Talebzadeh Cc: user @spark ; user Subject: Re: Presentation in London: Running Spark on Hive or Hive on Spark Dr Mich do you have any slides or videos available for

Re: Presentation in London: Running Spark on Hive or Hive on Spark

Dr Mich do you have any slides or videos available for the presentation you did @Canary Wharf? kindest regards marco On Wed, Jul 6, 2016 at 10:37 PM, Mich Talebzadeh wrote: > Dear forum members > > I will be presenting on the topic of "Running Spark on Hive or Hive on > Sp

Re: Presentation in London: Running Spark on Hive or Hive on Spark

Hi Mich, Would it be on YouTube , post session ? - Harmeet On Jul 7, 2016, at 3:07, Mich Talebzadeh wrote: > Dear forum members > > I will be presenting on the topic of "Running Spark on Hive or Hive on Spark, > your mileage varies" in Future of Data: London >

Re: Presentation in London: Running Spark on Hive or Hive on Spark

Thanks. Will this presentation recorded as well? Regards On Wednesday, 6 July 2016, 22:38, Mich Talebzadeh wrote: Dear forum members I will be presenting on the topic of "Running Spark on Hive or Hive on Spark, your mileage varies" in Future of Data: London DetailsOr

Presentation in London: Running Spark on Hive or Hive on Spark

Dear forum members I will be presenting on the topic of "Running Spark on Hive or Hive on Spark, your mileage varies" in Future of Data: London <http://www.meetup.com/futureofdata-london/events/232423292/> *Details* *Organized by: Hortonworks <http://hortonworks.com/>*

Re: Hive on Spark engine

Thanks Ted, More interested in general availability of Hive 2 on Spark 1.6 engine as opposed to Vendors specific custom built. Dr Mich Talebzadeh LinkedIn * https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw

Re: Hive on Spark engine

According to: https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.3.4/bk_HDP_RelNotes/bk_HDP_RelNotes-20151221.pdf Spark 1.5.2 comes out of box. Suggest moving questions on HDP to Hortonworks forum. Cheers On Sat, Mar 26, 2016 at 3:32 PM, Mich Talebzadeh wrote: > Thanks Jorn. > > Just to be

Re: Hive on Spark engine

Thanks Jorn. Just to be clear they get Hive working with Spark 1.6 out of the box (binary download)? The usual work-around is to build your own package and get the Hadoop-assembly jar file copied over to $HIVE_HOME/lib. Cheers Dr Mich Talebzadeh LinkedIn * https://www.linkedin.com/profile/

Re: Hive on Spark engine

If you check the newest Hortonworks distribution then you see that it generally works. Maybe you can borrow some of their packages. Alternatively it should be also available in other distributions. > On 26 Mar 2016, at 22:47, Mich Talebzadeh wrote: > > Hi, > > I am running Hive 2 and now Spar

Hive on Spark engine

Hi, I am running Hive 2 and now Spark 1.6.1 but I still do not see any sign that Hive can utilise a Spark engine higher than 1.3.1 My understanding was that there were miss-match on Hadoop assembly Jar files that cause Hive not being able to run on Spark using the binary downloads. I just tried H

Re: Hive on Spark performance

Depending on the version of Hive on Spark engine. As far as I am aware the latest version of Hive that I am using (Hive 2) has improvements compared to the previous versions of Hive (0.14,1.2.1) on Spark engine. As of today I have managed to use Hive 2.0 on Spark version 1.3.1. So it is not the

Re: Hive on Spark knobs

u run hive on spark with spark 1.5 and hive 1.1? > I think hive on spark doesn't support spark 1.5. There are compatibility > issues. > > > At 2016-01-28 01:51:43, "Ruslan Dautkhanov" wrote: > > > https://cwiki.apache.org/confluence/display/Hive/Hive+on+Spark%3

Hive on Spark knobs

https://cwiki.apache.org/confluence/display/Hive/Hive+on+Spark%3A+Getting+Started There are quite a lot of knobs to tune for Hive on Spark. Above page recommends following settings: mapreduce.input.fileinputformat.split.maxsize=75000 > hive.vectorized.execution.enabled=t

Re: adding jars - hive on spark cdh 5.4.3

gt; class: com.foursquare.hadoop.hive.io.HiveThriftSequenceFileInputFormat >>>> >>>> >>>> HiveThriftSequenceFileInputFormat is in one of the jars I'm trying to add. >>>> >>>> >>>> On Thu, Jan 7, 2016 at 9:58 PM, Prem Sure &g

Re: adding jars - hive on spark cdh 5.4.3

;>> class: com.foursquare.hadoop.hive.io.HiveThriftSequenceFileInputFormat >>> >>> >>> HiveThriftSequenceFileInputFormat is in one of the jars I'm trying to add. >>> >>> >>> On Thu, Jan 7, 2016 at 9:58 PM, Prem Sure wrote: >>&

Re: adding jars - hive on spark cdh 5.4.3

6 at 9:58 PM, Prem Sure wrote: >> >>> did you try -- jars property in spark submit? if your jar is of huge >>> size, you can pre-load the jar on all executors in a common available >>> directory to avoid network IO. >>> >>> On Thu, Jan 7, 2

Re: adding jars - hive on spark cdh 5.4.3

e-load the jar on all executors in a common available >> directory to avoid network IO. >> >> On Thu, Jan 7, 2016 at 4:03 PM, Ophir Etzion >> wrote: >> >>> I' trying to add jars before running a query using hive on spark on cdh >>> 5.4.3. >>>

Re: adding jars - hive on spark cdh 5.4.3

your jar is of huge size, > you can pre-load the jar on all executors in a common available directory > to avoid network IO. > > On Thu, Jan 7, 2016 at 4:03 PM, Ophir Etzion wrote: > >> I' trying to add jars before running a query using hive on spark on cdh >> 5.

Re: adding jars - hive on spark cdh 5.4.3

did you try -- jars property in spark submit? if your jar is of huge size, you can pre-load the jar on all executors in a common available directory to avoid network IO. On Thu, Jan 7, 2016 at 4:03 PM, Ophir Etzion wrote: > I' trying to add jars before running a query using hive on spar

adding jars - hive on spark cdh 5.4.3

I' trying to add jars before running a query using hive on spark on cdh 5.4.3. I've tried applying the patch in https://issues.apache.org/jira/browse/HIVE-12045 (manually as the patch is done on a different hive version) but still hasn't succeeded. did anyone manage to do ADD J

Re: hive on spark

Looks like a version mismatch, you need to investigate more and make sure the versions satisfies. Thanks Best Regards On Sat, Dec 19, 2015 at 2:15 AM, Ophir Etzion wrote: > During spark-submit when running hive on spark I get: > > Exception in thread "main" java.util.Servic

hive on spark

During spark-submit when running hive on spark I get: Exception in thread "main" java.util.ServiceConfigurationError: org.apache.hadoop.fs.FileSystem: Provider org.apache.hadoop.hdfs.HftpFileSystem could not be instantiated Caused by: java.lang.IllegalAccessError: tried to acc

Re: Hive on Spark - Error: Child process exited before connecting back

not be understood as given or endorsed by Peridale Technology > Ltd, its subsidiaries or their employees, unless expressly so stated. It is > the responsibility of the recipient to ensure that this email is virus > free, therefore neither Peridale Ltd, its subsidiaries nor their employees &g

Hive on Spark - Error: Child process exited before connecting back

Hi, when trying to do Hive on Spark on CDH5.4.3 I get the following error when trying to run a simple query using spark. I've tried setting everything written here ( https://cwiki.apache.org/confluence/display/Hive/Hive+on+Spark%3A+Getting+Started) as well as what the cdh recommends. an

Re: Hive on Spark orc file empty

> error:value write is not a member of org.apache.spark.sql.DataFrame > > Thanks > Deepak > > On Mon, Nov 16, 2015 at 4:10 PM, 张炜 wrote: > >> Dear all, >> I am following this article to try Hive on Spark >> >> http://hortonworks.com/hadoop-tutorial/usi

Re: Hive on Spark orc file empty

: > Dear all, > I am following this article to try Hive on Spark > > http://hortonworks.com/hadoop-tutorial/using-hive-with-orc-from-apache-spark/ > > My environment: > Hive 1.2.1 > Spark 1.5.1 > > in a nutshell, I ran spark-shell, created a hive table > > hiveConte

Hive on Spark orc file empty

Dear all, I am following this article to try Hive on Spark http://hortonworks.com/hadoop-tutorial/using-hive-with-orc-from-apache-spark/ My environment: Hive 1.2.1 Spark 1.5.1 in a nutshell, I ran spark-shell, created a hive table hiveContext.sql("create table yahoo_orc_table (date S

Re: Hive on Spark Vs Spark SQL

7;s a completely different path. >> >> >> On Sun, Nov 15, 2015 at 10:37 PM, kiran lonikar >> wrote: >> >>> I would like to know if Hive on Spark uses or shares the execution code >>> with Spark SQL or DataFrames? >>> >>> More specifi

Re: Hive on Spark Vs Spark SQL

So does not benefit from Project Tungsten right? On Mon, Nov 16, 2015 at 12:07 PM, Reynold Xin wrote: > It's a completely different path. > > > On Sun, Nov 15, 2015 at 10:37 PM, kiran lonikar wrote: > >> I would like to know if Hive on Spark uses or shares the execut

Re: Hive on Spark Vs Spark SQL

It's a completely different path. On Sun, Nov 15, 2015 at 10:37 PM, kiran lonikar wrote: > I would like to know if Hive on Spark uses or shares the execution code > with Spark SQL or DataFrames? > > More specifically, does Hive on Spark benefit from the changes made to >

Hive on Spark Vs Spark SQL

I would like to know if Hive on Spark uses or shares the execution code with Spark SQL or DataFrames? More specifically, does Hive on Spark benefit from the changes made to Spark SQL, project Tungsten? Or is it completely different execution path where it creates its own plan and executes on RDD

Re: hive on spark query error

Seems like you have "hive.server2.enable.doAs" enabled; you can either disable it, or configure hs2 so that the user running the service ("hadoop" in your case) can impersonate others. See: https://hadoop.apache.org/docs/r2.7.0/hadoop-project-dist/hadoop-common/Superusers.html On Fri, Sep 25, 201

RE: hive on spark query error

exited with code 1. -Original Message- From: Marcelo Vanzin [mailto:van...@cloudera.com] Sent: Friday, September 25, 2015 1:12 PM To: Garry Chen Cc: Jimmy Xiang ; user@spark.apache.org Subject: Re: hive on spark query error On Fri, Sep 25, 2015 at 10:05 AM, Garry Chen wrote: > In sp

Re: hive on spark query error

On Fri, Sep 25, 2015 at 10:05 AM, Garry Chen wrote: > In spark-defaults.conf the spark.master is spark://hostname:7077. From > hive-site.xml > spark.master > hostname > That's not a valid value for spark.master (as the error indicates). You should set it to "spark://hostname:7077"

RE: hive on spark query error

In spark-defaults.conf the spark.master is spark://hostname:7077. From hive-site.xml spark.master hostname From: Jimmy Xiang [mailto:jxi...@cloudera.com] Sent: Friday, September 25, 2015 1:00 PM To: Garry Chen Cc: user@spark.apache.org Subject: Re: hive on spark query error

Re: hive on spark query error

> Error: Master must start with yarn, spark, mesos, or local What's your setting for spark.master? On Fri, Sep 25, 2015 at 9:56 AM, Garry Chen wrote: > Hi All, > > I am following > https://cwiki.apache.org/confluence/display/Hive/Hive+on+Spark%3A+Getting

hive on spark query error

Hi All, I am following https://cwiki.apache.org/confluence/display/Hive/Hive+on+Spark%3A+Getting+Started? to setup hive on spark. After setup/configuration everything startup I am able to show tables but when executing sql statement within beeline I got error. Please help and

Re: Hive on Spark VS Spark SQL

is good if you're already using > Hive and need its language features and need it as it works today, and want > a faster batch execution version of it. > > On Wed, May 20, 2015 at 7:18 AM, Debasish Das > wrote: > >> SparkSQL was built to improve upon Hive on Spark

Re: Hive on Spark VS Spark SQL

#x27;re already using Hive and need its language features and need it as it works today, and want a faster batch execution version of it. On Wed, May 20, 2015 at 7:18 AM, Debasish Das wrote: > SparkSQL was built to improve upon Hive on Spark runtime further... > > On Tue, May 19

Re: Hive on Spark VS Spark SQL

SparkSQL was built to improve upon Hive on Spark runtime further... On Tue, May 19, 2015 at 10:37 PM, guoqing0...@yahoo.com.hk < guoqing0...@yahoo.com.hk> wrote: > Hive on Spark and SparkSQL which should be better , and what are the key > characteristics and the advantages and the d

Hive on Spark VS Spark SQL

Hive on Spark and SparkSQL which should be better , and what are the key characteristics and the advantages and the disadvantages between ? guoqing0...@yahoo.com.hk

Re: Hive on Spark with Spark as a service on CDH5.2

Hive on Spark and accessing HiveContext from the shall are seperate things. Hive on Spark - https://cwiki.apache.org/confluence/display/Hive/Hive+on+Spark%3A+Getting+Started To access hive on Spark you need to built with -Phive. http://spark.apache.org/docs/1.2.1/building-spark.html#building

Hive on Spark with Spark as a service on CDH5.2

*I am not clear if spark sql supports HIve on Spark when spark is run as a service in CDH 5.2? * Can someone please clarify this. If this is possible, how what configuration changes have I to make to import hive context in spark shell as well as to be able to do a spark-submit for the job to be

Benchmarks of 'Hive on Tez' vs 'Hive on Spark' vs Spark SQL

Hi I would like to share with you my comments on Hortonworks' benchmarks of 'Hive on Tez' vs 'Hive on Spark' vs 'Spark SQL'. Please check them in my related blog entry at http://goo.gl/K5mk0U Thanks Slim Baltagi Chicago, IL http://www.SparkBigData.com --

Re: How to integrate hive on spark

Hi Did you try these steps. https://cwiki.apache.org/confluence/display/Hive/Hive+on+Spark%3A+Getting+Started Thanks Arush On Wed, Feb 18, 2015 at 7:20 PM, sandeepvura wrote: > Hi , > > I am new to sparks.I had installed spark on 3 node cluster.I would like to > integrate h

How to integrate hive on spark

Hi , I am new to sparks.I had installed spark on 3 node cluster.I would like to integrate hive on spark . can anyone please help me on this, Regards, Sandeep.v -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/How-to-integrate-hive-on-spark-tp21702.html

Re: Hive on Spark vs. SparkSQL using Hive ?

I would characterize the difference as follows: Spark SQL <http://spark.apache.org/docs/latest/sql-programming-guide.html> is the native engine for processing structured data using Spark. In contrast to Shark or Hive on Spark is has its own optimizer that was designed for the RDD model.

Re: Hive on Spark vs. SparkSQL using Hive ?

Spark SQL on Hive 1. The purpose of Spark SQL is to allow Spark users to selectively use SQL expressions (with not a huge number of functions currently supported) when writing Spark jobs 2. Already Available Hive on Spark 1.Spark users will automatically get the whole set of Hive’s rich features

Hive on Spark vs. SparkSQL using Hive ?

Hello, probably this question was already asked but still I'd like to confirm from Spark users. This following blog shows 'hive on spark' : http://blog.cloudera.com/blog/2014/12/hands-on-hive-on-spark-in-the-aws-cloud/";. How is it different from using hive as data stora

spark worker nodes getting disassociated while running hive on spark

Hi, I have setup the spark 1.2 standalone cluster and trying to run hive on spark by following below link. https://cwiki.apache.org/confluence/display/Hive/Hive+on+Spark%3A+Getting+Started I got the latest build of hive on spark from git and was trying to running few queries. Queries are

Re: can't get smallint field from hive on spark

I mean the later... thanks On Thu, Nov 27, 2014 at 1:42 PM, Yin Huai wrote: > For "hive on spark", did you mean the thrift server of Spark SQL or > https://issues.apache.org/jira/browse/HIVE-7292? If you meant the latter > one, I think Hive's mailing list will be

Re: can't get smallint field from hive on spark

For "hive on spark", did you mean the thrift server of Spark SQL or https://issues.apache.org/jira/browse/HIVE-7292? If you meant the latter one, I think Hive's mailing list will be a good place to ask (see https://hive.apache.org/mailing_lists.html). Thanks, Yin On Wed, Nov 26

Re: can't get smallint field from hive on spark

now whether this question should be asked here, if not, please >> point me out, thanks. >> >> we are currently using hive on spark, when reading a small int field, it >> reports error: >> Cannot get field 'i16Val' because union is currently set to i

Re: can't get smallint field from hive on spark

This has been fixed in Spark 1.1.1 and Spark 1.2 https://issues.apache.org/jira/browse/SPARK-3704 On Wed, Nov 26, 2014 at 7:10 PM, 诺铁 wrote: > hi, > > don't know whether this question should be asked here, if not, please > point me out, thanks. > > we are currently us

can't get smallint field from hive on spark

hi, don't know whether this question should be asked here, if not, please point me out, thanks. we are currently using hive on spark, when reading a small int field, it reports error: Cannot get field 'i16Val' because union is currently set to i32Val I googled and find only