Re: Spark SQL and Hive tables

2014-09-30 Thread Chen Song
AT&T 4G LTE smartphone > > > Original message > From: Michael Armbrust > Date:07/25/2014 3:24 PM (GMT-08:00) > To: user@spark.apache.org > Subject: Re: Spark SQL and Hive tables > > [S]ince Hive has a large number of dependencies, it is not include

RE: Spark SQL and Hive tables

2014-07-25 Thread Andrew Lee
guess this only applies to standalone mode? Andrew L Date: Fri, 25 Jul 2014 15:25:42 -0700 Subject: RE: Spark SQL and Hive tables From: ssti...@live.com To: user@spark.apache.org Thanks! Will do. Sent via the Samsung GALAXY S®4, an AT&T 4G LTE smartphone Orig

RE: Spark SQL and Hive tables

2014-07-25 Thread sstilak
Thanks! Will do. Sent via the Samsung GALAXY S®4, an AT&T 4G LTE smartphone Original message From: Michael Armbrust Date:07/25/2014 3:24 PM (GMT-08:00) To: user@spark.apache.org Subject: Re: Spark SQL and Hive tables > > [S]ince Hive has a large number of de

Re: Spark SQL and Hive tables

2014-07-25 Thread Michael Armbrust
SQLContext SchemaRDD SchemaRDDLike api > catalystcolumnarexecution package parquet > test > > > ------ > Date: Fri, 25 Jul 2014 17:48:27 -0400 > > Subject: Re: Spark SQL and Hive tables > From: chiling...@gm

RE: Spark SQL and Hive tables

2014-07-25 Thread Sameer Tilak
columnarexecution package parquettest Date: Fri, 25 Jul 2014 17:48:27 -0400 Subject: Re: Spark SQL and Hive tables From: chiling...@gmail.com To: user@spark.apache.org Hi Sameer, The blog post you referred to is about Spark SQL. I don't think the intent of the article is mean

RE: Spark SQL and Hive tables

2014-07-25 Thread Sameer Tilak
Thanks, Michael. From: mich...@databricks.com Date: Fri, 25 Jul 2014 14:49:00 -0700 Subject: Re: Spark SQL and Hive tables To: user@spark.apache.org >From the programming guide: When working with Hive one must construct a HiveContext, which inherits from SQLContext, and adds support for find

RE: Spark SQL and Hive tables

2014-07-25 Thread Sameer Tilak
Thanks, Jerry. Date: Fri, 25 Jul 2014 17:48:27 -0400 Subject: Re: Spark SQL and Hive tables From: chiling...@gmail.com To: user@spark.apache.org Hi Sameer, The blog post you referred to is about Spark SQL. I don't think the intent of the article is meant to guide you how to read data from

Re: Spark SQL and Hive tables

2014-07-25 Thread Michael Armbrust
n u.age, > u.latitude, u.logitude FROM Users u JOIN Events e ON u.userId = > e.userId""") > > > > -- > Date: Fri, 25 Jul 2014 17:27:17 -0400 > Subject: Re: Spark SQL and Hive tables > From: chiling...@gma

Re: Spark SQL and Hive tables

2014-07-25 Thread Jerry Lam
are > talking about? > > -- > From: mich...@databricks.com > Date: Fri, 25 Jul 2014 14:34:53 -0700 > > Subject: Re: Spark SQL and Hive tables > To: user@spark.apache.org > > > In particular, have you put your hive-site.xml in the conf/ directory?

RE: Spark SQL and Hive tables

2014-07-25 Thread Sameer Tilak
Hi Michael,Thanks. I am not creating HiveContext, I am creating SQLContext. I am using CDH 5.1. Can you please let me know which conf/ directory you are talking about? From: mich...@databricks.com Date: Fri, 25 Jul 2014 14:34:53 -0700 Subject: Re: Spark SQL and Hive tables To: user

Re: Spark SQL and Hive tables

2014-07-25 Thread Michael Armbrust
In particular, have you put your hive-site.xml in the conf/ directory? Also, are you creating a HiveContext instead of a SQLContext? On Fri, Jul 25, 2014 at 2:27 PM, Jerry Lam wrote: > Hi Sameer, > > Maybe this page will help you: > https://spark.apache.org/docs/latest/sql-programming-guide.ht

RE: Spark SQL and Hive tables

2014-07-25 Thread Sameer Tilak
extracted from existing sources, // such as Apache Hive. val trainingDataTable = sql(""" SELECT e.action u.age, u.latitude, u.logitude FROM Users u JOIN Events e ON u.userId = e.userId""") Date: Fri, 25 Jul 2014 17:27:17 -0400 S

Re: Spark SQL and Hive tables

2014-07-25 Thread Jerry Lam
Hi Sameer, Maybe this page will help you: https://spark.apache.org/docs/latest/sql-programming-guide.html#hive-tables Best Regards, Jerry On Fri, Jul 25, 2014 at 5:25 PM, Sameer Tilak wrote: > Hi All, > I am trying to load data from Hive tables using Spark SQL. I am using > spark-shell. Her

Spark SQL and Hive tables

2014-07-25 Thread Sameer Tilak
Hi All,I am trying to load data from Hive tables using Spark SQL. I am using spark-shell. Here is what I see: val trainingDataTable = sql("""SELECT prod.prod_num, demographics.gender, demographics.birth_year, demographics.income_group FROM prod p JOIN demographics d ON d.user_id = p.user_id"""