Re: Spark SQL Thriftserver with HBase

and all responsibility for any loss, damage or destructionof data or any other property which may arise from relying on this email's technical content is explicitly disclaimed.The author will in no case be liable for any monetary damages arising from suchloss, damage or destruction. O

Re: Spark SQL Thriftserver with HBase

:47 AM, Mich Talebzadeh >>>> wrote: >>>> >>>> Like any other design what is your presentation layer and end users? >>>> >>>> Are they SQL centric users from Tableau background or they may use >>>> spark functional programming. &

Re: Spark SQL Thriftserver with HBase

t;> monetary damages arising from such loss, damage or destruction. >>>>>>>>> >>>>>>>>> >>>>>>>>>> On 8 October 2016 at 20:15, Benjamin Kim wrote: >>>>>>>>>> Mich,

Re: Spark SQL Thriftserver with HBase

Without naming specifics, there are at least 4 or 5 different implementations of HBASE sources, each at varying level of development and different requirements (HBASE release version, Kerberos support etc) _________ From: Benjamin Kim mailto:bbuil...@gmail.

Re: Spark SQL Thriftserver with HBase

a...@gmail.com>>, Felix Cheung mailto:felixcheun...@hotmail.com>>, "user@spark.apache.org<mailto:user@spark.apache.org>" mailto:user@spark.apache.org>> Subject: Re: Spark SQL Thriftserver with HBase Instead of (or additionally to) saving results somewhere, you just star

Re: Spark SQL Thriftserver with HBase

mailto:felixcheun...@hotmail.com>> wrote: I wouldn't be too surprised Spark SQL - JDBC data source - Phoenix JDBC server - HBASE would work better. Without naming specifics, there are at least 4 or 5 different implementations of HBASE sources, each at varying level of develo

Re: Spark SQL Thriftserver with HBase

erent requirements (HBASE release version, Kerberos support etc) _ From: Benjamin Kim mailto:bbuil...@gmail.com>> Sent: Saturday, October 8, 2016 11:26 AM Subject: Re: Spark SQL Thriftserver with HBase To: Mich Talebzadeh mailto:mich.talebza...@gmail.com>&

Re: Spark SQL Thriftserver with HBase

ey will connected to a located database >>>>>>>> server using JDBC. Either way, by using JDBC everywhere, it simplifies >>>>>>>> and unifies the code on the JDBC industry standard. >>>>>>>> >>>>>>>> Does this make sense? &g

Re: Spark SQL Thriftserver with HBase

cted to a located database >>>>>> server using JDBC. Either way, by using JDBC everywhere, it simplifies >>>>>> and unifies the code on the JDBC industry standard. >>>>>> >>>>>> Does this make sense? >>>>>>

Re: Spark SQL Thriftserver with HBase

;> LinkedIn * >>> https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw >>> <https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>* >>> >>> >>> http://talebzadehmich.wordpress.com >>>

Re: Spark SQL Thriftserver with HBase

er 17, 2016 at 1:53 PM > *To: *Benjamin Kim > *Cc: *Michael Segel , Jörn Franke < > jornfra...@gmail.com>, Mich Talebzadeh , Felix > Cheung , "user@spark.apache.org" < > user@spark.apache.org> > > *Subject: *Re: Spark SQL Thriftserver with HBase > > > > I

Re: Spark SQL Thriftserver with HBase

Talebzadeh , Felix Cheung , "user@spark.apache.org" Subject: Re: Spark SQL Thriftserver with HBase Instead of (or additionally to) saving results somewhere, you just start a thriftserver that expose the Spark tables of the SQLContext (or SparkSession now). That means you can implement any

Re: Spark SQL Thriftserver with HBase

>> JDBC directly within the same data-center. When not colocated in the same >>>>> data-center, they will connected to a located database server using JDBC. >>>>> Either way, by using JDBC everywhere, it simplifies and unifies the code >>>>> on >>>>> the JDB

Re: Spark SQL Thriftserver with HBase

t 8, 2016, at 11:47 AM, Mich Talebzadeh >>>> wrote: >>>> >>>> Like any other design what is your presentation layer and end users? >>>> >>>> Are they SQL centric users from Tableau background or they may use >>>> spark funct

Re: Spark SQL Thriftserver with HBase

On Oct 8, 2016, at 11:47 AM, Mich Talebzadeh >>>>> <mailto:mich.talebza...@gmail.com>> wrote: >>>>>> >>>>>> Like any other design what is your presentation layer and end users? >>>>>> >>>>>> Are they SQL cent

Re: Spark SQL Thriftserver with HBase

>> >>> LinkedIn * >>> https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw >>> <https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw>* >>> >>> >>> http://talebzadehmich.wordpress.com >&

Re: Spark SQL Thriftserver with HBase

nt and different requirements (HBASE release version, Kerberos support etc) _ From: Benjamin Kim mailto:bbuil...@gmail.com>> Sent: Saturday, October 8, 2016 11:26 AM Subject: Re: Spark SQL Thriftserver with HBase To: Mich Talebzadeh mailto:mich.talebza..

Re: Spark SQL Thriftserver with HBase

AEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw> >>>> >>>> http://talebzadehmich.wordpress.com <http://talebzadehmich.wordpress.com/> >>>> >>>> Disclaimer: Use it at your own risk. Any and all responsibility for any >>>> loss, damage

Re: Spark SQL Thriftserver with HBase

es >> arising from such loss, damage or destruction. >> >> >> >> On 8 October 2016 at 19:40, Felix Cheung >> wrote: >> >>> I wouldn't be too surprised Spark SQL - JDBC data source - Phoenix JDBC >>> server - HBASE would work better. >>>

Re: Spark SQL Thriftserver with HBase

from relying on this email's technical content is explicitly disclaimed. >>>> The author will in no case be liable for any monetary damages arising from >>>> such loss, damage or destruction. >>>> >>>> >>>>> On 8 October 2016 at 19:40, Felix Cheung >>>>> wrot

Re: Spark SQL Thriftserver with HBase

>> server - HBASE would work better. >> >> Without naming specifics, there are at least 4 or 5 different >> implementations of HBASE sources, each at varying level of development and >> different requirements (HBASE release version, Kerberos support etc) >> &g

Re: Spark SQL Thriftserver with HBase

d work better. >> >> Without naming specifics, there are at least 4 or 5 different >> implementations of HBASE sources, each at varying level of development and >> different requirements (HBASE release version, Kerberos support etc) >> >> >> ___

Re: Spark SQL Thriftserver with HBase

aming specifics, there are at least 4 or 5 different implementations > of HBASE sources, each at varying level of development and different > requirements (HBASE release version, Kerberos support etc) > > > _____ > From: Benjamin Kim mailto:bbuil...

Re: Spark SQL Thriftserver with HBase

ease version, Kerberos support etc) > > > _ > From: Benjamin Kim > Sent: Saturday, October 8, 2016 11:26 AM > Subject: Re: Spark SQL Thriftserver with HBase > To: Mich Talebzadeh > Cc: , Felix Cheung > > > > Mich, > > Are you talking about the Phoenix JDBC Server? If s

Re: Spark SQL Thriftserver with HBase

rsion, Kerberos support etc) _ From: Benjamin Kim mailto:bbuil...@gmail.com>> Sent: Saturday, October 8, 2016 11:26 AM Subject: Re: Spark SQL Thriftserver with HBase To: Mich Talebzadeh mailto:mich.talebza...@gmail.com>> Cc: mailto:user@spark.apache.or

Re: Spark SQL Thriftserver with HBase

gt; loss, damage or destruction. > > > On 8 October 2016 at 19:05, Felix Cheung <mailto:felixcheun...@hotmail.com>> wrote: > Great, then I think those packages as Spark data source should allow you to > do exactly that (replace org.apache.spark.sql.jdbc with HBASE o

Re: Spark SQL Thriftserver with HBase

experience with this! > > > _ > From: Benjamin Kim mailto:bbuil...@gmail.com>> > Sent: Saturday, October 8, 2016 11:00 AM > Subject: Re: Spark SQL Thriftserver with HBase > To: Felix Cheung <mailto:felixcheun...@hotmail.com>> > Cc: m

Re: Spark SQL Thriftserver with HBase

or destruction. On 8 October 2016 at 19:05, Felix Cheung wrote: > Great, then I think those packages as Spark data source should allow you > to do exactly that (replace org.apache.spark.sql.jdbc with HBASE one) > > I do think it will be great to get more examples around this thoug

Re: Spark SQL Thriftserver with HBase

! _ From: Benjamin Kim mailto:bbuil...@gmail.com>> Sent: Saturday, October 8, 2016 11:00 AM Subject: Re: Spark SQL Thriftserver with HBase To: Felix Cheung mailto:felixcheun...@hotmail.com>> Cc: mailto:user@spark.apache.org>> Felix, My goal is to use Spark SQL JDBC Thri

Re: Spark SQL Thriftserver with HBase

Thrift Server (with USING, > http://spark.apache.org/docs/latest/sql-programming-guide.html#tab_sql_10 > <http://spark.apache.org/docs/latest/sql-programming-guide.html#tab_sql_10>). > > > _ > From: Benjamin Kim mailto:bbuil...@gmail.com>&

Re: Spark SQL Thriftserver with HBase

tab_sql_10). _ From: Benjamin Kim mailto:bbuil...@gmail.com>> Sent: Saturday, October 8, 2016 10:40 AM Subject: Re: Spark SQL Thriftserver with HBase To: Felix Cheung mailto:felixcheun...@hotmail.com>> Cc: mailto:user@spark.apache.org>> Felix, The only alternative way is to create

Re: Spark SQL Thriftserver with HBase

Felix, The only alternative way is to create a stored procedure (udf) in database terms that would run Spark scala code underneath. In this way, I can use Spark SQL JDBC Thriftserver to execute it using SQL code passing the key, values I want to UPSERT. I wonder if this is possible since I cann

Re: Spark SQL Thriftserver with HBase

HBase has released support for Spark hbase.apache.org/book.html#spark And if you search you should find several alternative approaches. On Fri, Oct 7, 2016 at 7:56 AM -0700, "Benjamin Kim" mailto:bbuil...@gmail.com>> wrote: Does anyone know if Spark

Spark SQL Thriftserver with HBase

Does anyone know if Spark can work with HBase tables using Spark SQL? I know in Hive we are able to create tables on top of an underlying HBase table that can be accessed using MapReduce jobs. Can the same be done using HiveContext or SQLContext? We are trying to setup a way to GET and POST data

Re: Spark SQL Thriftserver

RK_HOME}/sbin/start-thriftserver.sh \ >>> --master \ >>> --hiveconf hive.server2.thrift.port=10055 \ >>> >>> and STS bypasses Spark optimiser and uses Hive optimizer and execution >>> engine. You will see this in hive.log file &g

Re: Spark SQL Thriftserver

ver.sh \ >>> --master \ >>> --hiveconf hive.server2.thrift.port=10055 \ >>> >>> and STS bypasses Spark optimiser and uses Hive optimizer and execution >>> engine. You will see this in hive.log file >>> >&

Re: Spark SQL Thriftserver

t;> >> HTH >> >> >> >> >> Dr Mich Talebzadeh >> >> >> LinkedIn * >> https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd6zP6AcPCCdOABUrV8Pw >> <https://www.linkedin.com/profile/view?id=AAEWh2gBxianrbJd

Re: Spark SQL Thriftserver

e > from relying on this email's technical content is explicitly disclaimed. > The author will in no case be liable for any monetary damages arising from > such loss, damage or destruction. > > > > On 13 September 2016 at 22:32, Benjamin Kim wrote: > >> Does an

Re: Spark SQL Thriftserver

own risk. Any and all responsibility for any loss, > damage or destruction of data or any other property which may arise from > relying on this email's technical content is explicitly disclaimed. The > author will in no case be liable for any monetary damages arising from such

Re: Spark SQL Thriftserver

uch loss, damage or destruction. On 13 September 2016 at 22:32, Benjamin Kim wrote: > Does anyone have any thoughts about using Spark SQL Thriftserver in Spark > 1.6.2 instead of HiveServer2? We are considering abando

Spark SQL Thriftserver

Does anyone have any thoughts about using Spark SQL Thriftserver in Spark 1.6.2 instead of HiveServer2? We are considering abandoning HiveServer2 for it. Some advice and gotcha’s would be nice to know. Thanks, Ben - To

Re: Spark SQL Thriftserver and Hive UDF in Production

ould be to create a Spark SQL UDF and launch the >> Spark SQL Thrift server programmatically. >> >> Mohammed >> >> -Original Message- >> From: ReeceRobinson [mailto:re...@therobinsons.gen.nz] >> Sent: Sunday, October 18, 2015 8:05 PM >> To: user

Re: Spark SQL Thriftserver and Hive UDF in Production

rammatically. > > Mohammed > > -Original Message- > From: ReeceRobinson [mailto:re...@therobinsons.gen.nz] > Sent: Sunday, October 18, 2015 8:05 PM > To: user@spark.apache.org > Subject: Spark SQL Thriftserver and Hive UDF in Production > > Does anyone have some advice on

RE: Spark SQL Thriftserver and Hive UDF in Production

, 2015 8:05 PM To: user@spark.apache.org Subject: Spark SQL Thriftserver and Hive UDF in Production Does anyone have some advice on the best way to deploy a Hive UDF for use with a Spark SQL Thriftserver where the client is Tableau using Simba ODBC Spark SQL driver. I have seen the hive

Spark SQL Thriftserver and Hive UDF in Production

Does anyone have some advice on the best way to deploy a Hive UDF for use with a Spark SQL Thriftserver where the client is Tableau using Simba ODBC Spark SQL driver. I have seen the hive documentation that provides an example of creating the function using a hive client ie: CREATE FUNCTION

Re: Spark SQL ThriftServer Impersonation Support

pache.org/jira/browse/SPARK-5159 > > I don't think it is yet supported until the HS2 code base is updated in > Spark hive-thriftserver project. > > -- > Date: Fri, 1 May 2015 15:56:30 +1000 > Subject: Spark SQL ThriftServer Impersonation Support

Spark SQL ThriftServer Impersonation Support

Hi guys, Trying to use the SparkSQL Thriftserver with hive metastore. It seems that hive meta impersonation works fine (when running Hive tasks). However spinning up SparkSQL thrift server, impersonation doesn't seem to work... What settings do I need to enable impersonation? I've copied the sa

Re: independent user sessions with a multi-user spark sql thriftserver (Spark 1.1)

at if you could share any findings on that JIRA. Thanks! On Mon, Nov 17, 2014 at 11:01 AM, Michael Allman wrote: > Hello, > > We're running a spark sql thriftserver that several users connect to with > beeline. One limitation we've run into is that the current working data

independent user sessions with a multi-user spark sql thriftserver (Spark 1.1)

Hello, We're running a spark sql thriftserver that several users connect to with beeline. One limitation we've run into is that the current working database (set with "use ") is shared across all connections. So changing the database on one connection changes the database