Thanks Jerry for the clarification. Ajay.
On Thu, Jul 11, 2019 at 12:48 PM Jerry Vinokurov <grapesmo...@gmail.com> wrote: > Hi Ajay, > > When a Spark SQL statement references a table, that table has to be > "registered" first. Usually the way this is done is by reading in a > DataFrame, then calling the createOrReplaceTempView (or one of a few other > functions) on that data frame, with the argument being the name under which > you'd like to register that table. You can then use the table in SQL > statements. As far as I know, you cannot directly refer to any external > data store without reading it in first. > > Jerry > > On Thu, Jul 11, 2019 at 1:27 PM infa elance <infa.ela...@gmail.com> wrote: > >> Sorry, i guess i hit the send button too soon.... >> >> This question is regarding a spark stand-alone cluster. My understanding >> is spark is an execution engine and not a storage layer. >> Spark processes data in memory but when someone refers to a spark table >> created through sparksql(df/rdd) what exactly are they referring to? >> >> Could it be a Hive table? If yes, is it the same hive store that spark >> uses? >> Is it a table in memory? If yes, how can an external app access this >> in-memory table? if JDBC what driver ? >> >> On a databricks cluster -- could they be referring spark table created >> through sparksql(df/rdd) as hive or deltalake table? >> >> Spark version with hadoop : spark-2.0.2-bin-hadoop2.7 >> >> Thanks and appreciate your help!! >> Ajay. >> >> >> >> On Thu, Jul 11, 2019 at 12:19 PM infa elance <infa.ela...@gmail.com> >> wrote: >> >>> This is stand-alone spark cluster. My understanding is spark is an >>> execution engine and not a storage layer. >>> Spark processes data in memory but when someone refers to a spark table >>> created through sparksql(df/rdd) what exactly are they referring to? >>> >>> Could it be a Hive table? If yes, is it the same hive store that spark >>> uses? >>> Is it a table in memory? If yes, how can an external app >>> >>> Spark version with hadoop : spark-2.0.2-bin-hadoop2.7 >>> >>> Thanks and appreciate your help!! >>> Ajay. >>> >> > > -- > http://www.google.com/profiles/grapesmoker >
