Re: Re: union two pyspark dataframes from different SparkSessions

2020-01-29 Thread Zong-han, Xie
Dear Yeikel I checked my code and it uses getOrCreate to create a SparkSession. Therefore, I should be retrieving the same SparkSession instance everytime I call that method. Thanks for your reminding. Best regard -- Sent from: http://apache-spark-user-list.1001560.n3.nabble.com/ ---

Re: union two pyspark dataframes from different SparkSessions

2020-01-29 Thread yeikel valdes
>From what I understand, the session is a singleton so even if you think you >are creating new instances you are just reusing it.  On Wed, 29 Jan 2020 02:24:05 -1100 icbm0...@gmail.com wrote Dear all I already had a python function which is used to query data from HBase and HDFS w