Re: Spark 3.2 - ReusedExchange not present in join execution plan

2022-01-06 Thread Albert
I happen to encounter something similar. it's probably because you are just `explain` it. when you actually `run` it. you will get the final spark plan in which case the exchange will be reused. right, this is different compared with 3.1 probably because the upgraded aqe. not sure whether this is

what's the potential security issue of reusing a spark session for multi users

2021-06-30 Thread Albert
Spark Devs, what's the potential security issue to reuse one spark session for running spark-hive queries, spark jobs ? suppose the session usage is isolated to different time chunks , and some clean up is done before each usage (for example hive cache is recreated before each usage). thanks.