Answering my own question. Global temp views get created in the global_temp
database, so can be accessed thusly.
Thanks
Dataset<Row> s = spark.read().parquet("/tmp/svampeatlas/*");
s.createOrReplaceGlobalTempView("occurrence_svampe");
spark.catalog().cacheTable("global_temp.occurrence_svampe");
On Sun, Feb 16, 2025 at 10:05 AM Tim Robertson <timrobertson...@gmail.com>
wrote:
> Hi folks
>
> Is it possible to cache a table for shared use across sessions with spark
> connect?
> I'd like to load a read only table once that many sessions will then
> query to improve performance.
>
> This is an example of the kind of thing that I have been trying, but have
> not succeeded with.
>
> SparkSession spark =
> SparkSession.builder().remote("sc://localhost").getOrCreate();
> Dataset<Row> s = spark.read().parquet("/tmp/svampeatlas/*");
>
> // this works if it is not "global"
> s.createOrReplaceGlobalTempView("occurrence_svampe");
> spark.catalog().cacheTable("occurrence_svampe");
>
> // this fails with a table not found when a global view is used
> spark
> .sql("SELECT * FROM occurrence_svampe")
> .write()
> .parquet("/tmp/export");
>
> Thank you
> Tim
>
