PS - issue doesn't happen if we don't use spark-connect and instead just
use spark-shell or pyspark as OP in github said as well. however stacktrace
desont seem to point any of the class from spark-connect jar
(org.apache.spark:spark-connect_2.12:3.5.0).
On Tue, Jan 9, 2024 at 4:52 PM Nirav Patel
Hi,
We are testing spark-connect with iceberg.
We tried spark 3.5, iceberg 1.4.x versions (all of
iceberg-spark-runtime-3.5_2.12-1.4.x.jar)
with all the 1.4.x jars we are having following issue when running iceberg
queries from sparkSession created using spark-connect (--remote
"sc://remote-master
It makes sense. Agree.
Regards
JB
On Mon, Jan 8, 2024 at 11:48 PM Daniel Weeks wrote:
>
> JB,
>
> I would draw a distinction between catalog and this proposed feature in that
> the catalog is actually not part of the spec, so it is entirely up to the
> engine and is optional.
>
> When it comes
