Great, thanks for letting us know it must have been a Spark bug. I think
that tracks with what Amogh's testing showed. He wasn't able to reproduce
it in 3.4.x, 3.5.x, or the latest in the 3.3.x line, only in 3.3.0.
On Wed, Dec 20, 2023 at 9:04 AM Harshith Bolar
wrote:
> There was an issue in my
Hi Ryan,
We have verified that the issue exists even after upgrading to Spark 3.5
and Iceberg 1.4.2. And explicitly casting the columns to String doesn't
seem to help either. The error goes away when we downgrade to Iceberg 1.0.0
and reappears in 1.1.0. So, we're thinking a bug was possibly introd
Thanks for reaching out about this. It does look like a bug somewhere. I'm
curious whether it works in Spark 3.5 if you have a chance to try it out.
It looks like the cause is a missing cast or incorrect schema somewhere in
Spark. The data coming in is a Spark string, but for some reason Spark
thi
Hey Folks,
My team members and me have been stuck in this issue for a long time.
Can you please check this issue once.
With the snippet given in the following question -
https://stackoverflow.com/questions/77655115/org-apache-spark-unsafe-types-utf8string-cannot-be-cast-to-java-lang-long-except
yo
