Great, thanks for letting us know it must have been a Spark bug. I think
that tracks with what Amogh's testing showed. He wasn't able to reproduce
it in 3.4.x, 3.5.x, or the latest in the 3.3.x line, only in 3.3.0.
On Wed, Dec 20, 2023 at 9:04 AM Harshith Bolar
wrote:
> There was an issue in my
Hi Ryan,
We have verified that the issue exists even after upgrading to Spark 3.5
and Iceberg 1.4.2. And explicitly casting the columns to String doesn't
seem to help either. The error goes away when we downgrade to Iceberg 1.0.0
and reappears in 1.1.0. So, we're thinking a bug was possibly introd
Thanks for reaching out about this. It does look like a bug somewhere. I'm
curious whether it works in Spark 3.5 if you have a chance to try it out.
It looks like the cause is a missing cast or incorrect schema somewhere in
Spark. The data coming in is a Spark string, but for some reason Spark
thi
