Hi all I've noticed following SO question about Spark 3.1.1 not working with Delta 0.7.0: https://stackoverflow.com/questions/66106096/delta-lake-insert-into-sql-in-pyspark-is-failing-with-java-lang-nosuchmethoder/66106800#66106800 - I checked with Delta 0.8.0 and it has the same problem. It brokes here: https://github.com/delta-io/delta/blob/v0.8.0/src/main/scala/org/apache/spark/sql/delta/DeltaAnalysis.scala#L204
I'm thinking that it's caused by following change: https://github.com/apache/spark/commit/a082f4600b1cb814442beed1b578bc3430a257a7#diff-cf96171d13fd77e670764766ae22afafbc4a396316bd758a89b60a6fe70d5b0dL150, but not 100% sure. If it's, then maybe we could add the backward compatible constructor for this case class? P.S. It's not the first time when 3.1 breaks compatibility with existing connectors, for example, Spark Cassandra Connector on 3.1 doesn't work without changes: https://github.com/datastax/spark-cassandra-connector/pull/1280 -- With best wishes, Alex Ott http://alexott.net/ Twitter: alexott_en (English), alexott (Russian)
