Error While Running Merge Statement With Iceberg

2024-07-30 Thread PRASHANT L
Hi I am trying to run merge statement with Iceberg with Spark I am getting below error THis is the statement MERGE INTO glue.euc_dp_prd.emp_merge_test AS t USING source AS s ON t.emp_id=s.emp_id WHEN MATCHED THEN UPDATE SET t.name=s.name , t.dept=s.dept , t.doj=s.doj pyspark.errors.exceptions.

A code change for spark ui in Sql tab

2024-07-30 Thread Donvi
Hi, Community I'm a new onboarder in the Spark community and find some lag between Spark and DBR in Spark UI. This is in DBR for cost based optimizer in Spark UI: https://docs.databricks.com/en/optimizations/cbo.html#spark-sql-ui. To implement similar thing in open source part, I've implmented t