Hello! Is there a way to set spark.sql.shuffle.partitions and spark.default.parallelism in Databricks? I checked the event log and can't find those parameters in the log either. Is it something that Databricks sets automatically?
Thanks, Erica
Hello! Is there a way to set spark.sql.shuffle.partitions and spark.default.parallelism in Databricks? I checked the event log and can't find those parameters in the log either. Is it something that Databricks sets automatically?
Thanks, Erica