Hi We are loading the data in spark dataset We don't do any repartitioning on the loaded dataset
If we want to load the in the iceberg partitioned table . If we do dataset.sortwithinpartition() before doing append or merge or overwrite. Will that be the best practice shall I go ahead? Or if I go with dataset.repartition on the partition column and then do the dataset.sortwithinpartitions() will that be the good practice. Kindly let me know
