comphead commented on code in PR #11218:
URL: https://github.com/apache/datafusion/pull/11218#discussion_r1676409356
##########
datafusion/core/tests/memory_limit/mod.rs:
##########
@@ -182,6 +182,24 @@ async fn merge_join() {
.await
}
+#[tokio::test]
+async fn sort_merge_join_spill() {
+ // Planner chooses MergeJoin only if number of partitions > 1
+ let config = SessionConfig::new()
+ .with_target_partitions(2)
+ .set_bool("datafusion.optimizer.prefer_hash_join", false);
+
+ TestCase::new()
+ .with_query(
+ "select t1.* from t t1 JOIN t t2 ON t1.pod = t2.pod AND t1.time =
t2.time",
+ )
+ .with_memory_limit(1_000)
+ .with_config(config)
+ .with_disk_manager_config(DiskManagerConfig::NewOs)
+ .run()
+ .await
Review Comment:
Yeah, that is great idea, I was overthinking how to check that file spilled
to disk but metrics is much easier, I'm adding it
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
