Re: [PR] Fix internal error in sort when hitting memory limit [datafusion]

2025-04-15 Thread via GitHub
alamb merged PR #15692: URL: https://github.com/apache/datafusion/pull/15692 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusi

Re: [PR] Fix internal error in sort when hitting memory limit [datafusion]

2025-04-14 Thread via GitHub
DerGut commented on code in PR #15692: URL: https://github.com/apache/datafusion/pull/15692#discussion_r2043010696 ## datafusion/physical-plan/src/sorts/sort.rs: ## @@ -759,12 +761,51 @@ impl ExternalSorter { if self.runtime.disk_manager.tmp_files_enabled() {

Re: [PR] Fix internal error in sort when hitting memory limit [datafusion]

2025-04-14 Thread via GitHub
DerGut commented on code in PR #15692: URL: https://github.com/apache/datafusion/pull/15692#discussion_r2041911140 ## datafusion/physical-plan/src/sorts/sort.rs: ## @@ -1552,6 +1593,62 @@ mod tests { Ok(()) } +#[tokio::test] +async fn test_batch_reservati

Re: [PR] Fix internal error in sort when hitting memory limit [datafusion]

2025-04-14 Thread via GitHub
DerGut commented on code in PR #15692: URL: https://github.com/apache/datafusion/pull/15692#discussion_r2041912243 ## datafusion/physical-plan/src/sorts/sort.rs: ## @@ -759,12 +761,51 @@ impl ExternalSorter { if self.runtime.disk_manager.tmp_files_enabled() {

Re: [PR] Fix internal error in sort when hitting memory limit [datafusion]

2025-04-14 Thread via GitHub
DerGut commented on code in PR #15692: URL: https://github.com/apache/datafusion/pull/15692#discussion_r2041254566 ## datafusion/physical-plan/src/sorts/sort.rs: ## @@ -759,12 +761,51 @@ impl ExternalSorter { if self.runtime.disk_manager.tmp_files_enabled() {

Re: [PR] Fix internal error in sort when hitting memory limit [datafusion]

2025-04-13 Thread via GitHub
DerGut commented on code in PR #15692: URL: https://github.com/apache/datafusion/pull/15692#discussion_r2041251862 ## datafusion/physical-plan/src/sorts/sort.rs: ## @@ -759,12 +761,51 @@ impl ExternalSorter { if self.runtime.disk_manager.tmp_files_enabled() {

Re: [PR] Fix internal error in sort when hitting memory limit [datafusion]

2025-04-13 Thread via GitHub
2010YOUY01 commented on code in PR #15692: URL: https://github.com/apache/datafusion/pull/15692#discussion_r2041385875 ## datafusion/physical-plan/src/sorts/sort.rs: ## @@ -765,6 +765,25 @@ impl ExternalSorter { Ok(()) } + +/// Reserves memory to be able to a

Re: [PR] Fix internal error in sort when hitting memory limit [datafusion]

2025-04-13 Thread via GitHub
2010YOUY01 commented on PR #15692: URL: https://github.com/apache/datafusion/pull/15692#issuecomment-2800424077 Thank you, it looks good to me 🙏🏼 Let's make the CI pass, I think we can change the assertion type for simplicity here, and do a separate PR for this utility `DataFusionError:

Re: [PR] Fix internal error in sort when hitting memory limit [datafusion]

2025-04-13 Thread via GitHub
2010YOUY01 commented on code in PR #15692: URL: https://github.com/apache/datafusion/pull/15692#discussion_r2041381039 ## datafusion/physical-plan/src/sorts/sort.rs: ## @@ -759,12 +761,51 @@ impl ExternalSorter { if self.runtime.disk_manager.tmp_files_enabled() {

Re: [PR] Fix internal error in sort when hitting memory limit [datafusion]

2025-04-13 Thread via GitHub
DerGut commented on code in PR #15692: URL: https://github.com/apache/datafusion/pull/15692#discussion_r2041234887 ## datafusion/physical-plan/src/sorts/sort.rs: ## @@ -765,6 +765,25 @@ impl ExternalSorter { Ok(()) } + +/// Reserves memory to be able to accom

Re: [PR] Fix internal error in sort when hitting memory limit [datafusion]

2025-04-13 Thread via GitHub
DerGut commented on code in PR #15692: URL: https://github.com/apache/datafusion/pull/15692#discussion_r2041251862 ## datafusion/physical-plan/src/sorts/sort.rs: ## @@ -759,12 +761,51 @@ impl ExternalSorter { if self.runtime.disk_manager.tmp_files_enabled() {

Re: [PR] Fix internal error in sort when hitting memory limit [datafusion]

2025-04-13 Thread via GitHub
DerGut commented on code in PR #15692: URL: https://github.com/apache/datafusion/pull/15692#discussion_r2041254566 ## datafusion/physical-plan/src/sorts/sort.rs: ## @@ -759,12 +761,51 @@ impl ExternalSorter { if self.runtime.disk_manager.tmp_files_enabled() {

Re: [PR] Fix internal error in sort when hitting memory limit [datafusion]

2025-04-13 Thread via GitHub
DerGut commented on code in PR #15692: URL: https://github.com/apache/datafusion/pull/15692#discussion_r2041251481 ## datafusion/physical-plan/src/sorts/sort.rs: ## @@ -1552,6 +1593,62 @@ mod tests { Ok(()) } +#[tokio::test] +async fn test_batch_reservati

Re: [PR] Fix internal error in sort when hitting memory limit [datafusion]

2025-04-13 Thread via GitHub
DerGut commented on code in PR #15692: URL: https://github.com/apache/datafusion/pull/15692#discussion_r2041251481 ## datafusion/physical-plan/src/sorts/sort.rs: ## @@ -1552,6 +1593,62 @@ mod tests { Ok(()) } +#[tokio::test] +async fn test_batch_reservati

Re: [PR] Fix internal error in sort when hitting memory limit [datafusion]

2025-04-13 Thread via GitHub
rluvaton commented on code in PR #15692: URL: https://github.com/apache/datafusion/pull/15692#discussion_r2041239380 ## datafusion/physical-plan/src/sorts/sort.rs: ## @@ -759,12 +761,51 @@ impl ExternalSorter { if self.runtime.disk_manager.tmp_files_enabled() {

Re: [PR] Fix internal error in sort when hitting memory limit [datafusion]

2025-04-13 Thread via GitHub
DerGut commented on code in PR #15692: URL: https://github.com/apache/datafusion/pull/15692#discussion_r2041234887 ## datafusion/physical-plan/src/sorts/sort.rs: ## @@ -765,6 +765,25 @@ impl ExternalSorter { Ok(()) } + +/// Reserves memory to be able to accom

Re: [PR] Fix internal error in sort when hitting memory limit [datafusion]

2025-04-13 Thread via GitHub
DerGut commented on code in PR #15692: URL: https://github.com/apache/datafusion/pull/15692#discussion_r2041222385 ## datafusion/physical-plan/src/sorts/sort.rs: ## @@ -765,6 +765,25 @@ impl ExternalSorter { Ok(()) } + +/// Reserves memory to be able to accom

Re: [PR] Fix internal error in sort when hitting memory limit [datafusion]

2025-04-13 Thread via GitHub
DerGut commented on code in PR #15692: URL: https://github.com/apache/datafusion/pull/15692#discussion_r2041181467 ## datafusion/physical-plan/src/sorts/sort.rs: ## @@ -529,6 +523,12 @@ impl ExternalSorter { /// Sorts the in-memory batches and merges them into a single sort

Re: [PR] Fix internal error in sort when hitting memory limit [datafusion]

2025-04-13 Thread via GitHub
DerGut commented on code in PR #15692: URL: https://github.com/apache/datafusion/pull/15692#discussion_r2041181467 ## datafusion/physical-plan/src/sorts/sort.rs: ## @@ -529,6 +523,12 @@ impl ExternalSorter { /// Sorts the in-memory batches and merges them into a single sort

Re: [PR] Fix internal error in sort when hitting memory limit [datafusion]

2025-04-12 Thread via GitHub
2010YOUY01 commented on code in PR #15692: URL: https://github.com/apache/datafusion/pull/15692#discussion_r2040878250 ## datafusion/physical-plan/src/sorts/sort.rs: ## @@ -529,6 +523,12 @@ impl ExternalSorter { /// Sorts the in-memory batches and merges them into a single

Re: [PR] Fix internal error in sort when hitting memory limit [datafusion]

2025-04-12 Thread via GitHub
DerGut commented on code in PR #15692: URL: https://github.com/apache/datafusion/pull/15692#discussion_r2040700866 ## datafusion/physical-plan/src/sorts/sort.rs: ## @@ -765,6 +765,25 @@ impl ExternalSorter { Ok(()) } + +/// Reserves memory to be able to accom

Re: [PR] Fix internal error in sort when hitting memory limit [datafusion]

2025-04-12 Thread via GitHub
DerGut commented on code in PR #15692: URL: https://github.com/apache/datafusion/pull/15692#discussion_r2040698187 ## datafusion/physical-plan/src/sorts/sort.rs: ## @@ -529,6 +523,12 @@ impl ExternalSorter { /// Sorts the in-memory batches and merges them into a single sort