Re: [PR] fix: Incorrect memory accounting in `array_agg` function [datafusion]

2025-06-30 Thread via GitHub
sfluor commented on code in PR #16519: URL: https://github.com/apache/datafusion/pull/16519#discussion_r2174894499 ## datafusion/functions-aggregate/src/array_agg.rs: ## @@ -341,12 +341,20 @@ impl Accumulator for ArrayAggAccumulator { Some(values) => {

Re: [PR] fix: Incorrect memory accounting in `array_agg` function [datafusion]

2025-06-25 Thread via GitHub
alamb commented on PR #16519: URL: https://github.com/apache/datafusion/pull/16519#issuecomment-3005370005 There appears to be an array_agg benchmark -- I will run that on this PR to see what it shows -- This is an automated message from the Apache Git Service. To respond to the message,

Re: [PR] fix: Incorrect memory accounting in `array_agg` function [datafusion]

2025-06-25 Thread via GitHub
alamb commented on PR #16519: URL: https://github.com/apache/datafusion/pull/16519#issuecomment-3005374575 🤖 `./gh_compare_branch.sh` [Benchmark Script](https://github.com/alamb/datafusion-benchmarking/blob/main/gh_compare_branch.sh) Running Linux aal-dev 6.11.0-1015-gcp #15~24.04.1-Ubun

Re: [PR] fix: Incorrect memory accounting in `array_agg` function [datafusion]

2025-06-25 Thread via GitHub
alamb commented on PR #16519: URL: https://github.com/apache/datafusion/pull/16519#issuecomment-3005932928 🤖: Benchmark completed Details ``` group main sa

Re: [PR] fix: Incorrect memory accounting in `array_agg` function [datafusion]

2025-06-25 Thread via GitHub
alamb commented on PR #16519: URL: https://github.com/apache/datafusion/pull/16519#issuecomment-3005913242 🤖 `./gh_compare_branch_bench.sh` [Benchmark Script](https://github.com/alamb/datafusion-benchmarking/blob/main/gh_compare_branch_bench.sh) Running Linux aal-dev 6.11.0-1015-gcp #15~

Re: [PR] fix: Incorrect memory accounting in `array_agg` function [datafusion]

2025-06-25 Thread via GitHub
alamb commented on PR #16519: URL: https://github.com/apache/datafusion/pull/16519#issuecomment-3005911800 🤖 `./gh_compare_branch_bench.sh` [Benchmark Script](https://github.com/alamb/datafusion-benchmarking/blob/main/gh_compare_branch_bench.sh) Running Linux aal-dev 6.11.0-1015-gcp #15~

Re: [PR] fix: Incorrect memory accounting in `array_agg` function [datafusion]

2025-06-25 Thread via GitHub
alamb commented on PR #16519: URL: https://github.com/apache/datafusion/pull/16519#issuecomment-3005910021 🤖 `./gh_compare_branch.sh` [Benchmark Script](https://github.com/alamb/datafusion-benchmarking/blob/main/gh_compare_branch.sh) Running Linux aal-dev 6.11.0-1015-gcp #15~24.04.1-Ubun

Re: [PR] fix: Incorrect memory accounting in `array_agg` function [datafusion]

2025-06-25 Thread via GitHub
gabotechs commented on PR #16519: URL: https://github.com/apache/datafusion/pull/16519#issuecomment-3004986202 EDIT: I'm seeing that there's cases where the `merge_batch` method is used for something other than merging states: https://github.com/apache/datafusion/blob/9278233e9fe34f7

Re: [PR] fix: Incorrect memory accounting in `array_agg` function [datafusion]

2025-06-25 Thread via GitHub
alamb commented on PR #16519: URL: https://github.com/apache/datafusion/pull/16519#issuecomment-3005367832 Thanks for the code and review @gabotechs and @sfluor -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] fix: Incorrect memory accounting in `array_agg` function [datafusion]

2025-06-25 Thread via GitHub
alamb commented on code in PR #16519: URL: https://github.com/apache/datafusion/pull/16519#discussion_r2167112157 ## datafusion/functions-aggregate/src/array_agg.rs: ## @@ -341,12 +341,20 @@ impl Accumulator for ArrayAggAccumulator { Some(values) => {

Re: [PR] fix: Incorrect memory accounting in `array_agg` function [datafusion]

2025-06-25 Thread via GitHub
gabotechs commented on code in PR #16519: URL: https://github.com/apache/datafusion/pull/16519#discussion_r2166863158 ## datafusion/functions-aggregate/src/array_agg.rs: ## @@ -341,12 +341,20 @@ impl Accumulator for ArrayAggAccumulator { Some(values) => {

Re: [PR] fix: Incorrect memory accounting in `array_agg` function [datafusion]

2025-06-25 Thread via GitHub
gabotechs commented on code in PR #16519: URL: https://github.com/apache/datafusion/pull/16519#discussion_r2165952424 ## datafusion/functions-aggregate/src/array_agg.rs: ## @@ -994,6 +1002,34 @@ mod tests { Ok(()) } +#[test] +fn does_not_over_account_memo

Re: [PR] fix: Incorrect memory accounting in `array_agg` function [datafusion]

2025-06-25 Thread via GitHub
gabotechs commented on code in PR #16519: URL: https://github.com/apache/datafusion/pull/16519#discussion_r2165952424 ## datafusion/functions-aggregate/src/array_agg.rs: ## @@ -994,6 +1002,34 @@ mod tests { Ok(()) } +#[test] +fn does_not_over_account_memo

Re: [PR] fix: Incorrect memory accounting in `array_agg` function [datafusion]

2025-06-25 Thread via GitHub
gabotechs commented on code in PR #16519: URL: https://github.com/apache/datafusion/pull/16519#discussion_r2165946809 ## datafusion/functions-aggregate/src/array_agg.rs: ## @@ -341,12 +341,20 @@ impl Accumulator for ArrayAggAccumulator { Some(values) => {

[PR] fix: Incorrect memory accounting in `array_agg` function [datafusion]

2025-06-23 Thread via GitHub
sfluor opened a new pull request, #16519: URL: https://github.com/apache/datafusion/pull/16519 ## Which issue does this PR close? See this issue: https://github.com/apache/datafusion/issues/16517 - Closes #. ## Rationale for this change ## What chan