dmitry-chirkov-dremio opened a new issue, #55281: URL: https://github.com/apache/arrow/issues/55281
### Describe the bug, including details regarding any error messages, version, and platform. Gandiva releases the arena after each row only if some function returns a string. `has_arena_allocs_` is set only for binary-like return types in `BuildFunctionCall`. `levenshtein` returns int32 but uses arena memory: `(shorter_len + 1) * 2 * sizeof(int)` bytes per row. In `levenshtein(f0, f1)` this memory is kept until `Evaluate` returns. In `levenshtein(upper(f0), upper(f1))` it is released after each row. To reproduce: 1. Make 262,144 rows, two utf8 columns, 16 chars each. 2. Build a projector with `levenshtein(f0, f1)`. 3. Call `Evaluate`. Compare `arrow::default_memory_pool()->total_bytes_allocated()` before and after. Expected: about 1 MB (the output buffers and one arena chunk). Actual: 36,876,288 bytes. Memory grows with rows x string length. Results are correct. No crash. Bigger batch, 2,097,152 rows of 64 chars: 1,178 MiB from the arena, peak RSS 1.6 GB. AI tools helped with the investigation. ### Component(s) C++, Gandiva -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
