UBarney commented on PR #16814: URL: https://github.com/apache/datafusion/pull/16814#issuecomment-3136567712
> Another advantage of this external profiling approach is that we can compare different memory allocators side by side (e.g., `jemalloc` and `mimalloc`). From what I can see, the new utility in this PR is designed to get RSS statistics specifically from binaries using `mimalloc`. If that's the case, we wouldn't be able to use it to perform a side-by-side RSS comparison between different memory allocators (like jemalloc and mimalloc). This [Python-based approach](https://gist.github.com/UBarney/7416e95841fc9c29b114c052ebf56a16#file-draft_get_mem_usage-py) was capable of this. It also seems we can already compare the timing of different allocators with the bench.sh script by setting this environment variable https://github.com/apache/datafusion/blob/483444af62d9458de0c4ff54f8283994a37bb398/benchmarks/bench.sh#L43 (e.g. `CARGO_COMMAND='cargo run --no-default-features --features snmalloc' ./benchmarks/bench.sh run tpch10`) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
