zclllyybb commented on issue #64106: URL: https://github.com/apache/doris/issues/64106#issuecomment-4619071402
Breakwater-GitHub-Analysis-Slot: slot_fd2c101f1810 Initial triage: this looks valid and actionable for Doris, even though the issue does not include a Doris-specific benchmark yet. What I checked: - Current upstream `master` still downloads `jemalloc-5.3.0` for both `JEMALLOC_DORIS_*` and `JEMALLOC_ARROW_*` in `thirdparty/vars.sh`. - BE startup/config defaults use a non-disabled muzzy decay setting: `muzzy_decay_ms:5000` in `conf/be.conf` and the fallback `JEMALLOC_CONF` in `bin/start_be.sh`. - That means the reported upstream jemalloc behavior is on Doris' default BE allocator path, not just an unused dependency. - The upstream jemalloc 5.3.1 release is available and includes bug fixes/performance optimizations: https://github.com/jemalloc/jemalloc/releases/tag/5.3.1 - The same `muzzy_decay_ms` regression was discussed in Polars' update PR, where 5.3.1 restored performance for enabled `muzzy_decay_ms`: https://github.com/pola-rs/polars/pull/27797 Suggested fix scope for a PR: - Update `JEMALLOC_DORIS_DOWNLOAD`, `JEMALLOC_DORIS_NAME`, `JEMALLOC_DORIS_SOURCE`, and `JEMALLOC_DORIS_MD5SUM` from 5.3.0 to 5.3.1 in `thirdparty/vars.sh`. - Also update the `JEMALLOC_ARROW_*` archive entries for consistency, since they are still listed in the thirdparty archive list. - The 5.3.1 tarball MD5 I get is `2b183326fcff2c087efd7efc0a808ba2`. - Check `thirdparty/download-thirdparty.sh`: it currently applies `jemalloc_hook.patch` only when `JEMALLOC_DORIS_SOURCE` is exactly `jemalloc-5.3.0`. A 5.3.1 PR should extend or relax that condition; the existing patch dry-runs successfully against the 5.3.1 source with only fuzz. - Update release/license metadata that still names jemalloc 5.3.0, for example `dist/LICENSE-dist.txt`, and add a thirdparty changelog entry if that is the local convention. Missing evidence for impact sizing: - No Doris workload, profile, BE logs, or before/after benchmark is attached, so I would not claim a measured Doris-side 7x regression from this issue alone. - Useful validation would be a BE build with the upgraded thirdparty, a startup smoke test with the default `JEMALLOC_CONF`, and either a memory-allocation-heavy query/load benchmark or a small allocator microbenchmark comparing `muzzy_decay_ms:5000` before and after the upgrade. Conclusion: this should be treated as a dependency/performance bug report. Upgrading Doris' bundled jemalloc from 5.3.0 to 5.3.1 is the right next step, with the patch gate and metadata updates included in the same PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
