[
https://issues.apache.org/jira/browse/IMPALA-14702?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18061180#comment-18061180
]
Joe McDonnell commented on IMPALA-14702:
----------------------------------------
Add jemalloc / google tcmalloc to the native toolchain with these two commits:
{noformat}
commit 679f827fdbef42637a5c4daf7adda743518ebd2f
Author: Joe McDonnell <[email protected]>
Date: Fri May 16 18:57:18 2025 -0700 IMPALA-14702: Add google tcmalloc and
jemalloc builds
This adds a build of google tcmalloc from a custom branch
that adds a very basic CMake build and pulls in a patch
to let it use huge pages on a system that has THP configured
in madvise mode. This branch is maintained in a public
github repository here: https://github.com/joemcdonnell/tcmalloc.git
This also adds jemalloc 5.3.0. It does not build it with
profiling yet, but that can be added later as an extension.
Change-Id: I701dc9d5e532fcc126fe6b0f5b46f55079ff00e2
Reviewed-on: http://gerrit.cloudera.org:8080/24016
Reviewed-by: Michael Smith <[email protected]>
Tested-by: Joe McDonnell <[email protected]>
commit d01d9e83e7f9dd9ec698016a5f845922cddd8485
Author: Joe McDonnell <[email protected]>
Date: Thu May 29 16:42:31 2025 -0700 IMPALA-14702 (prep): Bump Abseil
version and only build static
Abseil had some issues in its CMake build that resulted in
flakiness building against it. This bumps Abseil to a more
recent version (20250512.2) with those CMake build fixes.
Moving to a newer Abseil breaks the 2024-03-01 re2 build.
Since we aren't using that re2 version currently, this bumps
that re2 build to 2025-11-05. (The regular re2 version that
doesn't depend on Abseil continues to stay the same.)
Google tcmalloc has a dependency on Abseil. Abseil has many
small libraries with dependencies on each other. Consuming
the CMake files in lib/cmake/* makes this much easier.
We currently do a static build and then a shared build, but
that means that the shared build overwrites the CMake files
in lib/cmake/* so that it always uses shared libraries. That
makes it harder to ship it in a Docker build. This removes
the shared library build so that Abseil is always a static
library.
Testing:
- Built google tcmalloc and verified that it uses static
libraries for Abseil
Change-Id: Ie8f4a89ba44994b06b24b8fdef2d054a965cbd31
Reviewed-on: http://gerrit.cloudera.org:8080/24015
Reviewed-by: Joe McDonnell <[email protected]>
Tested-by: Joe McDonnell <[email protected]>{noformat}
> Add support for building against other malloc implementations (e.g. jemalloc)
> -----------------------------------------------------------------------------
>
> Key: IMPALA-14702
> URL: https://issues.apache.org/jira/browse/IMPALA-14702
> Project: IMPALA
> Issue Type: Task
> Components: Backend
> Affects Versions: Impala 5.0.0
> Reporter: Joe McDonnell
> Priority: Major
>
> Impala builds against gperftools tcmalloc. Other malloc implementations like
> jemalloc and Google's tcmalloc have different designs that would be
> interesting to test. We know that there are performance improvements related
> to avoiding gperftool tcmalloc's aggressive decommit setting. It would be
> nice to test that against other malloc implementations (or forgo overriding
> malloc altogether).
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
