Xavier Fernandis created HBASE-30472:
----------------------------------------
Summary: [hbase-thirdparty] Make netty/netty-tcnative
native-library relocation self-maintaining with a glob mapper
Key: HBASE-30472
URL: https://issues.apache.org/jira/browse/HBASE-30472
Project: HBase
Issue Type: Improvement
Components: thirdparty
Affects Versions: thirdparty-4.1.14
Reporter: Xavier Fernandis
Assignee: Xavier Fernandis
Fix For: thirdparty-4.1.15
h2. Summary
currently hbase-shaded-netty and hbase-shaded-netty-tcnative relocate netty
from io.netty to org.apache.hbase.thirdparty.io.netty. Netty's
NativeLibraryLoader derives the native library filename from the (relocated)
package of its own loader class at runtime, so the native .so/.jnilib/.dll
files bundled under META-INF/native/ must be physically renamed from
libnetty_*/netty_* to the shaded
liborg_apache_hbase_thirdparty_netty_*/org_apache_hbase_thirdparty_netty_* form
to match. Today this rename is done with a hard-coded list of
maven-antrun-plugin <move> entries one line per known native filename.
current state of manually renaming :
netty :
[https://github.com/apache/hbase-thirdparty/blob/master/hbase-shaded-netty/pom.xml#L130]
tcnative :
[https://github.com/apache/hbase-thirdparty/blob/master/hbase-shaded-netty-tcnative/pom.xml#L131]
h2. *Issue :*
The hard-coded list is not self-maintaining. It enumerates specific filenames,
so any native library netty adds or renames in a later release is silently left
un-relocated unless a committer remembers to add a matching <move> entry in
lockstep. This couples the pom to netty's internal, version-dependent set of
native artifacts.
||Native library||Relocated today?||
|{{netty_transport_native_epoll_x86_64.so}}|yes|
|{{netty_transport_native_epoll_aarch_64.so}}|yes|
|{{netty_transport_native_kqueue_x86_64.jnilib}}|yes|
|{{netty_resolver_dns_native_macos_x86_64.jnilib}}|yes|
|{{netty_transport_native_epoll_riscv64.so}}|no|
|{{netty_transport_native_kqueue_aarch_64.jnilib}}|no|
|{{netty_resolver_dns_native_macos_aarch_64.jnilib}}|no|
||Native library||Relocated today?||
|{{netty_tcnative_linux_x86_64.so}}|yes|
|{{netty_tcnative_linux_aarch_64.so}}|yes|
|{{netty_tcnative_osx_x86_64.jnilib}}|yes|
|{{netty_tcnative_osx_aarch_64.jnilib}}|yes|
|{{netty_tcnative_windows_x86_64.dll}}|no|
An un-relocated native keeps its original io.netty name, which no longer
matches the shaded name NativeLibraryLoader computes for the relocated classes
so that platform cannot load it.
Even where HBase does not request these natives today, the glob approach is
future-proof: the moment HBase enables a transport/arch it currently doesn't
(or netty adds a new native in an upgrade), the relocation covers it
automatically with no pom change.
h2. proposed change
Replace the hard-coded <move> entries with Ant <move> driven by a <fileset> and
<globmapper>, relocating whatever native libraries are present at build time:
{code:java}
<!-- Unix/macOS libs carry a 'lib' prefix: libnetty_* ->
liborg_apache_hbase_thirdparty_netty_* -->
<move todir="${project.build.directory}/unpacked/META-INF/native">
<fileset dir="${project.build.directory}/unpacked/META-INF/native">
<include name="libnetty_*"/>
</fileset>
<globmapper from="libnetty_*" to="liborg_apache_hbase_thirdparty_netty_*"/>
</move>{code}
For tcnative, a second mapper additionally handles the Windows DLL (no lib
prefix): netty_* -> org_apache_hbase_thirdparty_netty_*. An empty fileset is a
no-op, and the native artifacts are bundled in the dependency jars (not host-OS
dependent), so the step is correct on any build platform.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
