[
https://issues.apache.org/jira/browse/HDDS-17155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Siyao Meng updated HDDS-17155:
------------------------------
Status: Patch Available (was: Open)
> NativeLibraryLoader extracts libozone_rocksdb_tools.so into the working
> directory instead of java.io.tmpdir
> -----------------------------------------------------------------------------------------------------------
>
> Key: HDDS-17155
> URL: https://issues.apache.org/jira/browse/HDDS-17155
> Project: Apache Ozone
> Issue Type: Bug
> Components: Native
> Reporter: Siyao Meng
> Assignee: Siyao Meng
> Priority: Major
> Labels: pull-request-available
> Attachments: HDDS-17155.001.patch
>
>
> h3. Problem
> {{NativeLibraryLoader}} loads {{libozone_rocksdb_tools.so}} in two steps:
> # It tries {{System.loadLibrary}} from {{java.library.path}}. The
> distribution's {{lib/native}} only contains {{libhadoop}}, so this misses.
> # It extracts the library from {{hdds-rocks-native-*.jar}} into
> {{native.lib.tmp.dir}}. If that property is not set, it uses {{new
> File("").getAbsoluteFile()}}, the *current working directory*.
> No Ozone script sets {{native.lib.tmp.dir}}. Whenever the process cannot
> write to its working directory, extraction fails and the library is not
> loaded:
> {code}
> WARN utils.NativeLibraryLoader: Unable to load library: ozone_rocksdb_tools
> java.nio.file.AccessDeniedException:
> /opt/hadoop/ozone_rocksdb_tools7373108702252003876
> WARN snapshot.SnapshotDiffManager: Native Library for raw sst file reading
> loading failed. Fallback to performing a full diff instead. ...
> {code}
> This was hit in the compose runner image: the OM runs in {{/opt/hadoop}}, the
> mounted distribution, and that directory was not writable by the container
> user. The fallback is silent apart from the WARN, so snapshot diff quietly
> loses the native path. {{ozone debug checknative}} then reports
> {{rocks-tools: false}} for this reason, which would also mask the libstdc++
> problem in HDDS-17153.
> For comparison:
> * Hadoop's {{NativeCodeLoader}} only uses {{System.loadLibrary}} from
> {{lib/native}} and never extracts.
> * RocksJava's own {{NativeLibraryLoader}} extracts {{librocksdbjni}} into
> {{java.io.tmpdir}} by default ({{File.createTempFile}}), or into
> {{ROCKSDB_SHAREDLIB_DIR}}.
> h3. Fix
> When {{native.lib.tmp.dir}} is not set (or empty), create the extraction
> directory with {{Files.createTempDirectory(libraryName)}}, i.e. under
> {{java.io.tmpdir}}. An explicit {{native.lib.tmp.dir}} behaves as before.
> Cleanup is unchanged: the existing shutdown hook deletes the directory.
> h3. Testing done
> * {{TestNativeLibraryLoader#testDummyLibrary}} now expects the library under
> {{java.io.tmpdir}} when the property is {{null}} or empty. Previously it
> expected the working directory.
> ** Without the fix, those two cases fail ({{Expecting actual not to be
> empty}}). With the fix, they pass.
> ** The explicit-directory case still passes.
> * Native build ({{-Drocks_tools_native}}) on master: TestNativeLibraryLoader
> (6, including the real {{ozone_rocksdb_tools}} load),
> TestManagedRawSSTFileIterator (28), TestLatestVersionedKWayMergeIterator (10)
> and TestLatestVersionedKWayMergeIteratorOverSst (2) all pass. Nothing was
> extracted into the module directory.
> * Checkstyle is clean.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
