timosachsenberg opened a new issue, #51329:
URL: https://github.com/apache/arrow/issues/51329
### Describe the bug, including details regarding any error messages,
version, and platform.
## Summary
`libarrow_bundled_dependencies.a` contains `azure-storage-common`'s
`xml_wrapper.cpp.o`, which references 23 libxml2 symbols. Arrow's exported
CMake interface for the static targets names the bundle's other external
requirements but not libxml2, so consumers who link `Arrow::arrow_static` are
not told to link libxml2, and the link either fails outright or silently
produces a library with unresolved `xml*` symbols.
The same package's **shared** library declares the dependency correctly,
which is what makes this look like an oversight in the static packaging rather
than a deliberate choice.
## Version and platform
- Arrow 24.0.0 (`libarrow-dev` / `libparquet-dev` 24.0.0-1) from the Apache
APT repository
- Ubuntu 24.04.4 x86_64, GCC 13.4.0, GNU ld 2.42 with the distribution
default `--as-needed`
- Still present on `main`: `cpp/src/arrow/CMakeLists.txt` has no
`ARROW_AZURE` branch populating `ARROW_STATIC_INSTALL_INTERFACE_LIBS`, and
`grep -i libxml cpp/cmake_modules/ThirdpartyToolchain.cmake` returns nothing.
## The 23 symbols, and the single object that needs them
```console
$ cd /usr/lib/x86_64-linux-gnu
$ nm -A --undefined-only libarrow_bundled_dependencies.a | grep -E " U
xml[A-Z]" | head -3
libarrow_bundled_dependencies.a:xml_wrapper.cpp.o: U
xmlBufferCreate
libarrow_bundled_dependencies.a:xml_wrapper.cpp.o: U
xmlBufferFree
libarrow_bundled_dependencies.a:xml_wrapper.cpp.o: U
xmlCleanupParser
```
All 23 are in that one object: `xmlBufferCreate`, `xmlBufferFree`,
`xmlCleanupParser`, `xmlFreeTextReader`, `xmlFreeTextWriter`, `xmlInitParser`,
`xmlNewTextWriterMemory`, `xmlReaderForMemory`, `xmlTextReaderConstName`,
`xmlTextReaderConstValue`, `xmlTextReaderHasAttributes`,
`xmlTextReaderHasValue`, `xmlTextReaderIsEmptyElement`,
`xmlTextReaderMoveToNextAttribute`, `xmlTextReaderNodeType`,
`xmlTextReaderRead`, `xmlTextWriterEndDocument`, `xmlTextWriterEndElement`,
`xmlTextWriterStartDocument`, `xmlTextWriterStartElement`,
`xmlTextWriterWriteAttribute`, `xmlTextWriterWriteElement`,
`xmlTextWriterWriteString`.
Its owner is the Azure SDK:
```console
$ ar x libarrow_bundled_dependencies.a xml_wrapper.cpp.o
$ nm -C --defined-only --extern-only xml_wrapper.cpp.o
T Azure::Storage::_internal::XmlReader::XmlReader(char const*, unsigned long)
T Azure::Storage::_internal::XmlReader::Read()
T Azure::Storage::_internal::XmlWriter::GetDocument[abi:cxx11]()
W Azure::Storage::_internal::XmlGlobalInitializer::~XmlGlobalInitializer()
...
```
Not the AWS SDK, which is bundled too and contributes its own
`xml_parser.c.o` — that one is aws-c-common's parser (`aws_xml_node_*`,
`aws_xml_parse`) and has zero undefined `xml[A-Z]` symbols.
## Arrow's own code does not use libxml2
Counting undefined `xml[A-Z]` symbols in the static archives shipped by
`libarrow-dev` / `libparquet-dev`:
```
libarrow.a 0
libparquet.a 0
libarrow_bundled_dependencies.a 23
```
## Why this looks like a packaging inconsistency
The **shared** library from the very same package declares libxml2 itself:
```console
$ nm -D --undefined-only libarrow.so.2400.0.0 | grep -cE "xml[A-Z]"
23
$ readelf -d libarrow.so.2400.0.0 | grep NEEDED | grep -i xml
0x0000000000000001 (NEEDED) Shared library: [libxml2.so.2]
```
Same 23 symbols, dependency recorded. The static export, however, lists the
bundle's other external requirements and omits only libxml2:
```cmake
# /usr/lib/x86_64-linux-gnu/cmake/Arrow/ArrowTargets.cmake:80-84
set_target_properties(Arrow::arrow_static PROPERTIES
INTERFACE_LINK_LIBRARIES
"Arrow::arrow_bundled_dependencies;OpenSSL::Crypto;OpenSSL::SSL;
Brotli::brotlienc;Brotli::brotlidec;Brotli::brotlicommon;BZip2::BZip2;LZ4::lz4;
Snappy::snappy;ZLIB::ZLIB;zstd::libzstd_shared;CURL::libcurl;utf8proc::utf8proc;
re2::re2;protobuf::libprotobuf;Threads::Threads;dl;rt")
```
`grep -ril libxml` over `cmake/Arrow*` and `cmake/Parquet*` returns nothing.
The pkg-config side has the same gap — `arrow.pc`'s `Libs.private` is
`-larrow_bundled_dependencies /usr/lib/x86_64-linux-gnu/libbz2.so -ldl` — which
is the same family as #15139.
## The same file already establishes the pattern — for the AWS SDK
On `main`, `cpp/src/arrow/CMakeLists.txt` populates
`ARROW_STATIC_INSTALL_INTERFACE_LIBS` from fifteen `if(<dep>_SOURCE STREQUAL
"SYSTEM")` branches and exactly **one** `BUNDLED` branch:
```cmake
elseif(AWSSDK_SOURCE STREQUAL "BUNDLED")
if(UNIX)
list(APPEND ARROW_STATIC_INSTALL_INTERFACE_LIBS CURL::libcurl)
endif()
endif()
```
That is precisely the case at hand: a vendored third party goes into
`libarrow_bundled_dependencies.a`, so nothing about *it* needs declaring — but
*its own external requirements* do, and are declared. The bundled AWS SDK gets
that treatment for curl. The bundled Azure SDK never got it, for libxml2 or
curl. There is no `ARROW_AZURE` block in that list at all, and `grep -i libxml
cpp/cmake_modules/ThirdpartyToolchain.cmake` on `main` returns nothing, so
libxml2 is never even resolved as a dependency.
So this is not an open design question — it is a missing instance of a
pattern that already exists one `elseif` away in the same file.
## How it surfaces for consumers
In OpenMS we link `Arrow::arrow_static`. Because Arrow does not declare
libxml2, we added it as a direct dependency of our own library — but CMake
emits direct link libraries *before* the transitive archives that pull them in,
so `libxml2.so` landed on the link line well ahead of
`libarrow_bundled_dependencies.a`. With `--as-needed`, nothing had referenced
libxml2 by that point, so the linker dropped it, and the archive then arrived
with nothing left to resolve against. The result was a `libOpenMS.so` with 23
unresolved `xml*` symbols that only failed at `dlopen` time:
```
ImportError: .../libOpenMS.so: undefined symbol: xmlBufferFree
```
A consumer cannot fix this correctly from the outside without reaching into
Arrow's imported targets, because the ordering constraint is a property of
Arrow's own interface: the workaround is to append `LibXml2::LibXml2` to
`Arrow::arrow_bundled_dependencies`' `INTERFACE_LINK_LIBRARIES`, which is
exactly the declaration that appears to be missing.
## Already visible in Arrow's own CI, and a maintainer asked for this issue
#51280 (*"EXP: [C++][CI] Test static linking with S3"*, draft, open) added a
static-link smoke test and hit this on ARM64 Ubuntu 22.04:
> ```
> /usr/bin/ld: libarrow_bundled_dependencies.a(xml_wrapper.cpp.o): in
function
> `Azure::Storage::_internal::XmlReader::XmlReader(char const*, unsigned
long)':
> .../azure-storage-common/src/xml_wrapper.cpp:416: undefined reference to
`xmlReaderForMemory'
> ```
@raulcd's response there (2026-09-10) was:
> In my opinion it is better if we tackle this separately on its own issue.
I could not find that issue, so this is it.
#51280's diff already sketches the fix:
```cmake
# cpp/cmake_modules/ThirdpartyToolchain.cmake
if(AZURE_SDK_VENDORED AND NOT WIN32)
find_package(LibXml2 REQUIRED)
list(APPEND ARROW_SYSTEM_DEPENDENCIES LibXml2)
endif()
# cpp/src/arrow/CMakeLists.txt
if(ARROW_AZURE)
if(Azure_SOURCE STREQUAL "SYSTEM")
list(APPEND ARROW_STATIC_INSTALL_INTERFACE_LIBS
${AZURE_SDK_LINK_LIBRARIES})
elseif(Azure_SOURCE STREQUAL "BUNDLED")
if(UNIX)
list(APPEND ARROW_STATIC_INSTALL_INTERFACE_LIBS CURL::libcurl
LibXml2::LibXml2)
endif()
endif()
endif()
```
Two notes on that sketch, in case it is lifted as-is:
- It covers the CMake side only. `arrow.pc`'s `Requires.private` /
`Libs.private` come from `ARROW_PC_REQUIRES_PRIVATE` / `ARROW_PC_LIBS_PRIVATE`,
which the diff does not touch, so pkg-config consumers stay broken.
- The same CI run shows the gap is not libxml2-specific. On ARM64 macOS 14
the same static test fails on curl, pulled in by *two* bundled dependencies at
once:
> ```
> "_curl_easy_cleanup", referenced from:
> Azure::Core::_internal::UniqueHandleDeleter<void,
curl_easy_cleanup>::operator()(void*)
> in libarrow_bundled_dependencies.a[2](curl.cpp.o)
> google::cloud::rest_internal::v3_2::CurlPtrCleanup::operator()(void*)
const
> in libarrow_bundled_dependencies.a[214](curl_handle.cc.o)
> ```
So a fix scoped to "Azure + libxml2" would leave the GCS/curl case open.
Whether to widen this issue or split it is your call — libxml2 is simply the
one that reached a released package and broke a downstream consumer.
Related:
- #50753 (open) — *"`ARROW_STATIC_INSTALL_INTERFACE_LIBS` declarations are
usually forgotten because they are separate"*. Same list, and this report is
arguably another data point for it, but that issue is an enhancement about the
ergonomics of declaring **SYSTEM** dependencies (its driver was simdjson,
#50739/#50741). libxml2 is not a SYSTEM dependency of Arrow — it is an external
requirement of a **BUNDLED** third party, and Arrow never calls
`resolve_dependency` for it, so co-locating declarations with
`resolve_dependency` as proposed there would not surface this. Its only PR so
far, #50764, was closed unmerged on 2026-08-03.
Related but distinct:
- #50776 (closed) — Meson could not *find* libxml2 in order to *build* Azure
support; this issue is about the built artifact not *declaring* it to consumers.
- #15139 (closed) — `arrow.pc` missing dependencies for Windows static
builds; same family, pkg-config rather than CMake.
## Suggested fix
Declare libxml2 alongside the other bundled-dependency requirements, so
`Arrow::arrow_static` / `Arrow::arrow_bundled_dependencies` carry it in
`INTERFACE_LINK_LIBRARIES` (and `arrow.pc` in `Libs.private`) whenever Arrow is
built with Azure filesystem support and the Azure SDK is vendored into the
bundle — mirroring what the shared build already records in `DT_NEEDED`.
#51280's diff above is essentially that change.
### Component(s)
C++
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
