hi Joris, You probably ran into the conda-forge compiler migration. I'm not sure about Anaconda's Apache Arrow libraries since they maintain those recipes.
If you need shared libaries using the gcc 4.x ABI you may have to build them yourself right, or use the Linux packages for the platform where you are working. It would be useful to have a Dockerfile that produces "portable" shared libraries with the RedHat devtoolset-2 compiler - Wes On Tue, Mar 12, 2019 at 11:22 AM Joris Peeters <joris.mg.peet...@gmail.com> wrote: > > [A] Short background: > > We are working on a MEX library that converts a binary array (representing > an Arrow stream or file) into MATLAB structs. This is in > parallel/complement to what already exists in the main Arrow project, which > focuses on feather, but the hope is certainly to contribute back some of > this work. We talked to MathWorks about this already (as GSA Capital). > > The library (e.g. fromArrowStream.mexa64) gets published on > (company-internal) Anaconda, and upon installation the dependencies on > arrow-cpp, boost-cpp etc are resolved (from remote channels). All .so's end > up in a user-local conda environment's ../lib, which in MATLAB we make > available through addpath. Compilation uses -D_GLIBCXX_USE_CXX11_ABI=0. > > [B] The issue I'm facing ... > > For quite a while (when we depended on arrow-cpp=0.10 from conda-forge) > this has worked fine, but lately I've encountered increasing issues wrt ABI > compatibility, amongst arrow, gtest (both at build time) and MATLAB (at > run-time). > > Is arrow-cpp 0.11 only built with the new ABI? I loaded it from both > defaults and conda-forge channel, and it seems different in this regard > than conda-forge's 0.10. Either way, I'm now attempting to built my library > without the -D_GLIBCXX_USE_CXX11_ABI=0 compile flag, as that seems to be > the more sustainable way forward. > > Question: is it possible to load a MEX library that has been compiled with > the new ABI? When doing this naively, I get an error like: > > Invalid MEX-file > '/data/home/jpeeter/apps/anaconda3/envs/testarrow/lib/fromArrowStream.mexa64': > /data/home/jpeeter/apps/matlab/MATLAB/R2018a/bin/glnxa64/../../sys/os/glnxa64/libstdc++.so.6: > version `CXXABI_1.3.11' not found (required by > /data/home/jpeeter/apps/anaconda3/envs/testarrow/lib/libarrow.so.11). > > which is fair enough. > > Alternatively, though, when loading MATLAB with an LD_PRELOAD, like > LD_PRELOAD=/usr/lib64/libstdc++.so.6 ~/apps/matlab/MATLAB/R2018a/bin/matlab > > I get this error: > Invalid MEX-file > '/data/home/jpeeter/apps/anaconda3/envs/testarrow/lib/fromArrowStream.mexa64': > /data/home/jpeeter/apps/anaconda3/envs/testarrow/lib/fromArrowStream.mexa64: > undefined symbol: _ZNK5arrow6Status8ToStringEv. > > If this isn't possible, is there a reliable/recommended Anaconda-way to > only bring in libraries that have been compiled with the old ABI? My > impression was that conda-forge libraries satisfied that, but > - This no longer seems to be true for arrow-cpp=0.11? I might be mistaken > here. > - We are resolving our dependencies through Anaconda, so it could be quite > brittle for users to explicitly have to specify a channel for certain > libraries. There are further subtleties wrt resolving boost & arrow from > different channels etc. Ideally - but not necessarily - users don't depend > on conda-forge at all, but only on the default channels (as an aside: we > use Artifactory internally). > > Essentially, I'm happy with either, > - having a way for MATLAB to load MEX with the new ABI, or, > - reliably depending on libraries compiled with the old ABI > > but I've been struggling for a while now to achieve either one of those. > Any pointers would be most welcome! > > (For once, this turned out to be easier on Windows than Linux!) > > Best, > -Joris.
