Yeah, precisely. We're aiming to simultaneously - maintain compatibility with quite old versions of MATLAB (down to R2009 at the moment), - where possible, make use of prebuilt libraries from Anaconda, e.g. for arrow-cpp, boost, gtest etc
Our build infrastructure also doesn't easily support GCC 6.x anymore (we're also on 7.3 here), and whilst I can build packages inside a docker image set up with an old GCC, this isn't too pleasant. Overall, the best approach for us seems to be to compile-with/depend-upon a modern GCC and new ABI for everything, and use the LD_PRELOAD to shove libstdc++ in MATLAB. For us, even if MATLAB shipped with the very latest libstdc++, that still wouldn't quite work due to the requirement to support older versions as well. I've tested the above approach on Linux with R2009, R2014 and R2018 and it seems to work perfectly fine. Famous last words, though. -J On Wed, Mar 13, 2019 at 12:57 PM Uwe L. Korn <uw...@xhochy.com> wrote: > Hello all, > > the conda packages from conda-forge use gcc and libstdc++ 7.3, i.e. are a > major version newer than the ones that come with Matlab. Thus the missing > symbols and the need for LD_PRELOAD. As long as Matlab is using an older > version of gcc/libstdc++, the missing symbols will continue to pop up. > Thanks to symbol versioning in libstdc++ it should be save though to > preload a newer libstdc++. > > Uwe > > On Wed, Mar 13, 2019, at 11:35 AM, Hatem Helal wrote: > > Hi Joris, > > > > Nice to hear from you! I'd like the understand the need to use > > LD_PRELOAD for picking up a different libstdc++. In case haven't > > found this already, MATLAB even vendors a libstdc++ as per: > > > > (pwd = matlabroot) > > % more sys/os/glnxa64/README.libstdc++ > > > > The GCC runtime libraries included here: > > > > libstdc++.so.6.0.22 libgcc_s.so.1 libgfortran.so.3.0.0 > libquadmath.so.0.0.0 > > > > and associated symlinks are part of gcc-6.3.0, > > available from ftp.gnu.org. They are included with MATLAB > > in the event that your distribution does not provide them. > > > > This corresponds to the toolchain we use to compile MATLAB as well as > > the supported compiler for MEX. > > > > Could you share with us the details of your compiler / system? I think > > if you use gcc 6.3 for building both gsa-arrow-cpp and arrow-matlab we > > might be able to eliminate the LD_PRELOAD, which I think should be > > nicer for your end-users. What was the error you saw without > > LD_PRELOAD? > > > > Hatem > > > > On 13/03/2019, 09:59, "Joris Peeters" <joris.mg.peet...@gmail.com> > wrote: > > > > Hello Uwe, > > > > Yes, it would appear that was the case. Aggregating yours and Wes' > > suggestions, I'm now: > > > > - Compiling arrow-cpp myself, explicitly with the new ABI, and > publishing > > this on our internal conda channel (as gsa-arrow-cpp, so it can be > > explicitly depended upon). Did this by tweaking > > > https://github.com/conda-forge/arrow-cpp-feedstock/tree/master/recipe a > > little bit. > > - Compiling arrow-matlab (which has fromArrowStream and > fromArrowFile) also > > explicitly with the new ABI, depending on the aforementioned > gsa-arrow-cpp > > - Launch MATLAB with an explicit LD_PRELOAD for a sufficiently modern > > libstdc++ (also from Anaconda) > > > > And now it all works fine! > > I'm not sure if this is the most elegant solution - it does feel a > bit > > awkward - but it does allow us to make progress. > > > > Thanks, > > -J > > > > On Tue, Mar 12, 2019 at 8:49 PM Uwe L. Korn <uw...@xhochy.com> > wrote: > > > > > Hello Joris, > > > > > > > > > > > > '/data/home/jpeeter/apps/anaconda3/envs/testarrow/lib/fromArrowStream.mexa64': > > > > > > > > > /data/home/jpeeter/apps/anaconda3/envs/testarrow/lib/fromArrowStream.mexa64: > > > undefined symbol: _ZNK5arrow6Status8ToStringEv. > > > > > > sounds like fromArrowStream.mexa64 was still compiled using > > > -D_GLIBCXX_USE_CXX11_ABI=0, you might try to explicitly pass > > > -D_GLIBCXX_USE_CXX11_ABI=1 or when building an conda environment, > > be sure > > > to have the compilers package installed and use the $CC and $CXX > > > environment variables to pick the right compilers. You may also > > need to > > > LD_PRELOAD the libstdc++.so that is coming with conda and not the > > one > > > coming from the system. > > > > > > Anaconda/defaults and conda-forge based Arrow packages are both > > nowadays > > > built with the new ABI. But they are built with slightly different > > > toolchains, so it is best to only install packages from one of > > the two > > > repositories and don't mix them. > > > > > > Uwe > > > > > > > > > On Tue, Mar 12, 2019, at 5:32 PM, Wes McKinney wrote: > > > > hi Joris, > > > > > > > > You probably ran into the conda-forge compiler migration. I'm > > not sure > > > > about Anaconda's Apache Arrow libraries since they maintain > > those > > > > recipes. > > > > > > > > If you need shared libaries using the gcc 4.x ABI you may have > > to > > > > build them yourself right, or use the Linux packages for the > > platform > > > > where you are working. It would be useful to have a Dockerfile > > that > > > > produces "portable" shared libraries with the RedHat > > devtoolset-2 > > > > compiler > > > > > > > > - Wes > > > > > > > > On Tue, Mar 12, 2019 at 11:22 AM Joris Peeters > > > > <joris.mg.peet...@gmail.com> wrote: > > > > > > > > > > [A] Short background: > > > > > > > > > > We are working on a MEX library that converts a binary array > > > (representing > > > > > an Arrow stream or file) into MATLAB structs. This is in > > > > > parallel/complement to what already exists in the main Arrow > > project, > > > which > > > > > focuses on feather, but the hope is certainly to contribute > > back some > > > of > > > > > this work. We talked to MathWorks about this already (as GSA > > Capital). > > > > > > > > > > The library (e.g. fromArrowStream.mexa64) gets published on > > > > > (company-internal) Anaconda, and upon installation the > > dependencies on > > > > > arrow-cpp, boost-cpp etc are resolved (from remote channels). > > All > > > .so's end > > > > > up in a user-local conda environment's ../lib, which in > > MATLAB we make > > > > > available through addpath. Compilation uses > > > -D_GLIBCXX_USE_CXX11_ABI=0. > > > > > > > > > > [B] The issue I'm facing ... > > > > > > > > > > For quite a while (when we depended on arrow-cpp=0.10 from > > conda-forge) > > > > > this has worked fine, but lately I've encountered increasing > > issues > > > wrt ABI > > > > > compatibility, amongst arrow, gtest (both at build time) and > > MATLAB (at > > > > > run-time). > > > > > > > > > > Is arrow-cpp 0.11 only built with the new ABI? I loaded it > > from both > > > > > defaults and conda-forge channel, and it seems different in > > this regard > > > > > than conda-forge's 0.10. Either way, I'm now attempting to > > built my > > > library > > > > > without the -D_GLIBCXX_USE_CXX11_ABI=0 compile flag, as that > > seems to > > > be > > > > > the more sustainable way forward. > > > > > > > > > > Question: is it possible to load a MEX library that has been > > compiled > > > with > > > > > the new ABI? When doing this naively, I get an error like: > > > > > > > > > > Invalid MEX-file > > > > > > > > > > > '/data/home/jpeeter/apps/anaconda3/envs/testarrow/lib/fromArrowStream.mexa64': > > > > > > > > > > > /data/home/jpeeter/apps/matlab/MATLAB/R2018a/bin/glnxa64/../../sys/os/glnxa64/libstdc++.so.6: > > > > > version `CXXABI_1.3.11' not found (required by > > > > > > > /data/home/jpeeter/apps/anaconda3/envs/testarrow/lib/libarrow.so.11). > > > > > > > > > > which is fair enough. > > > > > > > > > > Alternatively, though, when loading MATLAB with an > > LD_PRELOAD, like > > > > > LD_PRELOAD=/usr/lib64/libstdc++.so.6 > > > ~/apps/matlab/MATLAB/R2018a/bin/matlab > > > > > > > > > > I get this error: > > > > > Invalid MEX-file > > > > > > > > > > > '/data/home/jpeeter/apps/anaconda3/envs/testarrow/lib/fromArrowStream.mexa64': > > > > > > > > > > > /data/home/jpeeter/apps/anaconda3/envs/testarrow/lib/fromArrowStream.mexa64: > > > > > undefined symbol: _ZNK5arrow6Status8ToStringEv. > > > > > > > > > > If this isn't possible, is there a reliable/recommended > > Anaconda-way to > > > > > only bring in libraries that have been compiled with the old > > ABI? My > > > > > impression was that conda-forge libraries satisfied that, but > > > > > - This no longer seems to be true for arrow-cpp=0.11? I might > > be > > > mistaken > > > > > here. > > > > > - We are resolving our dependencies through Anaconda, so it > > could be > > > quite > > > > > brittle for users to explicitly have to specify a channel for > > certain > > > > > libraries. There are further subtleties wrt resolving boost & > > arrow > > > from > > > > > different channels etc. Ideally - but not necessarily - users > > don't > > > depend > > > > > on conda-forge at all, but only on the default channels (as > > an aside: > > > we > > > > > use Artifactory internally). > > > > > > > > > > Essentially, I'm happy with either, > > > > > - having a way for MATLAB to load MEX with the new ABI, or, > > > > > - reliably depending on libraries compiled with the old ABI > > > > > > > > > > but I've been struggling for a while now to achieve either > > one of > > > those. > > > > > Any pointers would be most welcome! > > > > > > > > > > (For once, this turned out to be easier on Windows than > > Linux!) > > > > > > > > > > Best, > > > > > -Joris. > > > > > > > > > > > > > >
