So I was able to also reproduce simply as this:
works: oslc -IincA -IincB blah.osl
broken: DYLD_LIBRARY_PATH= oslc -IincA -IincB blah.osl
The only thing in my DYLD_LIBRARY_PATH was /usr/local/opt/llvm
It seems that what was happening was that in cases where I didn't have
/usr/local/opt/llvm/lib in my DYLD_LIBRARY_PATH (as was the case when I ran my
compiler from a subshell), it was ending up linking TWO copies of libc++.1.so,
one that was build with LLVM, and one system version in /usr/lib, and somehow
that led to this weird behavior.
There is apparently some known linkage problems related to this, with the
homebrew build of llvm when on Mojave. From what I understand, Catalina's
support for a newer xcode apparently also fixes, but neither specific xcode
version nor having DYLD_LIBRARY_PATH set a particular way is an assumption I
can make with respect to my users.
I also noticed that this isn't a problem when I statically link llvm and clang
libraries to my app, so my solution for now is to just force my app's build
system to use the static llvm+clang libs rather than dynamic when on OSX and
the llvm version is >= 10.
-- lg
> On Apr 28, 2020, at 2:23 PM, Larry Gritz <l...@larrygritz.com> wrote:
>
> I have some more clues here. I can reproduce this strange behavior with just
> bash, no python needed.
>
>
> $ oslc -IincA -IincB blah.osl
> adding 'incA' <-- my debug, shows that the '-IincA' was parsed on the
> command line
> adding 'incB' <-- my debug, shows that the '-IincB' was parsed on the
> command line
> #include "..." search starts here:
> #include <...> search starts here:
> incA <-- LLVM diagnostics, shows I passed incA in includedirs
> list
> incB <-- LLVM diagnostics, shows I passed incB in includedirs
> list
> End of search list.
> Compiled blah.osl -> blah.oso
>
>
> Works fine. Now what if I pass the same commands to 'bash -c':
>
>
> $ /bin/bash -c "oslc -IincA -IincB blah.osl"
> adding 'incA'
> adding 'incB'
> #include "..." search starts here:
> #include <...> search starts here:
> incA
> incB
> End of search list.
> error: blah.osl:3:10: fatal error: cannot open file 'incA/b.h': No such file
> or directory
> #include "b.h"
> ^
> FAILED blah.osl
>
>
> But this ONLY fails if I'm using llvm 10. If I rebuild my app with llvm@9
> from homebrew instead of llvm (which is llvm 10):
>
> $ /bin/bash -c "oslc -IincA -IincB blah.osl"
> adding 'incA'
> adding 'incB'
> #include "..." search starts here:
> #include <...> search starts here:
> incA
> incB
> End of search list.
> Compiled blah.osl -> blah.oso
>
>
> Works fine again. I don't have a working theory for what's going on here.
>
> So I know the -I commands are making it to my program, and I know I'm passing
> those paths to libclang, because its own diagnostics list those directories.
> But it nonetheless fails to find headers that aren't in the very first
> included searchpath -- ONLY for llvm 10, ONLY on Mac, ONLY if I'm doing it
> through a second spawned shell (works fine when I directly type the command).
>
> Any guesses?
>
>
>> On Apr 27, 2020, at 12:00 PM, Larry Gritz via cfe-users
>> <cfe-users@lists.llvm.org> wrote:
>>
>> Excuse if this is a tricky explanation; I'm not sure I understand what's
>> going on.
>>
>> I have a C-like language and compiler for which I use clang libraries to do
>> the preprocessing. My compiler lets users specify `-I` directories for
>> searchpaths for includes, per usual convention. I'm doing something like
>> this:
>>
>> clang::HeaderSearchOptions &headerOpts =
>> compilerinst.getHeaderSearchOpts();
>> headerOpts.UseBuiltinIncludes = 0;
>> headerOpts.UseStandardSystemIncludes = 0;
>> headerOpts.UseStandardCXXIncludes = 0;
>> for (auto&& inc : includepaths) {
>> headerOpts.AddPath (inc, clang::frontend::Quoted,
>> false /* not a framework */,
>> true /* ignore sys root */);
>> }
>>
>>
>> For the sake of a simple failure case, I have header a.h in directory incA/,
>> and header b.h in incB/, and my test program just consists of
>>
>> #include "a.h"
>> #include "b.h"
>>
>> Also, I have set this to turn on some debugging:
>> headerOpts.Verbose = 1; // DEBUGGING
>>
>> Now, when I invoke my compiler from the command line,
>>
>> oslc -IincA -IincB test.osl
>>
>> I get this output:
>>
>> #include "..." search starts here:
>> #include <...> search starts here:
>> incA
>> incB
>> End of search list.
>>
>> and my compile succeeds. As expected, and as it has for many many years.
>>
>> But, as part of my compiler's test suite, there is a python script involved
>> that boils down to:
>>
>> #!/usr/bin/env python
>> import subprocess
>> subprocess.call ('oslc -IincA -IincB test.osl', shell=True)
>>
>> When I run the python program,
>>
>> python mytest.py
>>
>> then I get this output:
>>
>> #include "..." search starts here:
>> #include <...> search starts here:
>> incA
>> incB
>> End of search list.
>> error: test.osl:3:10: fatal error: cannot open file 'incA/b.h': No such
>> file or directory
>> #include "b.h"
>> ^
>> FAILED test.osl
>>
>> Wha? So I've poked around a bit with the behavior, and near as I can tell,
>> even though the diagnostics say that both incA and incB are in the search
>> list, it's only actually searching the first directory listed.
>>
>> Now, this only happens on OSX, and only when I'm using clang 10 libraries
>> (installed via Homebrew, though also when I build clang from scratch). Works
>> fine on Linux. Works fine on all platforms for clang 9, 8, 7, 6, and I've
>> been using this since back to 3.3 or so. Only had this problem after
>> upgrading to clang/llvm 10, and only on OSX. Fails the same way for python
>> 2.7 and 3.7.
>>
>> If I change the subprocess.call to:
>>
>> subprocess.call (['oslc', '-IincA', '-IincB', 'blah.osl'], shell=False)
>>
>> it succeeds. (But in real life, this isn't an adequate workaround, because I
>> want to use shell=True and keep the whole command line together, because
>> it's really an arbitrary shell command that has output redirect.)
>>
>> Does any of this ring a bell for anybody? Or does anyone have suggestions
>> for what to try next?
>>
>>
>
> --
> Larry Gritz
> l...@larrygritz.com
>
>
>
>
--
Larry Gritz
l...@larrygritz.com
_______________________________________________
cfe-users mailing list
cfe-users@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-users
