[cfe-users] Strange problem with C preprocessor include file searching (Mac, llvm 10)
Excuse if this is a tricky explanation; I'm not sure I understand what's going
on.
I have a C-like language and compiler for which I use clang libraries to do the
preprocessing. My compiler lets users specify `-I` directories for searchpaths
for includes, per usual convention. I'm doing something like this:
clang::HeaderSearchOptions &headerOpts = compilerinst.getHeaderSearchOpts();
headerOpts.UseBuiltinIncludes = 0;
headerOpts.UseStandardSystemIncludes = 0;
headerOpts.UseStandardCXXIncludes = 0;
for (auto&& inc : includepaths) {
headerOpts.AddPath (inc, clang::frontend::Quoted,
false /* not a framework */,
true /* ignore sys root */);
}
For the sake of a simple failure case, I have header a.h in directory incA/,
and header b.h in incB/, and my test program just consists of
#include "a.h"
#include "b.h"
Also, I have set this to turn on some debugging:
headerOpts.Verbose = 1; // DEBUGGING
Now, when I invoke my compiler from the command line,
oslc -IincA -IincB test.osl
I get this output:
#include "..." search starts here:
#include <...> search starts here:
incA
incB
End of search list.
and my compile succeeds. As expected, and as it has for many many years.
But, as part of my compiler's test suite, there is a python script involved
that boils down to:
#!/usr/bin/env python
import subprocess
subprocess.call ('oslc -IincA -IincB test.osl', shell=True)
When I run the python program,
python mytest.py
then I get this output:
#include "..." search starts here:
#include <...> search starts here:
incA
incB
End of search list.
error: test.osl:3:10: fatal error: cannot open file 'incA/b.h': No such
file or directory
#include "b.h"
^
FAILED test.osl
Wha? So I've poked around a bit with the behavior, and near as I can tell, even
though the diagnostics say that both incA and incB are in the search list, it's
only actually searching the first directory listed.
Now, this only happens on OSX, and only when I'm using clang 10 libraries
(installed via Homebrew, though also when I build clang from scratch). Works
fine on Linux. Works fine on all platforms for clang 9, 8, 7, 6, and I've been
using this since back to 3.3 or so. Only had this problem after upgrading to
clang/llvm 10, and only on OSX. Fails the same way for python 2.7 and 3.7.
If I change the subprocess.call to:
subprocess.call (['oslc', '-IincA', '-IincB', 'blah.osl'], shell=False)
it succeeds. (But in real life, this isn't an adequate workaround, because I
want to use shell=True and keep the whole command line together, because it's
really an arbitrary shell command that has output redirect.)
Does any of this ring a bell for anybody? Or does anyone have suggestions for
what to try next?
--
Larry Gritz
l...@larrygritz.com
___
cfe-users mailing list
cfe-users@lists.llvm.org
https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-users
Re: [cfe-users] Strange problem with C preprocessor include file searching (Mac, llvm 10)
I have some more clues here. I can reproduce this strange behavior with just
bash, no python needed.
$ oslc -IincA -IincB blah.osl
adding 'incA'<-- my debug, shows that the '-IincA' was parsed on the
command line
adding 'incB'<-- my debug, shows that the '-IincB' was parsed on the
command line
#include "..." search starts here:
#include <...> search starts here:
incA<-- LLVM diagnostics, shows I passed incA in includedirs
list
incB<-- LLVM diagnostics, shows I passed incB in includedirs
list
End of search list.
Compiled blah.osl -> blah.oso
Works fine. Now what if I pass the same commands to 'bash -c':
$ /bin/bash -c "oslc -IincA -IincB blah.osl"
adding 'incA'
adding 'incB'
#include "..." search starts here:
#include <...> search starts here:
incA
incB
End of search list.
error: blah.osl:3:10: fatal error: cannot open file 'incA/b.h': No such file or
directory
#include "b.h"
^
FAILED blah.osl
But this ONLY fails if I'm using llvm 10. If I rebuild my app with llvm@9 from
homebrew instead of llvm (which is llvm 10):
$ /bin/bash -c "oslc -IincA -IincB blah.osl"
adding 'incA'
adding 'incB'
#include "..." search starts here:
#include <...> search starts here:
incA
incB
End of search list.
Compiled blah.osl -> blah.oso
Works fine again. I don't have a working theory for what's going on here.
So I know the -I commands are making it to my program, and I know I'm passing
those paths to libclang, because its own diagnostics list those directories.
But it nonetheless fails to find headers that aren't in the very first included
searchpath -- ONLY for llvm 10, ONLY on Mac, ONLY if I'm doing it through a
second spawned shell (works fine when I directly type the command).
Any guesses?
> On Apr 27, 2020, at 12:00 PM, Larry Gritz via cfe-users
> wrote:
>
> Excuse if this is a tricky explanation; I'm not sure I understand what's
> going on.
>
> I have a C-like language and compiler for which I use clang libraries to do
> the preprocessing. My compiler lets users specify `-I` directories for
> searchpaths for includes, per usual convention. I'm doing something like this:
>
>clang::HeaderSearchOptions &headerOpts =
> compilerinst.getHeaderSearchOpts();
>headerOpts.UseBuiltinIncludes = 0;
>headerOpts.UseStandardSystemIncludes = 0;
>headerOpts.UseStandardCXXIncludes = 0;
>for (auto&& inc : includepaths) {
>headerOpts.AddPath (inc, clang::frontend::Quoted,
>false /* not a framework */,
>true /* ignore sys root */);
>}
>
>
> For the sake of a simple failure case, I have header a.h in directory incA/,
> and header b.h in incB/, and my test program just consists of
>
>#include "a.h"
>#include "b.h"
>
> Also, I have set this to turn on some debugging:
>headerOpts.Verbose = 1; // DEBUGGING
>
> Now, when I invoke my compiler from the command line,
>
>oslc -IincA -IincB test.osl
>
> I get this output:
>
>#include "..." search starts here:
>#include <...> search starts here:
> incA
> incB
>End of search list.
>
> and my compile succeeds. As expected, and as it has for many many years.
>
> But, as part of my compiler's test suite, there is a python script involved
> that boils down to:
>
>#!/usr/bin/env python
>import subprocess
>subprocess.call ('oslc -IincA -IincB test.osl', shell=True)
>
> When I run the python program,
>
>python mytest.py
>
> then I get this output:
>
>#include "..." search starts here:
>#include <...> search starts here:
> incA
> incB
>End of search list.
>error: test.osl:3:10: fatal error: cannot open file 'incA/b.h': No such
> file or directory
>#include "b.h"
> ^
>FAILED test.osl
>
> Wha? So I've poked around a bit with the behavior, and near as I can tell,
> even though the diagnostics say that both incA and incB are in the search
> list, it's only actually searching the first directory listed.
>
> Now, this only happens on OSX, and only when I'm using clang 10 libraries
> (installed via Homebrew, though also when I build clang from scratch). Works
> fine on Linux. Works fine on all platforms for clang 9, 8, 7, 6, and I've
> been using this since back to 3.3 or so. Only had this problem af
Re: [cfe-users] Strange problem with C preprocessor include file searching (Mac, llvm 10)
So I was able to also reproduce simply as this:
works: oslc -IincA -IincB blah.osl
broken: DYLD_LIBRARY_PATH= oslc -IincA -IincB blah.osl
The only thing in my DYLD_LIBRARY_PATH was /usr/local/opt/llvm
It seems that what was happening was that in cases where I didn't have
/usr/local/opt/llvm/lib in my DYLD_LIBRARY_PATH (as was the case when I ran my
compiler from a subshell), it was ending up linking TWO copies of libc++.1.so,
one that was build with LLVM, and one system version in /usr/lib, and somehow
that led to this weird behavior.
There is apparently some known linkage problems related to this, with the
homebrew build of llvm when on Mojave. From what I understand, Catalina's
support for a newer xcode apparently also fixes, but neither specific xcode
version nor having DYLD_LIBRARY_PATH set a particular way is an assumption I
can make with respect to my users.
I also noticed that this isn't a problem when I statically link llvm and clang
libraries to my app, so my solution for now is to just force my app's build
system to use the static llvm+clang libs rather than dynamic when on OSX and
the llvm version is >= 10.
-- lg
> On Apr 28, 2020, at 2:23 PM, Larry Gritz wrote:
>
> I have some more clues here. I can reproduce this strange behavior with just
> bash, no python needed.
>
>
> $ oslc -IincA -IincB blah.osl
> adding 'incA'<-- my debug, shows that the '-IincA' was parsed on the
> command line
> adding 'incB'<-- my debug, shows that the '-IincB' was parsed on the
> command line
> #include "..." search starts here:
> #include <...> search starts here:
> incA<-- LLVM diagnostics, shows I passed incA in includedirs
> list
> incB<-- LLVM diagnostics, shows I passed incB in includedirs
> list
> End of search list.
> Compiled blah.osl -> blah.oso
>
>
> Works fine. Now what if I pass the same commands to 'bash -c':
>
>
> $ /bin/bash -c "oslc -IincA -IincB blah.osl"
> adding 'incA'
> adding 'incB'
> #include "..." search starts here:
> #include <...> search starts here:
> incA
> incB
> End of search list.
> error: blah.osl:3:10: fatal error: cannot open file 'incA/b.h': No such file
> or directory
> #include "b.h"
> ^
> FAILED blah.osl
>
>
> But this ONLY fails if I'm using llvm 10. If I rebuild my app with llvm@9
> from homebrew instead of llvm (which is llvm 10):
>
> $ /bin/bash -c "oslc -IincA -IincB blah.osl"
> adding 'incA'
> adding 'incB'
> #include "..." search starts here:
> #include <...> search starts here:
> incA
> incB
> End of search list.
> Compiled blah.osl -> blah.oso
>
>
> Works fine again. I don't have a working theory for what's going on here.
>
> So I know the -I commands are making it to my program, and I know I'm passing
> those paths to libclang, because its own diagnostics list those directories.
> But it nonetheless fails to find headers that aren't in the very first
> included searchpath -- ONLY for llvm 10, ONLY on Mac, ONLY if I'm doing it
> through a second spawned shell (works fine when I directly type the command).
>
> Any guesses?
>
>
>> On Apr 27, 2020, at 12:00 PM, Larry Gritz via cfe-users
>> wrote:
>>
>> Excuse if this is a tricky explanation; I'm not sure I understand what's
>> going on.
>>
>> I have a C-like language and compiler for which I use clang libraries to do
>> the preprocessing. My compiler lets users specify `-I` directories for
>> searchpaths for includes, per usual convention. I'm doing something like
>> this:
>>
>> clang::HeaderSearchOptions &headerOpts =
>> compilerinst.getHeaderSearchOpts();
>> headerOpts.UseBuiltinIncludes = 0;
>> headerOpts.UseStandardSystemIncludes = 0;
>> headerOpts.UseStandardCXXIncludes = 0;
>> for (auto&& inc : includepaths) {
>> headerOpts.AddPath (inc, clang::frontend::Quoted,
>> false /* not a framework */,
>> true /* ignore sys root */);
>> }
>>
>>
>> For the sake of a simple failure case, I have header a.h in directory incA/,
>> and header b.h in incB/, and my test program just consists of
>>
>> #include "a.h"
>> #include "b.h"
>>
>> Also, I have set this to turn on some debugging:
>> headerOpts.Verbose = 1; // DEBUGGING
>>
>> Now, w
[cfe-users] How to make #error or #warning work with libclang preprocessing?
I'm using clang libraries to implement the preprocessor for my language -- set up a CompilerInstance, setDiagnostics, ... DoPrintPreprocessedInput. When I preprocess code that has a #error or #warning, it just gets reported as "invalid preprocessing directive" on the clang side, no different than if it were a "malformed" directive like "#foobar". Can somebody point me in the right direction of how I can cause them to be treated as they are in C/C++ (print the error or warning as such, rather than complain about an unknown directive), or even better, to intercept #error or #warning preprocessor directives and pass them all the way to my compiler, just like #pragma gets passed along and I can handle it directly? Thanks for any help you can give. -- Larry Gritz l...@larrygritz.com ___ cfe-users mailing list cfe-users@lists.llvm.org https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-users
