Patrick,
> On 4/04/2020, at 1:33 AM, Patrick Schratz <patrick.schr...@gmail.com> wrote:
>
> Simon,
>
> thanks. I assume that you mean “clang from homebrew” = llvm (from homebrew)?
LLVM is the compiler infrastructure project, clang is the C/C++ compiler from
that project. We don't use any of the other compilers form LLVM.
> Indeed I am currently trying that out and it looks really robust for source
> installations (more than the systems clang (+ eventual openMP flags like
> suggested by Kevin) and other variants (gcc from homebrew or openMP enabled
> clang7 or 8).
>
> Note that I do not use R via homebrew.
>
> However, all in all I am essentially mixing the custom llvm from homebrew
> with the official R installer and the old 10.13 SDK.
> It looks quite complex and I’d wish everything would be easier. However, in
> the end I just want to have a stable “install from source” environment that
> works for all packages out there (I do not use the binaries).
>
Why don't use use Homebrew? That's exactly what Homebrew provides - its own
ecosystem, everything form source (with cached binaries if you want them).
> All in all I am a bit confused now about all the mixing and options
> available. Let’s see what the foreseeable future brings and where things are
> going.
>
Mixing is not available. You pick one system and go with it, but can't mix
between them, because of the different run-time libraries.
> Regarding the SDK issue: I think most users just use the binaries on macOS
> (across all OS versions) and rarely face such issues. And there are
> presumably not many people out there that do actual R-devel testing on
> Catalina (?), otherwise I’d expect way more people to jump into the
> discussion. However, I am not alone with these issues as you can see in the
> Rcpp issue we were talking about.
> Maybe you can even reproduce the issues by linking a custom install of the
> 10.15 SDK on a non-Catalina machine? I don’t know how much trouble that would
> bring up when trying to jump into the future - but this ist just an idea :)
>
That's an entirely different, unrelated topic. Testing cutting edge (or even
pre-release) systems makes sense, but that's not our current worry (there was a
separate thread about CI).
Cheers,
Simon
> Thanks for your work!
> On 3. Apr 2020, 14:13 +0200, Simon Urbanek <simon.urba...@r-project.org>,
> wrote:
>> Patrick,
>>
>> you were commenting on the thread where we talked about CRAN R - that one is
>> now compiled using Apple clang. You were talking about using clang from
>> Homebrew - those are incompatible as they use different run-time.
>> Unfortunately, the Intel OpenMP run-time varies by clang compiler version
>> and is known to fail when used with the wrong compiler. Analogously, mixing
>> gomp (from gcc) and iomp is problematic (and GNU Fortran uses gomp). As
>> discussed in the Homebrew thread, if you are compiling everything via
>> Homebrew including R then it's all good, you just can't mix Apple tools and
>> Homebrew in general.
>>
>> Also at the bottom I was only talking about 10.14 SDK - that's what we use
>> on CRAN and I have not seen any issues with it - you were claiming that it
>> doesn't work with Rcpp and igraph.
>>
>> Cheers,
>> Simon
>>
>>
>>
>>> On 4/04/2020, at 1:00 AM, Patrick Schratz <patrick.schr...@gmail.com> wrote:
>>>
>>> Simon,
>>>
>>> I don’t understand fully. I am using llvm for all C variants (just now
>>> shown) in combination with the 10.13 SDK.
>>> So far this combination works flawlessly for all “problematic” packages
>>> like data.table, igraph or Rcpp.
>>>
>>> I don’t have deeper knowledge about the “iomp” setup but as of right now I
>>> don’t know what I am mixing up here. Can you shine more light on this?
>>> If it is about llvm in general: data.table recommends to use llvm openly in
>>> their wiki due to the lack of openMP for the standard clang.
>>> And while this could be handled as “any other package” in R, we all know
>>> that it has quite some impact and probably devs who know what they do in
>>> terms of C configs.
>>>
>>> Regarding the SDK issue: Both Rcpp v1.0.4 and igraph 1.2.5 can’t be
>>> installed from source with SDK 10.15 and standard apple clang (or any C
>>> compiler). Switching to SDK 10.13 solves the issue. You were even
>>> commenting on the Rcpp issue.
>>> Since SDK 10.15 is the current release and many users have no choice on
>>> which OS version they are (for different reasons), this issue should imo be
>>> inspected in more detail. What do you think?
>>>
>>> Best, Patrick
>>> On 2. Apr 2020, 23:38 +0200, Simon Urbanek <simon.urba...@r-project.org>,
>>> wrote:
>>>> Patrick,
>>>>
>>>> you can't mix compilers - it really matters which iomp your'e using. llvm
>>>> has its own modified version which may not be the same as the official
>>>> Intel release. From your tests it looks like you're using the llvm one
>>>> which will likely work only with that compiler. Since Apple doesn't have
>>>> official omp support it's hard to say which versions work and which don't.
>>>>
>>>>
>>>>> On 3/04/2020, at 10:20 AM, Patrick Schratz <patrick.schr...@gmail.com>
>>>>> wrote:
>>>>>
>>>>> Thanks Kevin, interesting approach.
>>>>>
>>>>> However, when testing it against a few packages I get symbol pointer
>>>>> issues (e.g. for data.table and xgboost).
>>>>> Using llvm everything works.
>>>>> Could llvm be the best middle way? Easy installation via brew and still
>>>>> clang compliant.
>>>>>
>>>>> Currently my Makevars look as follows
>>>>>
>>>>> CFLAGS=-isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX10.13.sdk
>>>>> […]
>>>>>
>>>>> CC = ccache /usr/local/opt/llvm/bin/clang
>>>>> […]
>>>>>
>>>>> Where llvm was installed via `brew install llvm`.
>>>>> SDK 10.13 because of {igraph} and {Rcpp} issues with SDK 10.14 and SDK
>>>>> 10.15
>>>>
>>>>
>>>> I mentioned that before, but I do not see issues with 10.14 SDK.
>>>>
>>>> Cheers,
>>>> Simon
>>>>
>>>>
>>>>> On 2. Apr 2020, 23:01 +0200, Simon Urbanek <simon.urba...@r-project.org>,
>>>>> wrote:
>>>>>> Tim,
>>>>>>
>>>>>>
>>>>>>> On 3/04/2020, at 2:01 AM, BATES Timothy <tim.ba...@ed.ac.uk> wrote:
>>>>>>>
>>>>>>> Moving to a compiler that drops support for OpenMP seems a sad choice,
>>>>>>> especially now we’ve all climbed the learning curve of the non-Apple
>>>>>>> compiler (the real barrier was lack of a pkg installer and that’s done
>>>>>>> now).
>>>>>>>
>>>>>>
>>>>>> It has caused a lot of issues, it trips people on a daily basis which is
>>>>>> just not worth it. Also with Apple's increasingly strict rules about
>>>>>> what can be distributed we don't want to be in the business in
>>>>>> maintaining a separate toolchain. There have been numerous issues with
>>>>>> C++ exceptions so the fall out was much bigger than originally expected
>>>>>> and it would only get worse.
>>>>>>
>>>>>>
>>>>>>> Losing OpenMP for the CRAN version of OpenMx/umx (our SEM packages)
>>>>>>> would be a big loss for users (for whom the CRAN version now supports
>>>>>>> OpenMP giving them a 2-12x speed up). In general, R on Mac is made more
>>>>>>> viable by having OpenMP
>>>>>>>
>>>>>>> Re Brian’s points, I’d say that the distribution problem is crucial:
>>>>>>> Packages not on CRAN have dramatically diminished accessibility/useage.
>>>>>>>
>>>>>>
>>>>>> The idea is that if a package deems this critical, it can enable this
>>>>>> for the users. As it is now, packages can still supply iomp and use it.
>>>>>>
>>>>>> That said, I would open for discussion the ability to distribute iomp
>>>>>> with the R binary, but it would not be supported by R directly, i.e., it
>>>>>> would be on the package author to make sure that the way the package
>>>>>> operates is compatible with that binary. Let me know what you think.
>>>>>>
>>>>>>
>>>>>>> Second, a great range of compute-intensive problems are amenable to
>>>>>>> division amongst cores, including nearly all models that take more than
>>>>>>> a nominal amount of time to run: So simulations, CIs, bootstrapping,
>>>>>>> nearly everything in genetics all speeds up.
>>>>>>>
>>>>>>
>>>>>> Yes, but OpenMP is rarely used for those tasks. In R OpenMP can be only
>>>>>> used for very small subset of such tasks - which is why alternative
>>>>>> approaches are much more common.
>>>>>>
>>>>>> Cheers,
>>>>>> Simon
>>>>>>
>>>>>>
>>>>>>> I’d say especially on desktop/laptop. The big advantage of multi blade
>>>>>>> systems requires snowfall-type solutions, but desktops profit
>>>>>>> automatically from their multi-core structure and don;’t have multiple
>>>>>>> processors (except graphics, which no-one seems to be exploiting on
>>>>>>> CRAN-style R), so OpenMP is their one trick. I’d hope not to lose it.
>>>>>>>
>>>>>>> Best, t
>>>>>>>
>>>>>>>
>>>>>>>> On 2 Apr 2020, at 05:18, Prof Brian Ripley <rip...@stats.ox.ac.uk>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>> On 01/04/2020 22:02, Simon Urbanek wrote:
>>>>>>>>> JJB,
>>>>>>>>> 1. correct, there was too much trouble in this. But please feel free
>>>>>>>>> to start a new thread about this here if you have strong opinions.
>>>>>>>>
>>>>>>>> Also note that it is possible (and not hard) to install packages from
>>>>>>>> source with an OpenMP-supporting compiler, and how to do so is in the
>>>>>>>> R-admin manual. The problems come in distributing them.
>>>>>>>>
>>>>>>>> The benefits of OpenMP are often overestimated, especially on
>>>>>>>> desktop/laptop level hardware. But it is available for the small
>>>>>>>> (tiny?) proportion of users who need it.
>>>>>>> The University of Edinburgh is a charitable body, registered in
>>>>>>> Scotland, with registration number SC005336.
>>>>>>> _______________________________________________
>>>>>>> R-SIG-Mac mailing list
>>>>>>> R-SIG-Mac@r-project.org
>>>>>>> https://stat.ethz.ch/mailman/listinfo/r-sig-mac
>>>>>>
>>>>>> _______________________________________________
>>>>>> R-SIG-Mac mailing list
>>>>>> R-SIG-Mac@r-project.org
>>>>>> https://stat.ethz.ch/mailman/listinfo/r-sig-mac
>>>>
>>
_______________________________________________
R-SIG-Mac mailing list
R-SIG-Mac@r-project.org
https://stat.ethz.ch/mailman/listinfo/r-sig-mac
