Were you building on a KNL node or on the frontend? What architecture did you specify?
-erik On Thu, Oct 13, 2016 at 9:38 PM, Valentin Churavy <[email protected]> wrote: > Since KNL is just a new platform the default version of the LLVM compiler > that Julia is based on does not support it properly. > During our testing at MIT we found that we needed to switch to the current > upstream of LLVM (or if anybody reads this at a later time LLVM 4.0) > You can do that by putting > LLVM_VER:=svn > into your Make.user. > > - Valentin > > On Friday, 14 October 2016 09:55:16 UTC+9, ABB wrote: >> >> Sigh... build failed. I'm including the last part that worked and the >> error message which followed: >> >> JULIA usr/lib/julia/inference.ji >> essentials.jl >> generator.jl >> reflection.jl >> options.jl >> promotion.jl >> tuple.jl >> range.jl >> expr.jl >> error.jl >> bool.jl >> number.jl >> int.jl >> >> signal (4): Illegal instruction >> while loading int.jl, in expression starting on line 193 >> ! at ./bool.jl:16 >> jl_call_method_internal at /home1/04179/abean/julia/src/j >> ulia_internal.h:189 >> jl_apply_generic at /home1/04179/abean/julia/src/gf.c:1942 >> anonymous at ./<missing> (unknown line) >> jl_call_method_internal at /home1/04179/abean/julia/src/j >> ulia_internal.h:189 >> jl_toplevel_eval_flex at /home1/04179/abean/julia/src/toplevel.c:569 >> jl_parse_eval_all at /home1/04179/abean/julia/src/ast.c:717 >> jl_load at /home1/04179/abean/julia/src/toplevel.c:596 >> jl_load_ at /home1/04179/abean/julia/src/toplevel.c:605 >> include at ./boot.jl:231 >> jl_call_method_internal at /home1/04179/abean/julia/src/j >> ulia_internal.h:189 >> jl_apply_generic at /home1/04179/abean/julia/src/gf.c:1942 >> do_call at /home1/04179/abean/julia/src/interpreter.c:66 >> eval at /home1/04179/abean/julia/src/interpreter.c:190 >> jl_interpret_toplevel_expr at /home1/04179/abean/julia/src/i >> nterpreter.c:31 >> jl_toplevel_eval_flex at /home1/04179/abean/julia/src/toplevel.c:558 >> jl_eval_module_expr at /home1/04179/abean/julia/src/toplevel.c:196 >> jl_toplevel_eval_flex at /home1/04179/abean/julia/src/toplevel.c:465 >> jl_toplevel_eval at /home1/04179/abean/julia/src/toplevel.c:580 >> jl_toplevel_eval_in_warn at /home1/04179/abean/julia/src/builtins.c:590 >> jl_toplevel_eval_in at /home1/04179/abean/julia/src/builtins.c:556 >> eval at ./boot.jl:234 >> jl_call_method_internal at /home1/04179/abean/julia/src/j >> ulia_internal.h:189 >> jl_apply_generic at /home1/04179/abean/julia/src/gf.c:1942 >> do_call at /home1/04179/abean/julia/src/interpreter.c:66 >> eval at /home1/04179/abean/julia/src/interpreter.c:190 >> jl_interpret_toplevel_expr at /home1/04179/abean/julia/src/i >> nterpreter.c:31 >> jl_toplevel_eval_flex at /home1/04179/abean/julia/src/toplevel.c:558 >> jl_parse_eval_all at /home1/04179/abean/julia/src/ast.c:717 >> jl_load at /home1/04179/abean/julia/src/toplevel.c:596 >> exec_program at /home1/04179/abean/julia/ui/repl.c:66 >> true_main at /home1/04179/abean/julia/ui/repl.c:119 >> main at /home1/04179/abean/julia/ui/repl.c:232 >> __libc_start_main at /usr/lib64/libc.so.6 (unknown line) >> unknown function (ip: 0x401928) >> Allocations: 100373 (Pool: 100371; Big: 2); GC: 0 >> /bin/sh: line 1: 15078 Illegal instruction >> /home1/04179/abean/julia/usr/bin/julia-debug -C knl --output-ji >> /home1/04179/abean/julia/usr/lib/julia/inference.ji --startup-file=no >> coreimg.jl >> make[1]: *** [/home1/04179/abean/julia/usr/lib/julia/inference.ji] Error >> 132 >> make: *** [julia-inference] Error 2 >> >> >> >> Any advice for debugging that? I don't find any previous issues which >> are helpful. >> >> Thanks - >> >> Austin >> >> On Thursday, October 13, 2016 at 1:49:24 PM UTC-5, ABB wrote: >>> >>> Awesome. Thanks. I'll try it again then. I appreciate the help. >>> >>> (Austin is also my name. I save space in my memory by going to school >>> at, living in and being a guy with the same name.) >>> >>> On Thursday, October 13, 2016 at 1:40:09 PM UTC-5, Erik Schnetter wrote: >>>> >>>> AB >>>> >>>> You're speaking of Stampede, if I might guess from the "austin" prefix >>>> in your email address. I would treat the old and the new section of the >>>> machines as separate, since they are not binary compatible. If you are >>>> really interested in the KNL part, then I'd concentrate on these, and use >>>> the development mode to always log in, build on, and run on the KNL nodes, >>>> and ignore everything else. Mixing different architectures in a single >>>> Julia environment is something I'd tackle much later, if at all. >>>> >>>> Alternatively you can use "haswell" as CPU architecture (instead of >>>> "core2" above), which should work both on the front end as well as the KNL >>>> nodes. However, this way you will lose speed on the KNL nodes, except for >>>> linear algebra operations. >>>> >>>> -erik >>>> >>>> >>>> On Thu, Oct 13, 2016 at 2:26 PM, ABB <[email protected]> wrote: >>>> >>>>> This is great - thanks for getting back to me so quickly. >>>>> >>>>> To follow up, I have two small questions: >>>>> >>>>> - To build specifically for the KNL system I should include something >>>>> like "JULIA_CPU_TARGET = knl" in the Make.user file? >>>>> >>>>> - Part of the system is KNL, part of it is "Intel Xeon E5 Sandy >>>>> Bridge and the Intel Knights Corner (KNC) coprocessor" (the exact >>>>> system is this one: https://portal.tacc.utexa >>>>> s.edu/user-guides/stampede ). Is there a way to build for both of >>>>> the architectures? I think I read in another issue somewhere that it >>>>> wasn't possible to support the Knights Corner because of (if I recall >>>>> correctly) lack of LLVM support or something (maybe I am completely making >>>>> that up) so if it's not possible I wouldn't be surprised. (The two >>>>> sections of Stampede run different versions of Linux too, if that makes it >>>>> even more complicated. I'd just be happy to get it running one way or the >>>>> other.) >>>>> >>>>> Thanks again! >>>>> >>>>> AB >>>>> >>>>> >>>>> On Thursday, October 13, 2016 at 1:10:48 PM UTC-5, Erik Schnetter >>>>> wrote: >>>>>> >>>>>> AB >>>>>> >>>>>> Using "core2" is a fallback that will work on very old machines. In >>>>>> your case -- if this happens to be a more modern, uniform HPC system -- >>>>>> you >>>>>> might want to use a different architecture. For example, if you're >>>>>> building >>>>>> on the compute nodes, and never run on the front end, then the default >>>>>> should already work for you. Otherwise, choosing "knl" as architecture >>>>>> should also work (and would also make it impossible to run on the front >>>>>> end). >>>>>> >>>>>> -erik >>>>>> >>>>>> >>>>>> On Thu, Oct 13, 2016 at 1:18 PM, ABB <[email protected]> wrote: >>>>>> >>>>>>> I built Julia Version 0.5.1-pre+2 on a cluster I have access to. >>>>>>> >>>>>>> The login node on which I executed the build has this architecture: >>>>>>> >>>>>>> Intel Core i7-5000 Extreme Edition (Haswell R2) / Xeon E5-x600 v3 >>>>>>> (Haswell-EP C1/M1/R2), 22nm >>>>>>> >>>>>>> The compute node has this architecture: >>>>>>> >>>>>>> Intel Xeon Phi Coprocessor (Knights Landing), 14nm >>>>>>> >>>>>>> (Those are each the last line of the output of "cpuid") >>>>>>> >>>>>>> when I try to run anything, I get the error: >>>>>>> >>>>>>> ERROR: Target architecture mismatch. Please delete or regenerate >>>>>>> sys.{so,dll,dylib}. >>>>>>> >>>>>>> I found this old discussion: >>>>>>> >>>>>>> https://groups.google.com/forum/#!msg/julia-dev/Eqp0GhZWxME/ >>>>>>> 3mGKX1l_L9gJ >>>>>>> >>>>>>> which recommends using >>>>>>> >>>>>>> JULIA_CPU_TARGET = core2 >>>>>>> >>>>>>> in the Make.user file. >>>>>>> >>>>>>> Since that discussion is 2 years old, I am just double checking to >>>>>>> see if that's still the best advice or if there is something else I >>>>>>> should >>>>>>> try first and/or instead. >>>>>>> >>>>>>> Thanks! >>>>>>> >>>>>>> AB >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Erik Schnetter <[email protected]> http://www.perimeterinstitute. >>>>>> ca/personal/eschnetter/ >>>>>> >>>>> >>>> >>>> >>>> -- >>>> Erik Schnetter <[email protected]> http://www.perimeterinstitute. >>>> ca/personal/eschnetter/ >>>> >>> -- Erik Schnetter <[email protected]> http://www.perimeterinstitute.ca/personal/eschnetter/
