Hi Timo, Thanks for you suggestion. I tested step-1 and step-2 again and they ran successfully after I set the number of OpenMP threads to 1. I only plan plan on running MPI codes without any multi-threading. The BLAS implementation I am using is a part of the Intel MKL library which is already present within one of the default modules which loads intel compilers and intel implementations of other commonly used libraries. Would you recommend using another BLAS implementation or install one manually?
Installing deal.ii on the Cori system was definitely a bit convoluted since even the original *cmake* and *make install* had to be run on the compute node, and that was an extremely slow process taking a few hours to complete even with *make -j64 install(*unlike the login nodes where the entire thing would be done in about 30 minutes*)*. In one of their(NERSC) presentations(https://www.nersc.gov/assets/Uploads/06-Compiling-Codes.pdf, slide 14) they do recommend swapping a haswell module with mic-knl module and compiling on login nodes whenever possible(slide 20). That's what I had tried earlier and did so again yesterday, and then running step-1(interactively on a compute node) yielded the following error : *make[3]: *** No rule to make target '/usr/lib64/libopenblas.so', needed by 'step-1'. Stop.make[2]: *** [CMakeFiles/Makefile2:137: CMakeFiles/step-1.dir/all] Error 2make[1]: *** [CMakeFiles/Makefile2:181: CMakeFiles/run.dir/rule] Error 2make: *** [Makefile:157: run] Error 2 * the reason for which I don't know. I will probably go through the entire installation process that worked for me once again and document it. Eventually I need deal.ii as a prerequisite for another package that is built on it. Thank you for your help. Best, Aaditya On Monday, October 26, 2020 at 9:01:44 AM UTC-4 Timo Heister wrote: > Can you try setting > > export OMP_NUM_THREADS=1 > > before running any of the examples? > > If you are planning on running MPI-only codes, you don't need to use > OpenMP in your BLAS implementation. > > > On Sun, Oct 25, 2020 at 7:25 PM Aaditya Lakshmanan > <aadit...@gmail.com> wrote: > > > > Hi Timo, > > I cleaned the previous installation and started with a fresh build. This > time after a compilation and installation(with MPI, LAPACK, P4EST and > PETSC), trying to run examples/step-1 in the installation directory : > > > > cmake . > > make -j16 > > > > ran successfully after which > > > > make run > > > > resulted in the following output : > > > > [ 66%] Built target step-1 > > [100%] Run step-1 with Debug configuration > > > > OpenBLAS Warning : The number of CPU/Cores(272) is beyond the > limit(256). Terminated. > > make[3]: *** [CMakeFiles/run.dir/build.make:58: CMakeFiles/run] Error 1 > > make[2]: *** [CMakeFiles/Makefile2:174: CMakeFiles/run.dir/all] Error 2 > > make[1]: *** [CMakeFiles/Makefile2:181: CMakeFiles/run.dir/rule] Error 2 > > make: *** [Makefile:157: run] Error 2 > > > > Regarding your questions and suggestions, the libdeal_ii.g.so file is > about 1.6GB. Do you know what the above error signifies? Thank you. > > > > Best, > > Aaditya > > > > > > > > On Sunday, October 25, 2020 at 2:10:21 PM UTC-4 Timo Heister wrote: > >> > >> > Seems like there might then be some issue with the LAPACK, P4EST or > PETSC installation I am using. > >> > >> I doubt it, when looking at this error: > >> > >> >> /usr/bin/ld: > /global/project/projectdirs/m2360/packagesCPFE/dealii_install/lib/ > libdeal_II.g.so: file not recognized: file truncated > >> > >> This sounds like the linking of the .so file failed. Can you take a > >> look at this file (check the size?)? You can try deleting the file and > >> running "make install" again. Maybe the last linker step failed? > >> If this does not help, maybe try the release mode examples as well. > >> > >> > >> > >> > >> -- > >> Timo Heister > >> http://www.math.clemson.edu/~heister/ > > > > -- > > The deal.II project is located at http://www.dealii.org/ > > For mailing list/forum options, see > https://groups.google.com/d/forum/dealii?hl=en > > --- > > You received this message because you are subscribed to the Google > Groups "deal.II User Group" group. > > To unsubscribe from this group and stop receiving emails from it, send > an email to dealii+un...@googlegroups.com. > > To view this discussion on the web visit > https://groups.google.com/d/msgid/dealii/5facbeb5-33f3-4474-8ae0-bca848c20b6an%40googlegroups.com > . > > > > -- > Timo Heister > http://www.math.clemson.edu/~heister/ > -- The deal.II project is located at http://www.dealii.org/ For mailing list/forum options, see https://groups.google.com/d/forum/dealii?hl=en --- You received this message because you are subscribed to the Google Groups "deal.II User Group" group. To unsubscribe from this group and stop receiving emails from it, send an email to dealii+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/dealii/e37fe178-3231-4d1a-86e6-b601b41bed9an%40googlegroups.com.
