Hi Richard, I resolved the make error. Just needed to point my libcudart.so in /usr/local/cuda/lib to the 64-bit equivalent in /usr/local/cuda/lib64. Running into other cuda-related issues with gcc now, but that should be much easier to resolve.
-Derek ________________________________________ From: freesurfer-boun...@nmr.mgh.harvard.edu [freesurfer-boun...@nmr.mgh.harvard.edu] on behalf of Derek Pisner [dpis...@psychiatry.arizona.edu] Sent: Thursday, June 23, 2016 1:26 PM To: Freesurfer support list Subject: Re: [Freesurfer] Any updates on recon-all GPU support? Hi Richard, Hope this finds you well. I followed your instructions and compiled from source (you were right-- compiling Fs from source isn't too bad at all!) I also made your tweak to configure.in and then ran ./configure like so: ./configure --disable-Werror --enable-fermi-gpu --with-cuda="/usr/local/cuda" --with-pkgs-dir=/tmp/centos6-x86_64-packages --prefix=/usr/local/freesurfer That ran successfully. However, when I then ran 'make -j4', I get the following error and it exits: /usr/bin/ld: skipping incompatible /usr/local/cuda/lib/libcudart.so when searching for -lcudart /usr/bin/ld: cannot find -lcudart collect2: error: ld returned 1 exit status Makefile:568: recipe for target 'cudadetect' failed make[2]: *** [cudadetect] Error 1 make[2]: Leaving directory '/tmp/freesurfer/gpu_utils' Makefile:1053: recipe for target 'all-recursive' failed make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory '/tmp/freesurfer' Makefile:962: recipe for target 'all' failed make: *** [all] Error 2 It is very strange, because I don't think this is an issue with not being able to find the cuda libraries (i.e. libcudart*) since they are detected by FSL's CUDA wrappers and are exported in my ~/.bashrc: export PATH=$PATH:/usr/local/cuda/bin export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/usr/local/cuda/lib64:/usr/local/cuda/lib Curious to hear your thoughts on this. Also, if it helps to know, I am using cuda toolkit 6.5 and compiling on Debian 8.0. Thanks in advance, Derek ________________________________________ From: freesurfer-boun...@nmr.mgh.harvard.edu [freesurfer-boun...@nmr.mgh.harvard.edu] on behalf of R Edgar [freesurfer....@gmail.com] Sent: Wednesday, June 22, 2016 6:14 PM To: Freesurfer support list Subject: Re: [Freesurfer] Any updates on recon-all GPU support? On 22 June 2016 at 19:32, Derek Pisner <dpis...@psychiatry.arizona.edu> wrote: > Thanks for the very quick reply. Good to know that someone has successfully > compiled GPU binaries for newer CUDA versions. I took a quick look at the > developer's guide for building the cuda binaries from source, but it seems > like a very involved process? Do your compiled binaries only work for CUDA > SDK 7.5 or have you tested them with other CUDA versions? I only have one Linux box, running CentOS 7 (which isn't officially supported by Freesurfer - I had to tweak CFLAGS quite a bit too) with CUDA 7.5. I wouldn't say the process is very involved to compile from source, but I did have to tweak things a bit. Specifically, after running ./setup_configure, I had to open up the configure script, and find the section which looks like: ############################################################# # Nvidia CUDA enabling ############################################################ CUDA_DIR="" with_cuda="" And comment out the with_cuda="" line. Then I run configure with --enable-fermi-gpu --with-cuda="/usr/local/cuda" (plus some other options for my machine). Note that for me: $ which nvcc /usr/local/cuda-7.5/bin/nvcc > Also, would you mind sending me an example binary (e.g. mri_ca_register_cuda) > just so I cant test it myself and rule out whether this is just an issue with > the way our GPU is configured? (I can't even get cudadetect to acknowledge my > GPU as a cuda-enabled device (it just says 0 devices found), despite it > successfully acknowledging that the GPU "exists") Please let me know and I > cna send you a private email address. I can do, but there's a lot of things which could go wrong. Are you able to compile and run the sample CUDA programs which NVIDIA supplies as part of the SDK? If you haven't tried that yet, it would be a better place to start. > Thanks again for your help! And hopefully this thread will help others > interested in updating their gpu-capabilities with Freesurfer. Indeed - one Francis Thomas @arizona.edu was asking similar questions a few months ago. I'm afraid I'm not doing any extra GPU porting right now - I've been getting concerned about potential drift between the CPU and GPU pipelines, and trying to come up with a better way around the problem. HTH, Richard _______________________________________________ Freesurfer mailing list Freesurfer@nmr.mgh.harvard.edu https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Partners Compliance HelpLine at http://www.partners.org/complianceline . If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail. _______________________________________________ Freesurfer mailing list Freesurfer@nmr.mgh.harvard.edu https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer _______________________________________________ Freesurfer mailing list Freesurfer@nmr.mgh.harvard.edu https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer
