Hi again, Sorry I think that we have a missunderstanding. I was talking about the freesurfer mri_em_register_cuda and other mri_* binaries of freesurfer. Having the freesurfer files compiled in newer versions of cuda could include some optimizations at compiler level to run freesurfer processing in new architectures in the same way as you can compile the binaries for fermi support in the previous versions, there was a relevant improvement.
I'm not sure if Kepler can't work with 5.0 or newer versions. I will talk with a more experienced fellows in CUDA programing and I will let you know... We keep in touch! 2014-07-02 21:13 GMT+02:00 Chiu, Bryan (PHTH) <bryan.c...@ubc.ca>: > Hi, > > Thank you for your reply. > > The CUDA 5.x binaries can be found in the CUDA 5.x legacy drivers for > CUDA. > > As long as you set the PATH=/usr/local/cuda/bin:$PATH and > LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH, Freesurfer's > cudadetect will look into here for the binaries. I think you should note > that cudaDetect (belonging to nVidia) is different than Freesurfer's > cudadetect (found in usr/freesurfer/bin). > > Apologies for the previous misconception. > > My issue right now is that the GTX 780 (kepler) cant work with 5.0. Do > you know if it can work with 5.5? I might try my hand to see if I can > compile for a newer version of CUDA. > > Keep me posted. > - Bryan > > Hi there, > > I'm currently working on compiling FS source code trying to use CUDA 6, > and newer GPUs like Titan series. I haven't got lucky yet, but I will keep > on trying... In the meantime I would like to download the CUDA 5 binaries. > Are these binaries been published? I can't find anything on the FreeSurfer > wiki... > > In my previous work you mentioned "Improving Execution Performance of > FreeSurfer" we used CUDA 3 binaries that I had compiled with --enable-fermi > flag. Just to clarify your concern, we never used openmp on the test setup > of that study, we have implemented a scheduler to manage K simultaneous > recon-all executions. > > Thank you, > > > 2014-07-02 19:06 GMT+02:00 Chiu, Bryan (PHTH) <bryan.c...@ubc.ca>: > Hi, > > Turns out if you rename libcudart.so.6.0 to the 5.0 in your respective > lib64, you can pass the cudadetect for Freesurfer. However after it will > crash with a segfault error. This leads me to believe that I installed > correctly. > > I think Freesurfer only supports up to CUDA 5.0 so far (although I know > Freesurfer group stopped supporting the option and left the flags around). > Two options exist: Either in a future patch the program can just be enabled > to call the newer version of CUDA (or if I can find where it tries to find > CUDA from, edit it to accept any version), or to install any version of > CUDA less than or equal to 5.0 > > To others: The OpenMP and -use-gpu flag are quite promising. I took a > recon down from 12 hours to 3.5 hours using the -openmp 8 flag for a 4 core > processor (i7 4770k). I am currently trying to see if I can speed up faster > using the CUDA enabled option. There was a paper titled "Improving the > Execution Performance of Freesurfer" which implements a resource scheduler > on top of the CUDA and openMP option. No comment on the accuracy of the > recon-all using all of these in combination. > > - Bryan > > Date: Mon, 30 Jun 2014 16:34:45 +0000 > From: "Chiu, Bryan (PHTH)" <bryan.c...@ubc.ca> > Subject: Re: [Freesurfer] recon-all CUDA error (Chiu, Bryan (PHTH)) > To: "freesurfer@nmr.mgh.harvard.edu" <freesurfer@nmr.mgh.harvard.edu> > Message-ID: > <680d9b8d5feaaf40a1a5878265424ca44a594...@s-itsv-mbx06p.ead.ubc.ca > > > Content-Type: text/plain; charset="iso-8859-1" > > Hi all, > > I am still working on this problem. Does anyone know if Freesurfer is > asking for CUDA 5.0, except that I have CUDA 6.0 installed? > > My next steps will be to find out if Freesurfer is calling an older > version of CUDA that isn't installed, or trying the CUDA samples. I really > want to get this to work. > > -Bryan > > Date: Fri, 27 Jun 2014 17:59:37 +0000 > From: "Chiu, Bryan (PHTH)" <bryan.c...@ubc.ca> > Subject: [Freesurfer] recon-all CUDA error > To: "freesurfer@nmr.mgh.harvard.edu" <freesurfer@nmr.mgh.harvard.edu> > Message-ID: > <680d9b8d5feaaf40a1a5878265424ca44a594...@s-itsv-mbx06p.ead.ubc.ca > > > Content-Type: text/plain; charset="iso-8859-1" > > Hi, > > I'm fairly novice at Linux and I've been trying to setup CUDA for > recon-all. I get an error as follows: > > [TLA@COGMOB-linuxPC ~]$ recon-all -s bert -use-gpu -openmp 8 > Testing for CUDA device: > /usr/freesurfer/bin/mri_em_register_cuda: error while loading shared > libraries: libcudart.so.5.0: cannot open shared object file: No such file > or directory > Linux COGMOB-linuxPC 2.6.32-431.el6.x86_64 #1 SMP Fri Nov 22 03:15:09 UTC > 2013 x86_64 x86_64 x86_64 GNU/Linux > > recon-all -s bert exited with ERRORS at Fri Jun 27 10:53:29 PDT 2014 > > For more details, see the log file > To report a problem, see > http://surfer.nmr.mgh.harvard.edu/fswiki/BugReporting > > Any idea what is going on? I setup CUDA and my Nvidia drivers according to > this tutorial ( > http://aspratyush.wordpress.com/2012/05/06/install-nvidia-cuda-on-centos-6/). > With, of course some slight changes to install the latest toolkit. > > Thanks in advance, > > - Bryan > > _______________________________________________ > Freesurfer mailing list > Freesurfer@nmr.mgh.harvard.edu > https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer > > > The information in this e-mail is intended only for the person to whom it > is > addressed. If you believe this e-mail was sent to you in error and the > e-mail > contains patient information, please contact the Partners Compliance > HelpLine at > http://www.partners.org/complianceline . If the e-mail was sent to you in > error > but does not contain patient information, please contact the sender and > properly > dispose of the e-mail. > > > > > -- > Jordi Delgado Mengual > PIC (Port d'Informació Científica) > -- Jordi Delgado Mengual PIC (Port d'Informació Científica) Campus UAB, Edifici D E-08193 Bellaterra, Barcelona Tel: +34 93 586 82 32 Fax: +34 93 581 41 10 http://www.pic.es Avis - Aviso - Legal Notice: http://www.ifae.es/legal.html
_______________________________________________ Freesurfer mailing list Freesurfer@nmr.mgh.harvard.edu https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Partners Compliance HelpLine at http://www.partners.org/complianceline . If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail.
