Hi, I'm not sure which ones you want or are interested in. In the ~/freesurfer/bin/ you can find the mri_em_register_cuda as well as other *cuda binaries.
In ~/Freesurfer/lib/cuda you can find some other files that may be of importance to you. I haven't had time to dig through them so far. Have you taken at look at the CUDA Developer's Guide wiki on the Freesurfer site? - Bryan ________________________________ From: Jordi Delgado [jordi...@pic.es] Sent: Thursday, July 03, 2014 12:36 AM To: Chiu, Bryan (PHTH) Cc: freesurfer@nmr.mgh.harvard.edu Subject: Re: [Freesurfer] recon-all CUDA error Hi again, Sorry I think that we have a missunderstanding. I was talking about the freesurfer mri_em_register_cuda and other mri_* binaries of freesurfer. Having the freesurfer files compiled in newer versions of cuda could include some optimizations at compiler level to run freesurfer processing in new architectures in the same way as you can compile the binaries for fermi support in the previous versions, there was a relevant improvement. I'm not sure if Kepler can't work with 5.0 or newer versions. I will talk with a more experienced fellows in CUDA programing and I will let you know... We keep in touch! 2014-07-02 21:13 GMT+02:00 Chiu, Bryan (PHTH) <bryan.c...@ubc.ca<mailto:bryan.c...@ubc.ca>>: Hi, Thank you for your reply. The CUDA 5.x binaries can be found in the CUDA 5.x legacy drivers for CUDA. As long as you set the PATH=/usr/local/cuda/bin:$PATH and LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH, Freesurfer's cudadetect will look into here for the binaries. I think you should note that cudaDetect (belonging to nVidia) is different than Freesurfer's cudadetect (found in usr/freesurfer/bin). Apologies for the previous misconception. My issue right now is that the GTX 780 (kepler) cant work with 5.0. Do you know if it can work with 5.5? I might try my hand to see if I can compile for a newer version of CUDA. Keep me posted. - Bryan Hi there, I'm currently working on compiling FS source code trying to use CUDA 6, and newer GPUs like Titan series. I haven't got lucky yet, but I will keep on trying... In the meantime I would like to download the CUDA 5 binaries. Are these binaries been published? I can't find anything on the FreeSurfer wiki... In my previous work you mentioned "Improving Execution Performance of FreeSurfer" we used CUDA 3 binaries that I had compiled with --enable-fermi flag. Just to clarify your concern, we never used openmp on the test setup of that study, we have implemented a scheduler to manage K simultaneous recon-all executions. Thank you, 2014-07-02 19:06 GMT+02:00 Chiu, Bryan (PHTH) <bryan.c...@ubc.ca<mailto:bryan.c...@ubc.ca>>: Hi, Turns out if you rename libcudart.so.6.0 to the 5.0 in your respective lib64, you can pass the cudadetect for Freesurfer. However after it will crash with a segfault error. This leads me to believe that I installed correctly. I think Freesurfer only supports up to CUDA 5.0 so far (although I know Freesurfer group stopped supporting the option and left the flags around). Two options exist: Either in a future patch the program can just be enabled to call the newer version of CUDA (or if I can find where it tries to find CUDA from, edit it to accept any version), or to install any version of CUDA less than or equal to 5.0 To others: The OpenMP and -use-gpu flag are quite promising. I took a recon down from 12 hours to 3.5 hours using the -openmp 8 flag for a 4 core processor (i7 4770k). I am currently trying to see if I can speed up faster using the CUDA enabled option. There was a paper titled "Improving the Execution Performance of Freesurfer" which implements a resource scheduler on top of the CUDA and openMP option. No comment on the accuracy of the recon-all using all of these in combination. - Bryan Date: Mon, 30 Jun 2014 16:34:45 +0000 From: "Chiu, Bryan (PHTH)" <bryan.c...@ubc.ca<mailto:bryan.c...@ubc.ca>> Subject: Re: [Freesurfer] recon-all CUDA error (Chiu, Bryan (PHTH)) To: "freesurfer@nmr.mgh.harvard.edu<mailto:freesurfer@nmr.mgh.harvard.edu>" <freesurfer@nmr.mgh.harvard.edu<mailto:freesurfer@nmr.mgh.harvard.edu>> Message-ID: <680d9b8d5feaaf40a1a5878265424ca44a594...@s-itsv-mbx06p.ead.ubc.ca<mailto:680d9b8d5feaaf40a1a5878265424ca44a594...@s-itsv-mbx06p.ead.ubc.ca>> Content-Type: text/plain; charset="iso-8859-1" Hi all, I am still working on this problem. Does anyone know if Freesurfer is asking for CUDA 5.0, except that I have CUDA 6.0 installed? My next steps will be to find out if Freesurfer is calling an older version of CUDA that isn't installed, or trying the CUDA samples. I really want to get this to work. -Bryan Date: Fri, 27 Jun 2014 17:59:37 +0000 From: "Chiu, Bryan (PHTH)" <bryan.c...@ubc.ca<mailto:bryan.c...@ubc.ca>> Subject: [Freesurfer] recon-all CUDA error To: "freesurfer@nmr.mgh.harvard.edu<mailto:freesurfer@nmr.mgh.harvard.edu>" <freesurfer@nmr.mgh.harvard.edu<mailto:freesurfer@nmr.mgh.harvard.edu>> Message-ID: <680d9b8d5feaaf40a1a5878265424ca44a594...@s-itsv-mbx06p.ead.ubc.ca<mailto:680d9b8d5feaaf40a1a5878265424ca44a594...@s-itsv-mbx06p.ead.ubc.ca>> Content-Type: text/plain; charset="iso-8859-1" Hi, I'm fairly novice at Linux and I've been trying to setup CUDA for recon-all. I get an error as follows: [TLA@COGMOB-linuxPC ~]$ recon-all -s bert -use-gpu -openmp 8 Testing for CUDA device: /usr/freesurfer/bin/mri_em_register_cuda: error while loading shared libraries: libcudart.so.5.0: cannot open shared object file: No such file or directory Linux COGMOB-linuxPC 2.6.32-431.el6.x86_64 #1 SMP Fri Nov 22 03:15:09 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux recon-all -s bert exited with ERRORS at Fri Jun 27 10:53:29 PDT 2014 For more details, see the log file To report a problem, see http://surfer.nmr.mgh.harvard.edu/fswiki/BugReporting Any idea what is going on? I setup CUDA and my Nvidia drivers according to this tutorial (http://aspratyush.wordpress.com/2012/05/06/install-nvidia-cuda-on-centos-6/). With, of course some slight changes to install the latest toolkit. Thanks in advance, - Bryan _______________________________________________ Freesurfer mailing list Freesurfer@nmr.mgh.harvard.edu<mailto:Freesurfer@nmr.mgh.harvard.edu> https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Partners Compliance HelpLine at http://www.partners.org/complianceline . If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail. -- Jordi Delgado Mengual PIC (Port d'Informació Científica) -- Jordi Delgado Mengual PIC (Port d'Informació Científica) Campus UAB, Edifici D E-08193 Bellaterra, Barcelona Tel: +34 93 586 82 32 Fax: +34 93 581 41 10 http://www.pic.es Avis - Aviso - Legal Notice: http://www.ifae.es/legal.html
_______________________________________________ Freesurfer mailing list Freesurfer@nmr.mgh.harvard.edu https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Partners Compliance HelpLine at http://www.partners.org/complianceline . If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail.
