[Freesurfer] cuda compile help
cuda v8 freesurfer git download from today (v6?), gtx1080ti gpus, brief env recap: 999 cd /usr/local/freesurfer-gpu.git/ 1001 export PATH=/usr/local/cuda-8.0/bin:$PATH 1002 export LD_LIBRARY_PATH=/usr/local/cuda-8.0/lib64 1004 make clean 1008 ./setup_configure | tee -a setup_configure.log 1011 ./configure --disable-Werror --with-pkgs-dir=/usr/local/centos6-x86_64-packages --prefix=/usr/local/freesurfer-6.0.0 --with-cuda=/usr/local/cuda-8.0 | tee -a configure.log 1012 time make -j4 | tee -a make.log 1013 make install [hmeij@n78 freesurfer-gpu.git]$ grep nvcc Makefile NVCC = /usr/local/cuda-8.0/bin/nvcc Q1 what causes this and how to fix, make install fails /bin/install -c fs_spmreg.glnxa64 irepifitvol.glnx64 /usr/local/freesurfer-6.0.0/bin /bin/install: cannot stat â_spmreg.glnxa64âNo such file or directory /bin/install: cannot stat âepifitvol.glnx64âNo such file or directory make[2]: *** [install-data-local] Error 1 make[2]: Leaving directory `/usr/local/freesurfer-gpu.git/scripts' make[1]: *** [install-am] Error 2 make[1]: Leaving directory `/usr/local/freesurfer-gpu.git/scripts' make: *** [install-recursive] Error 1 Q2 no cuda packages, maybe related to Q1 [hmeij@n78 freesurfer-gpu.git]$ ls /usr/local/freesurfer-6.0.0/bin/*cuda* /usr/local/freesurfer-6.0.0/bin/cuda_setup Thanks for any help. -Henk ___ Freesurfer mailing list Freesurfer@nmr.mgh.harvard.edu https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Partners Compliance HelpLine at http://www.partners.org/complianceline . If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail.
Re: [Freesurfer] cuda compile help
I should have mentioned, i did not do any data dirs step, confused on this, just hoping to compile gpu code -Henk - Reply message - From: "Meij, Henk" To: "freesurfer@nmr.mgh.harvard.edu" Subject: [Freesurfer] cuda compile help Date: Fri, Nov 17, 2017 6:24 PM cuda v8 freesurfer git download from today (v6?), gtx1080ti gpus, brief env recap: 999 cd /usr/local/freesurfer-gpu.git/ 1001 export PATH=/usr/local/cuda-8.0/bin:$PATH 1002 export LD_LIBRARY_PATH=/usr/local/cuda-8.0/lib64 1004 make clean 1008 ./setup_configure | tee -a setup_configure.log 1011 ./configure --disable-Werror --with-pkgs-dir=/usr/local/centos6-x86_64-packages --prefix=/usr/local/freesurfer-6.0.0 --with-cuda=/usr/local/cuda-8.0 | tee -a configure.log 1012 time make -j4 | tee -a make.log 1013 make install [hmeij@n78 freesurfer-gpu.git]$ grep nvcc Makefile NVCC = /usr/local/cuda-8.0/bin/nvcc Q1 what causes this and how to fix, make install fails /bin/install -c fs_spmreg.glnxa64 irepifitvol.glnx64 /usr/local/freesurfer-6.0.0/bin /bin/install: cannot stat â_spmreg.glnxa64âNo such file or directory /bin/install: cannot stat âepifitvol.glnx64âNo such file or directory make[2]: *** [install-data-local] Error 1 make[2]: Leaving directory `/usr/local/freesurfer-gpu.git/scripts' make[1]: *** [install-am] Error 2 make[1]: Leaving directory `/usr/local/freesurfer-gpu.git/scripts' make: *** [install-recursive] Error 1 Q2 no cuda packages, maybe related to Q1 [hmeij@n78 freesurfer-gpu.git]$ ls /usr/local/freesurfer-6.0.0/bin/*cuda* /usr/local/freesurfer-6.0.0/bin/cuda_setup Thanks for any help. -Henk ___ Freesurfer mailing list Freesurfer@nmr.mgh.harvard.edu https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Partners Compliance HelpLine at http://www.partners.org/complianceline . If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail.
Re: [Freesurfer] cuda compile help
And that ofcourse was the partial answer, as listed on wiki page, I need to
apply a data dir,, these links are pointing to missing files. Will retry Monday.
-Henk
[hmeij@n78 freesurfer-gpu.git]$ find . -name *glnxa64* -exec ls -l {} \;
lrwxrwxrwx 1 hmeij its 195 Nov 17 08:14 ./scripts/fs_spmreg.glnxa64 ->
../.git/annex/objects/9g/2q/SHA256E-s13078893--f1e2a2ea60c0ad9ed4119bae45feed10cb866b93bf484340ada79a496ca5754a/SHA256E-s13078893--f1e2a2ea60c0ad9ed4119bae45feed10cb866b93bf484340ada79a496ca5754a
lrwxrwxrwx 1 hmeij its 196 Nov 17 08:14 ./fsfast/bin/fast_selxavg3b.glnxa64 ->
../../.git/annex/objects/5P/qx/SHA256E-s6879505--d26c8f55485cd9baea74afc6e34ae48e0384452ff76d498a08b648a4ecb26bce/SHA256E-s6879505--d26c8f55485cd9baea74afc6e34ae48e0384452ff76d498a08b648a4ecb26bce
lrwxrwxrwx 1 hmeij its 194 Nov 17 08:14 ./fsfast/bin/mkcontrast2m.glnxa64 ->
../../.git/annex/objects/P7/77/SHA256E-s105002--bd8037b6bf7bbb40ea5ca7b831b79cbbe5762d8c98f68b41efaf1a6065ec87f4/SHA256E-s105002--bd8037b6bf7bbb40ea5ca7b831b79cbbe5762d8c98f68b41efaf1a6065ec87f4
From: freesurfer-boun...@nmr.mgh.harvard.edu
on behalf of Meij, Henk
Sent: Friday, November 17, 2017 6:35:07 PM
To: Freesurfer support list
Subject: Re: [Freesurfer] cuda compile help
I should have mentioned, i did not do any data dirs step, confused on this,
just hoping to compile gpu code
-Henk
- Reply message -
From: "Meij, Henk"
To: "freesurfer@nmr.mgh.harvard.edu"
Subject: [Freesurfer] cuda compile help
Date: Fri, Nov 17, 2017 6:24 PM
cuda v8 freesurfer git download from today (v6?), gtx1080ti gpus, brief env
recap:
999 cd /usr/local/freesurfer-gpu.git/
1001 export PATH=/usr/local/cuda-8.0/bin:$PATH
1002 export LD_LIBRARY_PATH=/usr/local/cuda-8.0/lib64
1004 make clean
1008 ./setup_configure | tee -a setup_configure.log
1011 ./configure --disable-Werror
--with-pkgs-dir=/usr/local/centos6-x86_64-packages
--prefix=/usr/local/freesurfer-6.0.0 --with-cuda=/usr/local/cuda-8.0 | tee -a
configure.log
1012 time make -j4 | tee -a make.log
1013 make install
[hmeij@n78 freesurfer-gpu.git]$ grep nvcc Makefile
NVCC = /usr/local/cuda-8.0/bin/nvcc
Q1 what causes this and how to fix, make install fails
/bin/install -c fs_spmreg.glnxa64 irepifitvol.glnx64
/usr/local/freesurfer-6.0.0/bin
/bin/install: cannot stat â_spmreg.glnxa64âNo such file or directory
/bin/install: cannot stat âepifitvol.glnx64âNo such file or directory
make[2]: *** [install-data-local] Error 1
make[2]: Leaving directory `/usr/local/freesurfer-gpu.git/scripts'
make[1]: *** [install-am] Error 2
make[1]: Leaving directory `/usr/local/freesurfer-gpu.git/scripts'
make: *** [install-recursive] Error 1
Q2 no cuda packages, maybe related to Q1
[hmeij@n78 freesurfer-gpu.git]$ ls /usr/local/freesurfer-6.0.0/bin/*cuda*
/usr/local/freesurfer-6.0.0/bin/cuda_setup
Thanks for any help.
-Henk
___
Freesurfer mailing list
Freesurfer@nmr.mgh.harvard.edu
https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer
The information in this e-mail is intended only for the person to whom it is
addressed. If you believe this e-mail was sent to you in error and the e-mail
contains patient information, please contact the Partners Compliance HelpLine at
http://www.partners.org/complianceline . If the e-mail was sent to you in error
but does not contain patient information, please contact the sender and properly
dispose of the e-mail.
Re: [Freesurfer] cuda compile help
To close this thread, on centos7 'git annex' is an invalid command. That can be
fixed with
yum install epel-release
yum install git-annex
git-annex get --metadata fstags=makeinstall .
Now the 'make install' finishes without errors.
-Henk
From: freesurfer-boun...@nmr.mgh.harvard.edu
on behalf of Meij, Henk
Sent: Saturday, November 18, 2017 11:28:09 AM
To: Freesurfer support list
Subject: Re: [Freesurfer] cuda compile help
And that ofcourse was the partial answer, as listed on wiki page, I need to
apply a data dir,, these links are pointing to missing files. Will retry Monday.
-Henk
[hmeij@n78 freesurfer-gpu.git]$ find . -name *glnxa64* -exec ls -l {} \;
lrwxrwxrwx 1 hmeij its 195 Nov 17 08:14 ./scripts/fs_spmreg.glnxa64 ->
../.git/annex/objects/9g/2q/SHA256E-s13078893--f1e2a2ea60c0ad9ed4119bae45feed10cb866b93bf484340ada79a496ca5754a/SHA256E-s13078893--f1e2a2ea60c0ad9ed4119bae45feed10cb866b93bf484340ada79a496ca5754a
lrwxrwxrwx 1 hmeij its 196 Nov 17 08:14 ./fsfast/bin/fast_selxavg3b.glnxa64 ->
../../.git/annex/objects/5P/qx/SHA256E-s6879505--d26c8f55485cd9baea74afc6e34ae48e0384452ff76d498a08b648a4ecb26bce/SHA256E-s6879505--d26c8f55485cd9baea74afc6e34ae48e0384452ff76d498a08b648a4ecb26bce
lrwxrwxrwx 1 hmeij its 194 Nov 17 08:14 ./fsfast/bin/mkcontrast2m.glnxa64 ->
../../.git/annex/objects/P7/77/SHA256E-s105002--bd8037b6bf7bbb40ea5ca7b831b79cbbe5762d8c98f68b41efaf1a6065ec87f4/SHA256E-s105002--bd8037b6bf7bbb40ea5ca7b831b79cbbe5762d8c98f68b41efaf1a6065ec87f4
From: freesurfer-boun...@nmr.mgh.harvard.edu
on behalf of Meij, Henk
Sent: Friday, November 17, 2017 6:35:07 PM
To: Freesurfer support list
Subject: Re: [Freesurfer] cuda compile help
I should have mentioned, i did not do any data dirs step, confused on this,
just hoping to compile gpu code
-Henk
- Reply message -
From: "Meij, Henk"
To: "freesurfer@nmr.mgh.harvard.edu"
Subject: [Freesurfer] cuda compile help
Date: Fri, Nov 17, 2017 6:24 PM
cuda v8 freesurfer git download from today (v6?), gtx1080ti gpus, brief env
recap:
999 cd /usr/local/freesurfer-gpu.git/
1001 export PATH=/usr/local/cuda-8.0/bin:$PATH
1002 export LD_LIBRARY_PATH=/usr/local/cuda-8.0/lib64
1004 make clean
1008 ./setup_configure | tee -a setup_configure.log
1011 ./configure --disable-Werror
--with-pkgs-dir=/usr/local/centos6-x86_64-packages
--prefix=/usr/local/freesurfer-6.0.0 --with-cuda=/usr/local/cuda-8.0 | tee -a
configure.log
1012 time make -j4 | tee -a make.log
1013 make install
[hmeij@n78 freesurfer-gpu.git]$ grep nvcc Makefile
NVCC = /usr/local/cuda-8.0/bin/nvcc
Q1 what causes this and how to fix, make install fails
/bin/install -c fs_spmreg.glnxa64 irepifitvol.glnx64
/usr/local/freesurfer-6.0.0/bin
/bin/install: cannot stat â_spmreg.glnxa64âNo such file or directory
/bin/install: cannot stat âepifitvol.glnx64âNo such file or directory
make[2]: *** [install-data-local] Error 1
make[2]: Leaving directory `/usr/local/freesurfer-gpu.git/scripts'
make[1]: *** [install-am] Error 2
make[1]: Leaving directory `/usr/local/freesurfer-gpu.git/scripts'
make: *** [install-recursive] Error 1
Q2 no cuda packages, maybe related to Q1
[hmeij@n78 freesurfer-gpu.git]$ ls /usr/local/freesurfer-6.0.0/bin/*cuda*
/usr/local/freesurfer-6.0.0/bin/cuda_setup
Thanks for any help.
-Henk
___
Freesurfer mailing list
Freesurfer@nmr.mgh.harvard.edu
https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer
The information in this e-mail is intended only for the person to whom it is
addressed. If you believe this e-mail was sent to you in error and the e-mail
contains patient information, please contact the Partners Compliance HelpLine at
http://www.partners.org/complianceline . If the e-mail was sent to you in error
but does not contain patient information, please contact the sender and properly
dispose of the e-mail.
[Freesurfer] GPU compile cuda8/centos7
Just wanted to report back a success. I had to do some fixes so hope this helps anybody. My gpus are GTX1080Ti and the CPU involved is E5-2620 dual 10 core on this server. The vanilla cpu recon-all finished in 7 hours, the gpu run finished in 4.3 hours. Q: I seem to not have any errors or indications why mri_glmfit or mri_robust_register cuda binaries did not compile. Any ideas? -Henk In configure file (from previous post) comment out the with_cuda="" line https://mail.nmr.mgh.harvard.edu/pipermail//freesurfer/2016-June/046236.html CUDA_DIR="" with_cuda="" In utils/mrifilter.c uncomment line defining 'int width, weight, height' above the test if (width <=1 || height <=1 || depth <=1) on line 3005 then find Makefiles with incorrect gpu arch for the gpu packages change architecture from sm_?? to sm_61 (multiple times per file) which nvcc export PATH=/usr/local/cuda/bin:$PATH locate libcuda.so export LD_LIBRARY_PATH=/usr/lib64:/usr/local/cuda/lib64:$LD_LIBRARY_PATH ./configure --disable-Werror --with-pkgs-dir=/usr/local/centos6-x86_64-packages --prefix=/usr/local/freesurfer-6.0.0 --with-cuda=/usr/local/cuda make clean make -j 4 make install yields ./bin/mris_flatten_cuda ./bin/mris_sphere_cuda ./bin/mris_fix_topology_cuda ./bin/mri_segstats_cuda ./bin/mri_em_register_cuda ./bin/mri_vol2vol_cuda ./bin/mris_inflate_cuda ./bin/mri_ca_register_cuda ___ Freesurfer mailing list Freesurfer@nmr.mgh.harvard.edu https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Partners Compliance HelpLine at http://www.partners.org/complianceline . If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail.
Re: [Freesurfer] GPU compile cuda8/centos7
Well, it looks the *_cuda files for mri_glmfit and mri_robust_register and some others were dropped from the repository. Downloaded fresh copy today but the files are not there. So no errors. -Henk From: Meij, Henk Sent: Monday, November 27, 2017 11:02:44 AM To: freesurfer@nmr.mgh.harvard.edu Subject: GPU compile cuda8/centos7 Just wanted to report back a success. I had to do some fixes so hope this helps anybody. My gpus are GTX1080Ti and the CPU involved is E5-2620 dual 10 core on this server. The vanilla cpu recon-all finished in 7 hours, the gpu run finished in 4.3 hours. Q: I seem to not have any errors or indications why mri_glmfit or mri_robust_register cuda binaries did not compile. Any ideas? -Henk In configure file (from previous post) comment out the with_cuda="" line https://mail.nmr.mgh.harvard.edu/pipermail//freesurfer/2016-June/046236.html CUDA_DIR="" with_cuda="" In utils/mrifilter.c uncomment line defining 'int width, weight, height' above the test if (width <=1 || height <=1 || depth <=1) on line 3005 then find Makefiles with incorrect gpu arch for the gpu packages change architecture from sm_?? to sm_61 (multiple times per file) which nvcc export PATH=/usr/local/cuda/bin:$PATH locate libcuda.so export LD_LIBRARY_PATH=/usr/lib64:/usr/local/cuda/lib64:$LD_LIBRARY_PATH ./configure --disable-Werror --with-pkgs-dir=/usr/local/centos6-x86_64-packages --prefix=/usr/local/freesurfer-6.0.0 --with-cuda=/usr/local/cuda make clean make -j 4 make install yields ./bin/mris_flatten_cuda ./bin/mris_sphere_cuda ./bin/mris_fix_topology_cuda ./bin/mri_segstats_cuda ./bin/mri_em_register_cuda ./bin/mri_vol2vol_cuda ./bin/mris_inflate_cuda ./bin/mri_ca_register_cuda ___ Freesurfer mailing list Freesurfer@nmr.mgh.harvard.edu https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Partners Compliance HelpLine at http://www.partners.org/complianceline . If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail.
