Thanks for getting back. The patched mri_segstats works. Here are the results of our benchmark runs. Each result comes from a single run so the precision is questionable. But it seems clear that 8 processors makes little difference compared with 4. And parallelizing by side also makes little difference.
I've included the scripts both for the simple runs, i.e. recon-all -all, and
the parallized run.
The cores were Haswell processors with lots of local memory.
Serial run (4 cores): 5.465 hours
Serial run (8 cores): 5.251 hours
Parallel run (8 cores): 5.197 hours
The parallel run broke out as follows:
autorecon1(8 cores): 0.189 hours
autorecon2-volonly (8 cores): 1.136 hours
2 autorecon2-perhemi (4 cores): 0.745/0.659 hours
It was the longer of these which was used for the total run time of 5.197 hours
autorecon3: 3.127 hours
recon-all -autorecon3 includes no executables which use multiple cores.
Regards,
Don
Don Krieger, Ph.D.
Department of Neurological Surgery
University of Pittsburgh
> -----Original Message-----
> From: freesurfer-boun...@nmr.mgh.harvard.edu [mailto:freesurfer-
> boun...@nmr.mgh.harvard.edu] On Behalf Of Z K
> Sent: Friday, May 01, 2015 5:38 PM
> To: Freesurfer support list
> Subject: Re: [Freesurfer] problem with left/right parallelization
>
> Yup this is a known issue with the mri_segstats binary.
>
> Please go to the following page and download the mri_segstats file that
> corresponds to your version of freesurfer (see contents of
> $FREESURFER_HOME/build-stampt.txt file to determine your version).
> Replace the version of mri_segstats in your $FREESURFER_HOME/bin directory
> with the one you download from the following link. Also, it would be good to
> backup the version you currently have.
>
> ftp://surfer.nmr.mgh.harvard.edu/pub/dist/freesurfer/5.3.0-patch/
>
> -Zeke
>
> On 05/01/2015 05:02 PM, Krieger, Donald N. wrote:
> > I have completed runs of recon-all -autorecon1 and recon-all
> > -autorecon2-volonly per your suggested sequence:
> >
> > recon-all -s subj -autorecon1
> >
> > recon-all -s subj -autorecon2-volonly
> >
> > recon-all -s subj -autorecon2-perhemi -hemi rh -log recon-all-hemi
> > rh.log -notify rh-done.log -openmp 4 &
> >
> > recon-all -s subj -autorecon2-perhemi -hemi lh -log
> > recon-all-hemi-lh.log -notify lh-done.log -openmp 4 &
> >
> > while (rh-done.log AND lh-done.log do not both exist)
> >
> > do
> >
> > sleep 1
> >
> > end loop
> >
> > recon-all -s -autorecon3
> >
> > recon-all -s subj -autorecon2-volonly is completing but with an error.
> >
> > I've included below my signature the last 50 lines or so of
> > recon-all.log (file attached).
> >
> > It looks like it's completing execution of mri_fill successfully and
> > then going on to execute mri_segstats out of order.
> >
> > In the log for a standard run on this same image set, i.e. recon-all
> > -all, that mri_segstats cmd doesn't appear to occur till 3000 lines
> > later in the log file. And as the log file shows, no ribbon.mgz can
> > be found and inspection of the directories shows that surf is empty.
> >
> > It looks like something is missing between recon-all
> > -autorecon2-volonly and recon-all -autorecon2-perhemi
> >
> > What do you think?
> >
> > Thanks,
> >
> > Don
> >
> > Don Krieger, Ph.D.
> >
> > Department of Neurological Surgery
> >
> > Universityof Pittsburgh
> >
> > mri_pretess done
> >
> > #--------------------------------------------
> >
> > #@# Fill Fri May 1 12:28:01 PDT 2015
> >
> > /home/donkri/Contrib/freesurfer/subjects/173_9sh_lr/mri
> >
> > mri_fill -a ../scripts/ponscc.cut.log -xform transforms/talairach.lta
> > -segmentation aseg.auto_noCCseg.mgz wm.mgz filled.mgz
> >
> > logging cutting plane coordinates to ../scripts/ponscc.cut.log...
> >
> > INFO: Using transforms/talairach.lta and its offset for Talairach volume ...
> >
> > using segmentation aseg.auto_noCCseg.mgz...
> >
> > reading input volume...done.
> >
> > searching for cutting planes...voxel to talairach voxel transform
> >
> > 0.964 -0.026 -0.002 8.591;
> >
> > 0.022 1.002 0.185 -15.064;
> >
> > -0.012 -0.183 0.940 20.634;
> >
> > 0.000 0.000 0.000 1.000;
> >
> > voxel to talairach voxel transform
> >
> > 0.964 -0.026 -0.002 8.591;
> >
> > 0.022 1.002 0.185 -15.064;
> >
> > -0.012 -0.183 0.940 20.634;
> >
> > 0.000 0.000 0.000 1.000;
> >
> > reading segmented volume aseg.auto_noCCseg.mgz...
> >
> > Looking for area (min, max) = (350, 1400)
> >
> > area[0] = 2029 (min = 350, max = 1400), aspect = 0.86 (min = 0.10, max
> > =
> > 0.75)
> >
> > need search nearby
> >
> > using seed (128, 110, 91), TAL = (0.0, -37.0, 18.0)
> >
> > talairach voxel to voxel transform
> >
> > 1.037 0.026 -0.003 -8.449;
> >
> > -0.025 0.963 -0.190 18.642;
> >
> > 0.009 0.188 1.027 -18.439;
> >
> > 0.000 0.000 0.000 1.000;
> >
> > segmentation indicates cc at (128, 110, 91) --> (0.0, -37.0, 18.0)
> >
> > done.
> >
> > writing output to filled.mgz...
> >
> > filling took 0.5 minutes
> >
> > talairach cc position changed to (0.00, -37.00, 18.00)
> >
> > Erasing brainstem...done.
> >
> > seed_search_size = 9, min_neighbors = 5
> >
> > search rh wm seed point around talairach space:(18.00, -37.00, 18.00)
> > SRC: (108.24, 104.54, 96.63)
> >
> > search lh wm seed point around talairach space (-18.00, -37.00,
> > 18.00),
> > SRC: (145.58, 103.65, 96.94)
> >
> > compute mri_fill using aseg
> >
> > Erasing Brain Stem and Cerebellum ...
> >
> > Define left and right masks using aseg:
> >
> > Building Voronoi diagram ...
> >
> > Using the Voronoi diagram to separate WM into two hemispheres ...
> >
> > Find the largest connected component for each hemisphere ...
> >
> > #--------------------------------------------
> >
> > #@# ASeg Stats Fri May 1 12:28:33 PDT 2015
> >
> > INFO: mri_segstats will not calculate Supratentorial
> >
> > due to missing ribbon.mgz file
> >
> > INFO: ?h.white surfaces not available to mri_segstats.
> >
> > /home/donkri/Contrib/freesurfer/subjects/173_9sh_lr
> >
> > mri_segstats --seg mri/aseg.mgz --sum stats/aseg.stats --pv
> > mri/norm.mgz --empty --brainmask mri/brainmask.mgz
> > --brain-vol-from-seg --excludeid 0 --excl-ctxgmwm --subcortgray --in
> > mri/norm.mgz --in-intensity-name norm --in-intensity-units MR --etiv
> > --ctab /ho
> >
> > me/donkri/Contrib/freesurfer/ASegStatsLUT.txt --subject 173_9sh_lr
> >
> > $Id: mri_segstats.c,v 1.75.2.9 2013/02/16 00:09:33 greve Exp $
> >
> > cwd
> >
> > cmdline mri_segstats --seg mri/aseg.mgz --sum stats/aseg.stats --pv
> > mri/norm.mgz --empty --brainmask mri/brainmask.mgz
> > --brain-vol-from-seg --excludeid 0 --excl-ctxgmwm --subcortgray --in
> > mri/norm.mgz --in-intensity-name norm --in-intensity-units MR --etiv
> > --c
> >
> > tab /home/donkri/Contrib/freesurfer/ASegStatsLUT.txt --subject
> > 173_9sh_lr
> >
> > sysname Linux
> >
> > hostname comet-08-01.sdsc.edu
> >
> > machine x86_64
> >
> > user donkri
> >
> > UseRobust 0
> >
> > atlas_icv (eTIV) = 1971942 mm^3 (det: 0.987912 )
> >
> >
> MRISread(/home/donkri/Contrib/freesurfer/subjects/173_9sh_lr/surf/lh.white):
> > could not open file
> >
> > Loading mri/aseg.mgz
> >
> > Getting Brain Volume Statistics
> >
> > No such file or directory
> >
> > Linux comet-08-01.sdsc.edu 3.10.73-1.el6.elrepo.x86_64 #1 SMP Thu Mar
> > 26
> > 16:28:30 EDT 2015 x86_64 x86_64 x86_64 GNU/Linux
> >
> > recon-all -s 173_9sh_lr exited with ERRORS at Fri May 1 12:28:33 PDT
> > 2015
> >
> > To report a problem, see
> > http://surfer.nmr.mgh.harvard.edu/fswiki/BugReporting
> >
> >
> >
> > _______________________________________________
> > Freesurfer mailing list
> > Freesurfer@nmr.mgh.harvard.edu
> > https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer
> >
> _______________________________________________
> Freesurfer mailing list
> Freesurfer@nmr.mgh.harvard.edu
> https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer
>
>
> The information in this e-mail is intended only for the person to whom it is
> addressed. If you believe this e-mail was sent to you in error and the e-mail
> contains patient information, please contact the Partners Compliance HelpLine
> at http://www.partners.org/complianceline . If the e-mail was sent to you in
> error but does not contain patient information, please contact the sender and
> properly dispose of the e-mail.
freesurf_2hemis
Description: freesurf_2hemis
freesurf_simple
Description: freesurf_simple
_______________________________________________ Freesurfer mailing list Freesurfer@nmr.mgh.harvard.edu https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Partners Compliance HelpLine at http://www.partners.org/complianceline . If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail.
