Thanks Antonin,
We had suspected as much. For those who might want to use NITRC-CE (FS v6.0 via their AWS service in the future), please note that our initial (and memory-insufficient) "instance" was "c3.large" <failed>. As you can see in the table below, we should have paid more attention to the guidance regarding memory from the installation instructions. Since "c3.xlarge had only 7.5 GB memory, we opted for the next tier, "c3.2xlarge".
For for this initial benchmark, we used an ADNI subject (N3-corrected, downloaded from ADNI/LONI as *.nii), and the run completed in toto and took 9.038 hrs.
Much appreciated,
-Randy
Instance Types Matrix | |||||||||||
Instance Type | vCPU | Memory (GiB) | Storage (GB) | Networking Performance | Physical Processor | Clock Speed (GHz) | Intel AVX† | Intel AVX2† | Intel Turbo | EBS OPT | Enhanced Networking† |
c3.large | 2 | 3.75 | 2 x 16 SSD | Moderate | Intel Xeon E5-2680 v2 | 2.8 | Yes | - | Yes | - | Yes |
c3.xlarge | 4 | 7.5 | 2 x 40 SSD | Moderate | Intel Xeon E5-2680 v2 | 2.8 | Yes | - | Yes | Yes | Yes |
c3.2xlarge | 8 | 15 | 2 x 80 SSD | High | Intel Xeon E5-2680 v2 | 2.8 | Yes | - | Yes | Yes | Yes |
Re: [Freesurfer] FreeSurfer v6.0 in an AWS cloud configuration (hipposubfield/brainstem failures)
Antonin Skoch Thu, 02 Mar 2017 09:15:57 -0800
Antonin Skoch Thu, 02 Mar 2017 09:15:57 -0800
Dear Randy, from the log file you provided it seems that both of your jobs exited on the insufficient RAM. You should check your available RAM resources.
brainstem-structures: std::bad_alloc hippocampal subfiels: Out of memory. Regards, Antonin
Randolph D. Andrews, MS
Staff Scientist | ADM Diagnostics, LLC
phone +1 847 978 5007 | randr...@admdx.com
Staff Scientist | ADM Diagnostics, LLC
phone +1 847 978 5007 | randr...@admdx.com
-------- Original Message --------
Subject: FreeSurfer v6.0 in an AWS cloud configuration
(hipposubfield/brainstem failures)
From: <randr...@admdx.com>
Date: Thu, March 02, 2017 9:18 am
To: freesurfer@nmr.mgh.harvard.edu
Hello FreeSurfer Experts/Community,I am currently working with NITRC (as a novice user) to build and investigate the operability of FreeSurfer v6.0 in their Computational Environment (NITRC-CE).We have been able to get a "recon-all" run to work, but we are failing to get the sub-calls-hippocampal-subfields-T1 -brainstem-structures(see link earlier in thread) (see link earlier in thread) ... to run correctly / run to completion. We are aware that Matlab Runtime needed to be installed, and a partial completion is a testament to that.However, they seem to choke near the end, and it would be good to get some insight as to why. To wit, I am pasting the last dozen or so lines from each (see below) in the hopes we might get a tip or a comment from this forum.For this exercise, we used an ADNI subject (N3-corrected, downloaded from ADNI/LONI as *.nii)The syntax/command I used was:recon-all -s ADNI_003_S_2374_m00 \
-i $SUBJECTS_DIR/ADNI_003_S_2374__MRI_m00__EMCI-EMCI__20110506__S108554_I235638.nii \
-all -hippocampal-subfields-T1 -brainstem-structuresI'm simply not sure if there's something wrong with my syntax or if there is something else we're missing. Any help would be greatly appreciated, and I can pass on what is shared to my developer (and so that eventually everyone might begin to have a vetted version of v6.0 using NITRC-CE if they wish in the future)-RandyFrom:/scripts/brainstem-structures_______________________________________________________________<<<start excerpt>>>
Resolution level 3 iteration 3 deformation iterations 8
Did one deformation step of max. 1.6685e-05 voxels in 8.485 seconds
minLogLikelihoodTimesPrior =
7.5772e+06
Resolution level 3 iteration 3 deformation iterations 9
Did one deformation step of max. 5.3959e-05 voxels in 3.4185 seconds
minLogLikelihoodTimesPrior =
7.5772e+06
Resolution level 3 iteration 3 deformation iterations 10
Did one deformation step of max. 0 voxels in 6.782 seconds
minLogLikelihoodTimesPrior =
7.5772e+06
maximalDeformation is too small; stopping
Fitting mesh to image data mask took 919.7675 seconds
numberOfLabels: 21
Rasterizing mesh...here: 21
here2: 21
done
Error using kvlGEMSMatlab
std::bad_alloc
Error in kvlRasterizeAtlasMesh (line 11)
Error in segmentSubject (line 1151)____________________________________________________________________________________________________________<<<end of logfile>>>______________________________________________________From:/scripts/hippocampal-subfields-T1_______________________________________________________________<<<start excerpt>>>
Making alveus map to reduced label 3
--------------
Making Left-Lateral-Ventricle map to reduced label 4
--------------
Making hippocampal-fissure map to reduced label 5
--------------
Making Left-Pallidum map to reduced label 6
--------------Making Left-Putamen map to reduced label 7
--------------
Making Left-Caudate map to reduced label 8
--------------
Making Left-Thalamus-Proper map to reduced label 9
--------------
Making Left-choroid-plexus map to reduced label 10
--------------
Making Left-VentralDC map to reduced label 11
--------------
Making Left-Accumbens-area map to reduced label 12
--------------
Making Unknown map to reduced label 13
Computing hyperparameters for estimation of Gaussian parameters
Estimating typical intensities of alveus
numberOfLabels: 13
Rasterizing mesh...here: 13
here2: 13
done
Error using double
Out of memory. Type HELP MEMORY for your options.
Error in segmentSubjectT1_autoEstimateAlveusML (line 834)
MATLAB:nomem______________________________________________________<<<end of logfile>>>____________________________________________________________________________________________________________