At The Uni. of Chicago, we've been playing around with parallelizing freesurfer on a 128 node grid (256 processors). Developers have parallelized procedures (scripts) by unpacking "for" loops that rotate across left and right hemispheres [i.e., they fork the independent processing of left and right hemispheres to different nodes running in parallel, whenever possible].
The main point of this work is to acquire provenance records and therefore Freesurfer scripts are "wrapped" or expressed using a virtual data system language (VDS/VDL). The freesurfer implementation, AFAIK, is in its baby steps, but the general workflow model is pretty well established http://www.ci.uchicago.edu/wiki/bin/view/VDS/VDSWeb/WebMain Best, Uri On 12/6/06, Uri Hasson <[EMAIL PROTECTED]> wrote:
Here (Uni. of Chicago), we've been playing around with parallelizing freesurfer on a 128 node grid (256 processors), and developers have parallelized procedures (scripts) by unpacking "for" loops that rotate across left and right hemispheres [i.e., they fork the independent processing of left and right hemispheres to different nodes running in parallel, whenever possible]. The main point of this work is to acquire provenance records and therefore Freesurfer scripts are "wrapped" or expressed using a virtual data system language (VDS/VDL). The freesurfer implementation, AFAIK, is in its baby steps, but the general workflow model is pretty well established http://www.ci.uchicago.edu/wiki/bin/view/VDS/VDSWeb/WebMain Best, Uri On 12/6/06, Nick Schmansky <[EMAIL PROTECTED]> wrote: > Andries, > > I am not aware of usage of Freesurfer in a (Sun) Grid Engine environment > (such as that used by the Cohen group at UCLA). > > However, here at the MGH/MIT/HMS Martinos Center we use a cluster of > some 100+ nodes configured with Linux Centos 4, and governed by PBS > (Portable Batch System). Researchers here often conduct studies with > dozens to hundreds of brains, and for each subject, an instance of > Freesurfer's 'recon-all -s <subject> -all' script is submitted to the > batch system, which, under the hood, gets submitted to one computing > node. Thus, several dozen brains can be processed in a day (and a > half). > > Freesurfer does not currently support fine-grain parallelism. Some > coarse-grain parallelism, whereby each brain hemisphere is processed > independently (benefiting multiprocessor nodes) is possible, but not > currently implemented in our 'recon-all' script, as the error handling > and logging for doing so is somewhat tricky (and so this feature is in- > the-works-but-not-anytime-soon). > > In short, if you plan on using Freesurfer in studies with large numbers > of subjects, I would recommend some kind of computing cluster, and some > fairly simple batch software (like PBS) should be sufficient. For > instance, I know of one group that has successfully run Freesurfer on > their Altix Itanium Linux cluster. > > Groetjes, > > Nick > > > On Wed, 2006-12-06 at 18:44 +0100, Andries van der Leij wrote: > > > > > > > > > > > > ______________________________________________________________________ > > > > From: Andries van der Leij > > Sent: Wednesday, December 06, 2006 5:59 PM > > To: 'freesurfer@nmr.mgh.harvard.edu' > > Subject: Freesurfer and Grid computing > > > > > > > > > > Dear Freesurfer community, > > > > > > > > I'm a PHD student at the university of Amsterdam and I'm currently > > investigating the possibilities to streamline our MRI data processing > > stream. Next summer we'll obtain a research-only scanner. I'm trying > > to push the group to also invest in computing power and am currently > > investigating the applications that researchers will most probably > > use. > > > > > > > > I came across a project of the group of Cohen at UCLA. They have > > configured a Apple (unix) grid and have proposed a more or less > > standard setup specially designed for MRI analyses: > > > > > > > > http://airto.bmap.ucla.edu/mt- > > static/NICluster/archives/2005/06/welcome.html > > > > > > > > It is my understanding that one of the members has rewritten the FSL > > code which allow distributed parallel processing in a Grid. See the > > benchmarks here: > > > > > > > > http://airto.bmap.ucla.edu/bmcweb/bmc_bios/MarkCohen/Apple/Benchmarks.htm > > > > > > > > My question is fairly simple: Are similar steps taken in the > > Freesurfer community? I have no experience with this app myself, but > > it is my understanding that Freesurfer consumes a lot of resources. > > > > > > > > Thank you very much in advance, > > > > > > > > Andries van der Leij > > > > > > _______________________________________________ > > Freesurfer mailing list > > Freesurfer@nmr.mgh.harvard.edu > > https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer > > _______________________________________________ > Freesurfer mailing list > Freesurfer@nmr.mgh.harvard.edu > https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer >
_______________________________________________ Freesurfer mailing list Freesurfer@nmr.mgh.harvard.edu https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer
