Thanks again Pedro. I just ran 220 brains through in day or two using the cloud. Fantastic. The script people have been talking about would have been useful. One thing I didn't realize was that I should have moved the freesurfer subject directory from /usr/local/freesurfer/subjects to the larger disk that comes with the instance. I had disk-write errors because the the main partition ran out of space.
Anyway, ec2 rocks with Freesurfer. - Joshua Lee Graduate Student Center for Mind and Brain & Department of Psychology University of California, Davis 530.747.3805 2011/11/12 Nolan Nichols <bnn...@uw.edu> > If only running a small number of instances, I think it is reasonable > to leave the ami w/o a .license file since it is easy enough to scp it > up to the cloud. For running a cluster it would be nice to have a > script, perhaps using Fabric, to upload a .license file to each > instance in a cluster. As it stands, the starcluster FS AMI does not > have a .license file and requires one to be uploaded manually. > > Nolan > > 2011/11/12 Thomas Ballinger <thomasballin...@gmail.com>: > > Perhaps the AMI image could be without a licence file and uploading one > > (acquired though normal means) would be required? (I suppose that's the > > basic solution we're looking for a clean way around) > > > > 2011/11/12 Pedro Paulo de Magalhães Oliveira Junior < > p...@netfilter.com.br> > >> > >> Maybe we can set up a web server in the Martinos center and the AMI > Image > >> can authenticate in it. > >> --------------------------------------------------------------------- > >> Pedro Paulo de Magalhães Oliveira Junior > >> Netfilter & SpeedComm Telecom > >> -- www.netfilter.com.br > >> -- For mobile: http://itunes.apple.com/br/artist/netfilter/id365306441 > >> > >> > >> > >> > >> On Sat, Nov 12, 2011 at 15:28, Nick Schmansky < > ni...@nmr.mgh.harvard.edu> > >> wrote: > >>> > >>> I'm happy to see the work getting the cloud to be feasible. I still > need > >>> to catch up on this thread but one issue that I'd like to sort out is > >>> tracking the license count. Its fine for a new user to use an existing > >>> image, but I think there will need to be a way to remind people to > request > >>> our .license file even if they don't actually use the file so that > cloud > >>> usages get counted (license count is critical for us for grant renewal > >>> purposes). Ideas on a clean way to do this? > >>> N. > >>> > >>> On Nov 12, 2011, at 11:10 AM, Thomas Ballinger > >>> <thomasballin...@gmail.com> wrote: > >>> > >>> Satra, the ipython example looks great. The scripts I posted also use > >>> fabric for running Freesurfer, but don't download Freesurfer due to it > >>> already being installed on Pedro's image. I agree with Satra that > >>> individually administered instances might work better for those > wishing to > >>> run a few cases with Freesurfer, particularly due to the predictable > long > >>> running time (lessening the benefits of job distribution) and ability > of a > >>> recon-all job to efficiently use all of a ec2 small instance's cpu. I > think > >>> starcluster is pretty awesome, and I would love to push people in that > >>> direction, but it won't be quite as push-button, and the benefits may > be > >>> minimal for small use cases. That's not to say there aren't plenty of > folks > >>> who would rather use Nolan's solution (me for one :)) > >>> Tom > >>> > >>> > >>> On Sat, Nov 12, 2011 at 9:14 AM, Nolan Nichols <bnn...@uw.edu> wrote: > >>>> > >>>> Hi Satra, > >>>> > >>>> Fabric does look nice, thanks for pointing it out and for a link to > the > >>>> ipython_in_a_box example. > >>>> > >>>> That framework might work for what I had in mind for freesurfer and > >>>> starcluster, which uses the same idea to keep the initial AMI > footprint > >>>> smaller/flexible. > >>>> > >>>> Cheers, > >>>> > >>>> Nolan > >>>> > >>>> On Nov 12, 2011 7:32 AM, "Satrajit Ghosh" <sa...@mit.edu> wrote: > >>>>> > >>>>> hi nolan, > >>>>> i think starcluster is a great way to create a cluster on aws. but > for > >>>>> those wanting to create a single instance and install things on an > ami > >>>>> without actually having a large ami, fabric might be a good option. > >>>>> fabric: > >>>>> http://docs.fabfile.org/en/1.3.2/index.html > >>>>> example: (another neurodebian ami) > >>>>> > https://github.com/wholeslide/ipython_in_a_box/blob/master/fabfile.py > >>>>> cheers, > >>>>> > >>>>> satra > >>>>> > >>>>> On Sat, Nov 12, 2011 at 6:11 AM, Nolan Nichols <bnn...@uw.edu> > wrote: > >>>>>> > >>>>>> Thanks, Pedro! > >>>>>> > >>>>>> I am glad to see that you posted a public AMI for freesurfer. And > >>>>>> thanks to Thomas for producin documentation on how to use it. > >>>>>> > >>>>>> This thread inspired me share a StarCluster AMI (ami-27f1384e in the > >>>>>> us-east-1 region) that I built for batch FS jobs, which includes the > >>>>>> initial setup for installing tools from the NeuroDebian repository. > >>>>>> I've also been playing around with using s3fs, which allows Amazon's > >>>>>> S3 to be mounted as a drive and accessed directly as the > >>>>>> "subjects_dir". > >>>>>> > >>>>>> The AMI can be launched here: > >>>>>> > >>>>>> > https://console.aws.amazon.com/ec2/home?region=us-east-1#launchAmi=ami-27f1384e > >>>>>> > >>>>>> I did not include the .license for FS as I wasn't sure about > licensing > >>>>>> issues, but I would be happy to publish an AMI with a built-in > license > >>>>>> if I get the go ahead. For now you will need to start a cluster and > >>>>>> your license to each instance. > >>>>>> > >>>>>> A few details: > >>>>>> > >>>>>> Ubuntu 10.04 x86 > >>>>>> built from StarClusters ami-8cf913e5 > >>>>>> us-east-1 region > >>>>>> See the StarCluster details at http://web.mit.edu/stardev/cluster/ > >>>>>> Customizing StarCluster > >>>>>> > >>>>>> ( > http://web.mit.edu/stardev/cluster/docs/latest/manual/create_new_ami.html) > >>>>>> s3fs (http://code.google.com/p/s3fs/) > >>>>>> Freesurfer 5.1 32-bit > >>>>>> NeuroDebian ready (so you can apt-get install mricron, fsl, nipype, > >>>>>> etc.) > >>>>>> > >>>>>> Thanks again for starting this thread, and let me know if I can help > >>>>>> in any way. It would be great to see "cloud" resources be readily > >>>>>> accessibly to the community with minimal effort to get a cluster > with > >>>>>> freesurfer and tools available via neurodebian up and running. > >>>>>> > >>>>>> Cheers, > >>>>>> > >>>>>> Nolan > >>>>>> _______________________________________________ > >>>>>> Freesurfer mailing list > >>>>>> Freesurfer@nmr.mgh.harvard.edu > >>>>>> https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer > >>>>>> > >>>>>> > >>>>>> The information in this e-mail is intended only for the person to > whom > >>>>>> it is > >>>>>> addressed. If you believe this e-mail was sent to you in error and > the > >>>>>> e-mail > >>>>>> contains patient information, please contact the Partners Compliance > >>>>>> HelpLine at > >>>>>> http://www.partners.org/complianceline . If the e-mail was sent to > you > >>>>>> in error > >>>>>> but does not contain patient information, please contact the sender > >>>>>> and properly > >>>>>> dispose of the e-mail. > >>>>>> > >>>>> > >>>> > >>>> _______________________________________________ > >>>> Freesurfer mailing list > >>>> Freesurfer@nmr.mgh.harvard.edu > >>>> https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer > >>>> > >>>> > >>>> The information in this e-mail is intended only for the person to whom > >>>> it is > >>>> addressed. If you believe this e-mail was sent to you in error and the > >>>> e-mail > >>>> contains patient information, please contact the Partners Compliance > >>>> HelpLine at > >>>> http://www.partners.org/complianceline . If the e-mail was sent to > you > >>>> in error > >>>> but does not contain patient information, please contact the sender > and > >>>> properly > >>>> dispose of the e-mail. > >>>> > >>> > >>> _______________________________________________ > >>> Freesurfer mailing list > >>> Freesurfer@nmr.mgh.harvard.edu > >>> https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer > >>> > >>> _______________________________________________ > >>> Freesurfer mailing list > >>> Freesurfer@nmr.mgh.harvard.edu > >>> https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer > >>> > >>> > >>> The information in this e-mail is intended only for the person to whom > it > >>> is > >>> addressed. If you believe this e-mail was sent to you in error and the > >>> e-mail > >>> contains patient information, please contact the Partners Compliance > >>> HelpLine at > >>> http://www.partners.org/complianceline . If the e-mail was sent to > you in > >>> error > >>> but does not contain patient information, please contact the sender and > >>> properly > >>> dispose of the e-mail. > >>> > >> > > > > > > _______________________________________________ > Freesurfer mailing list > Freesurfer@nmr.mgh.harvard.edu > https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer >
_______________________________________________ Freesurfer mailing list Freesurfer@nmr.mgh.harvard.edu https://mail.nmr.mgh.harvard.edu/mailman/listinfo/freesurfer The information in this e-mail is intended only for the person to whom it is addressed. If you believe this e-mail was sent to you in error and the e-mail contains patient information, please contact the Partners Compliance HelpLine at http://www.partners.org/complianceline . If the e-mail was sent to you in error but does not contain patient information, please contact the sender and properly dispose of the e-mail.
