agreed on Rocker and it gives me an excuse to bug Dirk. Errr, I mean, test it
Statistics is the grammar of science. Karl Pearson <http://en.wikipedia.org/wiki/The_Grammar_of_Science> On Mon, Nov 10, 2014 at 10:04 AM, Dan Tenenbaum <dtene...@fredhutch.org> wrote: > > > ----- Original Message ----- > > From: "Laurent Gautier" <lgaut...@gmail.com> > > To: "Martin Morgan" <mtmor...@fredhutch.org> > > Cc: bioc-devel@r-project.org, "Dan Tenenbaum" <dtene...@fredhutch.org> > > Sent: Monday, November 10, 2014 9:57:00 AM > > Subject: Re: [Bioc-devel] PPA with built bioconductor packages (for > continuous integration) > > > > > > > > They would work in the context of well defined system such as the VM > > used by popular continuous integration providers (Travis or Drone > > for example). > > > > Then it would be easy as having the binaries built as artifacts by > > continuous integration and made available to other continuous > > integration processes. > > This sounds to me like a pretty good use case for docker/rocker. We just > need to define what packages should be installed on a given image; I don't > think we want the images to be too big (unlike the AMI). The images could > be rebuilt daily. So you'd still need to download the diffs from the > previous image but I imagine this would take less time than building those > packages from source. > > Dan > > > > > > On Nov 10, 2014 6:19 PM, "Martin Morgan" < mtmor...@fredhutch.org > > > wrote: > > > > > > On 11/09/2014 11:06 AM, Dan Tenenbaum wrote: > > > > > > > > > > ----- Original Message ----- > > > > > > From: "Martin Morgan" < mtmor...@fredhutch.org > > > To: "Laurent Gautier" < lgaut...@gmail.com >, > > bioc-devel@r-project.org > > Sent: Sunday, November 9, 2014 8:26:48 AM > > Subject: Re: [Bioc-devel] PPA with built bioconductor packages (for > > continuous integration) > > > > On 11/09/2014 07:23 AM, Laurent Gautier wrote: > > > > > > Hi, > > > > Continuous integration is a convenient way to automate some of the > > steps > > necessary to ensure quality software. > > > > Popular ways to do it create a vanilla virtual machine 9VM) with a > > Linux > > distribution, and scripts prepares the VM with 3rd-party > > dependencies > > required by the software. For example, the popular CI system Travis > > for > > github creates by default a VM running ubuntu, and dependencies can > > be > > installed with `apt-get install`. > > > > When developing software that requires CRAN/bioconductor, the > > latest R is > > available precompiled but the R packages must be downloaded > > installed from > > source. > > > > This can take a relatively long time. On a recent project over 80% > > of the > > time is spent downloading/installing the R/BioC packages. The > > remaining is > > building the code and running the unit tests. > > > > Having a Personal Package Archive (PPA) with bioconductor packages > > already > > compiled would both speed up the process and make the use of > > continuous > > integration by projects relying on bioconductor packages easier. > > > > Is this something others would like to have, and is this something > > that > > bioconductor would see to its mission to provide / help provide > > quality > > software and be able to host ? > > > > It would be interesting to catalogue objectives (e.g., development > > vs. > > reproducibility) and available alternatives (e.g., PPA, docker / > > Rocker, AMI, > > existing or possible cloud services [such as the Bioc 'single package > > builder' > > used to build and check new package submissions, or travis itself], > > the Becker > > repository management scheme Michael and Gabe mention, ...); > > > > > > Just to add to the mix of options, it's possible to run > > R CMD INSTALL --build on a source tarball on Linux and it will create > > a 'binary' version that is already compiled. > > > > These binaries are in general not portable, either within or between > > distributions, e.g., because the user has a different version of a > > system dependency than the one the binary was built against. > > > > Martin > > > > > > > > The problem with this is (AFAIK) there is no corresponding package > > type that can be used with install.packages(); > > otherwise the simplest solution would be to add a CRAN-style repos > > containing these "binaries". Maybe R could be patched to allow this? > > But it's possible that the requirements for Linux "binaries" could > > vary depending on many things: cpu type (intel or solaris, or...), > > architecture (i386, x64), presence/absence of BLAS/LAPACK, etc etc > > etc. This suggests that a vm or container-based approach might be > > better. > > > > Dan > > > > > > > > > > > > > > if there > > is a clear > > path forward satisfying some plurality of users without too many > > technical > > obstacles then it might fall within the Bioc purview; my initial > > sense is that > > there is not a consensus on use cases or viable implementations, but > > I can be > > convinced otherwise... > > > > In terms of Tim's post, getting your colleague to use a PPA / > > existing > > alternative (e.g., the Bioc AMI, > > http://bioconductor.org/help/ bioconductor-cloud-ami/ which comes > > with > > Rstudio > > server installed...) is not likely to be easier / faster than getting > > them to > > download / install relevant R / Bioc packages. One interesting > > possibility is a > > 'hosted' bioconductor with sufficient computational resources on the > > back-end > > and Rstudio server on the front end; this is not impossible to > > imaging seeking > > funding for. > > > > > > > > > > > > > > Martin > > > > > > > > > > Best, > > > > Laurent > > > > [[alternative HTML version deleted]] > > > > ______________________________ _________________ > > Bioc-devel@r-project.org mailing list > > https://stat.ethz.ch/mailman/ listinfo/bioc-devel > > > > > > > > -- > > Computational Biology / Fred Hutchinson Cancer Research Center > > 1100 Fairview Ave. N. > > PO Box 19024 Seattle, WA 98109 > > > > Location: Arnold Building M1 B861 > > Phone: (206) 667-2793 > > > > ______________________________ _________________ > > Bioc-devel@r-project.org mailing list > > https://stat.ethz.ch/mailman/ listinfo/bioc-devel > > > > > > > > -- > > Computational Biology / Fred Hutchinson Cancer Research Center > > 1100 Fairview Ave. N. > > PO Box 19024 Seattle, WA 98109 > > > > Location: Arnold Building M1 B861 > > Phone: (206) 667-2793 > > > > _______________________________________________ > Bioc-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/bioc-devel > [[alternative HTML version deleted]] _______________________________________________ Bioc-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/bioc-devel
