Re: [Bioc-devel] SummarizedExperiment with alternate back end

2015-09-18 Thread Ryan
For what it's worth, I've written a class which I have creatively named SubsettableListOfArrays which basically taking the "subset everything together" aspect of eSet and SummarizedExperiment and making it as generic as possible. It's basically like (non-ranged) SummarizedExperiment, except tha

Re: [Bioc-devel] SummarizedExperiment with alternate back end

2015-09-18 Thread Michael Lawrence
While it's useful (and often necessary) to store the big matrices out of core, it would be convenient to store the metadata (the other components of the object) along with the matrices. Something along the lines of HDF5, but we would want to keep things abstract. Other options include GDS (for geno

Re: [Bioc-devel] SummarizedExperiment with alternate back end

2015-09-18 Thread Peter Haverty
While we are on the topic, my GenoSet class will become a subclass of RangedSummarizedExperiment, rather than eSet, after this upcoming release. For this release both APIs work (colnames and sampleNames, etc.) I think the range-free SummarizedExperiment will be great. I've seen a lot of Expression

Re: [Bioc-devel] Merging git mirror and previous git repo ?

2015-09-18 Thread Robert M. Flight
I think the simplest course is to leave your remote as is, and not use the Bioconductor mirror as a true remote. Instead, keep using the git-svn bridge to push to Bioconductor. And put your github remote as the place to submit issues, etc Robert On Fri, Sep 18, 2015, 5:18 PM Steffen Neumann wrot

Re: [Bioc-devel] SummarizedExperiment with alternate back end

2015-09-18 Thread Ryan
In the dev version, SummarizedExperiment has been split into RangedSummarizedExperiment (equivalent to the current SummarizedExperiement, with rowRanges) and SummarizedExperiment (kind of like eSet, no rowRanges). Given that eSet objects also support multiple assayData elements, I believe the n

Re: [Bioc-devel] SummarizedExperiment with alternate back end

2015-09-18 Thread Kasper Daniel Hansen
Interesting, thanks for the pointer. In light of the existing (and future) work on this, may I suggest an eSet like class, but build using the technologies in SummarizedExperiment. Ie. a SummarizedExperiment without the rowRanges. I would very much like this for modern work using eSet like conta

Re: [Bioc-devel] SummarizedExperiment with alternate back end

2015-09-18 Thread Vincent Carey
thanks to all, lots of potential here. On Fri, Sep 18, 2015 at 3:28 PM, Peter Haverty wrote: > Yes, bigmemoryExtras::BigMatrix and genoset::RleDataFrame() are good > tricks for reducing the size of your eSets and SummarizedExperiments. Both > object types can go into assayData or assays. In fac

Re: [Bioc-devel] SummarizedExperiment with alternate back end

2015-09-18 Thread Peter Haverty
Yes, bigmemoryExtras::BigMatrix and genoset::RleDataFrame() are good tricks for reducing the size of your eSets and SummarizedExperiments. Both object types can go into assayData or assays. In fact, that's what they were designed for. At Genentech, we use these for our 2.5e6 x 1e3 rectangular dat

[Bioc-devel] Merging git mirror and previous git repo ?

2015-09-18 Thread Steffen Neumann
Hi, sorry for asking another git question, but I want to make sure I don't mess up. If something like this was answered elsewhere and I missed it, just point me there. I had developed a package up to Version 0.99.7 on github in https://github.com/sneumann/mtbls2/ , and now after acceptance to

Re: [Bioc-devel] SummarizedExperiment with alternate back end

2015-09-18 Thread Tim Triche, Jr.
bigmemoryExtras (Peter Haverty's extensions to bigMemory/bigMatrix) can be handy for this, as it works well as a backend, especially if you go about splitting by chromosome as for CNV segmentation, DMR finding, etc. It's not as seamless as one might like, but it's the closest thing I've found. S

Re: [Bioc-devel] SummarizedExperiment with alternate back end

2015-09-18 Thread Hervé Pagès
Hi Vince, This is actually on "our" TODO list for after the release. SummarizedExperiment was designed to support alternate back end for the assays slot. More details and proof of concept are in ?Assays. H. On 09/18/2015 01:29 PM, Vincent Carey wrote: i am dealing with ~700 450k arrays they a

[Bioc-devel] SummarizedExperiment with alternate back end

2015-09-18 Thread Vincent Carey
i am dealing with ~700 450k arrays they are derived from one study, so it makes sense to think of them holistically. both the load time and the memory consumption are not satisfactory. has anyone worked on an object type that implements the rangedSE API but has the assay data out of memory? >

Re: [Bioc-devel] Problem with the git mirror when committing to SVN

2015-09-18 Thread Diego Diez
Thanks Jesper for your reply. I am afraid however thats what I did before (see the details of my original message). The real problem is the file .Rbuildignore which causes a rebase that then finds the conflict in DESCRIPTION (and other files as I fix that one; I guess it is trying to go too far in

Re: [Bioc-devel] Problem with the git mirror when committing to SVN

2015-09-18 Thread Jesper Gådin
Hi Diego, I had the same problem some days ago. Can't tell if this is the right way to solve the problem, but it worked, and this is what I did, 1) modified the file under conflict, in your case DESCRIPTION. 2) git rebase --continue 3) git rebase --skip #This is the selected output from my bash h

Re: [Bioc-devel] Problem with the git mirror when committing to SVN

2015-09-18 Thread Diego Diez
Dear all, I am now having a similar issue (see bellow) with my other package rTRM and so would like to bring the attention to this thread again. Anyone using git mirrors experiencing something similar and/or knowing how to fix it? Thank you very much in advance. Best, Diego On Tue, Sep 8, 2015 a