good question some of the discussion on
http://sqlite.1065341.n5.nabble.com/Parallel-access-to-read-only-in-memory-database-td91814.html seems relevant. converting the relatively small annotation package content to pure R read-only tables on the master before parallelizing might be very simple? On Fri, Jan 19, 2018 at 11:43 AM, Ludwig Geistlinger < ludwig.geistlin...@sph.cuny.edu> wrote: > Hi, > > Within a package I am developing, I would like to enable parallel probe to > gene mapping for a compendium of microarray datasets. > > This accordingly makes use of annotation packages such as hgu133a.db, > which in turn connect to the SQLite database via AnnotationDbi. > > When running in multi-core mode (i.e. using a MulticoreParam with > BiocParallel) using more than 2 cores, this causes the error: > > database disk image is malformed > > > In a very similar problem: > > https://support.bioconductor.org/p/38541/ > > Adi Tarca and Dan Tenenbaum identified and resolved this problem by > ensuring that each process has its own unique database connection, i.e. > AnnotationDbi is not loaded before sending the job to the workers. > > This solution was easily realized as this analysis was carried out within > a script and not a package. > > However, within my package, AnnotationDbi is loaded as a dependency of my > package's imports. > > How to resolve this here? > I am not sure whether I perfectly understand the underlying mechanisms, > but is there a way to make my workers load their own version of > AnnotationDbi instead of using the one of the parent process? > Or am I supposed to unload all packages depending on AnnotationDbi, and > AnnotationDbi itself, before sending the job to the workers (and reload all > of them after the job has finished?) > > Thanks a lot, > Ludwig > > > > -- > Dr. Ludwig Geistlinger > CUNY School of Public Health > > [[alternative HTML version deleted]] > > _______________________________________________ > Bioc-devel@r-project.org mailing list > https://stat.ethz.ch/mailman/listinfo/bioc-devel > [[alternative HTML version deleted]] _______________________________________________ Bioc-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/bioc-devel
