To follow up on this: Please check the package code for adding/accessing resources in the netDx package. If the BiocFileCache options were used correctly there should not have been multiple entries added to the builders cache as Martin demonstrated below.
Once the package has been updated to ensure duplicate entries will not be added, we can clean up the builders. The package code should be adjusted first to ensure that users will not encounter the same situation. Lori Shepherd Bioconductor Core Team Roswell Park Comprehensive Cancer Center Department of Biostatistics & Bioinformatics Elm & Carlton Streets Buffalo, New York 14263 ________________________________ From: Bioc-devel <bioc-devel-boun...@r-project.org> on behalf of Martin Morgan <mtmorgan.b...@gmail.com> Sent: Friday, September 11, 2020 6:22 PM To: Pages, Herve <hpa...@fredhutch.org>; Shraddha Pai <shraddha....@utoronto.ca> Cc: bioc-devel@r-project.org <Bioc-devel@r-project.org> Subject: Re: [Bioc-devel] "length(url) == 1 is not TRUE" error for vignette only on build system bfcquery() just searches the cache, so if you've created two resources that match the name then you end up with two rids > xx = bfcnew(bfc, "foo") > bfcquery(bfc, "foo", "rname") # A tibble: 1 x 10 rid rname create_time access_time rpath rtype fpath last_modified_t� etag <chr> <chr> <chr> <chr> <chr> <chr> <chr> <dbl> <chr> 1 BFC3 foo 2020-09-11� 2020-09-11� /Use� rela� 64ea� NA NA # � with 1 more variable: expires <dbl> > xx = bfcnew(bfc, "foo_bar") > bfcquery(bfc, "foo", "rname") # A tibble: 2 x 10 rid rname create_time access_time rpath rtype fpath last_modified_t� etag <chr> <chr> <chr> <chr> <chr> <chr> <chr> <dbl> <chr> 1 BFC3 foo 2020-09-11� 2020-09-11� /Use� rela� 64ea� NA NA 2 BFC4 foo_� 2020-09-11� 2020-09-11� /Use� rela� 64ea� NA NA # � with 1 more variable: expires <dbl> and even > xx = bfcnew(bfc, "foo_bar") > bfcquery(bfc, "foo", "rname") # A tibble: 3 x 10 rid rname create_time access_time rpath rtype fpath last_modified_t� etag <chr> <chr> <chr> <chr> <chr> <chr> <chr> <dbl> <chr> 1 BFC3 foo 2020-09-11� 2020-09-11� /Use� rela� 64ea� NA NA 2 BFC4 foo_� 2020-09-11� 2020-09-11� /Use� rela� 64ea� NA NA 3 BFC5 foo_� 2020-09-11� 2020-09-11� /Use� rela� 64ea� NA NA # � with 1 more variable: expires <dbl> If the cache is under netDx control, then you can be careful about creating new resources by checking first whether the resource exists, as outlined in http://secure-web.cisco.com/1t9-aFwD7wK-0S-MuqPB5uIzT5hQ2gyAD8KMGooS3HThd0k45RZ-frPKl-6Yl2EwZFFcmutTlgEODe0vTjmGHdvrtnhcf5beUQTSS3KiUAl4eWVWv4m3kjsTCUxTVQ2Ll5cGeus1QPlv2osdgHIycOGo6WKDjPqb6PoCxqWOMpEB_x8cSenUxF2N0OkDSEKuwlgJzHwqGjVOwqZQ_odXuqKEnKXz5moLlJO0XSz5XXkTj4kTGwtMAjqKAk_tR8DujPGfYr6Cc70ReKMZUWNVIOc4JgCOAmkILBW94Rp0x_3CX-8DEDeaRhlrgoM7MxkqHGyB0UxHyCXGepxXY196Uglz7WmH1uY02-XQsQ_6e8og/http%3A%2F%2Fbioconductor.org%2Fpackages%2Fdevel%2Fbioc%2Fvignettes%2FBiocFileCache%2Finst%2Fdoc%2FBiocFileCache.html%23cache-to-manage-package-data If one is managing internet resources, just use bfcadd > bfc <- BiocFileCache() > path <- bfcrpath(bfc, url) adding rname 'https://secure-web.cisco.com/1lUwUbgjSoXsv6f7JfdXhRG2f7_ISpNGSvQkNDjD3wQSEEMkTAVHfopgsowEo1GgsXrbByzyoKcAqYa3Kw2bxocuFRTwCIF5UxwWY7fw_FCeNByJcX4L4A9pGyPD1Il4OWJRoYewqI9SOxbFsK6XqqYQmOxhzALdAwDrJw1cOOpppzIIoz5unFJOO2Ihuca8QuIswQZGQqxSBXbeuu8WxSl9QoenolmV7PHovZu_sUMd1DYSflBtliuDz8hRJHSdp-N4wLsvl5GZXxTcJH4ZruhecvOJPR7aGrHCRLZln3wWoMKctL_Oc2xnUYu0ZYrCrVD5MQ8Wf7fgFEl7RPTt5RbTMxrbLO0zXxZZaEwhL5AU/https%3A%2F%2Fbioconductor.org%2Findex.html' |======================================================================| 100% > path <- bfcrpath(bfc, url) > where the first time 'url' is used (as a unique key) the resource is downloaded to the cache; the second time it is simply accessed from the cache. It might be necessary to manually 'clean up' the builders Martin On 9/11/20, 4:10 PM, "Bioc-devel on behalf of Pages, Herve" <bioc-devel-boun...@r-project.org on behalf of hpa...@fredhutch.org> wrote: I'm not a BiocFileCache expert, sorry. You will probably get better help by opening a BiocFileCache issue on GitHub. Cheers, H. On 9/11/20 12:00, Shraddha Pai wrote: > Hi Herve, > Sorry - hit shortcut to "send" by mistake. > > This is the code snippet that got cut off: > if (length(rid)>1) { > x <- as.POSIXct(rid_rec$last_modified_time) > y <- order(x,decreasing=TRUE) > rid <- rid[y[1]] > } > > Overkill? I could just take rid[1] but that would not be methodical. > > Will file issue once you confirm that the problem is that there should > be a unique rid associated with each rname. > > Thanks again - appreciate it, > Shraddha > > On Fri, Sep 11, 2020 at 2:57 PM Shraddha Pai <shraddha....@utoronto.ca > <mailto:shraddha....@utoronto.ca>> wrote: > > Hi Herve, > Thank you for pinpointing this. > Just so I'm clear: rid should not have two IDs correct? > For each rname, the cache should only have a unique ID, right? Or is > that not the case? > > To get the package to build I will take the rid with the latest > "last_modified_time". > Looks like this can be achieved with: > if (length(rid)>1) { > x <- rid_rec$last_modified > > On Fri, Sep 11, 2020 at 1:22 PM Pages, Herve <hpa...@fredhutch.org > <mailto:hpa...@fredhutch.org>> wrote: > > Hi Shraddha, > > Seems to be a BiocFileCache issue. > > On my laptop the following code (taken from your > Predict_CaseControl_from_CNV.Rnw vignette): > > require(BiocFileCache) > geneURL <- paste("http://secure-web.cisco.com/1q6dletAEEmECRlpAh7w_w4AmdNAhzos1ijnImWIdzsuXHz3kMIav7Urj3RzYO0S8BzLMKG_0Zcp4ytMlPfA9Ht6IFyJIoCQY-9mRDq-BQDDX_zer8B-bkuIy0hvpOFRkd4a0lSKnviwzssrArVnDcHSCh7EQZsCSMgdEeFsojc3WO3wt8cG5GJztsLKILl9NFGnr2g19zjUcuvqtOBAyec62CVmCfonqPnOWy_VEUPd4WcDd2XwFUW7vEzrFcpC3V6K5pAc0bWjjqEz-3RLU0cWFYgNOGa7zIBwHCX8XMu1JcpLCVoXB-W_RXWMiOwtaHwy8vGei3fBckztfRFgZnA/http%3A%2F%2Fdownload.baderlab.org%2FnetDx%2F > <https://secure-web.cisco.com/1pBAaf568Sh8T9WwtZnai1icCdNVYUY8FRTQfjYtlK3gI5radqfCsKfzfJz-0BFk_9sTpehBBo60575Jx0MEnZeJRJIKZxiwx2eQ0X90O_CRwVziGst9BfByxNmW5LhvoYTWWSYm1G7ILPw13lka_Q4K2MiNJUX9s0gU6JDcCjBZ6000t2wqGJ_YEBcAGdt30Uv9ElTiYcAXps8tQzLNc9DGwx8ErTvnVmMjvbBVbBRQEBe3Gvo7OcE4-TS_nIEzsZhyvsT1V8xJ3KU9bTb_0J2S1AWC3ETe48ipc0WcJ_cA0OkCYMR7v5dIhE58VGsox/https%3A%2F%2Furldefense.proofpoint.com%2Fv2%2Furl%3Fu%3Dhttp-3A__download.baderlab.org_netDx_%26d%3DDwMFaQ%26c%3DeRAMFD45gAfqt84VtBcfhQ%26r%3DBK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA%26m%3DV-k2jGkEksU1mmcALiO9azNiQrGrVrfwlL4vMvJZwHQ%26s%3DDG1UzWYSS8kbCIEYszKms6FBBQ-GZKWplwuOOOIf88c%26e%3D>", > "supporting_data/refGene.hg18.bed",sep="") > cache <- rappdirs::user_cache_dir(appname = "netDx") > bfc <- BiocFileCache::BiocFileCache(cache,ask=FALSE) > rid_rec <- bfcquery(bfc, "hg18_genes", "rname") > rid <- rid_rec$rid > > returns only one rid: > > > rid > [1] "BFC4" > > whereas on malbec1 it returns two: > > > rid > [1] "BFC6" "BFC11" > > In both cases, I'm using the latest version of BiocFileCache > (1.13.1). > > I suspect this has to do with the cache being in different > states on the > two machines. > > The fact that 'rid' has a length of 2 causes the failure a few > lines > later when bfcdownload() is called: > > rid <- rid_rec$rid > if (!length(rid)) { > rid <- names(bfcadd(bfc, "hg18_genes", geneURL)) > } > if (!isFALSE(bfcneedsupdate(bfc, rid))){ > bfcdownload(bfc, rid,ask=FALSE) > } > > I suggest that you open a BiocFileCache issue on GitHub. > > Thanks, > H. > > > On 9/11/20 08:51, Shraddha Pai wrote: > > Hello BioC-devel team, > > Any idea why netDx 1.1.4 fails on the bioc devel build system? > > > > This is the error: > > > > --- re-building 'Predict_CaseControl_from_CNV.Rmd' using knitr > > Quitting from lines 173-195 (Predict_CaseControl_from_CNV.Rmd) > > Error: processing vignette 'Predict_CaseControl_from_CNV.Rmd' > failed > > with diagnostics: > > length(url) == 1 is not TRUE > > --- failed re-building 'Predict_CaseControl_from_CNV.Rmd' > > > > > > I just set up the Bioc-devel docker, cloned the repo using > > git clone g...@git.bioconductor.org:packages/netDx > > and ran the vignette, and it runs through successfully. > > > > However the build error occurs across all platforms so > something needs to > > change. > > > > Thank you, > > Shraddha > > > > [[alternative HTML version deleted]] > > > > _______________________________________________ > > Bioc-devel@r-project.org <mailto:Bioc-devel@r-project.org> > mailing list > > > https://secure-web.cisco.com/1Q1gRqlh1AbjlYoYoW-Tao_IKz59ZWyREYyVSz_yAJWP75JqkTLfdvqwGnEwk-lQEa0GIswGLe7jTmscS0aE9zb844BYJvAdTWLUneOW1-5ZUjtTOWpbCs88HMYRlKLIrA9cP4HevAvVxueol9bqn0j0LHw-UZX-k72wAWe5tjjd54VbNWOM5kKAPNuti9OB4nxoSRxmrVXUKKx2oQB0wT_yROTrgBTc1EAjjJ1P3fI85SfYyYA2SwxLeltUGcEkWoxzDXL1y7atMucMCnMJ2fv8KN41qlhh0uuv8VaHXyIXcxcg76La78NT4MVLsfX2D/https%3A%2F%2Furldefense.proofpoint.com%2Fv2%2Furl%3Fu%3Dhttps-3A__stat.ethz.ch_mailman_listinfo_bioc-2Ddevel%26d%3DDwICAg%26c%3DeRAMFD45gAfqt84VtBcfhQ%26r%3DBK7q3XeAvimeWdGbWY_wJYbW0WYiZvSXAJJKaaPhzWA%26m%3DzwoVT5i-NLz7FqpRg0pSLIqNJBJiI_Ac8FigI43cwRc%26s%3DYhavBW80Elhyr-R7RuyrA2bwYPfI08HAsraBEs2FRU0%26e%3D > > > > -- > Herv� Pag�s > > Program in Computational Biology > Division of Public Health Sciences > Fred Hutchinson Cancer Research Center > 1100 Fairview Ave. N, M1-B514 > P.O. Box 19024 > Seattle, WA 98109-1024 > > E-mail: hpa...@fredhutch.org <mailto:hpa...@fredhutch.org> > Phone: (206) 667-5791 > Fax: (206) 667-1319 > -- Herv� Pag�s Program in Computational Biology Division of Public Health Sciences Fred Hutchinson Cancer Research Center 1100 Fairview Ave. N, M1-B514 P.O. Box 19024 Seattle, WA 98109-1024 E-mail: hpa...@fredhutch.org Phone: (206) 667-5791 Fax: (206) 667-1319 _______________________________________________ Bioc-devel@r-project.org mailing list https://secure-web.cisco.com/1e58U71LbG0FMxJdQhxAXm6tEq4DjFc60_hb90-31Y7uIV0IbSMcYaN13j95FzoALCkV6H1go-g1bCAFCzKjDJtyWH1ZTXg2zMKy1xg74eX9mvyC7enPrBgvEDMpkrKaaUuqVPrSkFhAO4BikyNDvl6a_whzJzZXuRi2q5nKEr1JFdIv2pNXCZnMVgARPrdL8AT7dhjPrEPv1r7zAUJ1FMF7h7UsPWABFV1myjK_Thqh0_oXgySzvERAN7bKu5QStc-kXMzxLnj16qYsIycsLBpUZ4b5xvt2xnQfTcJxYfo_zSC2Pgrw2c3wYXBTi69XCs1jdxZRu7qId_lNuFbZ_tg/https%3A%2F%2Fstat.ethz.ch%2Fmailman%2Flistinfo%2Fbioc-devel _______________________________________________ Bioc-devel@r-project.org mailing list https://secure-web.cisco.com/1e58U71LbG0FMxJdQhxAXm6tEq4DjFc60_hb90-31Y7uIV0IbSMcYaN13j95FzoALCkV6H1go-g1bCAFCzKjDJtyWH1ZTXg2zMKy1xg74eX9mvyC7enPrBgvEDMpkrKaaUuqVPrSkFhAO4BikyNDvl6a_whzJzZXuRi2q5nKEr1JFdIv2pNXCZnMVgARPrdL8AT7dhjPrEPv1r7zAUJ1FMF7h7UsPWABFV1myjK_Thqh0_oXgySzvERAN7bKu5QStc-kXMzxLnj16qYsIycsLBpUZ4b5xvt2xnQfTcJxYfo_zSC2Pgrw2c3wYXBTi69XCs1jdxZRu7qId_lNuFbZ_tg/https%3A%2F%2Fstat.ethz.ch%2Fmailman%2Flistinfo%2Fbioc-devel This email message may contain legally privileged and/or confidential information. If you are not the intended recipient(s), or the employee or agent responsible for the delivery of this message to the intended recipient(s), you are hereby notified that any disclosure, copying, distribution, or use of this email message is prohibited. If you have received this message in error, please notify the sender immediately by e-mail and delete this email message from your computer. Thank you. [[alternative HTML version deleted]]
_______________________________________________ Bioc-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/bioc-devel
