[R-pkg-devel] Segmentation Fault cause 'memory not mapped' on Debian only
Hi all, I wanted to publish my RITCH package (https://github.com/DavZim/RITCH) to CRAN, which has Rcpp code. It has, in it�s CICD Pipeline, Tests for Macos, Windows, and Ubuntu (devel, release,, and oldrel-1), which all succeed on Github Actions. When I submitted the package to CRAN, I was notified, that I got the following error on Debian (the tests on Windows throw no such bug, neither do the Github Actions tests): *** caught segfault *** address 0x55939b7ca000, cause 'memory not mapped' Segmentation fault The full Report is currently available here: https://win-builder.r-project.org/incoming_pretest/RITCH_0.1.14_20230420_223157/Debian/00check.log (note the error happens in the test_filter_itch.R unit tests). I looked for the error online but couldn�t find anything useful, and as I am not able to reproduce the segfault locally, cannot debug it. Do you have any ideas or hunches what might cause this and how to resolve it? Thank you, David [[alternative HTML version deleted]] __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
Re: [R-pkg-devel] Segmentation Fault cause 'memory not mapped' on Debian only
Thank you Dirk and Joshua,
Using Valgrind, I have found several memory leaks in the program that required
some smaller rewrites.
Valgrind (with Makevars flags -O0 and --ggdb) now reports no "definitely" and
no "indirectly" lost bytes so that's a small win already.
But two questions remain:
1. I have 960 bytes "possibly lost", where the callstack does not touch any
of my code but seems to originate from subset.c (base R?) and libgomp, is this
ok/expected, or am I misreading the output?
2. For all tests that I ran, there are still 105mln bytes still reachable. I
read online that this can be problematic, but usually isn't. Is this something
that I should invest more time in or can I ignore this?
Thank you very much!
David
To reproduce the valgrind output using bash (this is using the latest code
commit
https://github.com/DavZim/RITCH/commit/13ea3f478717edf24f8db709f0cac1e22b501afc)
echo "PKG_CXXFLAGS=-O0 -ggdb\nPKG_LIBS=-O0 -ggdb -lz" >> src/Makevars.win
echo "PKG_CXXFLAGS=-O0 -ggdb\nPKG_LIBS=-O0 -ggdb -lz" >> src/Makevars
R CMD build .
R CMD INSTALL RITCH_0.1.15.tar.gz
R -d "valgrind --tool=memcheck --leak-check=full" -e
"tinytest::test_package('RITCH')"
# omitted for brevity...
tinytest::test_package('RITCH')
test_filename_helpers.R... 13 tests OK 1.1s
test_filter_itch.R 60 tests OK 30.1s
test_gz_functions.R...5 tests OK 6.0s
test_read_functions.R. 122 tests OK 14.1s
test_write_itch.R. 18 tests OK 33.0s
All ok, 218 results (1m 24.5s)
>
==1535==
==1535== HEAP SUMMARY:
==1535== in use at exit: 105,616,149 bytes in 18,292 blocks
==1535== total heap usage: 562,268 allocs, 543,976 frees, 6,834,040,088 bytes
allocated
==1535==
==1535== 960 bytes in 3 blocks are possibly lost in loss record 199 of 2,000
==1535==at 0x483DD99: calloc (in
/usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
==1535==by 0x40149DA: allocate_dtv (dl-tls.c:286)
==1535==by 0x40149DA: _dl_allocate_tls (dl-tls.c:532)
==1535==by 0x5741322: allocate_stack (allocatestack.c:622)
==1535==by 0x5741322: pthread_create@@GLIBC_2.2.5 (pthread_create.c:660)
==1535==by 0x5710DEA: ??? (in /usr/lib/x86_64-linux-gnu/libgomp.so.1.0.0)
==1535==by 0x57088E0: GOMP_parallel (in
/usr/lib/x86_64-linux-gnu/libgomp.so.1.0.0)
==1535==by 0x10882EB9: subsetVectorRaw (subset.c:54)
==1535==by 0x10883B50: subsetDT (subset.c:309)
==1535==by 0x494253D: ??? (in /usr/lib/R/lib/libR.so)
==1535==by 0x49861A6: ??? (in /usr/lib/R/lib/libR.so)
==1535==by 0x4999D77: Rf_eval (in /usr/lib/R/lib/libR.so)
==1535==by 0x499BC2E: ??? (in /usr/lib/R/lib/libR.so)
==1535==by 0x499CB21: Rf_applyClosure (in /usr/lib/R/lib/libR.so)
==1535==
==1535== LEAK SUMMARY:
==1535==definitely lost: 0 bytes in 0 blocks
==1535==indirectly lost: 0 bytes in 0 blocks
==1535== possibly lost: 960 bytes in 3 blocks
==1535==still reachable: 105,615,189 bytes in 18,289 blocks
==1535== of which reachable via heuristic:
==1535== newarray : 4,264 bytes in 1 blocks
==1535== suppressed: 0 bytes in 0 blocks
==1535== Reachable blocks (those to which a pointer was found) are not shown.
==1535== To see them, rerun with: --leak-check=full --show-leak-kinds=all
==1535==
==1535== For lists of detected and suppressed errors, rerun with: -s
==1535== ERROR SUMMARY: 1 errors from 1 contexts (suppressed: 0 from 0)
________
From: Joshua Ulrich
Sent: Friday, April 21, 2023 7:48 PM
To: Dirk Eddelbuettel
Cc: D Z ; r-package-devel@r-project.org
Subject: Re: [R-pkg-devel] Segmentation Fault cause 'memory not mapped' on
Debian only
Hi David,
In addition to running under valgrind as Dirk suggests below, you
should also run under UBSAN and ASAN.
You can do all 3 (and more) using Winston Cheng's docker images:
https://github.com/wch/r-debug/
That should help you narrow down the problematic code.
Best,
Josh
On Fri, Apr 21, 2023 at 2:33 PM Dirk Eddelbuettel wrote:
>
>
> Hi David,
>
> On 21 April 2023 at 19:17, D Z wrote:
> | Hi all,
> | I wanted to publish my RITCH package (https://github.com/DavZim/RITCH) to
> CRAN, which has Rcpp code.
>
> That would be great!
>
> | It has, in it’s CICD Pipeline, Tests for Macos, Windows, and Ubuntu (devel,
> release,, and oldrel-1), which all succeed on Github Actions.
> | When I submitted the package to CRAN, I was notified, that I got the
> following error on Debian (the tests on Windows throw no such bug, neither do
> the Github Actions tests):
> |
> |
> |*** caught segfault ***
> |
> | address 0x55939b7ca000, cause 'memory not mapped'
> |
> | Segmentation fault
> |
> | The full Report is currently availa
[R-pkg-devel] "Examples with CPU time > 2.5 times elapsed time" and other NOTEs on CRAN and rhub checks
Hi all, I submitted my package RITCH (https://github.com/DavZim/RITCH) to CRAN (used to be archived but I wanted to revive it again) but I got a NOTE (Question 1 below). Besides this NOTE from CRAN, I got two other NOTEs from rhub (Q2 and Q3). Q1) The CRAN NOTE (Debian only, does not surface on Windows or other platforms) reads *checking examples ... [7s/3s] NOTE Examples with CPU time > 2.5 times elapsed time user system elapsed ratio read_functions 3.968 0.092 0.831 4.886 (https://github.com/DavZim/RITCH/blob/master/R/read_functions.R in case you need the source code, the full CRAN report can be found here https://win-builder.r-project.org/incoming_pretest/RITCH_0.1.23_20240110_120457/Debian/00check.log) The package itself has no parallelism built-in, but Imports data.table. This NOTE does not surface on other platforms (eg using rhub or on my GitHub actions runners). My unit tests already limit data.table to 2 cores using setDTthreads(2), but I would like to keep this line out of the help files for my functions. Is there anything that I can do or can I ignore the result and argue for an exception using the false positive argument? Q2) A second question that I have is that on rhub Ubuntu Linux 20.04.1 LTS, R-release, GCC (https://artifacts.r-hub.io/RITCH_0.1.22.tar.gz-d2b925faf6b24497abbfa6ff60e51d34/RITCH.Rcheck/00check.log) I receive the NOTE that my libs/ sub-directory is at 7.7Mb. Can I ignore this or do I need to figure out how to reduce the binary size of the package? * checking installed package size ... NOTE installed size is 8.6Mb sub-directories of 1Mb or more: libs 7.7Mb My code uses Rcpp and has some classes and interdependencies between C++ functions, therefore a rewrite to make the binary size smaller might take a lot of work. From looking around online I find that other packages are a lot bigger. Are there any low-hanging fruits that I can use to reduce the size or should I ignore this NOTE? Q3) And last but not least, on some rhub instances (Fedora and Ubuntu GCC) I receive a NOTE that the package runs its examples too slowly (eg above 5secs). I have already tweaked the example code already that it runs reliably <4 secs on my development laptop Ubuntu Linux 20.04.1 LTS, R-release, GCC (https://builder.r-hub.io/status/original/RITCH_0.1.22.tar.gz-d2b925faf6b24497abbfa6ff60e51d34) * checking examples ... [6s/37s] NOTE Examples with CPU (user + system) or elapsed time > 5s user system elapsed read_functions 2.51 0.028 12.57 and on Fedora Linux, R-devel, clang, gfortran (https://builder.r-hub.io/status/original/RITCH_0.1.22.tar.gz-01bf475551eb4b30a722ea79ce421788) * checking examples ... [6s/26s] NOTE Examples with CPU (user + system) or elapsed time > 5s user system elapsed read_functions 1.896 0.018 8.891 As this does not surface on the CRAN checks, I would ignore it for now and concentrate only on the CRAN checks. Is this correct or should I pay more attention to these NOTEs? Any help/comment is appreciated. Thank you for your time and best regards, David [[alternative HTML version deleted]] __ R-package-devel@r-project.org mailing list https://stat.ethz.ch/mailman/listinfo/r-package-devel
[R-pkg-devel] Installation took CPU time elapsed time for rust-based package
Hi all,
I have a new package that I submitted to CRAN, that uses rust (for reference,
it's called rtiktoken and can be found here:
https://github.com/DavZim/rtiktoken).
The submission failed as 1-2 NOTEs were found, most importantly: Installation
took CPU time 7.2 times elapsed time.
In both my Makevars and Makevars.win files, I specify to use 2 cores for cargo
build on CRAN machines (see below or here
https://github.com/DavZim/rtiktoken/blob/master/src/Makevars#L21C2-L25C4) but I
still get the NOTE. Furthermore, I only get this on Debian but not on Windows.
if [ -n "$_R_CHECK_LIMIT_CORES" ]; then \
export BUILDFLAGS="-j 2 --offline"; \
else \
export BUILDFLAGS="--offline"; \
fi
...
cargo build ${BUILDFLAGS} --lib --release --manifest-path=./rust/Cargo.toml
--target-dir $(TARGET_DIR)
The check results are here
*
Windows:
https://win-builder.r-project.org/incoming_pretest/rtiktoken_0.0.3_20241016_135446/Windows/00check.log
(1 NOTE about misspelled words and a URL not being available)
*
Debian:https://win-builder.r-project.org/incoming_pretest/rtiktoken_0.0.3_20241016_135446/Debian/00check.log
(2 NOTEs: installation time and misspelled words)
Now I am wondering if there is anything that I can do about this NOTE or if I
should ignore it.
I did not find any reference to this in the "Using Rust in CRAN Packages" nor
in other CRAN packages that use rust (eg gifski, string2path, arcpbf), but I
might have missed something.
I found also the CARGO_BUILD_JOBS flag for rust
(https://doc.rust-lang.org/cargo/reference/environment-variables.html) but as I
can't reproduce the error locally (or on other GitHub Actions runners), I don't
want to trial-and-test-it on CRAN servers.
Thanks, any help is appreciated!
David
[[alternative HTML version deleted]]
__
R-package-devel@r-project.org mailing list
https://stat.ethz.ch/mailman/listinfo/r-package-devel
