[Rd] R crashes when using huge data sets with character string variables

2020-12-12 Thread Arne Henningsen
When working with a huge data set with character string variables, I experienced that various commands let R crash. When I run R in a Linux/bash console, R terminates with the message "Killed". When I use RStudio, I get the message "R Session Aborted. R encountered a fatal error. The session was te

Re: [Rd] R crashes when using huge data sets with character string variables

2020-12-12 Thread Ben Bolker
On Windows you can use memory.limit. https://stackoverflow.com/questions/12582793/limiting-memory-usage-in-r-under-linux Not sure how much that helps. On 12/12/20 6:19 PM, Arne Henningsen wrote: When working with a huge data set with character string variables, I experienced that various

Re: [Rd] R crashes when using huge data sets with character string variables

2020-12-12 Thread brodie gaslam via R-devel
> On Saturday, December 12, 2020, 6:33:33 PM EST, Ben Bolker > wrote: > >  On Windows you can use memory.limit. > > https://stackoverflow.com/questions/12582793/limiting-memory-usage-in-r-under-linux > >    Not sure how much that helps. > >On 12/12/20 6:19 PM, Arne Henningsen wrote: >> When wor

Re: [Rd] [External] R crashes when using huge data sets with character string variables

2020-12-12 Thread luke-tierney
If R is receiving a kill signal there is nothing it can do about it. I am guessing you are running into a memory over-commit issue in your OS. https://en.wikipedia.org/wiki/Memory_overcommitment https://engineering.pivotal.io/post/virtual_memory_settings_in_linux_-_the_problem_with_overcommit/ I

Re: [Rd] [External] R crashes when using huge data sets with character string variables

2020-12-12 Thread Dirk Eddelbuettel
On 12 December 2020 at 21:26, luke-tier...@uiowa.edu wrote: | If R is receiving a kill signal there is nothing it can do about it. | | I am guessing you are running into a memory over-commit issue in your OS. | https://en.wikipedia.org/wiki/Memory_overcommitment | https://engineering.pivotal.io