Then you don't have enough memory to process the whole thing at once. Not unlike stuffing your mouth with cookies and not being able to chew for lack of space to move the food around in your mouth.
Now, can you answer my question? On November 8, 2024 5:38:37 PM PST, Val <valkr...@gmail.com> wrote: >The data was read. The problem is with processing. > >On Fri, Nov 8, 2024 at 7:30 PM Bert Gunter <bgunter.4...@gmail.com> wrote: >> >> Is the problem reading the file in or processing it after it has been read >> in? >> >> Bert >> >> On Fri, Nov 8, 2024 at 5:13 PM Jeff Newmiller via R-help >> <r-help@r-project.org> wrote: >>> >>> Can you tell us what is wrong with the "chunked" package which comes up >>> when you Google "r read large file in chunks"? >>> >>> On November 8, 2024 4:58:18 PM PST, Val <valkr...@gmail.com> wrote: >>> >Hi All, >>> > >>> >I am reading data file ( > 1B rows) and do some date formatting like >>> > dat=fread(mydatafile) >>> > dat$date1 <- as.Date(ymd(dat$date1)) >>> > >>> >However, I am getting an error message saying that >>> > Error: cons memory exhausted (limit reached?) >>> > >>> >The script was working when the number rows were around 650M. >>> > >>> >Is there another way to handle a big data set in R? >>> > >>> > >>> >Thank you. >>> > >>> >______________________________________________ >>> >R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see >>> >https://stat.ethz.ch/mailman/listinfo/r-help >>> >PLEASE do read the posting guide >>> >https://www.R-project.org/posting-guide.html >>> >and provide commented, minimal, self-contained, reproducible code. >>> >>> -- >>> Sent from my phone. Please excuse my brevity. >>> >>> ______________________________________________ >>> R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see >>> https://stat.ethz.ch/mailman/listinfo/r-help >>> PLEASE do read the posting guide >>> https://www.R-project.org/posting-guide.html >>> and provide commented, minimal, self-contained, reproducible code. -- Sent from my phone. Please excuse my brevity. ______________________________________________ R-help@r-project.org mailing list -- To UNSUBSCRIBE and more, see https://stat.ethz.ch/mailman/listinfo/r-help PLEASE do read the posting guide https://www.R-project.org/posting-guide.html and provide commented, minimal, self-contained, reproducible code.
