Re: [R-pkg-devel] libarrow

2024-11-27 Thread Dewey Dunnington
I'll also add that you can write R level code that processes files lazily chunk-by-chunk using arrow::open_dataset() |> arrow::map_batches(), and that you can write C level code that processes batches chunk-by-chunk using arrow::open_dataset() |> nanoarrow::as_nanoarrow_array_stream(), which wi

Re: [R-pkg-devel] libarrow

2024-11-27 Thread Pham, T.V. via R-package-devel
It is clear. Thank you very much. Thang -Original Message- From: R-package-devel On Behalf Of r-package-devel-requ...@r-project.org Sent: Wednesday, November 27, 2024 12:00 PM To: r-package-devel@r-project.org Subject: R-package-devel Digest, Vol 115, Issue 19 Send R-package-devel mail

Re: [R-pkg-devel] libarrow

2024-11-27 Thread Bryce Mecum
The work in progress to add libarrow to MXE is located at [1]. I don't think it's complete but I'm sharing it here in case it's a useful starting point for someone. Continuing this work would be welcome. Thang: Feel free to file an issue on the Arrow issue tracker [2] or email the user@ mailing li