Arrow GCS filesystem library question

2025-03-13 Thread Jerry Adair
Hi, Currently, I use the Arrow filesystem library to establish an object that I then use to connect to a GCS server. The credentials are specified via a Json object, using the FromServiceAccountCredentials() method within the GcsOptions class. I have a request (an escalation actually) from a

RE: [C++] Arrow S3 filesystem init/finalize

2024-12-01 Thread Jerry Adair
less you are unloading the library, it's not clear that you will be freeing all resources anyways. [1] https://protect.checkpoint.com/v2/r01/___https://docs.aws.amazon.com/sdk-for-cpp/v1/developer-guide/basic-use.html___.YzJ1OnNhc2luc3RpdHV0ZTpjOm86MDNmYTg4MzEwZTE0ZjVlOGY3YmIzNmQxN

[C++] Arrow S3 filesystem init/finalize

2024-12-01 Thread Jerry Adair
Hi, I have a question regarding the initialization/finalization of the S3 filesystem within the Arrow filesystem library. Apologies if this question has been raised in the past; I did perform a search but that search didn't turn up anything. I did read the thread that discussed the issue of i

RE: Group rows in a stream of record batches by group id?

2023-06-14 Thread Jerry Adair
Hi Weston (and dev group), Speaking of the grouper in the C++ library and writing partitioned data, I had a tangential question if I may. I noticed in the example C++ source that an Arrow table, then in-memory dataset were created first, followed by a writing of the data to a partitioned data

ADLS C++ support in next release (version 11)

2023-01-06 Thread Jerry Adair
I am curious to know if the ADLS support for the Parquet C++ library will be included in the version 11 release that is scheduled for mid-January (at last check). Does anyone have feedback? We are in need of that capability. Thanks! Jerry

RE: Apache Arrow filesystem question

2022-10-27 Thread Jerry Adair
552 > 373a0c2f%7C0%7C0%7C638025038018602431%7CUnknown%7CTWFpbGZsb3d8eyJWIjoi > MC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7 > C%7C&sdata=Zhzd2XQa1pAkKYolRG4%2Bvgpr%2Fk7tmKSv6fL11Kg5xIg%3D& > reserved=0 > > -Original Message- > From: J

RE: Apache Arrow filesystem question

2022-10-26 Thread Jerry Adair
Hi- I am curious about the Apache Arrow filesystem support. I noticed that there are classes for S3 and GCS. I am currently using those classes to read/write from/to those cloud storage options. However I noticed that there is no class for Azure (ADLS). And so I am curious why that is? Doe