Re: [DISCUSS][C++] Store C++ shared_ptr in arrow table

2024-10-11 Thread Jorge Cardoso Leitão
AFAIK uintptr_t being internally stored as an integer does not make it equivalent to uint64_t - compilers use the type to set them apart, see the example in [1]. ptr2int2ptr can result in UB in subtle ways, due to how C/C++ are specified and translated to LLVM IR. Storying pointers as arrow intege

Re: Extract objects from CompressedOutputStream

2024-10-11 Thread Antoine Pitrou
Hi Robert, On Thu, 10 Oct 2024 08:33:28 -0700 Robert McLeod wrote: > > I think my trouble is coming from the fact that a `CompressedOutputStream` > closes the `BufferedOutputStream` when it exits its context manager. I > don't know if that is required or if it is simply an oversight because no

Re: Extract objects from CompressedOutputStream

2024-10-11 Thread Aldrin
Alrighty, I looked at a bunch of code and then I decided to play around with it to see if I could figure it out. I have some code that should be easy to copy-paste and play around with at [1]. The short story is I learned a lot (meaning I internalized it) but none of it is anything you didn't ne

Re: [DISCUSS][C++] Store C++ shared_ptr in arrow table

2024-10-11 Thread Yi Cao
First of all, thank you so much for your inputs and great insights! Integer-Pointer round trip does not seem a reliable way to me. We experienced subtle UB in some case before, which is one of the reasons we look at Arrow. Regarding Jorge's 4 options, Option 1-3 is not considered due to (de)serial

Re: Extract objects from CompressedOutputStream

2024-10-11 Thread Felipe Oliveira Carvalho
Hi Robert, I hit the same problem recently but there’s a Python-only workaround you can use. https://github.com/apache/arrow-experiments/pull/35/files#r1797397257 — Felipe On Fri, 11 Oct 2024 at 05:13 Antoine Pitrou wrote: > > Hi Robert, > > On Thu, 10 Oct 2024 08:33:28 -0700 > Robert McLeod