00 00:01.104
>
> -Original Message-
> From: Antoine Pitrou [mailto:anto...@python.org]
> Sent: Friday, May 11, 2018 4:54 AM
> To: dev@arrow.apache.org
> Subject: Re: Question about streaming to memorymapped files
>
>
> If you write your own auto-growing memory
[mailto:anto...@python.org]
Sent: Friday, May 11, 2018 4:54 AM
To: dev@arrow.apache.org
Subject: Re: Question about streaming to memorymapped files
If you write your own auto-growing memory mapped file implementation,
I'd be curious about performance measurements vs. FileOutputStream (and
pos
port
> dynamically growing the mmap file I'll have to write my own impl
>
> -Original Message-
> From: Antoine Pitrou [mailto:anto...@python.org]
> Sent: Wednesday, May 09, 2018 11:42 AM
> To: dev@arrow.apache.org
> Subject: Re: Question about streaming to memo
y 09, 2018 11:42 AM
To: dev@arrow.apache.org
Subject: Re: Question about streaming to memorymapped files
As for buffering data before making a call to write(): in Arrow 0.10.0
you'll be able to use BufferedOutputStream for this:
https://urldefense.proofpoint.com
, I do this in my applications and it works fine.
>> I want the efficiency of writing via memory maps, so would prefer to avoid
>> FileOutputStream
>>
>> -Original Message-
>> From: Antoine Pitrou [mailto:anto...@python.org]
>> Sent: Wednesday, May 09,
y maps, so would prefer to avoid
> FileOutputStream
>
> -Original Message-
> From: Antoine Pitrou [mailto:anto...@python.org]
> Sent: Wednesday, May 09, 2018 10:37 AM
> To: dev@arrow.apache.org
> Subject: Re: Question about streaming to memorymapped files
>
>
> Hi,
>
fer to avoid
> FileOutputStream
>
> -Original Message-
> From: Antoine Pitrou [mailto:anto...@python.org]
> Sent: Wednesday, May 09, 2018 10:37 AM
> To: dev@arrow.apache.org
> Subject: Re: Question about streaming to memorymapped files
>
>
> Hi,
>
>
Message-
From: Antoine Pitrou [mailto:anto...@python.org]
Sent: Wednesday, May 09, 2018 10:37 AM
To: dev@arrow.apache.org
Subject: Re: Question about streaming to memorymapped files
Hi,
If you don't know the output size upfront then should probably use a
FileOutputStream instead
Hi,
If you don't know the output size upfront then should probably use a
FileOutputStream instead. By definition, memory mapped files must have
a fixed size (since they are mapped to a fixed area in virtual memory).
Regards
Antoine.
Le 09/05/2018 à 16:31, Ambalu, Robert a écrit :
> Hey, I'm
Hey, I'm looking into streaming table updates into a memory mapped file ( C++ )
I think I have everything I need ( MemoryMappedFile output streamer,
RecordBatchStreamWriter ) but I don't understand how to properly create the
memmap file. It looks like it requires you to preset a size to the file
10 matches
Mail list logo
