[jira] [Created] (ARROW-6051) [C++][Python] Parquet float column writing performance regression from 0.13.0 to 0.14.1

[Wes McKinney (JIRA)]

Wes McKinney created ARROW-6051:
-----------------------------------

             Summary: [C++][Python] Parquet float column writing performance 
regression from 0.13.0 to 0.14.1
                 Key: ARROW-6051
                 URL: https://issues.apache.org/jira/browse/ARROW-6051
             Project: Apache Arrow
          Issue Type: Bug
          Components: C++
            Reporter: Wes McKinney


I'm not sure the origin of the regression but I have with

pyarrow 0.13.0 from conda-forge

{code}
import pyarrow as pa
import pyarrow.parquet as pq
import numpy as np
import pandas as pd
arr = pa.array([np.nan] * 10000000)
t = pa.Table.from_arrays([arr], names=['f0'])

%timeit pq.write_table(t, '/home/wesm/tmp/nans.parquet')
28.7 ms ± 570 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)
{code}

but in pyarrow 0.14.1 from conda-forge

{code}
%timeit pq.write_table(t, '/home/wesm/tmp/nans.parquet')
88.1 ms ± 1.68 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)
{code}

I'm sorry to say, but this is what happens when benchmark data is not tracked 
and monitored



--
This message was sent by Atlassian JIRA
(v7.6.14#76016)