Re: predict whether pa.array() will produce ChunkedArray

2019-12-03 Thread Wes McKinney
hi John, The documentation says array : pyarrow.Array or pyarrow.ChunkedArray A ChunkedArray instead of an Array is returned if: - the object data overflowed binary storage. - the object's ``__arrow_array__`` protocol method returned a chunked array. Overfl

predict whether pa.array() will produce ChunkedArray

2019-12-03 Thread John Muehlhausen
Given input data and a type, how do we predict whether array() will produce ChunkedArray? I figure the formula involves: - the length of input - the type, and max length (to be conservative) for variable length types - some constant(s) that Arrow knows internally... that may change in the future?