Re: Question on handling API changes when upgrading Pyarrow

2022-09-10 Thread Li Jin
Thanks Weston. I don’t own the code that broke so it’s possible that it was ignored. Thanks for the links! On Fri, Sep 9, 2022 at 8:44 PM Weston Pace wrote: > Breaking changes should be documented in the release notes which are > announced on the Arrow blog[1][2]. In addition, in pyarrow, chan

Re: Question on handling API changes when upgrading Pyarrow

2022-09-09 Thread Weston Pace
Breaking changes should be documented in the release notes which are announced on the Arrow blog[1][2]. In addition, in pyarrow, changes to non-experimental APIs (and often also those made to experimental APIs) should go through a deprecation cycle where a warning is emitted for at least one relea

Re: Question on handling API changes when upgrading Pyarrow

2022-09-09 Thread Li Jin
After digging the code a bit, it looks like: (1) pyarrow.read_schema should be changed to pyarrow.ipc.read_schema (2) chunksize should be changed to max_chunksize (it was passed in as a generic kwargs before and I am guessing it was a wrong in the first place) These seem to be easy enough to fix b

Question on handling API changes when upgrading Pyarrow

2022-09-09 Thread Li Jin
Hi, I am trying to update Pyarrow from 7.0 to 9.0 and hit a couple of issues that I believe are because of some API changes. In particular, two issues I saw seems to be (1) pyarrow.read_schema is removed (2) pa.Table.to_batches no longer takes a keyword argument (chunksize) What's the best way t