Hi,
It seems that there are no objections for these patch releases.
I'll initiate these patch releases. Here is my plan. If
there are any objections/concerns, please share them.
* We'll release 6.0.2, 7.0.1 and 8.0.1
* These releases only include
https://github.com/apache/arrow/pull/1332
Hi,
Yes. It's zero-copy.
See also the documentation of
arrow::ipc::StreamDecoder::next_required_size():
https://arrow.apache.org/docs/cpp/api/ipc.html#_CPPv4NK5arrow3ipc13StreamDecoder18next_required_sizeEv
Thanks,
--
kou
In
"StreamDecoder zero-copy (?) for pre-framed contiguous Messages" o
Hi,
How about using arrow::ipc::StreamDecoder instead of
arrow::ipc::StreamReader?
class MyListener : public arrow::ipc::Listener {
public:
arrow::Status
OnRecordBatchDecoded(std::shared_ptr record_batch)
override {
ArrowFilter arrow_filter = ArrowFilter(record_batch);
Thanks Antoine! I got this fixed (although it is not the ARROW_COMPUTE flag
but some other error I made)
Li
On Thu, Jul 7, 2022 at 4:23 PM Antoine Pitrou wrote:
>
> I don't think you need anything more on the PyArrow side, but you need
> to (re)compile Arrow C++ with ARROW_COMPUTE enabled, is t
I don't think you need anything more on the PyArrow side, but you need
to (re)compile Arrow C++ with ARROW_COMPUTE enabled, is that the case?
Le 07/07/2022 à 22:16, Li Jin a écrit :
Hello,
I am trying to build Arrow/Pyarrow with our internal build system (cmake
based) and encounter and e
Hello,
I am trying to build Arrow/Pyarrow with our internal build system (cmake
based) and encounter and error when running pyarrow test:
ImportError while importing test module
'/home/ljin/vats/add-arrowpython-master/ext/public/python/pyarrow/master/dist/lib/python3.9/pyarrow/tests/test_table.py
Thanks! This is very helpful.
On Thu, Jul 7, 2022 at 11:33 AM Jacob Wujciak wrote:
> The crossbow tarballs do not contain the arrow source, they only contain
> the crossbow source (aka a few yaml files).
>
> On Thu, Jul 7, 2022 at 5:29 PM Li Jin wrote:
>
> > Thanks both!
> >
> > This looks use
The crossbow tarballs do not contain the arrow source, they only contain
the crossbow source (aka a few yaml files).
On Thu, Jul 7, 2022 at 5:29 PM Li Jin wrote:
> Thanks both!
>
> This looks useful - I wonder if the version between the crossbow tarball
> and pypi dist matches (i.e. are they pub
Hello Li,
The jobs that create the wheels and sdist a part of the nightly packaging
builds [1].
Within the log [2] you can check the exact sha the wheels/sdist are built
with and use https://github.com/apache/arrow/tarball/ to get the
matching source tarball.
Best
Jacob
[1]:
https://github.com/u
Thanks both!
This looks useful - I wonder if the version between the crossbow tarball
and pypi dist matches (i.e. are they published together)? i.e. the latest
crossbow source is "Nightly-test-2022-07-07" and the pypi fury is
"pyarrow-9.0.0.dev341.tar.gz"
On Thu, Jul 7, 2022 at 11:19 AM Joris V
We do upload nightly wheels to an alternative PyPI index (see
https://arrow.apache.org/docs/python/install.html#installing-nightly-packages),
at https://pypi.fury.io/arrow-nightlies/pyarrow, and it seems we actually
also upload an sdist there.
(it could still be more reliable to used HEAD, though,
Ah.
Hmm, while the Crossbow releases do include wheels, they don't include the
source (the source link there is for the CI scripts, not Arrow itself). I'm not
sure if there's anything else that packages the Python sources.
On Thu, Jul 7, 2022, at 11:10, Li Jin wrote:
> Thanks David - I am curr
Thanks David - I am currently using [1]. A slight issue is that I don't
have the corresponding sdist for "pyarrow" so I'm wondering if there is an
easy way for me to grab matching [1] source and pyarrow source tarball.
Not a big issue, I can create the pyarrow dist from [1]
On Thu, Jul 7, 2022 at
Does it work to just use the HEAD [1]?
Otherwise, the nightly CI jobs do generate releases, but I don't think there's
much of a guarantee around the naming/format being stable [2].
I'm not sure if there's an explicit/dedicated "nightly source release" (I'm not
aware of one), or how that would d
Hello,
I wonder if we have nightly source tarball published somewhere?
Li
Now at https://github.com/apache/arrow-nanoarrow
Dewey: you can use .asf.yaml to enable issues and such:
https://cwiki.apache.org/confluence/display/INFRA/Git+-+.asf.yaml+features#Git.asf.yamlfeatures-GitHubsettings
On Thu, Jul 7, 2022, at 09:06, David Li wrote:
> I'll go ahead and set up arrow-
I'll go ahead and set up arrow-nanoarrow for convenience.
In the medium term we should think about whether arrow-adbc and arrow-nanoarrow
should be folded back into the arrow monorepo, in order to potentially reduce
the release/CI maintenance burden, or document why we've chosen to split those
> However, from my understanding, the implementation effort is quite
> manageable. Please correct me if I'm wrong,..
Yes, this was discussed earlier in this thread. The protobuf compiler would
need to be ran by the PyArrow build; currently, only the Arrow C++ build runs
it. There would be a bit
Hi Yaron,
Le 07/07/2022 à 10:48, Yaron Gvili a écrit :
It looks like the main decision to make is whether accessing Substrait protobuf
Python classes from PyArrow is what we want here, because it would require some
effort to implement as discussed.
I'm not the best person to comment on thi
It looks like the main decision to make is whether accessing Substrait protobuf
Python classes from PyArrow is what we want here, because it would require some
effort to implement as discussed.
> These are two reasons to need the protobuf in python. However, I still don't
> fully understand wh
20 matches
Mail list logo
