Thanks Jacques. I agree that none of the ways forward on this problem
are wholly satisfactory. We should encourage users of this C API to
prefer emitting byte-aligned / 0-offset in line with the IPC spec
wherever possible. It will be interesting to see after a period of
time how downstream projects
Upon further reflection (and as I've noted on the PR), I think merging the
ABI as a general feature of Arrow is preferable to making this be a
subinterface of the C++ part of the project. While the offset field is
awkward given its absence from the IPC spec, it's better to avoid
fragmenting the com
Independent of the particulars of the discussion, the C++ project
needs to be free to create a C API for itself. If you want to try to
block the C++ contributors from doing this we may be barreling toward
a governance crisis in the project. I'm stepping back from this
discussion for a time now to a
I don't see this as an endogenous concern of the C++ project. I appreciate
your goal with saying so but I think this has broader ramifications around
fragmentation of the project.
The core challenge that we're dealing with is we introduced foundational
concepts in some implementations that go beyo
hi Jacques,
Taking a step back from the discussion, the original problem statement
was to enable third party projects to produce the data structure used
by C++ Array classes in C without depending on the C++ code
That's the ArrayData class here
https://github.com/apache/arrow/blob/master/cpp/src
As I noted on the pull request, I think fundamentally this work is at odds
with the Arrow specification and being used to introduce a shadow
specification.
I don't think our intentions about how people should use something really
influence how people will actually use or perceive it. They'll just
hi folks,
I just made a comment in https://github.com/apache/arrow/pull/6026
that I wanted to surface here on the mailing list.
It seems that to reach consensus for a C interface that is intended to
be broadly used by multiple programming languages, we may make some
compromises that harm or outri
Thanks for addressing my comments. I'm actively reviewing the proposal. It
is taking me more time than I would like given the time of the year but I
want to make sure that you know that I'm looking at it and hope to provide
additional feedback beyond that which I've provided thus far on the PR.
Wil
Hello,
Following Jacques's feedback, I drafted a new version of the C data
interface spec.
The spec PR is here:
https://github.com/apache/arrow/pull/6040
Direct link to the RST file:
https://github.com/apache/arrow/blob/5d8669d371401f9db12326b079e13c0058ba972b/docs/source/format/CDataInterface.
