> > A way to alleviate this is to change our status table from "pass / not > pass" to a "since version X.Y". This would offer the transparency into > which version it was implemented since, and would also align well with the > idea that Rust X.Y.Z is bundled as part of the arrow release.
Great idea we should make this change (at least reset all check marks to 4.0). > This does not address backward incompatible changes of the format, which is > a whole different beast (e.g. do we require all implementations to change > prior to releasing the new version of the format?) When versioning (more specifically release 1.0 of the spec) came up, we were hoping (and I think most people) would be opposed to breaking changes at this point. So this is a possibility, but I'd rather cross this road when we come to it. -Micah On Wed, Apr 28, 2021 at 9:56 PM Jorge Cardoso Leitão < jorgecarlei...@gmail.com> wrote: > Hi, > > Thanks for bringing this up. > > AFAIK, this has always been an issue because we do not require all > implementations to support a type prior to releasing that new type in the > spec. Concrete example: map has been released for a while, but Rust does > not support it yet. Same for interval, where we still do not test in > integration. > > My understanding is that once we accept that we can bump the spec without > having all implementations implement it, people will need to consult *some > table*. > > A related issue is the *some table* bit: someone using pyarrow==2 can't > tell whether they can send type X to Rust v2 via IPC. If they go to the > latest table [1], they can only see the latest status (4.0 in this case). > AFAIK they would need to download the source, go to the table in .rst for > that release, and check whether type X is supported by Rust on that > release. > > A way to alleviate this is to change our status table from "pass / not > pass" to a "since version X.Y". This would offer the transparency into > which version it was implemented since, and would also align well with the > idea that Rust X.Y.Z is bundled as part of the arrow release. > > This does not address backward incompatible changes of the format, which is > a whole different beast (e.g. do we require all implementations to change > prior to releasing the new version of the format?) > > [1] https://arrow.apache.org/docs/status.html > > > > On Thu, Apr 29, 2021 at 2:27 AM Weston Pace <weston.p...@gmail.com> wrote: > > > We now have independent releases. There has been some discussion (not > > sure if it was formalized) around aligning major release versions > > across the languages. > > > > There is also a potential format change coming up (new interval type). > > > > I think this brings up a few questions... > > > > Can an arrow library adopt a new format version in a minor release? > > Semantic versioning would imply yes but then I think we could get in a > > situation where Arrow-C++ X.Y.Z Might not be able to communicate via > > flight using Arrow-C++ X.A.B. > > > > In addition, is there value in aligning format adoption across languages? > > > > For example, if Rust adopts format version 1.1 in version 5 and > > pyarrow does not then users will need to consult a table to figure out > > which versions are interoperable. > > > > -Weston > > >
