On 11/16/2015 08:10 AM, Sam Groth wrote:
So I have developed for a similar scenario where we had APIs in 3 different
languages that needed to be kept in sync. The releases were split, and we only
had some high level tests to verify compatibility and therefore had to be very
careful what changes we made to avoid failures across many different use cases
of our APIs. There were a few people who were required to review any changes to
this API. I feel that in a project with as many implementations as Avro, it
will be difficult to have this same level of review. If it's possible to write
some linked tests for most of the implementations like Niels suggested, it
should be ok to split the releases. We would still have to pay close attention
to changes in these test cases.
Sam
I agree with the need to validate that files are to spec, but we
currently have that problem regardless of whether we release components
as a group or individually. Right now we don't do much cross-language
testing at all so I think this should be a goal (an important one!), not
a requirement for changing our releases.
On Saturday, November 14, 2015 7:54 AM, Niels Basjes <ni...@basjes.nl>
wrote:
Hi,
First of all a +1 from me to move to git.
Sounds like we have consensus around moving to git, so at least we can
get that started.
Regarding the "how many parts and the release cycle";
In general I'm in the "release often" camp.
Yet I understand the needless confusion if a certain language has not
changed.
How about this idea:
We create a separate project (avro-spec) with the specification/testcases
and such that guard the language specification. All language specific
implementations must reference this (perhaps by using the git feature of a
"linked submodule") and run the tests during the language specific deploy.
The version of this module therefor the version of the Avro format
specification.
This is similar to what we do in Parquet: we have a parquet-format
project that has meaningful versions for file compatibility. The version
for this dependency doesn't necessarily determine the version of a file
because the API can choose what underlying version it uses.
I think it would be good if these modules use a version that essentially is
<avro-spec version>-<library version>
This way the libraries can have a separate release cycle and still clearly
indicate the supported Avro specifcation.
I think this makes sense. Right now, while we have only one version of
the file format, do we need to include the format version in all version
numbers? We could put off that step when we have more than one format
version to worry about.
rb
--
Ryan Blue
Software Engineer
Cloudera, Inc.
