A good test to add would be to include the object serialized by avro 1.9
and ensure we can read it with avro 1.10.
On Tue, Mar 16, 2021 at 8:13 AM Enrico Olivelli <eolive...@gmail.com> wrote:
> Sijie.
> let me write an example.
>
> if you have this Pojo class
> class Pojo {
> String foo;
> String bar;
> }
>
> if you use Schema.AVRO(Pojo.class)
>
> with Avro 1.9 you will probably see a Schema with the fields in this
> order: foo, bar
> with Avro 1.10 you will see exactly the schema with bar, foo. (in
> alphabetical order)
>
> This is a potential problem because the same Java code that used to
> write/read the "foo,bar" schema after upgrading to the new version of
> Pulsar (with Avro 1.10) it will start writing with a different schema
> ("bar,foo")
> and those two schemas are not compatible to each other. Because you
> cannot change the order of fields.
>
> We have tests that verify that Schema.AVRO(Pojo.class) produces a
> schema in the expected order of fields, and they are failing.
> I had started to fix all of the tests (the fix is trivial), but as far
> as I was fixing them I realized that this change may cause unexpected
> problems to users.
>
> you can see the errors here on CI on the Avro upgrade patch
> https://github.com/apache/pulsar/pull/9898/checks?check_run_id=2106075182
>
> I have tried to reproduce a problem with real clients but I am not
> able to produce an error.
>
> I am missing some part of the story, do you have an explanation ?
>
> Enrico
>
>
> Il giorno mar 16 mar 2021 alle ore 02:13 Sijie Guo
> <guosi...@gmail.com> ha scritto:
> >
> > I don't quite understand the compatibility issue here. Looking into
> > AVRO-2579, it seems that only the order of fields returned will be
> > different. We don't depend on the ordering for the compatibility checks.
> >
> > Can you explain more about the compatibility issue you refer to?
> >
> > Thanks,
> > Sijie
> >
> > On Mon, Mar 15, 2021 at 3:52 AM Enrico Olivelli <eolive...@gmail.com>
> wrote:
> >
> > > Hello,
> > > I am working on the upgrade of Avro from 1.9 to 1.10.1.
> > >
> > > I noticed a bad behaviour change about Schema.AVRO(Pojo.class).
> > > Basically the new Avro version creates the Schema by sorting the
> > > fields in alphabetical order and this is an incompatible schema
> > > change.
> > >
> > > More details here
> > > https://github.com/apache/pulsar/pull/9898
> > >
> > > and here
> > > https://issues.apache.org/jira/browse/AVRO-2579
> > >
> > > Am I missing something?
> > >
> > > Thoughts ?
> > >
> > > Enrico
> > >
>
--
--
Matteo Merli
<mme...@apache.org>
