Sijie.
let me write an example.
if you have this Pojo class
class Pojo {
String foo;
String bar;
}
if you use Schema.AVRO(Pojo.class)
with Avro 1.9 you will probably see a Schema with the fields in this
order: foo, bar
with Avro 1.10 you will see exactly the schema with bar, foo. (in
alphabetical order)
This is a potential problem because the same Java code that used to
write/read the "foo,bar" schema after upgrading to the new version of
Pulsar (with Avro 1.10) it will start writing with a different schema
("bar,foo")
and those two schemas are not compatible to each other. Because you
cannot change the order of fields.
We have tests that verify that Schema.AVRO(Pojo.class) produces a
schema in the expected order of fields, and they are failing.
I had started to fix all of the tests (the fix is trivial), but as far
as I was fixing them I realized that this change may cause unexpected
problems to users.
you can see the errors here on CI on the Avro upgrade patch
https://github.com/apache/pulsar/pull/9898/checks?check_run_id=2106075182
I have tried to reproduce a problem with real clients but I am not
able to produce an error.
I am missing some part of the story, do you have an explanation ?
Enrico
Il giorno mar 16 mar 2021 alle ore 02:13 Sijie Guo
<guosi...@gmail.com> ha scritto:
>
> I don't quite understand the compatibility issue here. Looking into
> AVRO-2579, it seems that only the order of fields returned will be
> different. We don't depend on the ordering for the compatibility checks.
>
> Can you explain more about the compatibility issue you refer to?
>
> Thanks,
> Sijie
>
> On Mon, Mar 15, 2021 at 3:52 AM Enrico Olivelli <eolive...@gmail.com> wrote:
>
> > Hello,
> > I am working on the upgrade of Avro from 1.9 to 1.10.1.
> >
> > I noticed a bad behaviour change about Schema.AVRO(Pojo.class).
> > Basically the new Avro version creates the Schema by sorting the
> > fields in alphabetical order and this is an incompatible schema
> > change.
> >
> > More details here
> > https://github.com/apache/pulsar/pull/9898
> >
> > and here
> > https://issues.apache.org/jira/browse/AVRO-2579
> >
> > Am I missing something?
> >
> > Thoughts ?
> >
> > Enrico
> >
