Re: Problems with Avro upgrade from 1.9 to 1.10

2021-03-16 Thread Enrico Olivelli
Sijie, Thanks for your explanation. Now it is clear to me that it won't be a problem for users. I will fix all of the tests on my PR Enrico Il Mar 16 Mar 2021, 19:15 Sijie Guo ha scritto: > I would be surprised that it would break compatibility just because it > writes the schema definition

Re: Problems with Avro upgrade from 1.9 to 1.10

2021-03-16 Thread Sijie Guo
I would be surprised that it would break compatibility just because it writes the schema definition in a different order. The test validating how schema definition was written should be improved. I don't see that breaks the compatibility. The test case you have is problematic. If you want to veri

Re: Problems with Avro upgrade from 1.9 to 1.10

2021-03-16 Thread Enrico Olivelli
Matteo, actually it breaks @Test public void helloCompatibility() throws Exception { // sampled data created using Pulsar with Avro 1.9 String schemaAvro19 = "{\"type\":\"record\",\"name\":\"MyPojo\",\"namespace\":\"SenderTest\",\"fields\":[{\"name\":\"foo\",\"type\":[\"null\",\"string\"],\"

Re: Problems with Avro upgrade from 1.9 to 1.10

2021-03-16 Thread Matteo Merli
A good test to add would be to include the object serialized by avro 1.9 and ensure we can read it with avro 1.10. On Tue, Mar 16, 2021 at 8:13 AM Enrico Olivelli wrote: > Sijie. > let me write an example. > > if you have this Pojo class > class Pojo { > String foo; > String bar; > } >

Re: Problems with Avro upgrade from 1.9 to 1.10

2021-03-16 Thread Enrico Olivelli
Sijie. let me write an example. if you have this Pojo class class Pojo { String foo; String bar; } if you use Schema.AVRO(Pojo.class) with Avro 1.9 you will probably see a Schema with the fields in this order: foo, bar with Avro 1.10 you will see exactly the schema with bar, foo. (in alp

Re: Problems with Avro upgrade from 1.9 to 1.10

2021-03-15 Thread Sijie Guo
I don't quite understand the compatibility issue here. Looking into AVRO-2579, it seems that only the order of fields returned will be different. We don't depend on the ordering for the compatibility checks. Can you explain more about the compatibility issue you refer to? Thanks, Sijie On Mon, M

Problems with Avro upgrade from 1.9 to 1.10

2021-03-15 Thread Enrico Olivelli
Hello, I am working on the upgrade of Avro from 1.9 to 1.10.1. I noticed a bad behaviour change about Schema.AVRO(Pojo.class). Basically the new Avro version creates the Schema by sorting the fields in alphabetical order and this is an incompatible schema change. More details here https://github.