Re: Upgrading Avro dependencies

2023-11-15 Thread Alexey Romanenko
As I mentioned before, we always can generate version-dependent Avro classes by running “org.apache.avro.tool.Main" directly with “JavaExec” Gradle task. Please, see this implementation in Avro extension: https://github.com/apache/beam/blob/c713425e1ac2cdc3ec2ec264c9bf61f7356856bd/sdks/java/exte

Re: Upgrading Avro dependencies

2023-11-15 Thread John Casey via dev
Alright, it looks like something was broken with my setup. I think a straightforward upgrade is actually possible, so I'm going to continue on that On Wed, Nov 15, 2023 at 10:17 AM John Casey wrote: > So, thats the thing. I've upgraded to 1.11.3, but the plugin still seems > to be generating Avr

Re: Upgrading Avro dependencies

2023-11-15 Thread John Casey via dev
So, thats the thing. I've upgraded to 1.11.3, but the plugin still seems to be generating Avro code that isn't compatible with 1.11.3. Looking at the PR https://github.com/apache/beam/pull/29390, it looks like it generates avro code based on 1.9.2, which ends up being incompatible. I don't think

Re: Upgrading Avro dependencies

2023-11-14 Thread Alexey Romanenko
Thanks! Please, let me know if you need any help on this. — Alexey > On 14 Nov 2023, at 17:52, John Casey wrote: > > The vulnerability said to upgrade to 1.11.3, so I think that would be my > starting point. > > > On Mon, Nov 13, 2023 at 12:23 PM Alexey Romanenko

Re: Upgrading Avro dependencies

2023-11-14 Thread John Casey via dev
The vulnerability said to upgrade to 1.11.3, so I think that would be my starting point. On Mon, Nov 13, 2023 at 12:23 PM Alexey Romanenko wrote: > > > On 10 Nov 2023, at 19:23, John Casey wrote: > > I guess I'm a bit confused as to why specifically generateTestAvroJava > seems to use the wron

Re: Upgrading Avro dependencies

2023-11-13 Thread Alexey Romanenko
> On 10 Nov 2023, at 19:23, John Casey wrote: > > I guess I'm a bit confused as to why specifically generateTestAvroJava seems > to use the wrong version. I see our version specific generated code, but this > action appears to be inherited from the plugin, and is configured with > whichever

Re: Upgrading Avro dependencies

2023-11-10 Thread John Casey via dev
I guess I'm a bit confused as to why specifically generateTestAvroJava seems to use the wrong version. I see our version specific generated code, but this action appears to be inherited from the plugin, and is configured with whichever avro version is provided. Given that I tried to just change to

Re: Upgrading Avro dependencies

2023-11-10 Thread Alexey Romanenko
Hi John, This old Avro version in Beam is a very long story. Briefly, since initially it was toughly integrated into Java SDK “core” module then it was not possible to upgrade an Avro version without breaking changes for users (because of some Avro incompatible changes, as you have noticed befo

Upgrading Avro dependencies

2023-11-10 Thread John Casey via dev
Hi All, There was a CVE detected in Avro 1.8.2 (CVE-2023-39410), so I'm trying to upgrade to avro 1.11.3. Unfortunately, it seems that our auto-generated Avro test classes aren't being generated properly with this new version. I've updated our avro generation plugin as well, but for whatever reas