Re: Flink runner - restore from savepoint after evolving POJOs

2021-07-10 Thread Pavel Solomin
Ok, this is clear. For now, I went a different path though - having my business logic-related POJOs still Avro-generated, but I introduced another, generic one, which just stores schema & payload bytes, and does not need to change. Having that, I use https://beam.apache.org/releases/javadoc/2.30.0

Re: Flink runner - restore from savepoint after evolving POJOs

2021-07-09 Thread Pavel Solomin
Hello Luke, thanks for your reply. I do not set serialVersionUID myself, but when I change Avro schema of my PoJo, the generated class has that VersionUID changed. So, as I understand your suggestion, it is basically this: 1 - don't use Avro-generated PoJo, cause there is no control of serialVers

Flink runner - restore from savepoint after evolving POJOs

2021-07-06 Thread Pavel Solomin
Hello! I am using Beam 2.30.0 with Flink runner 1.11. The app gets input data as some POJOs, creates fixed windows of such, and then writes window files to AWS S3. As the data evolves, I want to add new fields to my POJOs (which are actually avro-generated ones), and, given the new POJO is fully