Hi Chen,
I think in a long-term perspective it makes sense to support things like
this. The next big step is dynamic scaling without stopping the
execution. Partial upgrades could be addressed afterwards, but I'm not
aware of any plans.
Until then, I would recommend a different architecture by using
connect() and stream in a new logic dynamically. This is especially
interesting for ML models etc.
Regards,
Timo
Am 10/1/17 um 3:03 AM schrieb Chen Qin:
Hi there,
So far, flink job is interpreted and deployed during bootstrap phase. Once
pipeline runs, it's very hard to do partial upgrade without stop execution.
(like savepoint is heavy) Is there any plan to allow upload annotated jar
package which hints which stream tasks implementation CAN BE partial
upgraded after next checkpoint succeed without worry about backfill.
Thanks,
Chen
