Re: Upgrades of streaming jobs

Yes, all checkpoints are forward compatible. However, you do need to restart the query if you want to update the code of the query. This downtime can be in less than a second (if you just restart the query without stopping the application/Spark driver) or 10s of seconds (if you have to stop the ap

Upgrades of streaming jobs

Hi What is the state of spark structured streaming jobs and upgrades? Can checkpoints of version 1 be read by version 2 of a job? Is downtime required to upgrade the job? Thanks