How are folks here managing deployments in production? We are deploying Flink jobs on EMR manually at the moment but would like to move towards some form of automation before anything goes into production. Adding additional EMR Steps to a long running cluster to deploy or update jobs seems like the most accessible method, however getting feedback output (i.e. listing currently running jobs, detecting transient failures) is lacking.
I've briefly looked into Ververica platform which is an ideal solution except for the licensing aspect. Ideally I'd like to work with something open source. A UI is not necessary. Has anyone tried ING-Banks open source tool? https://github.com/ing-bank/flink-deployer Any and all ideas are welcome, thanks!
