Hi, I remember there was a discussion about better supporting auto scaling for structured streaming. Is there anything happening with that for the upcoming Spark 4.0 release? Will there be support for auto scaling (at least on K8s) spark structured streaming apps?
Thanks, Nimrod