Hey all, I have been working on adding operational metrics for monitoring the health and performance of Structured Streaming applications. The detailed design and the WIP Github PR is here.
*JIRA *- SPARK-17731 <https://issues.apache.org/jira/browse/SPARK-17731> *Design Doc* - https://docs.google.com/document/d/ 1NIdcGuR1B3WIe8t7VxLrt58TJB4DtipWEbj5I_mzJys/edit?usp=sharing *PR* - https://github.com/apache/spark/pull/15307 Would be wonderful if you can give me feedback on the selection of metrics and the overall design. TD
