[
https://issues.apache.org/jira/browse/SPARK-47733?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Anish Shrigondekar resolved SPARK-47733.
----------------------------------------
Resolution: Fixed
> Add operational metrics for TWS operators
> -----------------------------------------
>
> Key: SPARK-47733
> URL: https://issues.apache.org/jira/browse/SPARK-47733
> Project: Spark
> Issue Type: Task
> Components: Structured Streaming
> Affects Versions: 4.0.0
> Reporter: Jing Zhan
> Assignee: Anish Shrigondekar
> Priority: Major
> Labels: pull-request-available
>
> Add metrics to improve observability for the newly added operator
> TransformWithState and some changes we've made into RocksDB.
> Proposed metrics to add:
> * on the RocksDB StateStore metrics side, we will add the following:
> ** num external col families
> ** num internal col families
> * on the operator side, we will add the following:
> ** number of state vars
> ** count of state vars by type
> ** output mode
> ** timeout mode
> ** registered timers in batch
> ** expired timers in batch
> ** initial state enabled or not
> ** number of state vars removed in batch
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
