[
https://issues.apache.org/jira/browse/KAFKA-7223?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16719672#comment-16719672
]
ASF GitHub Bot commented on KAFKA-7223:
---------------------------------------
guozhangwang closed pull request #6024: KAFKA-7223: document suppression buffer
metrics
URL: https://github.com/apache/kafka/pull/6024
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/docs/ops.html b/docs/ops.html
index f48791181c2..64f71593d5d 100644
--- a/docs/ops.html
+++ b/docs/ops.html
@@ -1441,7 +1441,7 @@ <h4><a id="kafka_streams_monitoring"
href="#kafka_streams_monitoring">Streams Mo
<pre>metrics.recording.level="info"</pre>
<h5><a id="kafka_streams_thread_monitoring"
href="#kafka_streams_thread_monitoring">Thread Metrics</a></h5>
-All the following metrics have a recording level of ``info``:
+All the following metrics have a recording level of <code>info</code>:
<table class="data-table">
<tbody>
<tr>
@@ -1563,7 +1563,7 @@ <h5><a id="kafka_streams_thread_monitoring"
href="#kafka_streams_thread_monitori
</table>
<h5><a id="kafka_streams_task_monitoring"
href="#kafka_streams_task_monitoring">Task Metrics</a></h5>
-All the following metrics have a recording level of ``debug``:
+All the following metrics have a recording level of <code>debug</code>:
<table class="data-table">
<tbody>
<tr>
@@ -1605,7 +1605,7 @@ <h5><a id="kafka_streams_task_monitoring"
href="#kafka_streams_task_monitoring">
</table>
<h5><a id="kafka_streams_node_monitoring"
href="#kafka_streams_node_monitoring">Processor Node Metrics</a></h5>
- All the following metrics have a recording level of ``debug``:
+ All the following metrics have a recording level of <code>debug</code>:
<table class="data-table">
<tbody>
<tr>
@@ -1703,13 +1703,35 @@ <h5><a id="kafka_streams_node_monitoring"
href="#kafka_streams_node_monitoring">
<td>The total number of of records being forwarded downstream, from
source nodes only. </td>
<td>kafka.streams:type=stream-processor-node-metrics,client-id=([-.\w]+),task-id=([-.\w]+),processor-node-id=([-.\w]+)</td>
</tr>
+ <tr>
+ <td>suppression-emit-rate</td>
+ <td>
+ The rate at which records that have been emitted downstream from
suppression operation nodes.
+ Compare with the <code>process-rate</code> metric to determine how
many updates are being suppressed.
+ </td>
+
<td>kafka.streams:type=stream-processor-node-metrics,client-id=([-.\w]+),task-id=([-.\w]+),processor-node-id=([-.\w]+)</td>
+ </tr>
+ <tr>
+ <td>suppression-emit-total</td>
+ <td>
+ The total number of records that have been emitted downstream from
suppression operation nodes.
+ Compare with the <code>process-total</code> metric to determine how
many updates are being suppressed.
+ </td>
+
<td>kafka.streams:type=stream-processor-node-metrics,client-id=([-.\w]+),task-id=([-.\w]+),processor-node-id=([-.\w]+)</td>
+ </tr>
</tbody>
</table>
<h5><a id="kafka_streams_store_monitoring"
href="#kafka_streams_store_monitoring">State Store Metrics</a></h5>
- All the following metrics have a recording level of ``debug``. Note that the
``store-scope`` value is specified in <code>StoreSupplier#metricsScope()</code>
for user's customized
- state stores; for built-in state stores, currently we have
<code>in-memory-state</code>, <code>in-memory-lru-state</code>,
<code>rocksdb-state</code> (for RocksDB backed key-value store),
- <code>rocksdb-window-state</code> (for RocksDB backed window store) and
<code>rocksdb-session-state</code> (for RocksDB backed session store).
+ All the following metrics have a recording level of <code>debug</code>. Note
that the <code>store-scope</code> value is specified in
<code>StoreSupplier#metricsScope()</code> for user's customized
+ state stores; for built-in state stores, currently we have:
+ <ul>
+ <li><code>in-memory-state</code></li>
+ <li><code>in-memory-lru-state</code></li>
+ <li><code>rocksdb-state</code> (for RocksDB backed key-value store)</li>
+ <li><code>rocksdb-window-state</code> (for RocksDB backed window
store)</li>
+ <li><code>rocksdb-session-state</code> (for RocksDB backed session
store)</li>
+ </ul>
<table class="data-table">
<tbody>
@@ -1902,7 +1924,7 @@ <h5><a id="kafka_streams_store_monitoring"
href="#kafka_streams_store_monitoring
</table>
<h5><a id="kafka_streams_cache_monitoring"
href="#kafka_streams_cache_monitoring">Record Cache Metrics</a></h5>
- All the following metrics have a recording level of ``debug``:
+ All the following metrics have a recording level of <code>debug</code>:
<table class="data-table">
<tbody>
@@ -1929,6 +1951,44 @@ <h5><a id="kafka_streams_cache_monitoring"
href="#kafka_streams_cache_monitoring
</tbody>
</table>
+ <h5><a id="kafka_streams_buffer_monitoring"
href="#kafka_streams_buffer_monitoring">Suppression Buffer Metrics</a></h5>
+ All the following metrics have a recording level of <code>debug</code>:
+
+ <table class="data-table">
+ <tbody>
+ <tr>
+ <td>suppression-buffer-size-current</td>
+ <td>The current total size, in bytes, of the buffered data.</td>
+
<td>kafka.streams:type=stream-buffer-metrics,client-id=([-.\w]+),task-id=([-.\w]+),buffer-id=([-.\w]+)</td>
+ </tr>
+ <tr>
+ <td>suppression-buffer-size-avg</td>
+ <td>The average total size, in bytes, of the buffered data over the
sampling window.</td>
+
<td>kafka.streams:type=stream-buffer-metrics,client-id=([-.\w]+),task-id=([-.\w]+),buffer-id=([-.\w]+)</td>
+ </tr>
+ <tr>
+ <td>suppression-buffer-size-max</td>
+ <td>The maximum total size, in bytes, of the buffered data over the
sampling window.</td>
+
<td>kafka.streams:type=stream-buffer-metrics,client-id=([-.\w]+),task-id=([-.\w]+),buffer-id=([-.\w]+)</td>
+ </tr>
+ <tr>
+ <td>suppression-buffer-count-current</td>
+ <td>The current number of records buffered.</td>
+
<td>kafka.streams:type=stream-buffer-metrics,client-id=([-.\w]+),task-id=([-.\w]+),buffer-id=([-.\w]+)</td>
+ </tr>
+ <tr>
+ <td>suppression-buffer-size-avg</td>
+ <td>The average number of records buffered over the sampling
window.</td>
+
<td>kafka.streams:type=stream-buffer-metrics,client-id=([-.\w]+),task-id=([-.\w]+),buffer-id=([-.\w]+)</td>
+ </tr>
+ <tr>
+ <td>suppression-buffer-size-max</td>
+ <td>The maximum number of records buffered over the sampling
window.</td>
+
<td>kafka.streams:type=stream-buffer-metrics,client-id=([-.\w]+),task-id=([-.\w]+),buffer-id=([-.\w]+)</td>
+ </tr>
+ </tbody>
+ </table>
+
<h4><a id="others_monitoring" href="#others_monitoring">Others</a></h4>
We recommend monitoring GC time and other stats and various server stats
such as CPU utilization, I/O service time, etc.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> KIP-328: Add in-memory Suppression
> ----------------------------------
>
> Key: KAFKA-7223
> URL: https://issues.apache.org/jira/browse/KAFKA-7223
> Project: Kafka
> Issue Type: Improvement
> Components: streams
> Reporter: John Roesler
> Assignee: John Roesler
> Priority: Major
>
> As described in
> [https://cwiki.apache.org/confluence/display/KAFKA/KIP-328%3A+Ability+to+suppress+updates+for+KTables.]
>
> This ticket is to implement Suppress, but only for in-memory buffers.
> (depends on KAFKA-7222)
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
