tikimims commented on code in PR #12941:
URL: https://github.com/apache/kafka/pull/12941#discussion_r1042943030
##########
docs/connect.html:
##########
@@ -369,6 +369,114 @@ <h4><a id="connect_errorreporting"
href="#connect_errorreporting">Error Reportin
# Tolerate all errors.
errors.tolerance=all</pre>
+ <h4><a id="connect_exactlyonce" href="#connect_exactlyonce">Exactly-once
support</a></h4>
+
+ <p>Kafka Connect is capable of providing exactly-once delivery guarantees
for sink connectors (as of version 0.11.0) and source connectors (as of version
3.3.0). Please note that <b>support for exactly-once delivery is highly
dependent on the type of connector being run.</b> Even if all the correct
worker properties are set in the config for each node in a cluster, if a
connector is not designed to or simply cannot take advantage of the
capabilities of the Kafka Connect framework, exactly-once may not be
possible.</p>
+
+ <h5><a id="connect_exactlyoncesink" href="#connect_exactlyoncesink">Sink
connectors</a></h5>
+
+ <p>If a sink connector supports exactly-once delivery, all that is
necessary to enable exactly-once delivery at the Connect worker level is
ensuring that its consumer group is configured to ignore records in aborted
transactions. This can be done by setting the worker property
<code>consumer.isolation.level</code> to <code>read_committed</code> or, if
running a version of Kafka Connect that supports it, using a <a
href="#connectconfigs_connector.client.config.override.policy">connector client
config override policy</a> that allows the
<code>consumer.override.isolation.level</code> property to be set to
<code>read_committed</code> in individual connector configs. There are no
additional ACL requirements.</p>
+
+ <h5><a id="connect_exactlyoncesource"
href="connect_exactlyoncesource">Source connectors</a></h5>
+
+ <p>If a source connector supports exactly-once delivery, your Connect
cluster must also be configured to enable framework-level support for
exactly-once delivery for source connectors, and additional ACLs will be
necessary if running against a secured Kafka cluster. Note that exactly-once
support for source connectors is currently only available in distributed mode;
standalone Connect workers cannot provide exactly-once guarantees.</p>
+
+ <h6>Worker configuration</h6>
+
+ <p>For new Connect clusters, set the
<code>exactly.once.source.support</code> property to <code>enabled</code> in
the worker config for each node in the cluster. For existing clusters, two
rolling upgrades are necessary. During the first upgrade, the
<code>exactly.once.source.support</code> property should be set to
<code>preparing</code>, and during the second, it should be set to
<code>enabled</code>.</p>
+
+ <h6>ACL requirements</h6>
+
+ <p>With exactly-once source support enabled, the principal for each
Connect worker will require these ACLs:</p>
+
+ <table class="data-table">
+ <thead>
+ <tr>
+ <th>Operation</th>
+ <th>Resource Type</th>
+ <th>Resource Name</th>
+ <th>Note</th>
+ </tr>
+ </thead>
+ <tbody>
+ <tr>
+ <td>Write</td>
+ <td>TransactionalId</td>
+ <td><code>connect-cluster-${groupId}</code>, where
<code>${groupId}</code> is the <code>group.id</code> of the cluster</td>
+ <td></td>
+ </tr>
+ <tr>
+ <td>Describe</td>
+ <td>TransactionalId</td>
+ <td><code>connect-cluster-${groupId}</code>, where
<code>${groupId}</code> is the <code>group.id</code> of the cluster</td>
+ <td></td>
+ </tr>
+ <tr>
+ <td>IdempotentWrite</td>
+ <td>Cluster</td>
+ <td>ID of the Kafka cluster that hosts the worker's config
topic</td>
+ <td>The IdempotentWrite ACL has been deprecated as of 2.8 and
will only be necessary for Connect clusters running on pre-2.8 Kafka
clusters</td>
+ </tr>
+ </tbody>
+ </table>
+
+ <p>And the principal for each individual connector will require these
ACLs:</p>
Review Comment:
```suggestion
<p>And the principal for each individual connector will require the
following ACLs:</p>
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
