maksaska commented on code in PR #11497: URL: https://github.com/apache/ignite/pull/11497#discussion_r1976395112
########## docs/_docs/extensions-and-integrations/change-data-capture-extensions.adoc: ########## @@ -193,6 +315,17 @@ Now, you have additional binary `$IGNITE_HOME/bin/kafka-to-ignite.sh` and `$IGNI NOTE: Please, enable `ignite-cdc-ext` to be able to run `kafka-to-ignite.sh`. +==== Kafka Installation Review Comment: Changed it ########## docs/_docs/extensions-and-integrations/change-data-capture-extensions.adoc: ########## @@ -165,14 +237,64 @@ section of the official Kafka documentation. | `MarkersCount` | Count of metadata markers sent to Kafka. |=== +=== Configuration example + +Add link:https://github.com/apache/ignite-extensions/blob/master/modules/cdc-ext/src/main/java/org/apache/ignite/cdc/kafka/IgniteToKafkaCdcStreamer.java[Ignite2KafkaCdcStreamer] bean configuration to the source ignite configuration. + +```xml +<!--Kafka properties for CDC streamer--> +<util:properties id="kafkaProperties" location="file:/config/path/kafka.properties"/> + +<!--IgniteToKafkaCdcStreamer--> +<bean id="cdc.streamer" class="org.apache.ignite.cdc.kafka.IgniteToKafkaCdcStreamer"> + <property name="topic" value="dc1_to_dc2"/> + <property name="metadataTopic" value="metadata_from_dc1"/> + <property name="kafkaPartitions" value="16"/> + <property name="caches"> + <list> + <value>terminator</value> + </list> + </property> + <property name="maxBatchSize" value="1024"/> + <property name="onlyPrimary" value="false"/> + <property name="kafkaProperties" ref="kafkaProperties"/> +</bean> +``` + +NOTE: Use the resulting configuation with `ignite-cdc.sh` to start CDC client on the source Ignite instance. + +NOTE: You need to preactivate source cluster before starting the CDC client. You can use `command.sh` for that + +``` +./control.sh --set-state ACTIVE --host localhost:server_connector_port --yes +``` + +You can specify Kafka producer properties in the separate file `kafka.properties` + +```xml +bootstrap.servers=xxx.x.x.x:9092 +request.timeout.ms=10000 +``` + +NOTE: Configure Kafka topics beforehand. CDC clients will fail on trying to connect, if Kafka topics were not started. + +In this CDC scenario we use Kafka with Zookeeper. For that to work, set two Kafka topics prior to CDC start-up. We used 'dc1_to_dc2' and 'metadata_from_dc1' naming respectively. You can use the following commands: + +``` +./kafka-topics.sh --create --partitions 16 --replication-factor 1 --topic dc1_to_dc2 --bootstrap-server localhost:9092 +./kafka-topics.sh --create --partitions 1 --replication-factor 1 --topic metadata_from_dc1 --bootstrap-server localhost:9092 +``` + +NOTE: For Active-Passive replication regime with Kafka two topics would suffice. For Active-Active replication regime you need another two topics to transfer data from the second cluster. Review Comment: Done! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@ignite.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
