I'm looking for a comprehensive example for deploying a new connector plugin into an existing Kafka cluster. Is there any standard solution for distributing a connector jar across nodes, and then starting the connector ? Or is it a manual copy process (e.g., via pdcp), and then run the Connect REST API ?
At present, my understanding of the the procedure looks like: Copy the connector to all kafka nodes: pdcp -w kafka-node[0-8] shiny-new-connector.jar $KAFKA_HOME/lib # copy other resources as needed Use REST to start the connector: curl -H "Content-Type: application/json" -H "Accept: application/json" \ -X POST http://kafka-node0:8083/connectors \ --data-binary @shiny-new-connector.json Presumably, connect-distributed has already been executed to spawn worker processes. How does the Kafka Connect CLASSPATH get updated to include the connector jar when using the REST interface ? Or do all the workers have to be stopped/restarted to update the CLASSPATH ?
