Hi, we are currently working on improving the PLC4X Kafka Connect plugin. While the current version offered a pretty simple configuration, this is not quite suitable for production scenarios.
In contrast to normal Kafka Connect sources/sinks the PLC4X source and sink should distribute the load that a connection to an industrial controller is only initiated by one Kafka connect node at a time (The reason for this is, that industrial controllers are only able to accept a very limited number of connections) So the Kafka Connect system should distribute these sources to it’s nodes. For each source we also are able to run multiple jobs which collect given sets of parameters in individual intervals and push the data to individual Kafka topics. We came up with the following configuration: name=plc-source-test connector.class=org.apache.plc4x.kafka.Plc4xSourceConnector defaults.topic=some/default sources.machineA.connectionString=s7://1.2.3.4/1/1 sources.machineA.jobReferences.s7-dashboard.enabled=true sources.machineA.jobReferences.s7-heartbeat.enabled=true sources.machineA.jobReferences.s7-heartbeat.topic=heartbeat sources.machineB.connectionString=s7://10.20.30.40/1/1 sources.machineB.topic=heartbeat sources.machineB.jobReferences.s7-heartbeat.enabled=true sources.machineC.connectionString=ads://1.2.3.4.5.6 sources.machineC.topic=heartbeat sources.machineC.jobReferences.ads-heartbeat.enabled=true jobs.s7-dashboard.interval=500 jobs.s7-dashboard.fields.inputPreasure=%DB.DB1.4:INT jobs.s7-dashboard.fields.outputPreasure=%Q1:BYT jobs.s7-dashboard.fields.temperature=%I3:INT jobs.s7-heartbeat.interval=1000 jobs.s7-heartbeat.fields.active=%I0.2:BOOL jobs.ads-heartbeat.interval=1000 jobs.ads-heartbeat.fields.active=Main.running So we define the individual sources which map to PLC connections and each connection references a set of jobs. “Source” and “Job” are PLC4X terms, so if they collide with Kafka Connect terms, please don’t be confused. We are able to successfully process this into a working configuration. The problem is, that we would like to certify the driver and therefore are required to use the ConfigDef objects to describe the configuration structure. What would be the best way to do this? Or is our approach completely wrong? As far as I can see it, I can only configure my Kafka Connect plugins via property structures (Even if they’re transferred via JAVA in the distributed mode) We would greatly appreciate some help here. Chris
