Re: validate identity of producer in each record

2017-03-20 Thread Matt Magoffin
OK, thank you for that. I looked at org.apache.kafka.connect.transforms.Transformation and org.apache.kafka.connect.source.SourceRecord, but am not discovering where the authenticated username (or Principal) might be available for the call to Transformation.apply()… or does Connect even support

Re: validate identity of producer in each record

2017-03-20 Thread Hans Jespersen
Nothing on the broker today but if you use Kafka Connect API in 0.10.2 and above there is a pluggable interface called Transformations. See org.apache.kafka.connect.transforms in https://kafka.apache.org/0102/javadoc/index.html?org/apache/kafka/connect Source Connector transformations happen b

Re: validate identity of producer in each record

2017-03-20 Thread Matt Magoffin
Thanks, Hans. Signing messages is a good idea. Other than that, is there possibly an extension point in Kafka itself on the receiving of records, before they are stored/distributed? I was thinking along the lines of org.apache.kafka.clients.producer.ProducerInterceptor but on the server side?

Re: validate identity of producer in each record

2017-03-20 Thread Matt Magoffin
Thanks, Hans. Signing messages is a good idea. Other than that, is there possibly an extension point in Kafka itself on the receiving of records, before they are stored/distributed? I was thinking along the lines of org.apache.kafka.clients.producer.ProducerInterceptor but on the server side?

Re: validate identity of producer in each record

2017-03-20 Thread Hans Jespersen
You can configure Kafka with ACLs that only allow certain users to produce/consume to certain topics but if multiple producers are allowed to produce to a shared topic then you cannot identify them without adding something to the messages. For example, you can have each producer digitally sign (or