We are using Kafka Connect consumers that consume from the raw unredacted topic and apply transformations and produce to a redacted topic. Using kafka connect allows us to set it all up with an HTTP request and doesn't require additional infrastructure.
Then we wrote a KafkaPrincipal builder to authenticate each consumer to their service names. KafkaPrincipal class is specified in the server.properties file on the brokers. To provide topic level access control we just configured SimpleAclAuthorizer. The net result is, some consumers can only read redacted topic and very few have consumers can read unredacted. On Wed, Nov 22, 2017 at 10:47 AM David Espinosa <espi...@gmail.com> wrote: > Hi all, > I would like to double check with you how we want to apply some GDPR into > my kafka topics. In concrete the "right to be forgotten", what forces us to > delete some data contained in the messages. So not deleting the message, > but editing it. > For doing that, my intention is to replicate the topic and apply a > transformation over it. > I think that frameworks like Kafka Streams or Apache Storm. > > Did anybody had to solve this problem? > > Thanks in advance. > -- Scott Reynolds Principal Engineer [image: twilio] <http://www.twilio.com/?utm_source=email_signature> MOBILE (630) 254-2474 EMAIL sreyno...@twilio.com
