Hi Scott and thanks for your reply. For what you say, I guess that when you are asked to delete some "data user" (that's the "right to be forgotten" in GDPR), what you are really doing is blocking the access to it. I had a similar approach, based on the idea of Greg Young's solution of encrypting any private data and forgetting the key when data has to deleted. Sadly, our legal department after some checkins has conclude that this approach is "to block" data but not deleting it, as a consequence it can take us problems. If my guess about your solution is right, you could have the same problems.
Thanks 2017-11-22 19:59 GMT+01:00 Scott Reynolds <sreyno...@twilio.com.invalid>: > We are using Kafka Connect consumers that consume from the raw unredacted > topic and apply transformations and produce to a redacted topic. Using > kafka connect allows us to set it all up with an HTTP request and doesn't > require additional infrastructure. > > Then we wrote a KafkaPrincipal builder to authenticate each consumer to > their service names. KafkaPrincipal class is specified in the > server.properties file on the brokers. To provide topic level access > control we just configured SimpleAclAuthorizer. The net result is, some > consumers can only read redacted topic and very few have consumers can read > unredacted. > > On Wed, Nov 22, 2017 at 10:47 AM David Espinosa <espi...@gmail.com> wrote: > > > Hi all, > > I would like to double check with you how we want to apply some GDPR into > > my kafka topics. In concrete the "right to be forgotten", what forces us > to > > delete some data contained in the messages. So not deleting the message, > > but editing it. > > For doing that, my intention is to replicate the topic and apply a > > transformation over it. > > I think that frameworks like Kafka Streams or Apache Storm. > > > > Did anybody had to solve this problem? > > > > Thanks in advance. > > > -- > > Scott Reynolds > Principal Engineer > [image: twilio] <http://www.twilio.com/?utm_source=email_signature> > MOBILE (630) 254-2474 > EMAIL sreyno...@twilio.com >
