I think the best way to implement this is via envelope encryption: your system manages a key encryption key (kek) which is used to encrypt data encryption keys (dek) per user/customer which are used to encrypt the user's/customer's data.
If the user/customer walks away, simply drop the dek. His data becomes undecryptable. You do have to implement reencryption in case keks or deks become compromised. If you run in the cloud, AWS and GCloud have basic services Key Management Services (KMS) to manage the KEKs esp. The access to it and versioning it. Their docs explain such a setup very well. https://cloud.google.com/kms/docs/envelope-encryption HTH -wim On Thu, Nov 23, 2017, 09:55 David Espinosa <espi...@gmail.com> wrote: > Hi Scott and thanks for your reply. > For what you say, I guess that when you are asked to delete some "data > user" (that's the "right to be forgotten" in GDPR), what you are really > doing is blocking the access to it. I had a similar approach, based on the > idea of Greg Young's solution of encrypting any private data and forgetting > the key when data has to deleted. > Sadly, our legal department after some checkins has conclude that this > approach is "to block" data but not deleting it, as a consequence it can > take us problems. If my guess about your solution is right, you could have > the same problems. > > Thanks > > 2017-11-22 19:59 GMT+01:00 Scott Reynolds <sreyno...@twilio.com.invalid>: > > > We are using Kafka Connect consumers that consume from the raw unredacted > > topic and apply transformations and produce to a redacted topic. Using > > kafka connect allows us to set it all up with an HTTP request and doesn't > > require additional infrastructure. > > > > Then we wrote a KafkaPrincipal builder to authenticate each consumer to > > their service names. KafkaPrincipal class is specified in the > > server.properties file on the brokers. To provide topic level access > > control we just configured SimpleAclAuthorizer. The net result is, some > > consumers can only read redacted topic and very few have consumers can > read > > unredacted. > > > > On Wed, Nov 22, 2017 at 10:47 AM David Espinosa <espi...@gmail.com> > wrote: > > > > > Hi all, > > > I would like to double check with you how we want to apply some GDPR > into > > > my kafka topics. In concrete the "right to be forgotten", what forces > us > > to > > > delete some data contained in the messages. So not deleting the > message, > > > but editing it. > > > For doing that, my intention is to replicate the topic and apply a > > > transformation over it. > > > I think that frameworks like Kafka Streams or Apache Storm. > > > > > > Did anybody had to solve this problem? > > > > > > Thanks in advance. > > > > > -- > > > > Scott Reynolds > > Principal Engineer > > [image: twilio] <http://www.twilio.com/?utm_source=email_signature> > > MOBILE (630) 254-2474 > > EMAIL sreyno...@twilio.com > > >
