Hi Mike, that sounds good! I've not yet received any other feedback, but worst case scenario is that just the two of us discuss this over a cup of coffee :)
I'll talk to the Summit organizers again about some sort of venue and get back to you once I know more. Maybe we can get a few more people to join the discussion once some more details are known. Other than that, I am also very happy to gather feedback on the mailing list from people who won't be able to make it to the Summit. So anybody who can come up with some thoughts or requirements around encryption functionality for Kafka, please don't hesitate to chime in! Best regards, Sönke On Wed, Oct 3, 2018 at 3:03 AM mikegray...@gmail.com <mikegray...@gmail.com> wrote: > Hi Sönke, > > I would be very interested in participating in this conversation. Very > interested in how TDE might work in Kafka! I’m coming with several > colleagues and will see if they’re interested in participating as well. > > Thanks, > Mike Grayson > > On 2018/10/02 11:19:36, Sönke Liebau <soenke.lie...@opencore.com.INVALID> > wrote: > > Hi all, > > > > I have created KIP-317 [1] a while ago, which outlines an implementation > > proposal to add transparent data encryption functionality to Kafka. The > KIP > > in its current form is somewhat rigid in its implementation, I will > rework > > this to become extensible over the next few days to allow for additional > > implementations. > > > > I have discussed the current method of providing keys with a colleague > and > > while we agree that this is a valid use case for some people, there are > > certainly a lot of other valid use cases out there as well. > > To ensure that the initial implementation provides the necessary > > flexibility I'd like some feedback from the community on what > requirements > > they would have around data encryption and key management. > > > > The following questions should serve as a starting point for the > > discussion, please feel free to address anything that comes to mind > which I > > have not mentioned here: > > > > - Should encryption be configurable rather on the client or on the broker > > and be pushed down to the client? > > - Where should keys be stored? > > - How much flexibility around keys is necessary - is there for example a > > use case that would decide on a per message basis which key to use? > > (imagine a topic containing top secret, secret and public data with three > > different keys) > > - Do we need functionality to prohibit publishing unencrypted messages to > > topics based on that topics setup? > > > > Of course the mailing list is the first place that discussions like these > > should take place, but sometimes I find a face to face discussion can be > > quite useful as well, especially when discussing non-trivial topics (like > > encryption). I have reached out to the organizers of the upcoming Kafka > > Summit in SF and there might be a chance for us to get a room with a > > whiteboard at some point (probably during lunch, when the room is > otherwise > > unused). Would people be interested in meeting up for 20 minutes to > discuss > > this in person? I'd be happy to provide a summary on the mailing list > > afterwards of course. > > > > Look forward to hearing from all of you! > > > > Best regards, > > Sönke > > > > [1] > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-317%3A+Add+transparent+data+encryption+functionality > > > -- Sönke Liebau Partner Tel. +49 179 7940878 OpenCore GmbH & Co. KG - Thomas-Mann-Straße 8 - 22880 Wedel - Germany
