Yes, I think encryption at the message level is a workable solution, as long as you don't care about exposing the meta data that goes with it (e.g. topic names, kafka broker/zk server locations, etc.).
Jason On Tue, Apr 23, 2013 at 10:02 AM, Fergal Somers <fergal.som...@workday.com>wrote: > Hi > > We are planning to use Kafka, but like others on this list we have a need > to be able to secure communication. The approaches people have suggested on > this list are: > > - Encrypt the messages at the producer (e.g > > http://search-hadoop.com/m/1AfXKcZIk52/message+encryption&subj=Re+Secure+communication > ) > - Add SSL to Kafka protocol - > > http://mail-archives.apache.org/mod_mbox/kafka-users/201304.mbox/ajax/%3CCAA%2BBczQ_dMXUTNndSu4d%2B6aRo%3DSLiFa4iGMu_78OWKub_CTScw%40mail.gmail.com%3E > > Adding SSL support to Kafka, probably means adding SSLEngine support the > the nio socket handling ( > https://groups.google.com/forum/#!msg/kafka-dev/ZmJrB_plu1I/_9cmGlLCSVEJ). > I don't think there are any immediate plans to provide this, but it's > potentially something that Kafka would support in the future? > > In theory this is something we could look at, but we would need to go > further. We also need to separate producers from consumers. The aim would > be to ensure that a Kafka producer couldn't also act as a consumer. > Essentially producers can write to Kafka, but not read. > > From looking at the Kafka source, achieving producer/consumer separation > looks to me like it would be quite a change to the Kafka server (0.7). So > are there any plans in the (near) future in this area (producer / consumer > separation) ? > > Message encryption (at the application layer) would allow us to achieve > both aims of securing communication and separating consumers from > producers. Producers get the public cert (so they can encrypt messages as > they place them on the bus). Only consumers get the private cert - so only > they can decrypt messages consumed. This seems like something we can do > ourselves - I just wanted to sanity check the approach with this group. > > Cheers, > > Fergal. >
