Hi We are planning to use Kafka, but like others on this list we have a need to be able to secure communication. The approaches people have suggested on this list are:
- Encrypt the messages at the producer (e.g http://search-hadoop.com/m/1AfXKcZIk52/message+encryption&subj=Re+Secure+communication ) - Add SSL to Kafka protocol - http://mail-archives.apache.org/mod_mbox/kafka-users/201304.mbox/ajax/%3CCAA%2BBczQ_dMXUTNndSu4d%2B6aRo%3DSLiFa4iGMu_78OWKub_CTScw%40mail.gmail.com%3E Adding SSL support to Kafka, probably means adding SSLEngine support the the nio socket handling ( https://groups.google.com/forum/#!msg/kafka-dev/ZmJrB_plu1I/_9cmGlLCSVEJ). I don't think there are any immediate plans to provide this, but it's potentially something that Kafka would support in the future? In theory this is something we could look at, but we would need to go further. We also need to separate producers from consumers. The aim would be to ensure that a Kafka producer couldn't also act as a consumer. Essentially producers can write to Kafka, but not read. >From looking at the Kafka source, achieving producer/consumer separation looks to me like it would be quite a change to the Kafka server (0.7). So are there any plans in the (near) future in this area (producer / consumer separation) ? Message encryption (at the application layer) would allow us to achieve both aims of securing communication and separating consumers from producers. Producers get the public cert (so they can encrypt messages as they place them on the bus). Only consumers get the private cert - so only they can decrypt messages consumed. This seems like something we can do ourselves - I just wanted to sanity check the approach with this group. Cheers, Fergal.
