>The client-broker protocol would have to be augmented to carry the
>encrypted encryption key, plus logic to handle redistribution to existing
>clients due to key rotation.
This is a good point. HDFS has the encryption zone concept, which could be
akin to a topic. The keys in HDFS are per file leve
Doing encryption on the client has the following benefits (most of them
already mentioned in the thread):
* brokers don't have additional CPU load
* brokers never see the data in unencrypted form (Kafka admins cannot snoop)
* secure multi-tenancy (keys are 100% on the client space)
* no need to se
If I understand you correctly, you are saying that the kerberos keytab that
the broker uses to authenticate with the KMS will be somewhere on the
broker node and can be used by a malicious admin.
I agree this is a valid concern.
I am not opposed to client-side encryption, I am more concerned that
>Any reason you think its better to let the clients handle it?
Gwen, I agree with Todd, depending on the goal, the requirements might
vary. If the goal is that someone stills the disk, then they should be
able to access the data, then encrypting at Broker is enough. However, if
the requirement is t
I agree with Todd, the major concern I have is still the complexity on
broker which can kill the performance - which a key advantage of Kafka. I
think there are two separate issues here:
1. Key management
2. the actual encryption/decryption work.
Personally I think it might be OK to have [1] suppo
1 - Yes, authorization combined with encryption does get us most of the way
there. However, depending on the auditor it might not be good enough. The
problem is that if you are encrypting at the broker, then by definition
anyone who has access to the broker (i.e. operations staff) have access to
th
Good points :)
1) Kafka already (pending commit) has an authorization layer, so
theoretically we are good for SOX, HIPAA, PCI, etc. Transparent broker
encryption will support PCI
never-let-unencrypted-card-number-hit-disk.
2) Agree on Key Management being complete PITA. It may better to
centraliz
It does limit it to clients that have an implementation for encryption,
however encryption on the client side is better from an auditing point of
view (whether that is SOX, HIPAA, PCI, or something else). Most of those
types of standards are based around allowing visibility of data to just the
peop
I've seen interest in HDFS-like "encryption zones" in Kafka.
This has the advantage of magically encrypting data at rest regardless
of which client is used as a producer.
Adding it on the client side limits the feature to the java client.
Gwen
On Fri, Jul 31, 2015 at 1:20 PM, eugene miretsky
wr
I think that Hadoop and Cassandra do [1] (Transparent Encryption)
We're doing [2] (on a side note, for [2] you still need authentication on
the producer side - you don't want an unauthorized user writing garbage).
Right now we have the 'user' doing the encryption and submitting raw bytes
to the p
I think the goal here is to make the actual message stored on broker to be
encrypted, because after we have SSL, the transmission would be encrypted.
In general there might be tow approaches:
1. Broker do the encryption/decryption
2. Client do the encryption/decryption
>From performance point of
11 matches
Mail list logo
