just a clarification on the part "... we are not left with any choice but to use the file based config .." - That is "if" we want material rotations to be picked up automatically.
On Tue, Mar 22, 2022 at 1:57 PM Maulin Vasavada <maulin.vasav...@gmail.com> wrote: > Hi Dejan and Rajini > > I tend to agree with both the arguments when I read them in isolation :) > However, what I feel is - we have to also take an operator's view of this > setup. If we have encrypted keys/certs already it may be easier to fit with > file based material but if we don't have material encrypted already, it > would mean we have to additionally get into the encrypted material's > management to use the file based configuration OR adapt to in-line PEM > config. If the inline config is not able to take advantage of > auto-refreshes based on the file's modified timestamp, we are not left with > any choice but to use the file based config and it adds > operations/maintenance overhead. > > I can imagine a system where the unencrypted PEM keys are rotated fairly > frequently, like once every few hours or once a week and keeping them > protected with file permissions. In this case, it would be best if the > Kafka system is operator friendly. > > So all in all I am thinking if it is a norm to have PEM certificates > unencrypted outside of the Kafka world, we should try to stick to that > without adding new process overhead requirements for the Kafka users. > > Thanks > Maulin > > > On Mon, Mar 21, 2022 at 10:03 AM Rajini Sivaram <rajinisiva...@gmail.com> > wrote: > >> For the background on the current implementation: We use Java's keystore >> loading for JKS/PKCS12 keystore files and these files require passwords. >> We >> retained the same requirement for PEM files as well for consistency, even >> though it is not a language restriction anymore. When PEM keys are >> provided >> as actual config values rather than files, we do allow non-encrypted keys >> with the expectation that config externalization will be used to protect >> these, similar to SASL configs (e.g. private keys may be in vault or >> encrypted using a secure config provider). We could take the view that >> file >> protection should be up to the user as well and permit non-encrypted keys, >> making it simpler to adopt PEM. But since this is a deviation from the >> other SSL file configs, it will be good to know what others think. >> >> Regards, >> >> Rajini >> >> >> On Mon, Mar 21, 2022 at 2:16 PM Dejan Maric <chrom...@gmail.com> wrote: >> >> > Regarding this KIP, we've created an MR that removes the need for >> > specifying *keyPassword *when PEM certificates and private key are >> provided >> > as files: https://github.com/apache/kafka/pull/11916 >> > >> > We think that Kafka should not enforce the use of passwords on private >> > keys. It would be sufficient to *recommend* encrypted private keys. >> > >> > The reason we suggested the PR is because this issue is currently >> blocking >> > us from using PEM-formatted certificates issued by cert-manager >> > <https://github.com/cert-manager/cert-manager> with Kafka - since >> > cert-manager does not support encrypted private keys. >> > >> > We asked one of the main cert-manager maintainers about this, and got >> the >> > following comments: >> > >> > >There is a benefit to encrypting this stuff if the cert is being >> persisted >> > to disk and if the decryption key isn't written to the same disk, which >> is >> > probably a pretty common setup in a lot of older non-cloud systems. The >> > issue in a lot of more modern or cloud environments - especially in k8s >> - >> > is that certs are ideally never going to be written to disk and wherever >> > they are stored, they'd likely be stored next to their decryption keys. >> > >> > > >> > >So I guess everyone's right - in some environments encrypting the key >> is >> > probably worth it, and in some it's a false sense of security and wasted >> > CPU cycles. That to me means that being able to choose to not encrypt >> the >> > key is desirable. >> > >(This is assuming that the encryption is actually secure, which might >> not >> > be true for all methods of encrypting private keys) >> > >> > >> > We'd like to hear other opinions on this. >> > >> > Dejan >> > >> > >> > On 2020/08/03 11:28:42 Rajini Sivaram wrote: >> > > Hi all, >> > > >> > > I have submitted KIP-651 to support PEM format for SSL key and trust >> > stores: >> > > >> > > - >> > > >> > >> > >> https://cwiki.apache.org/confluence/display/KAFKA/KIP-651+-+Support+PEM+format+for+SSL+certificates+and+private+key >> > >> > > >> > > This enables better integration with other Kafka features like dynamic >> > > config update and password protection. >> > > >> > > Feedback and suggestions are welcome. >> > > >> > > Thank you... >> > > >> > > Regards, >> > > >> > > Rajini >> > > >> > >> >
