Raghav, Perhaps what you want to do is:
*You do (for the brokers):* Generate key-pair for broker: keytool -keystore kafka.server.keystore.jks -alias localhost -validity 365 -genkey Export certificate to a file to send to your customers: keytool -exportcert -file server-cert-file -keystore kafka.server.keystore.jks -alias localhost And you send server-cert-file to your customers. Once you get your customer's client-cert-file, you do: keytool -importcert -file client-cert-file -keystore kafka.server.truststore.jks -alias customerA If you are using SSL for inter-broker communication, your broker certificate also needs to be in the server truststore: keytool -importcert -file server-cert-file -keystore kafka.client.truststore.jks -alias broker *Your customers do (for the clients):* Generate key-pair for client: keytool -keystore kafka.client.keystore.jks -alias localhost -validity 365 -genkey Export certificate to a file to send to to you: keytool -exportcert -file client-cert-file -keystore kafka.client.keystore.jks -alias localhost Your customers send you their client-cert-file Your customers create their truststore using the broker certificate server-cert-file that you send to them: keytool -importcert -file server-cert-file -keystore kafka.client.truststore.jks -alias broker You then configure your brokers with (kafka.server.keystore.jks, kafka.server.truststore.jks).Your customers configure their clients with ( kafka.client.keystore.jks, kafka.client.truststore.jks). Hope that helps. Regards, Rajini On Thu, May 18, 2017 at 10:33 AM, Raghav <raghavas...@gmail.com> wrote: > Rajini, > > Sure, will submit a PR shortly. > > Your answer is very helpful, but I think I did not put the question > correctly. Pardon my ignore but I am still trying to get my ways around > Kafka security. > > I was trying to understand, can we (Kafka Broker) just add the certificate > (unsigned or signed) from customer to our trust store without adding the CA > cert to trust store... could that work ? > > 1. Let's say Kafka broker (there is only 1 for simplicity) generates a > keystore and generates a key using the command below > > keytool -keystore kafka.server.keystore.jks -alias localhost -validity *365* > -genkey > > keytool -keystore kafka.server.keystore.jks -alias localhost -certreq -file > server-cert-file > > 2. Similarly, Kafka Client (Producer) does the same > > keytool -keystore kafka.client.keystore.jks -alias localhost -validity *365* > -genkey > > keytool -keystore kafka.client.keystore.jks -alias localhost -certreq -file > client-cert-file > > > 3. Now, we add *client-cert-file* into the trust store of server, and > *server-cert-file* into the trust store of client. Given that each trust > store has other party's certificate in their trust store, does CA > certificate come into the picture ? > > On Thu, May 18, 2017 at 6:26 AM, Rajini Sivaram <rajinisiva...@gmail.com> > wrote: > >> Raghav, >> >> Yes, you can create a truststore with your customers' certificates and >> vice-versa. It will be best to give your CA certificate to your customers >> and get the CA certificate from each of your customers and add them to your >> broker's truststore. You can both then create additional certificates if >> you need without any changes to your truststore as long as the CA >> certificates are valid. Unlike certificates signed by a trusted authority, >> you will need to add the CAs of every customer to your truststore. Kafka >> brokers don't reload certificates, so if you wanted to add another >> customer's certificate to your truststore, you will need to restart your >> broker. >> >> Would you like to submit a PR with the information that is missing in the >> Apache Kafka documentation that you think may be useful? >> >> Regards, >> >> Rajini >> >> On Wed, May 17, 2017 at 6:21 PM, Raghav <raghavas...@gmail.com> wrote: >> >>> Another quick question: >>> >>> Say we chose to add our customer's certificates directly to our brokers >>> trust store and vice verse, could that work ? There is no documentation on >>> Kafka or Confluent site for this ? >>> >>> Thanks. >>> >>> >>> On Wed, May 17, 2017 at 1:56 PM, Rajini Sivaram <rajinisiva...@gmail.com >>> > wrote: >>> >>>> Raghav, >>>> >>>> 1. Yes, your customers can use certificates signed by a trusted >>>> authority. You can simply omit the truststore configuration for your broker >>>> in server.properties, and Kafka would use the default, which will trust the >>>> client certificates. If your brokers are using SSL for inter-broker >>>> communication and you are still using your private CA for broker's >>>> keystore, then you will need two separate endpoints in your listener >>>> configuration, one for your customer's clients and another for inter-broker >>>> communication so that you can specify a truststore with your private >>>> ca-cert for your broker connections. >>>> >>>> 2. Yes, all the commands can specify password on the command line, so >>>> you should be able to generate all the stores using a script without any >>>> interactions. >>>> >>>> Regards, >>>> >>>> Rajini >>>> >>>> >>>> On Wed, May 17, 2017 at 2:49 PM, Raghav <raghavas...@gmail.com> wrote: >>>> >>>>> One follow up questions Rajini: >>>>> >>>>> 1. Can we use some other mechanism like have our customer's use a well >>>>> known CA which JKS understands, and in that case we don't have to ask our >>>>> customers to do this certificate-in and certificate-out thing ? I am just >>>>> trying to understand if we can make our customer's workflow easier. >>>>> Anything else that you can suggest here.... >>>>> >>>>> 2. Can we automate the key gen steps mentioned on apache website and >>>>> adding to keystone and trust store so that we don't have to manually >>>>> supply >>>>> password ? Currently, everytime I tried to do steps mentioned in >>>>> https://kafka.apache.org/documentation/#security I have to manually >>>>> give password. It would be great if we can automate this process either >>>>> through script or Java code. Any suggestions ... >>>>> >>>>> >>>>> Many thanks. >>>>> >>>>> On Tue, May 16, 2017 at 10:58 AM, Raghav <raghavas...@gmail.com> >>>>> wrote: >>>>> >>>>>> Many thanks, Rajini. >>>>>> >>>>>> On Tue, May 16, 2017 at 8:43 AM, Rajini Sivaram < >>>>>> rajinisiva...@gmail.com> wrote: >>>>>> >>>>>>> Hi Raghav, >>>>>>> >>>>>>> If your Kafka broker is configured with *ssl.client.auth=required,* your >>>>>>> customer's clients need to provide a keystore. In any case, they need a >>>>>>> truststore since your broker is using SSL. For the truststore, you can >>>>>>> given them ca-cert, as you mentioned. Client keystore contains a >>>>>>> certificate and a private key. >>>>>>> >>>>>>> In the round-trip you described, customers generate the keys and >>>>>>> give you the certificate signing request, keeping their private key >>>>>>> private. You then send them back a signed certificate that goes into >>>>>>> their >>>>>>> keystore. This is the standard way of signing and is secure. >>>>>>> >>>>>>> In the single step scenario that you described, you generate the >>>>>>> customer's key-pair consisting of certificate and private key. You then >>>>>>> need to send them both the signed certificate and the private key. This >>>>>>> is >>>>>>> less secure. Unlike the round-trip, you now have the private key of the >>>>>>> customer. >>>>>>> >>>>>>> Regards, >>>>>>> >>>>>>> Rajini >>>>>>> >>>>>>> >>>>>>> On Tue, May 16, 2017 at 10:47 AM, Raghav <raghavas...@gmail.com> >>>>>>> wrote: >>>>>>> >>>>>>>> Hi Rajini >>>>>>>> >>>>>>>> This was very helpful. I have another questions on similar lines. >>>>>>>> >>>>>>>> We host Kafka Broker, and we also have our own private CA. We want >>>>>>>> our customers to setup their Kafka Clients (Producer and Consumer) >>>>>>>> using >>>>>>>> SSL using *ssl.client.auth=required*. >>>>>>>> >>>>>>>> Is there a way, we can generate certificate for our clients, sign >>>>>>>> it using our private CA, and then hand over our customers these two >>>>>>>> certificates (1. ca-cert 2. cert-signed), which if they add to their >>>>>>>> keystroke and truststore, they can send message to our Kafka brokers >>>>>>>> while >>>>>>>> keeping *ssl.client.auth=required*. >>>>>>>> >>>>>>>> We are looking to minimize our customer's pre-setup steps. For >>>>>>>> example in normal scenario, customers will need to generate >>>>>>>> certificate, >>>>>>>> and hand over their certificate request to our private CA, which we >>>>>>>> then >>>>>>>> sign it, and send them signed certificate and private CA's >>>>>>>> certificate. So >>>>>>>> there is one round trip. Just wondering if we can reduce this 2 step >>>>>>>> into 1 >>>>>>>> step. >>>>>>>> >>>>>>>> Thanks. >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> On Fri, May 12, 2017 at 8:53 AM, Rajini Sivaram < >>>>>>>> rajinisiva...@gmail.com> wrote: >>>>>>>> >>>>>>>>> Raqhav, >>>>>>>>> >>>>>>>>> 1. Clients need a keystore if you are using TLS client >>>>>>>>> authentication. To >>>>>>>>> enable client authentication, you need to configure >>>>>>>>> ssl.client.auth in >>>>>>>>> server.properties. This can be set to required|requested|none. If >>>>>>>>> you don't >>>>>>>>> enable client authentication, any client will be able to connect >>>>>>>>> to your >>>>>>>>> broker. You could alternatively use SASL for client authentication. >>>>>>>>> . >>>>>>>>> 2. Client keystore is mandatory if ssl.client.auth=required, >>>>>>>>> optional for >>>>>>>>> requested and not used for none. The truststore configured on the >>>>>>>>> client is >>>>>>>>> used to authenticate the server. So you have to provide it unless >>>>>>>>> your >>>>>>>>> broker is using certificates signed by a trusted authority. >>>>>>>>> >>>>>>>>> Hope that helps. >>>>>>>>> >>>>>>>>> Rajini >>>>>>>>> >>>>>>>>> On Fri, May 12, 2017 at 11:35 AM, Raghav <raghavas...@gmail.com> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>> > Hi >>>>>>>>> > >>>>>>>>> > I read the documentation here: >>>>>>>>> > https://kafka.apache.org/documentation/#security_ssl >>>>>>>>> > >>>>>>>>> > I have few questions about trust store and keystore based on >>>>>>>>> this scenario: >>>>>>>>> > >>>>>>>>> > We have 5 Kafka Brokers in our cluster. We want our clients to >>>>>>>>> write to our >>>>>>>>> > Kafka brokers in a secure way. Suppose, we also host a private >>>>>>>>> CA as >>>>>>>>> > mentioned in the documentation above, and provide our clients >>>>>>>>> the *ca-cert* >>>>>>>>> > file, which they add it to their trust store. >>>>>>>>> > >>>>>>>>> > 1. Do we require our clients to generate their certificate and >>>>>>>>> have it >>>>>>>>> > signed by our private CA, and add it to their keystore? >>>>>>>>> > >>>>>>>>> > 2. When is keystore used by clients, and when is truststore used >>>>>>>>> by clients >>>>>>>>> > ? >>>>>>>>> > >>>>>>>>> > >>>>>>>>> > Thanks. >>>>>>>>> > >>>>>>>>> > -- >>>>>>>>> > R >>>>>>>>> > >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Raghav >>>>>>>> >>>>>>> >>>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Raghav >>>>>> >>>>> >>>>> >>>>> >>>>> -- >>>>> Raghav >>>>> >>>> >>>> >>> >>> >>> -- >>> Raghav >>> >> >> > > > -- > Raghav >
