Rajini I just tried this. It turns out that I can't import cert-file by itself in trust store until it is signed by a CA. Could be because of the format ? Any idea here ...
In the above steps, if I sign the server-cert-file and client-cert-file by a private CA then I can add them to trust store and key store. In this test, I did not add the CA cert in either keystone or trust store. Thanks for all your help. On Thu, May 18, 2017 at 8:26 AM, Rajini Sivaram <rajinisiva...@gmail.com> wrote: > Raghav, > > Perhaps what you want to do is: > > *You do (for the brokers):* > > Generate key-pair for broker: > > keytool -keystore kafka.server.keystore.jks -alias localhost -validity 365 > -genkey > > Export certificate to a file to send to your customers: > > keytool -exportcert -file server-cert-file -keystore > kafka.server.keystore.jks -alias localhost > > > And you send server-cert-file to your customers. > > Once you get your customer's client-cert-file, you do: > > keytool -importcert -file client-cert-file -keystore > kafka.server.truststore.jks -alias customerA > > If you are using SSL for inter-broker communication, your broker > certificate also needs to be in the server truststore: > > keytool -importcert -file server-cert-file -keystore > kafka.client.truststore.jks -alias broker > > > *Your customers do (for the clients):* > > Generate key-pair for client: > > keytool -keystore kafka.client.keystore.jks -alias localhost -validity 365 > -genkey > > Export certificate to a file to send to to you: > > keytool -exportcert -file client-cert-file -keystore > kafka.client.keystore.jks -alias localhost > > > Your customers send you their client-cert-file > > Your customers create their truststore using the broker certificate > server-cert-file that you send to them: > > keytool -importcert -file server-cert-file -keystore > kafka.client.truststore.jks -alias broker > > > > You then configure your brokers with (kafka.server.keystore.jks, ka > fka.server.truststore.jks).Your customers configure their clients with ( > kafka.client.keystore.jks, kafka.client.truststore.jks). > > > Hope that helps. > > Regards, > > Rajini > > > > On Thu, May 18, 2017 at 10:33 AM, Raghav <raghavas...@gmail.com> wrote: > >> Rajini, >> >> Sure, will submit a PR shortly. >> >> Your answer is very helpful, but I think I did not put the question >> correctly. Pardon my ignore but I am still trying to get my ways around >> Kafka security. >> >> I was trying to understand, can we (Kafka Broker) just add the >> certificate (unsigned or signed) from customer to our trust store without >> adding the CA cert to trust store... could that work ? >> >> 1. Let's say Kafka broker (there is only 1 for simplicity) generates a >> keystore and generates a key using the command below >> >> keytool -keystore kafka.server.keystore.jks -alias localhost -validity *365* >> -genkey >> >> keytool -keystore kafka.server.keystore.jks -alias localhost -certreq -file >> server-cert-file >> >> 2. Similarly, Kafka Client (Producer) does the same >> >> keytool -keystore kafka.client.keystore.jks -alias localhost -validity *365* >> -genkey >> >> keytool -keystore kafka.client.keystore.jks -alias localhost -certreq -file >> client-cert-file >> >> >> 3. Now, we add *client-cert-file* into the trust store of server, and >> *server-cert-file* into the trust store of client. Given that each trust >> store has other party's certificate in their trust store, does CA >> certificate come into the picture ? >> >> On Thu, May 18, 2017 at 6:26 AM, Rajini Sivaram <rajinisiva...@gmail.com> >> wrote: >> >>> Raghav, >>> >>> Yes, you can create a truststore with your customers' certificates and >>> vice-versa. It will be best to give your CA certificate to your customers >>> and get the CA certificate from each of your customers and add them to your >>> broker's truststore. You can both then create additional certificates if >>> you need without any changes to your truststore as long as the CA >>> certificates are valid. Unlike certificates signed by a trusted authority, >>> you will need to add the CAs of every customer to your truststore. Kafka >>> brokers don't reload certificates, so if you wanted to add another >>> customer's certificate to your truststore, you will need to restart your >>> broker. >>> >>> Would you like to submit a PR with the information that is missing in >>> the Apache Kafka documentation that you think may be useful? >>> >>> Regards, >>> >>> Rajini >>> >>> On Wed, May 17, 2017 at 6:21 PM, Raghav <raghavas...@gmail.com> wrote: >>> >>>> Another quick question: >>>> >>>> Say we chose to add our customer's certificates directly to our brokers >>>> trust store and vice verse, could that work ? There is no documentation on >>>> Kafka or Confluent site for this ? >>>> >>>> Thanks. >>>> >>>> >>>> On Wed, May 17, 2017 at 1:56 PM, Rajini Sivaram < >>>> rajinisiva...@gmail.com> wrote: >>>> >>>>> Raghav, >>>>> >>>>> 1. Yes, your customers can use certificates signed by a trusted >>>>> authority. You can simply omit the truststore configuration for your >>>>> broker >>>>> in server.properties, and Kafka would use the default, which will trust >>>>> the >>>>> client certificates. If your brokers are using SSL for inter-broker >>>>> communication and you are still using your private CA for broker's >>>>> keystore, then you will need two separate endpoints in your listener >>>>> configuration, one for your customer's clients and another for >>>>> inter-broker >>>>> communication so that you can specify a truststore with your private >>>>> ca-cert for your broker connections. >>>>> >>>>> 2. Yes, all the commands can specify password on the command line, so >>>>> you should be able to generate all the stores using a script without any >>>>> interactions. >>>>> >>>>> Regards, >>>>> >>>>> Rajini >>>>> >>>>> >>>>> On Wed, May 17, 2017 at 2:49 PM, Raghav <raghavas...@gmail.com> wrote: >>>>> >>>>>> One follow up questions Rajini: >>>>>> >>>>>> 1. Can we use some other mechanism like have our customer's use a >>>>>> well known CA which JKS understands, and in that case we don't have to >>>>>> ask >>>>>> our customers to do this certificate-in and certificate-out thing ? I am >>>>>> just trying to understand if we can make our customer's workflow easier. >>>>>> Anything else that you can suggest here.... >>>>>> >>>>>> 2. Can we automate the key gen steps mentioned on apache website and >>>>>> adding to keystone and trust store so that we don't have to manually >>>>>> supply >>>>>> password ? Currently, everytime I tried to do steps mentioned in >>>>>> https://kafka.apache.org/documentation/#security I have to manually >>>>>> give password. It would be great if we can automate this process either >>>>>> through script or Java code. Any suggestions ... >>>>>> >>>>>> >>>>>> Many thanks. >>>>>> >>>>>> On Tue, May 16, 2017 at 10:58 AM, Raghav <raghavas...@gmail.com> >>>>>> wrote: >>>>>> >>>>>>> Many thanks, Rajini. >>>>>>> >>>>>>> On Tue, May 16, 2017 at 8:43 AM, Rajini Sivaram < >>>>>>> rajinisiva...@gmail.com> wrote: >>>>>>> >>>>>>>> Hi Raghav, >>>>>>>> >>>>>>>> If your Kafka broker is configured with *ssl.client.auth=required,* >>>>>>>> your >>>>>>>> customer's clients need to provide a keystore. In any case, they need a >>>>>>>> truststore since your broker is using SSL. For the truststore, you can >>>>>>>> given them ca-cert, as you mentioned. Client keystore contains a >>>>>>>> certificate and a private key. >>>>>>>> >>>>>>>> In the round-trip you described, customers generate the keys and >>>>>>>> give you the certificate signing request, keeping their private key >>>>>>>> private. You then send them back a signed certificate that goes into >>>>>>>> their >>>>>>>> keystore. This is the standard way of signing and is secure. >>>>>>>> >>>>>>>> In the single step scenario that you described, you generate the >>>>>>>> customer's key-pair consisting of certificate and private key. You then >>>>>>>> need to send them both the signed certificate and the private key. >>>>>>>> This is >>>>>>>> less secure. Unlike the round-trip, you now have the private key of the >>>>>>>> customer. >>>>>>>> >>>>>>>> Regards, >>>>>>>> >>>>>>>> Rajini >>>>>>>> >>>>>>>> >>>>>>>> On Tue, May 16, 2017 at 10:47 AM, Raghav <raghavas...@gmail.com> >>>>>>>> wrote: >>>>>>>> >>>>>>>>> Hi Rajini >>>>>>>>> >>>>>>>>> This was very helpful. I have another questions on similar lines. >>>>>>>>> >>>>>>>>> We host Kafka Broker, and we also have our own private CA. We want >>>>>>>>> our customers to setup their Kafka Clients (Producer and Consumer) >>>>>>>>> using >>>>>>>>> SSL using *ssl.client.auth=required*. >>>>>>>>> >>>>>>>>> Is there a way, we can generate certificate for our clients, sign >>>>>>>>> it using our private CA, and then hand over our customers these two >>>>>>>>> certificates (1. ca-cert 2. cert-signed), which if they add to their >>>>>>>>> keystroke and truststore, they can send message to our Kafka brokers >>>>>>>>> while >>>>>>>>> keeping *ssl.client.auth=required*. >>>>>>>>> >>>>>>>>> We are looking to minimize our customer's pre-setup steps. For >>>>>>>>> example in normal scenario, customers will need to generate >>>>>>>>> certificate, >>>>>>>>> and hand over their certificate request to our private CA, which we >>>>>>>>> then >>>>>>>>> sign it, and send them signed certificate and private CA's >>>>>>>>> certificate. So >>>>>>>>> there is one round trip. Just wondering if we can reduce this 2 step >>>>>>>>> into 1 >>>>>>>>> step. >>>>>>>>> >>>>>>>>> Thanks. >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> On Fri, May 12, 2017 at 8:53 AM, Rajini Sivaram < >>>>>>>>> rajinisiva...@gmail.com> wrote: >>>>>>>>> >>>>>>>>>> Raqhav, >>>>>>>>>> >>>>>>>>>> 1. Clients need a keystore if you are using TLS client >>>>>>>>>> authentication. To >>>>>>>>>> enable client authentication, you need to configure >>>>>>>>>> ssl.client.auth in >>>>>>>>>> server.properties. This can be set to required|requested|none. If >>>>>>>>>> you don't >>>>>>>>>> enable client authentication, any client will be able to connect >>>>>>>>>> to your >>>>>>>>>> broker. You could alternatively use SASL for client >>>>>>>>>> authentication. >>>>>>>>>> . >>>>>>>>>> 2. Client keystore is mandatory if ssl.client.auth=required, >>>>>>>>>> optional for >>>>>>>>>> requested and not used for none. The truststore configured on the >>>>>>>>>> client is >>>>>>>>>> used to authenticate the server. So you have to provide it unless >>>>>>>>>> your >>>>>>>>>> broker is using certificates signed by a trusted authority. >>>>>>>>>> >>>>>>>>>> Hope that helps. >>>>>>>>>> >>>>>>>>>> Rajini >>>>>>>>>> >>>>>>>>>> On Fri, May 12, 2017 at 11:35 AM, Raghav <raghavas...@gmail.com> >>>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>> > Hi >>>>>>>>>> > >>>>>>>>>> > I read the documentation here: >>>>>>>>>> > https://kafka.apache.org/documentation/#security_ssl >>>>>>>>>> > >>>>>>>>>> > I have few questions about trust store and keystore based on >>>>>>>>>> this scenario: >>>>>>>>>> > >>>>>>>>>> > We have 5 Kafka Brokers in our cluster. We want our clients to >>>>>>>>>> write to our >>>>>>>>>> > Kafka brokers in a secure way. Suppose, we also host a private >>>>>>>>>> CA as >>>>>>>>>> > mentioned in the documentation above, and provide our clients >>>>>>>>>> the *ca-cert* >>>>>>>>>> > file, which they add it to their trust store. >>>>>>>>>> > >>>>>>>>>> > 1. Do we require our clients to generate their certificate and >>>>>>>>>> have it >>>>>>>>>> > signed by our private CA, and add it to their keystore? >>>>>>>>>> > >>>>>>>>>> > 2. When is keystore used by clients, and when is truststore >>>>>>>>>> used by clients >>>>>>>>>> > ? >>>>>>>>>> > >>>>>>>>>> > >>>>>>>>>> > Thanks. >>>>>>>>>> > >>>>>>>>>> > -- >>>>>>>>>> > R >>>>>>>>>> > >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> -- >>>>>>>>> Raghav >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Raghav >>>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> Raghav >>>>>> >>>>> >>>>> >>>> >>>> >>>> -- >>>> Raghav >>>> >>> >>> >> >> >> -- >> Raghav >> > > -- Raghav
