Raghav, If you send me the full command sequence, I can take a look. Also, which JRE are you using?
Regards, Rajini On Thu, May 18, 2017 at 12:19 PM, Raghav <raghavas...@gmail.com> wrote: > Rajini > > I just tried this. It turns out that I can't import cert-file by itself in > trust store until it is signed by a CA. Could be because of the format ? > Any idea here ... > > In the above steps, if I sign the server-cert-file and client-cert-file by > a private CA then I can add them to trust store and key store. In this > test, I did not add the CA cert in either keystone or trust store. > > Thanks for all your help. > > > > > On Thu, May 18, 2017 at 8:26 AM, Rajini Sivaram <rajinisiva...@gmail.com> > wrote: > >> Raghav, >> >> Perhaps what you want to do is: >> >> *You do (for the brokers):* >> >> Generate key-pair for broker: >> >> keytool -keystore kafka.server.keystore.jks -alias localhost -validity >> 365 -genkey >> >> Export certificate to a file to send to your customers: >> >> keytool -exportcert -file server-cert-file -keystore >> kafka.server.keystore.jks -alias localhost >> >> >> And you send server-cert-file to your customers. >> >> Once you get your customer's client-cert-file, you do: >> >> keytool -importcert -file client-cert-file -keystore >> kafka.server.truststore.jks -alias customerA >> >> If you are using SSL for inter-broker communication, your broker >> certificate also needs to be in the server truststore: >> >> keytool -importcert -file server-cert-file -keystore >> kafka.client.truststore.jks -alias broker >> >> >> *Your customers do (for the clients):* >> >> Generate key-pair for client: >> >> keytool -keystore kafka.client.keystore.jks -alias localhost -validity >> 365 -genkey >> >> Export certificate to a file to send to to you: >> >> keytool -exportcert -file client-cert-file -keystore >> kafka.client.keystore.jks -alias localhost >> >> >> Your customers send you their client-cert-file >> >> Your customers create their truststore using the broker certificate >> server-cert-file that you send to them: >> >> keytool -importcert -file server-cert-file -keystore >> kafka.client.truststore.jks -alias broker >> >> >> >> You then configure your brokers with (kafka.server.keystore.jks, ka >> fka.server.truststore.jks).Your customers configure their clients with ( >> kafka.client.keystore.jks, kafka.client.truststore.jks). >> >> >> Hope that helps. >> >> Regards, >> >> Rajini >> >> >> >> On Thu, May 18, 2017 at 10:33 AM, Raghav <raghavas...@gmail.com> wrote: >> >>> Rajini, >>> >>> Sure, will submit a PR shortly. >>> >>> Your answer is very helpful, but I think I did not put the question >>> correctly. Pardon my ignore but I am still trying to get my ways around >>> Kafka security. >>> >>> I was trying to understand, can we (Kafka Broker) just add the >>> certificate (unsigned or signed) from customer to our trust store without >>> adding the CA cert to trust store... could that work ? >>> >>> 1. Let's say Kafka broker (there is only 1 for simplicity) generates a >>> keystore and generates a key using the command below >>> >>> keytool -keystore kafka.server.keystore.jks -alias localhost -validity >>> *365* -genkey >>> >>> keytool -keystore kafka.server.keystore.jks -alias localhost -certreq -file >>> server-cert-file >>> >>> 2. Similarly, Kafka Client (Producer) does the same >>> >>> keytool -keystore kafka.client.keystore.jks -alias localhost -validity >>> *365* -genkey >>> >>> keytool -keystore kafka.client.keystore.jks -alias localhost -certreq -file >>> client-cert-file >>> >>> >>> 3. Now, we add *client-cert-file* into the trust store of server, and >>> *server-cert-file* into the trust store of client. Given that each >>> trust store has other party's certificate in their trust store, does CA >>> certificate come into the picture ? >>> >>> On Thu, May 18, 2017 at 6:26 AM, Rajini Sivaram <rajinisiva...@gmail.com >>> > wrote: >>> >>>> Raghav, >>>> >>>> Yes, you can create a truststore with your customers' certificates and >>>> vice-versa. It will be best to give your CA certificate to your customers >>>> and get the CA certificate from each of your customers and add them to your >>>> broker's truststore. You can both then create additional certificates if >>>> you need without any changes to your truststore as long as the CA >>>> certificates are valid. Unlike certificates signed by a trusted authority, >>>> you will need to add the CAs of every customer to your truststore. Kafka >>>> brokers don't reload certificates, so if you wanted to add another >>>> customer's certificate to your truststore, you will need to restart your >>>> broker. >>>> >>>> Would you like to submit a PR with the information that is missing in >>>> the Apache Kafka documentation that you think may be useful? >>>> >>>> Regards, >>>> >>>> Rajini >>>> >>>> On Wed, May 17, 2017 at 6:21 PM, Raghav <raghavas...@gmail.com> wrote: >>>> >>>>> Another quick question: >>>>> >>>>> Say we chose to add our customer's certificates directly to our >>>>> brokers trust store and vice verse, could that work ? There is no >>>>> documentation on Kafka or Confluent site for this ? >>>>> >>>>> Thanks. >>>>> >>>>> >>>>> On Wed, May 17, 2017 at 1:56 PM, Rajini Sivaram < >>>>> rajinisiva...@gmail.com> wrote: >>>>> >>>>>> Raghav, >>>>>> >>>>>> 1. Yes, your customers can use certificates signed by a trusted >>>>>> authority. You can simply omit the truststore configuration for your >>>>>> broker >>>>>> in server.properties, and Kafka would use the default, which will trust >>>>>> the >>>>>> client certificates. If your brokers are using SSL for inter-broker >>>>>> communication and you are still using your private CA for broker's >>>>>> keystore, then you will need two separate endpoints in your listener >>>>>> configuration, one for your customer's clients and another for >>>>>> inter-broker >>>>>> communication so that you can specify a truststore with your private >>>>>> ca-cert for your broker connections. >>>>>> >>>>>> 2. Yes, all the commands can specify password on the command line, so >>>>>> you should be able to generate all the stores using a script without any >>>>>> interactions. >>>>>> >>>>>> Regards, >>>>>> >>>>>> Rajini >>>>>> >>>>>> >>>>>> On Wed, May 17, 2017 at 2:49 PM, Raghav <raghavas...@gmail.com> >>>>>> wrote: >>>>>> >>>>>>> One follow up questions Rajini: >>>>>>> >>>>>>> 1. Can we use some other mechanism like have our customer's use a >>>>>>> well known CA which JKS understands, and in that case we don't have to >>>>>>> ask >>>>>>> our customers to do this certificate-in and certificate-out thing ? I am >>>>>>> just trying to understand if we can make our customer's workflow easier. >>>>>>> Anything else that you can suggest here.... >>>>>>> >>>>>>> 2. Can we automate the key gen steps mentioned on apache website and >>>>>>> adding to keystone and trust store so that we don't have to manually >>>>>>> supply >>>>>>> password ? Currently, everytime I tried to do steps mentioned in >>>>>>> https://kafka.apache.org/documentation/#security I have to manually >>>>>>> give password. It would be great if we can automate this process either >>>>>>> through script or Java code. Any suggestions ... >>>>>>> >>>>>>> >>>>>>> Many thanks. >>>>>>> >>>>>>> On Tue, May 16, 2017 at 10:58 AM, Raghav <raghavas...@gmail.com> >>>>>>> wrote: >>>>>>> >>>>>>>> Many thanks, Rajini. >>>>>>>> >>>>>>>> On Tue, May 16, 2017 at 8:43 AM, Rajini Sivaram < >>>>>>>> rajinisiva...@gmail.com> wrote: >>>>>>>> >>>>>>>>> Hi Raghav, >>>>>>>>> >>>>>>>>> If your Kafka broker is configured with >>>>>>>>> *ssl.client.auth=required,* your customer's clients need to >>>>>>>>> provide a keystore. In any case, they need a truststore since your >>>>>>>>> broker >>>>>>>>> is using SSL. For the truststore, you can given them ca-cert, as >>>>>>>>> you mentioned. Client keystore contains a certificate and a private >>>>>>>>> key. >>>>>>>>> >>>>>>>>> In the round-trip you described, customers generate the keys and >>>>>>>>> give you the certificate signing request, keeping their private key >>>>>>>>> private. You then send them back a signed certificate that goes into >>>>>>>>> their >>>>>>>>> keystore. This is the standard way of signing and is secure. >>>>>>>>> >>>>>>>>> In the single step scenario that you described, you generate the >>>>>>>>> customer's key-pair consisting of certificate and private key. You >>>>>>>>> then >>>>>>>>> need to send them both the signed certificate and the private key. >>>>>>>>> This is >>>>>>>>> less secure. Unlike the round-trip, you now have the private key of >>>>>>>>> the >>>>>>>>> customer. >>>>>>>>> >>>>>>>>> Regards, >>>>>>>>> >>>>>>>>> Rajini >>>>>>>>> >>>>>>>>> >>>>>>>>> On Tue, May 16, 2017 at 10:47 AM, Raghav <raghavas...@gmail.com> >>>>>>>>> wrote: >>>>>>>>> >>>>>>>>>> Hi Rajini >>>>>>>>>> >>>>>>>>>> This was very helpful. I have another questions on similar lines. >>>>>>>>>> >>>>>>>>>> We host Kafka Broker, and we also have our own private CA. We >>>>>>>>>> want our customers to setup their Kafka Clients (Producer and >>>>>>>>>> Consumer) >>>>>>>>>> using SSL using *ssl.client.auth=required*. >>>>>>>>>> >>>>>>>>>> Is there a way, we can generate certificate for our clients, sign >>>>>>>>>> it using our private CA, and then hand over our customers these two >>>>>>>>>> certificates (1. ca-cert 2. cert-signed), which if they add to their >>>>>>>>>> keystroke and truststore, they can send message to our Kafka brokers >>>>>>>>>> while >>>>>>>>>> keeping *ssl.client.auth=required*. >>>>>>>>>> >>>>>>>>>> We are looking to minimize our customer's pre-setup steps. For >>>>>>>>>> example in normal scenario, customers will need to generate >>>>>>>>>> certificate, >>>>>>>>>> and hand over their certificate request to our private CA, which we >>>>>>>>>> then >>>>>>>>>> sign it, and send them signed certificate and private CA's >>>>>>>>>> certificate. So >>>>>>>>>> there is one round trip. Just wondering if we can reduce this 2 step >>>>>>>>>> into 1 >>>>>>>>>> step. >>>>>>>>>> >>>>>>>>>> Thanks. >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> On Fri, May 12, 2017 at 8:53 AM, Rajini Sivaram < >>>>>>>>>> rajinisiva...@gmail.com> wrote: >>>>>>>>>> >>>>>>>>>>> Raqhav, >>>>>>>>>>> >>>>>>>>>>> 1. Clients need a keystore if you are using TLS client >>>>>>>>>>> authentication. To >>>>>>>>>>> enable client authentication, you need to configure >>>>>>>>>>> ssl.client.auth in >>>>>>>>>>> server.properties. This can be set to required|requested|none. >>>>>>>>>>> If you don't >>>>>>>>>>> enable client authentication, any client will be able to connect >>>>>>>>>>> to your >>>>>>>>>>> broker. You could alternatively use SASL for client >>>>>>>>>>> authentication. >>>>>>>>>>> . >>>>>>>>>>> 2. Client keystore is mandatory if ssl.client.auth=required, >>>>>>>>>>> optional for >>>>>>>>>>> requested and not used for none. The truststore configured on >>>>>>>>>>> the client is >>>>>>>>>>> used to authenticate the server. So you have to provide it >>>>>>>>>>> unless your >>>>>>>>>>> broker is using certificates signed by a trusted authority. >>>>>>>>>>> >>>>>>>>>>> Hope that helps. >>>>>>>>>>> >>>>>>>>>>> Rajini >>>>>>>>>>> >>>>>>>>>>> On Fri, May 12, 2017 at 11:35 AM, Raghav <raghavas...@gmail.com> >>>>>>>>>>> wrote: >>>>>>>>>>> >>>>>>>>>>> > Hi >>>>>>>>>>> > >>>>>>>>>>> > I read the documentation here: >>>>>>>>>>> > https://kafka.apache.org/documentation/#security_ssl >>>>>>>>>>> > >>>>>>>>>>> > I have few questions about trust store and keystore based on >>>>>>>>>>> this scenario: >>>>>>>>>>> > >>>>>>>>>>> > We have 5 Kafka Brokers in our cluster. We want our clients to >>>>>>>>>>> write to our >>>>>>>>>>> > Kafka brokers in a secure way. Suppose, we also host a private >>>>>>>>>>> CA as >>>>>>>>>>> > mentioned in the documentation above, and provide our clients >>>>>>>>>>> the *ca-cert* >>>>>>>>>>> > file, which they add it to their trust store. >>>>>>>>>>> > >>>>>>>>>>> > 1. Do we require our clients to generate their certificate and >>>>>>>>>>> have it >>>>>>>>>>> > signed by our private CA, and add it to their keystore? >>>>>>>>>>> > >>>>>>>>>>> > 2. When is keystore used by clients, and when is truststore >>>>>>>>>>> used by clients >>>>>>>>>>> > ? >>>>>>>>>>> > >>>>>>>>>>> > >>>>>>>>>>> > Thanks. >>>>>>>>>>> > >>>>>>>>>>> > -- >>>>>>>>>>> > R >>>>>>>>>>> > >>>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> Raghav >>>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>>> >>>>>>>> -- >>>>>>>> Raghav >>>>>>>> >>>>>>> >>>>>>> >>>>>>> >>>>>>> -- >>>>>>> Raghav >>>>>>> >>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> Raghav >>>>> >>>> >>>> >>> >>> >>> -- >>> Raghav >>> >> >> > > > -- > Raghav >
