Hi Jakub, Thanks for the detailed note...
here is the update -> I was able to convert the host.cert.pem to PKCS#12 & import the cert into the kafka.server.keystore.jks (also into kafka.server.truststore.jks) wrt the host.root.pem & host.intermed.pem certs - i'm assuming i need to convert them to PKCS#12 as well ? Or do i import them as PEM file ? I was not able to convert them to PKCS#12(since i understand the PKCS#12 format requires a private key & the root.pem & intermed.pem donot have a private key associated) the error is shown below - openssl pkcs12 -export -out intermed.p12 -in hostname.issuer.pem -inkey > privatekey-kafka04-0920.key.pem -password pass:<password> > Enter pass phrase for privatekey-kafka04-0920.key.pem: > No certificate matches private key What i did next -> The host.chain.cert has all the 3 certs (server, intermediate, root) and i converted this to PKCS format & also was able to import into the keystore/truststore. I'm able to see the 3 certificates in the keystore/truststore under ONE alias. Now i get a different error shown below -> Command : openssl s_client -debug -connect <hostname>:9192 -tls1 Error : depth=0 CN = <hostname>, OU = <OU>, O = <Organization>, ST = <State>, C = US > verify error:num=20:unable to get local issuer certificate > verify return:1 > depth=0 CN = <hostname>, OU = <OU>, O = <Organization>, ST = <State>, C = > US > verify error:num=27:certificate not trusted > verify return:1 > depth=0 CN = <hostname>, OU = <OU>, O = <Organization>, ST = <State>, C = > US > verify error:num=21:unable to verify the first certificate > verify return:1 > write to 0xd47830 [0xd98100] (12 bytes => 12 (0xC)) > --------- > --------- > 0930 - 74 20 47 6c 6f 62 61 6c-20 43 41 0e t Global CA. > 093f - <SPACES/NULS> > write to 0xd47830 [0xd98100] (37 bytes => -1 (0xFFFFFFFFFFFFFFFF)) > 140456850737056:error:1408E0F4:SSL routines:SSL3_GET_MESSAGE:unexpected > message:s3_both.c:491: > -------- > -------- > SSL-Session: > Protocol : TLSv1 > ------ > ------ > Timeout : 7200 (sec) > Verify return code: 21 (unable to verify the first certificate) Any ideas on how to fix this ? On Wed, Sep 20, 2017 at 2:22 PM, Jakub Scholz <ja...@scholz.cz> wrote: > ad 1) The problem is that the signed certificate (host.cert.pem) which the > CA provides is only the public key. You have to combine it with the private > key which you created when requesting the signed certificate. The private > key is never sent to the CA so they cannot provide it back. You or whoever > created the signing request should have it. > > ad 2) To create the keystore with your signed key, you have to take the > signed public key and the private key and create P12 file: > $ openssl pkcs12 -export -out server.p12 -in host.cert.pem -inkey host.key > -password pass:somepassword > > And afterwards convert it to keystore format: > $ keytool -importkeystore -srckeystore server.p12 -srcstoretype PKCS12 > -srcstorepass somepassword -destkeystore server.keystore -deststoretype JKS > -deststorepass somepassword -noprompt > > You can add the intermediate and root to the keystore as well. But when it > tells you that you already have them in the system wide keystore say "yes" > that you want to add them to the keystore again. > > ad 3) Do you want your clients to authenticate with a client certificates? > If not, you should not need the truststore for the broker. You only need a > truststore for the client which should contain the root and the > intermediate certificates. If you want clients to authenticate using > certificates you have to do basically the same as above for the client (You > should use different certificate for cleints from the one in server - for > security reasons ;-)). > > ad 4) As I said, I think what you did before was that the broker basically > uses the self-signed cert you generated. This is probably the default > subject distinguished name of the self signed certificate. And since it is > self-signed openssl cannot verify its identity (unless you pass it the > public key of this exact self signed certificate). > > Jakub > > On Wed, Sep 20, 2017 at 10:46 PM, karan alang <karan.al...@gmail.com> > wrote: > > > Hello > > - thanks for the response > > Here is the update on the issue. > > > > I'm using certs signed/provided by org-wide CA (geotrust, not a > self-signed > > cert) > > The Signed(by the CA - geotrust) cert provided has 3 certificates > > - host.chain.pem (certificate chain - contains the Root, Intermediate, > > Signed Server cert ) > > - host.intermediate.pem (intermediate certificate) > > - host.cert.pem (signed server sert) > > Steps : > > 1) Added the signed cert to kafka.server.keystore.jks > > > > > > > keytool -import -alias servercert -trustcacerts -file host.cert.pem > > > -keystore kafka.server.keystore.jks > > > > > > *btw, Is the command above correct or needs to be changed ? * > > > > 2) Added the Intermediate & Signed cert to kafka.server.keystore.jks > > > > > keytool -import -alias intermediate -trustcacerts -file host.issuer.pem > > > -keystore kafka.server.truststore.jks > > > keytool -import -alias servercert -trustcacerts -file host.cert.pem > > > -keystore kafka.server.truststore.jks > > > > > > When i try to add the Root Cert, it shows the following message. > > > > *Enter keystore password: * > > * Certificate already exists in system-wide CA keystore under alias > > <geotrustglobalca [jdk]>* > > * Do you still want to add it to your own keystore? [no]: * > > > > So, i assume the CA is already trusted .. do i still need to add this to > > the kafka.server.truststore.jks ? > > (I think i did that, and it did not fix the issue) > > 3) In the Broker side (server.properties file), the setting is as shown > > below > > (this is the same setting when i enabled SSL using OpenSSL, which is > > working fine btw) > > > > > ssl.keystore.location=/usr/hdp/2.5.3.0-37/confluent-3.2. > > > 2/kafkaSSL/kafka.server.keystore.jks > > > ssl.keystore.password=<passwd> > > > ssl.key.password=<passwd> > > > ssl.truststore.location=/usr/hdp/2.5.3.0-37/confluent-3.2. > > > 2/kafkaSSL/kafka.server.truststore.jks > > > ssl.truststore.password=<passwd> > > > ssl.client.auth=required > > > > 4) when i verify the SSL using command below > > > > Command : > > > > > openssl s_client -debug -connect <hostname>:9192 -tls1 > > > > > > Error snippet : > > > > > > > > > > > > > depth=0 C = US, ST = <State Name>, L = Unknown, O = <org name>., OU = > > > <OU>, CN = <localhost> > > > > > verify error:num=18:self signed certificate > > > > > > verify return:1 > > > .................... > > > > > ----------------- > > ------------------ > > > > > > > 063d - <SPACES/NULS> > > > write to 0x860830 [0x8b1100] (37 bytes => -1 (0xFFFFFFFFFFFFFFFF)) > > > 140487373043616:error:1408E0F4:SSL routines:SSL3_GET_MESSAGE: > unexpected > > > message:s3_both.c:491: > > > > > > Any ideas on this ? > > What needs to be done to debug or fix this ? > > > > On Wed, Sep 20, 2017 at 12:33 AM, Jakub Scholz <ja...@scholz.cz> wrote: > > > > > Hi, > > > > > > Looking at your commands it looks as if you generated a self signed key > > for > > > server, self signed key for client and then imported the CA keys public > > > keys into the truststores. I don’t think this will work because now you > > > have two different self signed keys in the keystores and the presumably > > the > > > CA public key in your truststores. > > > > > > Does the cert.pem actualy contain the private key? In case it doesn’t > > > contain the private key you need to get the private key first. If yes, > > you > > > have to convert it into the keystore format using this guide: > > > https://docs.oracle.com/cd/E35976_01/server.740/es_admin/ > > > src/tadm_ssl_convert_pem_to_jks.html > > > You basically have to first use OpenSSL to create PKCS12 key and that > can > > > be converted into keystore. > > > > > > BTW: If you run your application with system property “javax.net.debug” > > set > > > to “ssl” it will generate a lot of useful debug information which will > > help > > > to understand what is going on and fix this. > > > > > > Jakub > > > > > > On Tue, 19 Sep 2017 at 23:44, karan alang <karan.al...@gmail.com> > wrote: > > > > > > > Hello All - > > > > I was able to set up SSL for the Kafka brokers, using OpenSSL. > > > > > > > > however, I'm having issues with setting up SSL using the pem file > (i.e. > > > SSL > > > > certificate - certified by CA, provided by the company) > > > > > > > > Here is what i've done - > > > > created the server/client keystore & truststore files and imported > the > > > > provided cert.pem file > > > > > > > > keytool -keystore kafka.server.keystore.jks -alias localhost > -validity > > > 365 > > > > -genkey > > > > keytool -keystore kafka.server.truststore.jks -alias CARoot -import > > -file > > > > cert.pem > > > > keytool -keystore kafka.client.truststore.jks -alias CARoot -import > > -file > > > > cert.pem > > > > keytool -keystore kafka.server.keystore.jks -alias CARoot -import > -file > > > > cert.pem > > > > keytool -keystore kafka.client.keystore.jks -alias localhost > -validity > > > 365 > > > > -genkey > > > > keytool -keystore kafka.client.keystore.jks -alias CARoot -import > -file > > > > cert.pem > > > > > > > > I've a console producer pushing data in to the topic, and gives error > > as > > > > shown below -> > > > > > > > > > > > > Caused by: javax.net.ssl.SSLProtocolException: Handshake message > > > sequence > > > > > violation, state = 1, type = 1 > > > > > at > > > > > > > > > sun.security.ssl.ServerHandshaker.processMessage( > > > ServerHandshaker.java:213) > > > > > at sun.security.ssl.Handshaker.processLoop(Handshaker.java:1026) > > > > > at sun.security.ssl.Handshaker$1.run(Handshaker.java:966) > > > > > at sun.security.ssl.Handshaker$1.run(Handshaker.java:963) > > > > > at java.security.AccessController.doPrivileged(Native Method) > > > > > at sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker. > > java:1416) > > > > > at > > > > > > > > > org.apache.kafka.common.network.SslTransportLayer.runDelegatedTasks( > > > SslTransportLayer.java:336) > > > > > at > > > > > > > > > org.apache.kafka.common.network.SslTransportLayer.handshakeUnwrap( > > > SslTransportLayer.java:417) > > > > > ... 7 more > > > > > > > > > > > > > > > > Any ideas on what the issue might be ? > > > > thanks for help in advance! > > > > > > > > > >
