Another point .. on adding the chain.p12 (PKCS#12 format) .. order of the certs in the keystore is as shown below
Alias : 1 Cert[1] -> server cert Cert[2] -> Intermediate cert Cert[3] -> Root cert mentioning that, since one of the articles i read mentioned that the certs imported should be in following order -> 1) Root 2) Intermediate 3) Server On Wed, Sep 20, 2017 at 4:48 PM, karan alang <karan.al...@gmail.com> wrote: > Hi Jakub, > > Thanks for the detailed note... > > here is the update -> > > I was able to convert the host.cert.pem to PKCS#12 & import the cert into > the kafka.server.keystore.jks > (also into kafka.server.truststore.jks) > wrt the host.root.pem & host.intermed.pem certs - i'm assuming i need to > convert them to PKCS#12 as well ? > Or do i import them as PEM file ? > > I was not able to convert them to PKCS#12(since i understand the PKCS#12 > format requires a private key & the root.pem & intermed.pem donot have a > private key associated) > the error is shown below - > > openssl pkcs12 -export -out intermed.p12 -in hostname.issuer.pem -inkey >> privatekey-kafka04-0920.key.pem -password pass:<password> >> Enter pass phrase for privatekey-kafka04-0920.key.pem: >> No certificate matches private key > > > > What i did next -> > > The host.chain.cert has all the 3 certs (server, intermediate, root) > and i converted this to PKCS format & also was able to import into the > keystore/truststore. > I'm able to see the 3 certificates in the keystore/truststore under ONE > alias. > > > Now i get a different error shown below -> > > Command : > openssl s_client -debug -connect <hostname>:9192 -tls1 > > Error : > > depth=0 CN = <hostname>, OU = <OU>, O = <Organization>, ST = <State>, C = >> US >> verify error:num=20:unable to get local issuer certificate >> verify return:1 >> depth=0 CN = <hostname>, OU = <OU>, O = <Organization>, ST = <State>, C = >> US >> verify error:num=27:certificate not trusted >> verify return:1 >> depth=0 CN = <hostname>, OU = <OU>, O = <Organization>, ST = <State>, C = >> US >> verify error:num=21:unable to verify the first certificate >> verify return:1 >> write to 0xd47830 [0xd98100] (12 bytes => 12 (0xC)) >> --------- >> --------- >> 0930 - 74 20 47 6c 6f 62 61 6c-20 43 41 0e t Global CA. >> 093f - <SPACES/NULS> >> write to 0xd47830 [0xd98100] (37 bytes => -1 (0xFFFFFFFFFFFFFFFF)) >> 140456850737056:error:1408E0F4:SSL routines:SSL3_GET_MESSAGE:unexpected >> message:s3_both.c:491: >> -------- >> -------- >> SSL-Session: >> Protocol : TLSv1 >> ------ >> ------ >> Timeout : 7200 (sec) >> Verify return code: 21 (unable to verify the first certificate) > > > > Any ideas on how to fix this ? > > > On Wed, Sep 20, 2017 at 2:22 PM, Jakub Scholz <ja...@scholz.cz> wrote: > >> ad 1) The problem is that the signed certificate (host.cert.pem) which the >> CA provides is only the public key. You have to combine it with the >> private >> key which you created when requesting the signed certificate. The private >> key is never sent to the CA so they cannot provide it back. You or whoever >> created the signing request should have it. >> >> ad 2) To create the keystore with your signed key, you have to take the >> signed public key and the private key and create P12 file: >> $ openssl pkcs12 -export -out server.p12 -in host.cert.pem -inkey host.key >> -password pass:somepassword >> >> And afterwards convert it to keystore format: >> $ keytool -importkeystore -srckeystore server.p12 -srcstoretype PKCS12 >> -srcstorepass somepassword -destkeystore server.keystore -deststoretype >> JKS >> -deststorepass somepassword -noprompt >> >> You can add the intermediate and root to the keystore as well. But when it >> tells you that you already have them in the system wide keystore say "yes" >> that you want to add them to the keystore again. >> >> ad 3) Do you want your clients to authenticate with a client certificates? >> If not, you should not need the truststore for the broker. You only need a >> truststore for the client which should contain the root and the >> intermediate certificates. If you want clients to authenticate using >> certificates you have to do basically the same as above for the client >> (You >> should use different certificate for cleints from the one in server - for >> security reasons ;-)). >> >> ad 4) As I said, I think what you did before was that the broker basically >> uses the self-signed cert you generated. This is probably the default >> subject distinguished name of the self signed certificate. And since it is >> self-signed openssl cannot verify its identity (unless you pass it the >> public key of this exact self signed certificate). >> >> Jakub >> >> On Wed, Sep 20, 2017 at 10:46 PM, karan alang <karan.al...@gmail.com> >> wrote: >> >> > Hello >> > - thanks for the response >> > Here is the update on the issue. >> > >> > I'm using certs signed/provided by org-wide CA (geotrust, not a >> self-signed >> > cert) >> > The Signed(by the CA - geotrust) cert provided has 3 certificates >> > - host.chain.pem (certificate chain - contains the Root, Intermediate, >> > Signed Server cert ) >> > - host.intermediate.pem (intermediate certificate) >> > - host.cert.pem (signed server sert) >> > Steps : >> > 1) Added the signed cert to kafka.server.keystore.jks >> > >> > >> > > keytool -import -alias servercert -trustcacerts -file host.cert.pem >> > > -keystore kafka.server.keystore.jks >> > >> > >> > *btw, Is the command above correct or needs to be changed ? * >> > >> > 2) Added the Intermediate & Signed cert to kafka.server.keystore.jks >> > >> > > keytool -import -alias intermediate -trustcacerts -file >> host.issuer.pem >> > > -keystore kafka.server.truststore.jks >> > > keytool -import -alias servercert -trustcacerts -file host.cert.pem >> > > -keystore kafka.server.truststore.jks >> > >> > >> > When i try to add the Root Cert, it shows the following message. >> > >> > *Enter keystore password: * >> > * Certificate already exists in system-wide CA keystore under alias >> > <geotrustglobalca [jdk]>* >> > * Do you still want to add it to your own keystore? [no]: * >> > >> > So, i assume the CA is already trusted .. do i still need to add this to >> > the kafka.server.truststore.jks ? >> > (I think i did that, and it did not fix the issue) >> > 3) In the Broker side (server.properties file), the setting is as shown >> > below >> > (this is the same setting when i enabled SSL using OpenSSL, which is >> > working fine btw) >> > >> > > ssl.keystore.location=/usr/hdp/2.5.3.0-37/confluent-3.2. >> > > 2/kafkaSSL/kafka.server.keystore.jks >> > > ssl.keystore.password=<passwd> >> > > ssl.key.password=<passwd> >> > > ssl.truststore.location=/usr/hdp/2.5.3.0-37/confluent-3.2. >> > > 2/kafkaSSL/kafka.server.truststore.jks >> > > ssl.truststore.password=<passwd> >> > > ssl.client.auth=required >> > >> > 4) when i verify the SSL using command below >> > >> > Command : >> > >> > > openssl s_client -debug -connect <hostname>:9192 -tls1 >> > >> > >> > Error snippet : >> > >> > >> > > >> > > >> > > depth=0 C = US, ST = <State Name>, L = Unknown, O = <org name>., OU = >> > > <OU>, CN = <localhost> >> > > >> > verify error:num=18:self signed certificate >> > > >> > > verify return:1 >> > > .................... >> > > >> > ----------------- >> > ------------------ >> > >> > >> > > 063d - <SPACES/NULS> >> > > write to 0x860830 [0x8b1100] (37 bytes => -1 (0xFFFFFFFFFFFFFFFF)) >> > > 140487373043616:error:1408E0F4:SSL routines:SSL3_GET_MESSAGE:unex >> pected >> > > message:s3_both.c:491: >> > >> > >> > Any ideas on this ? >> > What needs to be done to debug or fix this ? >> > >> > On Wed, Sep 20, 2017 at 12:33 AM, Jakub Scholz <ja...@scholz.cz> wrote: >> > >> > > Hi, >> > > >> > > Looking at your commands it looks as if you generated a self signed >> key >> > for >> > > server, self signed key for client and then imported the CA keys >> public >> > > keys into the truststores. I don’t think this will work because now >> you >> > > have two different self signed keys in the keystores and the >> presumably >> > the >> > > CA public key in your truststores. >> > > >> > > Does the cert.pem actualy contain the private key? In case it doesn’t >> > > contain the private key you need to get the private key first. If yes, >> > you >> > > have to convert it into the keystore format using this guide: >> > > https://docs.oracle.com/cd/E35976_01/server.740/es_admin/ >> > > src/tadm_ssl_convert_pem_to_jks.html >> > > You basically have to first use OpenSSL to create PKCS12 key and that >> can >> > > be converted into keystore. >> > > >> > > BTW: If you run your application with system property >> “javax.net.debug” >> > set >> > > to “ssl” it will generate a lot of useful debug information which will >> > help >> > > to understand what is going on and fix this. >> > > >> > > Jakub >> > > >> > > On Tue, 19 Sep 2017 at 23:44, karan alang <karan.al...@gmail.com> >> wrote: >> > > >> > > > Hello All - >> > > > I was able to set up SSL for the Kafka brokers, using OpenSSL. >> > > > >> > > > however, I'm having issues with setting up SSL using the pem file >> (i.e. >> > > SSL >> > > > certificate - certified by CA, provided by the company) >> > > > >> > > > Here is what i've done - >> > > > created the server/client keystore & truststore files and imported >> the >> > > > provided cert.pem file >> > > > >> > > > keytool -keystore kafka.server.keystore.jks -alias localhost >> -validity >> > > 365 >> > > > -genkey >> > > > keytool -keystore kafka.server.truststore.jks -alias CARoot -import >> > -file >> > > > cert.pem >> > > > keytool -keystore kafka.client.truststore.jks -alias CARoot -import >> > -file >> > > > cert.pem >> > > > keytool -keystore kafka.server.keystore.jks -alias CARoot -import >> -file >> > > > cert.pem >> > > > keytool -keystore kafka.client.keystore.jks -alias localhost >> -validity >> > > 365 >> > > > -genkey >> > > > keytool -keystore kafka.client.keystore.jks -alias CARoot -import >> -file >> > > > cert.pem >> > > > >> > > > I've a console producer pushing data in to the topic, and gives >> error >> > as >> > > > shown below -> >> > > > >> > > > >> > > > Caused by: javax.net.ssl.SSLProtocolException: Handshake message >> > > sequence >> > > > > violation, state = 1, type = 1 >> > > > > at >> > > > > >> > > > sun.security.ssl.ServerHandshaker.processMessage( >> > > ServerHandshaker.java:213) >> > > > > at sun.security.ssl.Handshaker.processLoop(Handshaker.java:1026) >> > > > > at sun.security.ssl.Handshaker$1.run(Handshaker.java:966) >> > > > > at sun.security.ssl.Handshaker$1.run(Handshaker.java:963) >> > > > > at java.security.AccessController.doPrivileged(Native Method) >> > > > > at sun.security.ssl.Handshaker$DelegatedTask.run(Handshaker. >> > java:1416) >> > > > > at >> > > > > >> > > > org.apache.kafka.common.network.SslTransportLayer.runDelegat >> edTasks( >> > > SslTransportLayer.java:336) >> > > > > at >> > > > > >> > > > org.apache.kafka.common.network.SslTransportLayer.handshakeUnwrap( >> > > SslTransportLayer.java:417) >> > > > > ... 7 more >> > > > >> > > > >> > > > >> > > > Any ideas on what the issue might be ? >> > > > thanks for help in advance! >> > > > >> > > >> > >> > >
