I read thru the security_ssl page when I started this, it doesn't apply
much to me because I'm running this in AWS MSK, where I can't access the
broker. so my hands are tied there when it come to certificate.
however, this morning, I decided to work on creating a self sign cert for
the CURL command. I was able to get past the ssl handshake error and now
running into another issue.
* Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 8443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* Cipher selection:
ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* successfully set certificate verify locations:
* CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
* TLSv1.2 (OUT), TLS header, Certificate Status (22):
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (OUT), TLS alert, unknown CA (560):
** SSL certificate problem: self signed certificate* Closing connection 0*
so I tried passing the -k flag and got the following message
* Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to localhost (127.0.0.1) port 8443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* Cipher selection:
ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* successfully set certificate verify locations:
* CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
* TLSv1.2 (OUT), TLS header, Certificate Status (22):
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES256-GCM-SHA384
* ALPN, server did not agree to a protocol
* Server certificate:
* subject: C=us; ST=il; L=Unknown; O=usfoods; OU=Unknown; CN=Unknown
* start date: Apr 14 16:52:07 2021 GMT
* expire date: Apr 12 16:52:07 2031 GMT
* issuer: C=US; ST=MI; L=Unknown; O=FUD; OU=Unknown; CN=Unknown
* SSL certificate verify result: self signed certificate (18), continuing
anyway.
> PUT /connectors HTTP/1.1
> Host: localhost:8443
> User-Agent: curl/7.61.1
> Accept: */*
> Content-Type: application/json
> Content-Length: 1251
> Expect: 100-continue
>
< HTTP/1.1 100 Continue
* We are completely uploaded and fine
< HTTP/1.1 405 Method Not Allowed
< Date: Wed, 14 Apr 2021 19:07:01 GMT
< Content-Length: 58
< Server: Jetty(9.4.33.v20201020)
<
* Connection #0 to host localhost left intact
*{"error_code":405,"message":"HTTP 405 Method Not Allowed"}*
with the error, the mm2 process still isn't running. i'm not sure how to
go about troubleshooting this HTTP 405 msg. Maybe the cert can't be self
signed and need an official one.
On Tue, Apr 13, 2021 at 8:41 PM Ning Zhang <ning2008w...@gmail.com> wrote:
> assume your target / destination kafka cluster is SSL enabled. If your MM2
> wants to write to such cluster, you may have the following config in your
> MM2:
>
>
> https://github.com/ning2008wisc/minikube-mm2-demo/blob/master/kafka-mm/values.yaml#L79-L80
>
> on the broker (even client) side, you may refer to:
>
> http://kafka.apache.org/090/documentation.html#security_ssl
>
> On 2021/04/09 15:01:26, Men Lim <zulu...@gmail.com> wrote:
> > Hi Ning,
> >
> > thanks for the response. This self sign cert stays on the ec2 instance,
> > specifically for the curl command and I don't have to share it with the
> > brokers correct?
> >
> > thanks,
> >
> >
> >
> > On Fri, Apr 9, 2021 at 7:55 AM Ning Zhang <ning2008w...@gmail.com>
> wrote:
> >
> > > Hi Men,
> > >
> > > I used to deploy MM2 on EC2 with SSL and IIRC, probably give a try of
> > > self-signing certs and key for testing purpose:
> > > https://linuxize.com/post/creating-a-self-signed-ssl-certificate/
> > >
> > > On 2021/04/09 03:14:30, Men Lim <zulu...@gmail.com> wrote:
> > > > Hi Ryanne,
> > > >
> > > > thanks for the reply. My kafka clusters are on AWS, their serverless
> > > > platform, MSK. I'm stuck with using the default java cacerts unless
> I
> > > use
> > > > their AWS PCA which is pretty pricey.
> > > >
> > > > I ran the CURL command yesterday with the -v and --tlsv1.2 flag and
> got
> > > the
> > > > following verbose message:
> > > >
> > > > curl -s -X POST -H 'Content-Type: application/json' --data
> > > @connector.json
> > > > https://localhost:8443/connectors -v --tlsv1.2
> > > > * Trying 127.0.0.1...
> > > > * TCP_NODELAY set
> > > > * Connected to localhost (127.0.0.1) port 8443 (#0)
> > > > * ALPN, offering h2
> > > > * ALPN, offering http/1.1
> > > > * Cipher selection:
> > > > ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
> > > > * successfully set certificate verify locations:
> > > > * CAfile: /etc/pki/tls/certs/ca-bundle.crt
> > > > CApath: none
> > > > * TLSv1.2 (OUT), TLS header, Certificate Status (22):
> > > > * TLSv1.2 (OUT), TLS handshake, Client hello (1):
> > > > * TLSv1.2 (IN), TLS header, Unknown (21):
> > > > * TLSv1.2 (IN), TLS alert, handshake failure (552):
> > > > * error:14077410:SSL routines:SSL23_GET_SERVER_HELLO:sslv3 alert
> > > handshake
> > > > failure
> > > >
> > > > Thanks
> > > >
> > > > On Mon, Apr 5, 2021 at 7:26 AM Ryanne Dolan <ryannedo...@gmail.com>
> > > wrote:
> > > >
> > > > > Yes it's possible. The most common issue in my experience is the
> > > location
> > > > > of the trust store and key store being different or absent on some
> > > hosts.
> > > > > You need to make sure that these locations are consistent across
> all
> > > hosts
> > > > > in your Connect cluster, or use a ConfigProvider to provide the
> > > location
> > > > > dynamically. Otherwise, a task will get scheduled on some host and
> > > fail to
> > > > > find these files.
> > > > >
> > > > > Ryanne
> > > > >
> > > > >
> > > > > On Wed, Mar 31, 2021, 8:22 PM Men Lim <zulu...@gmail.com> wrote:
> > > > >
> > > > > > Hello. I was wondering if someone can help answer my question.
> I'm
> > > > > trying
> > > > > > to run MirrorMaker 2 in distributed mode using SSL. I have the
> > > > > distributor
> > > > > > running in SSL but when I can't get the curl REST api to do so.
> I saw
> > > > > that
> > > > > > kif-208 fixed this but I can't seem to implement it.
> > > > > >
> > > > > > in my mm2-dist.prop file I have set:
> > > > > > ////
> > > > > > listeners=https://localhost:8443
> > > > > > security.protocol=SSL
> > > > > >
> > > > > >
> > > > >
> > >
> ssl.truststore.location=/home/ec2-user/kafka_2.13-2.7.0/cert/kafka.client.truststore.jks
> > > > > > ////
> > > > > > my connector.json file look like this:
> > > > > >
> > > > > > ////
> > > > > > {
> > > > > > "name": "mm2-connect-cluster",
> > > > > > "config":{
> > > > > > "connector.class":
> > > > > "org.apache.kafka.connect.mirror.MirrorSourceConnector",
> > > > > > "connector.client.config.override.policy": "All",
> > > > > > "name": "mm2-connect-cluster",
> > > > > > "topics": "test.*",
> > > > > > "tasks.max": "1",
> > > > > > "source.cluster.alias": "source",
> > > > > > "target.cluster.alias": "target",
> > > > > > "source.cluster.bootstrap.servers": "source:9094",
> > > > > > "target.cluster.bootstrap.servers": "target:9094",
> > > > > > "source->target.enabled": "true",
> > > > > > "target->source.enabled": "false",
> > > > > > "offset-syncs.topic.replication.factor": "4",
> > > > > > "topics.exclude": ".*[\\-\\.]internal, .*\\.replica,
> > > > > > __consumer_offsets",
> > > > > > "groups.blacklist": "console-consumer-.*, connect-.*,
> __.*",
> > > > > > "topic.creation.enabled": "true",
> > > > > > "topic.creation.default.replication.factor": "4",
> > > > > > "topic.creation.default.partitions": "1"
> > > > > > "key.converter":
> > > "org.apache.kafka.connect.json.JsonConverter",
> > > > > > "value.converter":
> > > "org.apache.kafka.connect.json.JsonConverter",
> > > > > > "security.protocol": "SSL",
> > > > > > "ssl.truststore.password":
> > > > > >
> "/home/ec2-user/kafka_2.13-2.7.0/cert/kafka.client.truststore.jks"
> > > > > > }
> > > > > > }
> > > > > > ////
> > > > > >
> > > > > > I would then start up the distributor and it launched fine. So I
> > > try to
> > > > > > run the CURl command
> > > > > >
> > > > > > ////
> > > > > > curl -s -X POST -H 'Content-Type: application/json' --data
> > > > > @connector.json
> > > > > > https://localhost:8443/connectors
> > > > > > ////
> > > > > > nada. nothing. no error. no reasons for not starting.
> > > > > >
> > > > > > Is it possible to run MM2 with SSL? If so, can someone point me
> to a
> > > > > > working example?
> > > > > >
> > > > > > thanks.
> > > > > >
> > > > >
> > > >
> > >
> >
>
