Re: [DISCUSS] PIP-158: Split client TLS transport encryption from authentication
Hi Michael, It's not the same here.
If you use AuthenticationTLS, which means you enable TLS authentication and
transport.
```
PulsarClient client = PulsarClient.builder()
.serviceUrl("pulsar://my-host:6651")
.tlsTrustCertsFilePath("/path/to/cacert.pem")
.tlsKeyFilePath("/path/to/client-key.pem")
.tlsCertificateFilePath("/path/to/client-cert.pem")
.authentication(AuthenticationTls.class.getName()) //
AuthenticationTls will uses the above certificate.
.build();
```
If you remove AuthenticationTLS, means we only use TLS transport.
Thanks,
Zixuan
Michael Marshall 于2022年5月14日周六 13:27写道：
> Thanks for your responses, Zixuan.
>
> I think it might make sense to eventually deprecate the
> AuthenticationTLS class, if only because I think it can be confusing
> to give users two ways to configure the same thing. However, that is a
> minor detail. For now, we'll need to support both.
>
> Thanks,
> Michael
>
> On Thu, May 12, 2022 at 4:43 AM Zixuan Liu wrote:
> >
> > You can see the code in the implementation part, this will be consistent
> > with the actual document.
> >
> > Zixuan Liu 于2022年5月12日周四 17:03写道：
> >
> > > Hi Michael,
> > >
> > > Thanks for your feedback!
> > >
> > > > I notice that the PIP doesn't
> > > mention documentation. Since we're adding another way to configure
> > > mTLS, please make sure to document the recommended way that users
> > > should take advantage of this feature and how this feature relates to
> the
> > > existing AuthenticationTLS feature.
> > >
> > > Good idea, let me add a simple document that how to use TLS transport
> and
> > > TLS authentication.
> > >
> > > > We are removing the client's need to use the AuthenticationTLS class
> > > to perform TLS authentication of clients by the server.
> > >
> > > We don't remove the use of the AuthenticationTLS.
> > >
> > > > If a user wants to use TLS certificates for authorization, they can
> > > still put
> > > roles in their client certificates and continue to use the
> > > AuthenticationProviderTLS class to map a TLS certificate to a role on
> > > the server side.
> > >
> > > You are right, the users still can use the AuthenticationTLS to perform
> > > the TLS transport and TLS authentication.
> > >
> > > Currently, the AuthenticationTLS includes TLS transport and TLS
> > > authentication, if the user only uses the TLS transport, not use the
> TLS
> > > authentication, it is confusing, so I want to add a TLS transport
> config in
> > > `ClientBuilder`.
> > >
> > > Thanks,
> > > Zixuan
> > >
> > >
> > > Michael Marshall 于2022年5月12日周四 01:51写道：
> > >
> > >> I agree that the current state of this feature is a bit confusing, and
> > >> I think the proposed changes make sense. I notice that the PIP doesn't
> > >> mention documentation. Since we're adding another way to configure
> > >> mTLS, please make sure to document the recommended way that users
> > >> should take advantage of this feature and how this feature relates to
> the
> > >> existing AuthenticationTLS feature.
> > >>
> > >> In order to make sure I understand the feature correctly, can you
> > >> confirm that the following is correct?
> > >>
> > >> We are removing the client's need to use the AuthenticationTLS class
> > >> to perform TLS authentication of clients by the server. If a user
> > >> wants to use TLS certificates for authorization, they can still put
> > >> roles in their client certificates and continue to use the
> > >> AuthenticationProviderTLS class to map a TLS certificate to a role on
> > >> the server side.
> > >>
> > >> Thanks,
> > >> Michael
> > >>
> > >>
> > >>
> > >>
> > >>
> > >>
> > >> On Mon, May 9, 2022 at 12:58 AM Yunze Xu >
> > >> wrote:
> > >> >
> > >> > Thanks for your clarification. Let’s continue maintaining these
> configs
> > >> in
> > >> > `ClientBuilder`.
> > >> >
> > >> > Thanks,
> > >> > Yunze
> > >> >
> > >> >
> > >> >
> > >> >
> > >> > > 2022年5月9日 13:54，Zixuan Liu 写道：
> > >> > >
> > >> > > Hi Yunze,
> > >> > >
> > >> > > Thanks for your suggestion, your idea is great, but we have the
> > >> > > `tlsProtocols()` and `tlsCiphers()` in `ClientBuilder`, so I use
> this
> > >> style.
> > >> > >
> > >> > > Thanks,
> > >> > > Zixuan
> > >> > >
> > >> > > Yunze Xu 于2022年5月9日周一 13:31写道：
> > >> > >
> > >> > >> It totally LGTM. I have a suggestion that it might be better to
> > >> configure a
> > >> > >> class like `TlsConfiguration` instead of multiple TLS related
> configs
> > >> > >> added to
> > >> > >> `ClientBuilder`.
> > >> > >>
> > >> > >> Thanks,
> > >> > >> Yunze
> > >> > >>
> > >> > >>
> > >> > >>
> > >> > >>
> > >> > >>> 2022年4月24日 14:15，Zixuan Liu 写道：
> > >> > >>>
> > >> > >>> Hi Pulsar community,
> > >> > >>>
> > >> > >>> I open a https://github.com/apache/pulsar/issues/15289 for
> Split
> > >> client
> > >> > >> TLS
> > >> > >>> transport encryption from authentication.
> > >> > >>>
> > >> > >>> Let me know what you think.
> > >> > >>>
> > >> > >>> Thanks,
> > >> > >>> Zix
Re: [VOTE] [PIP-158] Split client TLS transport encryption from authentication
+1 (non-binding) Thanks, Yunze
[jira] [Updated] (PULSAR-20) Exception for "Topic not exist" should name the topic and server
[ https://issues.apache.org/jira/browse/PULSAR-20?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jason Kania updated PULSAR-20: -- Description: I am getting the following exception while using Flink with Pulsar where I believe the topic does not exist but I cannot determine which topic it is because the job is complicated. I get the following error: org.apache.pulsar.client.admin.PulsarAdminException$NotFoundException: Topic not exist at org.apache.pulsar.client.admin.internal.BaseResource.getApiException(BaseResource.java:230) at org.apache.pulsar.client.admin.internal.TopicsImpl$7.failed(TopicsImpl.java:529) at org.apache.pulsar.shade.org.glassfish.jersey.client.JerseyInvocation$1.failed(JerseyInvocation.java:882) at org.apache.pulsar.shade.org.glassfish.jersey.client.JerseyInvocation$1.completed(JerseyInvocation.java:863) at org.apache.pulsar.shade.org.glassfish.jersey.client.ClientRuntime.processResponse(ClientRuntime.java:229) at org.apache.pulsar.shade.org.glassfish.jersey.client.ClientRuntime.access$200(ClientRuntime.java:62) at org.apache.pulsar.shade.org.glassfish.jersey.client.ClientRuntime$2.lambda$response$0(ClientRuntime.java:173) at org.apache.pulsar.shade.org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) at org.apache.pulsar.shade.org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) at org.apache.pulsar.shade.org.glassfish.jersey.internal.Errors.process(Errors.java:292) at org.apache.pulsar.shade.org.glassfish.jersey.internal.Errors.process(Errors.java:274) at org.apache.pulsar.shade.org.glassfish.jersey.internal.Errors.process(Errors.java:244) at org.apache.pulsar.shade.org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:288) at org.apache.pulsar.shade.org.glassfish.jersey.client.ClientRuntime$2.response(ClientRuntime.java:173) at org.apache.pulsar.client.admin.internal.http.AsyncHttpConnector.lambda$apply$1(AsyncHttpConnector.java:228) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:859) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:837) at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506) at java.base/java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:2073) at org.apache.pulsar.client.admin.internal.http.AsyncHttpConnector.lambda$retryOperation$4(AsyncHttpConnector.java:270) at java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:859) at java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:837) at java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506) at java.base/java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:2073) at org.apache.pulsar.shade.org.asynchttpclient.netty.NettyResponseFuture.loadContent(NettyResponseFuture.java:222) at org.apache.pulsar.shade.org.asynchttpclient.netty.NettyResponseFuture.done(NettyResponseFuture.java:257) at org.apache.pulsar.shade.org.asynchttpclient.netty.handler.AsyncHttpClientHandler.finishUpdate(AsyncHttpClientHandler.java:241) at org.apache.pulsar.shade.org.asynchttpclient.netty.handler.HttpHandler.handleChunk(HttpHandler.java:114) at org.apache.pulsar.shade.org.asynchttpclient.netty.handler.HttpHandler.handleRead(HttpHandler.java:143) at org.apache.pulsar.shade.org.asynchttpclient.netty.handler.AsyncHttpClientHandler.channelRead(AsyncHttpClientHandler.java:78) at org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at org.apache.pulsar.shade.io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) at org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) at org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) at org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) at org.apache.pulsar.shade.io.netty.channel.CombinedChannelDuplexHandler$DelegatingChannelHandlerContext.fireChannelRead(CombinedChannelDuplexHandler.java:436) at org.apache.pulsar.shade.io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:327) at org.apache.pulsar.shade.io.nett
[jira] [Commented] (PULSAR-20) Exception for "Topic not exist" should name the topic and server
[ https://issues.apache.org/jira/browse/PULSAR-20?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17537128#comment-17537128 ] Penghui Li commented on PULSAR-20: -- [~longtimer] Thanks for your feedback. We will improve this part to add the topic name in the error message. And I think you have disabled the topic auto-creation? You can try to enable the topic auto-creation to workaround. > Exception for "Topic not exist" should name the topic and server > > > Key: PULSAR-20 > URL: https://issues.apache.org/jira/browse/PULSAR-20 > Project: Pulsar > Issue Type: Improvement >Reporter: Jason Kania >Priority: Major > > I am getting the following exception while using Flink with Pulsar where I > believe the topic does not exist but I cannot determine which topic it is > because the job is complicated. I get the following error: > org.apache.pulsar.client.admin.PulsarAdminException$NotFoundException: Topic > not exist > at > org.apache.pulsar.client.admin.internal.BaseResource.getApiException(BaseResource.java:230) > at > org.apache.pulsar.client.admin.internal.TopicsImpl$7.failed(TopicsImpl.java:529) > at > org.apache.pulsar.shade.org.glassfish.jersey.client.JerseyInvocation$1.failed(JerseyInvocation.java:882) > at > org.apache.pulsar.shade.org.glassfish.jersey.client.JerseyInvocation$1.completed(JerseyInvocation.java:863) > at > org.apache.pulsar.shade.org.glassfish.jersey.client.ClientRuntime.processResponse(ClientRuntime.java:229) > at > org.apache.pulsar.shade.org.glassfish.jersey.client.ClientRuntime.access$200(ClientRuntime.java:62) > at > org.apache.pulsar.shade.org.glassfish.jersey.client.ClientRuntime$2.lambda$response$0(ClientRuntime.java:173) > at > org.apache.pulsar.shade.org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) > at > org.apache.pulsar.shade.org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) > at > org.apache.pulsar.shade.org.glassfish.jersey.internal.Errors.process(Errors.java:292) > at > org.apache.pulsar.shade.org.glassfish.jersey.internal.Errors.process(Errors.java:274) > at > org.apache.pulsar.shade.org.glassfish.jersey.internal.Errors.process(Errors.java:244) > at > org.apache.pulsar.shade.org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:288) > at > org.apache.pulsar.shade.org.glassfish.jersey.client.ClientRuntime$2.response(ClientRuntime.java:173) > at > org.apache.pulsar.client.admin.internal.http.AsyncHttpConnector.lambda$apply$1(AsyncHttpConnector.java:228) > at > java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:859) > at > java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:837) > at > java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506) > at > java.base/java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:2073) > at > org.apache.pulsar.client.admin.internal.http.AsyncHttpConnector.lambda$retryOperation$4(AsyncHttpConnector.java:270) > at > java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:859) > at > java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:837) > at > java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506) > at > java.base/java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:2073) > at > org.apache.pulsar.shade.org.asynchttpclient.netty.NettyResponseFuture.loadContent(NettyResponseFuture.java:222) > at > org.apache.pulsar.shade.org.asynchttpclient.netty.NettyResponseFuture.done(NettyResponseFuture.java:257) > at > org.apache.pulsar.shade.org.asynchttpclient.netty.handler.AsyncHttpClientHandler.finishUpdate(AsyncHttpClientHandler.java:241) > at > org.apache.pulsar.shade.org.asynchttpclient.netty.handler.HttpHandler.handleChunk(HttpHandler.java:114) > at > org.apache.pulsar.shade.org.asynchttpclient.netty.handler.HttpHandler.handleRead(HttpHandler.java:143) > at > org.apache.pulsar.shade.org.asynchttpclient.netty.handler.AsyncHttpClientHandler.channelRead(AsyncHttpClientHandler.java:78) > at > org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) > at > org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) > at > org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) > at > org.apache.pulsar.shade.io.netty.handler.codec.MessageToMessageDe
[jira] [Commented] (PULSAR-20) Exception for "Topic not exist" should name the topic and server
[ https://issues.apache.org/jira/browse/PULSAR-20?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17537134#comment-17537134 ] Penghui Li commented on PULSAR-20: -- [~longtimer] I have pushed out a PR [https://github.com/apache/pulsar/pull/15606] to fix the issue. And the fix will available in the next release. > Exception for "Topic not exist" should name the topic and server > > > Key: PULSAR-20 > URL: https://issues.apache.org/jira/browse/PULSAR-20 > Project: Pulsar > Issue Type: Improvement >Reporter: Jason Kania >Priority: Major > > I am getting the following exception while using Flink with Pulsar where I > believe the topic does not exist but I cannot determine which topic it is > because the job is complicated. I get the following error: > org.apache.pulsar.client.admin.PulsarAdminException$NotFoundException: Topic > not exist > at > org.apache.pulsar.client.admin.internal.BaseResource.getApiException(BaseResource.java:230) > at > org.apache.pulsar.client.admin.internal.TopicsImpl$7.failed(TopicsImpl.java:529) > at > org.apache.pulsar.shade.org.glassfish.jersey.client.JerseyInvocation$1.failed(JerseyInvocation.java:882) > at > org.apache.pulsar.shade.org.glassfish.jersey.client.JerseyInvocation$1.completed(JerseyInvocation.java:863) > at > org.apache.pulsar.shade.org.glassfish.jersey.client.ClientRuntime.processResponse(ClientRuntime.java:229) > at > org.apache.pulsar.shade.org.glassfish.jersey.client.ClientRuntime.access$200(ClientRuntime.java:62) > at > org.apache.pulsar.shade.org.glassfish.jersey.client.ClientRuntime$2.lambda$response$0(ClientRuntime.java:173) > at > org.apache.pulsar.shade.org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) > at > org.apache.pulsar.shade.org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) > at > org.apache.pulsar.shade.org.glassfish.jersey.internal.Errors.process(Errors.java:292) > at > org.apache.pulsar.shade.org.glassfish.jersey.internal.Errors.process(Errors.java:274) > at > org.apache.pulsar.shade.org.glassfish.jersey.internal.Errors.process(Errors.java:244) > at > org.apache.pulsar.shade.org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:288) > at > org.apache.pulsar.shade.org.glassfish.jersey.client.ClientRuntime$2.response(ClientRuntime.java:173) > at > org.apache.pulsar.client.admin.internal.http.AsyncHttpConnector.lambda$apply$1(AsyncHttpConnector.java:228) > at > java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:859) > at > java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:837) > at > java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506) > at > java.base/java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:2073) > at > org.apache.pulsar.client.admin.internal.http.AsyncHttpConnector.lambda$retryOperation$4(AsyncHttpConnector.java:270) > at > java.base/java.util.concurrent.CompletableFuture.uniWhenComplete(CompletableFuture.java:859) > at > java.base/java.util.concurrent.CompletableFuture$UniWhenComplete.tryFire(CompletableFuture.java:837) > at > java.base/java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506) > at > java.base/java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:2073) > at > org.apache.pulsar.shade.org.asynchttpclient.netty.NettyResponseFuture.loadContent(NettyResponseFuture.java:222) > at > org.apache.pulsar.shade.org.asynchttpclient.netty.NettyResponseFuture.done(NettyResponseFuture.java:257) > at > org.apache.pulsar.shade.org.asynchttpclient.netty.handler.AsyncHttpClientHandler.finishUpdate(AsyncHttpClientHandler.java:241) > at > org.apache.pulsar.shade.org.asynchttpclient.netty.handler.HttpHandler.handleChunk(HttpHandler.java:114) > at > org.apache.pulsar.shade.org.asynchttpclient.netty.handler.HttpHandler.handleRead(HttpHandler.java:143) > at > org.apache.pulsar.shade.org.asynchttpclient.netty.handler.AsyncHttpClientHandler.channelRead(AsyncHttpClientHandler.java:78) > at > org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) > at > org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) > at > org.apache.pulsar.shade.io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) > at > org.apache.pulsar.shade.io.netty.handler.codec.MessageToMessageDecoder.channelRead(MessageToMessageDecoder.java:103) > at > org.apache.pulsar.
