Re: [DISCUSS] Releasing Pulsar-client-go 0.8.0

[Ming Luo]

+1 non binding

> On Feb 10, 2022, at 6:37 PM, Neng Lu  wrote:
> 
> +1 non-binding
> 
> On Wed, Feb 9, 2022 at 6:46 PM Matteo Merli  wrote:
> 
>> +1
>> 
>> 
>> --
>> Matteo Merli
>> 
>> 
>> On Wed, Feb 9, 2022 at 6:44 PM r...@apache.org 
>> wrote:
>>> 
>>> Hello Everyone:
>>> 
>>> 
>>> I hope you’ve all been doing well. In the past two months, we have
>>> 
>>> fixed a number of bugs related to connection leaks and added
>>> 
>>> some new features. For more information refer to:
>>> 
>>> 
>>> https://github.com/apache/pulsar-client-go/milestone/9
>>> 
>>> 
>>> 
>>> For that reason, I think we should be releasing a 0.8.0 version with
>>> 
>>> what we have today.
>>> 
>>> 
>>> --
>>> 
>>> 
>>> Thanks
>>> 
>>> Xiaolong Ran
>> 
> 
> 
> -- 
> Best Regards,
> Neng

PIP 55 Refresh Authentication Credentials

[Ming Luo]

Hi,

Would PIP 55 propose how the client side plugin  can implement a
credential/token refresh mechanism? The use case would be the client
proactively refresh the bearer token before disconnected due to expiration
by the broker.

I am thinking some additions to Authentication/AuthenticationDataProvider
interface that the client can registered with a callback either time based
trigger or failure retrigger.

Ming
Kafkaesque.io

Re: [DISCUSS] PIP 64: Introduce REST endpoints for producing, consuming and reading messages

[Ming Luo]

Hi,

Thanks for Sijie's proposal. I resonate Rajan's comment that this project
should become a peripheral to Pulsar because of the stateless nature of
HTTP and scaling up a web service inside Pulsar might be affecting
its current design. For that reason, I developed a project with an HTTP
interface to Pulsar and webhook notification that pushes messages to
consumers.

It is written in Go. The current repo is at
https://github.com/kafkaesque-io/pulsar-beam It can be deployed in
standalone or inside a Pulsar cluster. I know someone has started using it
in their production trial. It makes sense for the HTTP interface to
become an optional add-on to Pulsar. I would like to contribute this back
to Apache Software Foundation as a standalone repo like Pulsar Manager.
What do you think? Let me know how I can help.

Cheers
Ming

On Wed, 13 May 2020 at 17:00, Rajan Dhabalia  wrote:

> *Hi,I would like to add little history about Websocket vs REST API
> implementation in Pulsar. Before open-sourcing Pulsar and WebSocket proxy
> in Yahoo, we had implemented and evaluated both WebSocket and REST-HTTP
> APIs options and then decided to not introduce REST-API and kept Websocket
> proxy solution in Pulsar. Let me add a few points to understand why we
> should avoid adding REST API in Pulsar. 1. One of the advantages of REST
> architecture is statelessness which means that any server can handle any
> request. However, it’s true for Websocket protocol as well but Websoket is
> a bi-directional protocol with full duplex communication and client-server
> can communicate on a single TCP connection which avoids different overheads
> such as authentication on every request that needs to be done on REST-Http
> api.2. If client doesn’t use persist connection correctly on REST-HTTP then
> every produce/consume request can go to a different server and it will
> create a lot of complexity to manage the control flow, producer/consumer
> queuing at REST-server which internally manages pulsar-client, clean up
> unused resources, etc.3. Performance and scalability: Websocket is highly
> scalable and provides higher performance and throughput compare to
> REST-HTTP.4. Almost all programming languages and web-browsers
>  support WebSocket.5. Pulsar is
> known
> for performance and scalability, so, we try to select solutions that work
> with scale and provide better performance.6. Pulsar broker gives an option
> to start WebSocket proxy along with broker and that doesn’t require extra
> component to manage. So, if one doesn’t want to manage additional component
> then one can start with the broker process as well. 7. Other messaging
> system also moving away from their legacy REST-HTTP api to WebSocket
> because of performance and scalability concerns.*
> *So, I think let’s not add this feature in Pulsar but it can be added a
> utility project outside of pulsar if it is needed for specific usecases.*
>
> *Thanks,*
>
> *Rajan*
>
> On Wed, May 13, 2020 at 3:25 AM Sijie Guo  wrote:
>
> > Hi all,
> >
> > Some of the people have been asking for a REST-proxy like equivalent
> > component in Pulsar. Pulsar already has a web service that provides the
> > management API over a Pulsar cluster. We don't actually need a separate
> > component or service for supporting streaming events in and out of Pulsar
> > using HTTP. We can just add endpoints to the existing web server to
> support
> > those functionalities.
> >
> > I'd like to start a discussion on adding REST endpoints to the current
> web
> > server for producing, consuming and reading messages. So people can use
> > HTTP for streaming events in and out of Pulsar without setting up
> > additional components.
> >
> > The full proposal is written at
> >
> >
> https://github.com/apache/pulsar/wiki/PIP-64:-Introduce-REST-endpoints-for-producing,-consuming-and-reading-messages
> > .
> >
> > Feedback and comments are welcome!
> >
> > - Sijie
> >
>


-- 
Powered by Pulsar
Engineering, Kafkaesque

Re: [DISCUSS] Protobuf 2.4.1

[Ming Luo]

Hi Sijie,

I just want to clarify if it is to upgrade protobuf 2 to 3. Would that lead
to a backward compatibility issue for clients? Proto3 has removed the
custom default, which pulsar api proto uses, and a few others in proto 2.
Would that be a concern for clients?

Ming

On Fri, 22 May 2020 at 13:52, Sijie Guo  wrote:

> Hi all,
>
> I would like to kick off a discussion about what we should do for protobuf
> 2.4.1 and figure out what is the long term plan.
>
> Currently, Pulsar is using a customized version of protobuf 2.4.1 for wire
> protocol. The customization was made to leverage netty object pooling to
> reduce object generation and cause bad JVM GC behavior. However, protobuf
> 2.4.1 is a many-years old release and there is also a vulnerability issue
> reported for protobuf-java 2.4.1 (
> https://nvd.nist.gov/vuln/detail/CVE-2015-5237). We need to figure out a
> plan to upgrade this dependency.
>
> Here are some thoughts. Happy to see what others think.
>
> a) We upgrade to protobuf 3.x without customization.
>
> - Pros: be able to catch up with changes in protobuf and avoid any
> vulnerabilities.
> - Cons: we might have to suffer the pains from object generations. But it
> might not be too bad since we don't store the message payloads in protobuf.
>
> b) We upgrade to protobuf 3.x with new customization.
>
> - Pros: We are able to control object generations.
> - Cons: Maintaining a customization makes harder to catch up changes in
> protobuf community and avoid vulnerabilities.
>
> c) Look for other alternatives to protobuf such as flatbuffers.
>
> - Cons: it is not a backward-compatible solution with existing Pulsar
> clients.
>
> - Sijie
>


-- 
Powered by Pulsar
Engineering, Kafkaesque

Re: [VOTE] Pulsar Client Go Release 0.1.1 Candidate 1

[Ming Luo]

+1
Are we doing additional testing for each release as supposed to each PR's
CI test?

On Fri, 12 Jun 2020 at 05:41, xiaolong ran  wrote:

> Hi everyone,
>
> Please review and vote on the release candidate #1 for the version 0.1.1,
> as follows:
> [ ] +1, Approve the release
> [ ] -1, Do not approve the release (please provide specific comments)
>
> This is the first release candidate for Apache Pulsar Go client, version
> 0.1.1.
>
> It fixes the following issues:
> https://github.com/apache/pulsar-client-go/milestone/2?closed=1 <
> https://github.com/apache/pulsar-client-go/milestone/2?closed=1>
>
> Pulsar Client Go's KEYS file contains PGP keys we used to sign this
> release:
>
> https://dist.apache.org/repos/dist/dev/pulsar/KEYS <
> https://dist.apache.org/repos/dist/dev/pulsar/KEYS>
>
> Please download these packages and review this release candidate:
>
> - Download the source package (verify shasum, and asc) and follow the
> README.md to build and run the pulsar-client-go.
>
> The vote will be open for at least 72 hours. It is adopted by majority
> approval, with at least 3 PMC affirmative votes.
>
> Source file:
>
>
> https://dist.apache.org/repos/dist/dev/pulsar/pulsar-client-go-0.1.1-candidate-1/apache-pulsar-client-go-0.1.1-src.tar.gz
> <
> https://dist.apache.org/repos/dist/dev/pulsar/pulsar-client-go-0.1.0-candidate-1/apache-pulsar-client-go-0.1.0-src.tar.gz
> >
>
> The tag to be voted upon:
> v0.1.1-candidate-1
> https://github.com/apache/pulsar-client-go/releases/tag/v0.1.1-candidate-1
> <
> https://github.com/apache/pulsar-client-go/releases/tag/v0.1.1-candidate-1
> >
>
> SHA-512 checksums:
> 2fed052f49364ccadbf912b360b4ce62ae63540d99b2d497eb8947808eceb8cea058e66f3968919c5f1b53937c3f5155a81b87b6160ea6a84309899481aaffa4
> apache-pulsar-client-go-0.1.1-src.tar.gz

Re: [DISCUSS] Apache pulsar-client-go v0.4.0 release

[Ming Luo]

+1

Xiaolong,

Thank you for organizing this. Is there any additional regression for 0.4.0
release? There were a few bug fixes around concurrency issues that were
introduced in earlier releases. We probably need more tests to cover.

Also can you confirm whether these three open PR (from the link) will be in
0.4.0?

Cheers
Ming

On Sat, 30 Jan 2021 at 02:11, xiaolong ran  wrote:

> Hello all:
>
> I hope you've all been doing well. Since the last v0.3.0 release,
> almost two months and a half have passed. So I will start to release
> v0.4.0 next week.
>
> You can find the whole change list of the v0.4.0 release from:
>
>
> https://github.com/apache/pulsar-client-go/pulls?q=is%3Aopen+is%3Apr+milestone%3A0.4.0
>
> If I missed some features, please let me know. And if
> some PRs can't complete in a few days, the owner can
> help check if we should include them in v0.4.0 or push to
> the next release version(v0.5.0).
>
> --
> Thanks
> Xiaolong Ran
>


-- 
Powered by Pulsar
Engineering, Kesque

Re: [DISCUSS] Releasing Pulsar-client-go 0.6.0

[Ming Luo]

+1

Thanks
Ming

On Mon, 19 Jul 2021 at 18:55, Neng Lu  wrote:

> +1
>
> Neng Lu
>
> On 2021/07/19 08:44:11 "r...@apache.org" wrote:
> > Hello Everyone:
> >
> > I hope you’ve all been doing well. In the past two months, we have
> > fixed a number of bugs related to connection leaks and added
> > some new features. For more information refer to:
> >
> > https://github.com/apache/pulsar-client-go/milestone/7?closed=1
> >
> > For that reason, I think we should be releasing a 0.6.0 version with
> > what we have today.
> >
> > --
> > Thanks
> > Xiaolong Ran
> >
>


-- 
Powered by Pulsar
Engineering, Kesque

Re: [DISCUSS] Changed pulsar-go-client minimum Go SDK version from 1.15 to 1.16

[Ming Luo]

Have you thought about using ldflags to pass git commit or git tag which
can be the actual version. This will tie into a build system automatically
without additional files. We don't have to maintain the version
information in git and a separate file.
Here is an example.
https://stackoverflow.com/questions/11354518/application-auto-build-versioning/55090713#55090713

On Thu, 29 Sept 2022 at 02:41, r...@apache.org 
wrote:

> Hello Matteo:
>
> Considering that some users are still using the Go 1.15 version, in the
> 0.9.0 release, we still use the form of hard code and set the form of
> pulsar-go-client to the version of 0.9.0. In order to ensure that the 0.9.0
> release maintains backward compatibility, we may continue to do this for a
> while. When we scrap Go 1.15, we will use the official Go embed to
> dynamically configure pulsar Version information for -client-go.
>
> --
> Thanks
> Xiaolong Ran
>
> Matteo Merli  于2022年9月29日周四 11:33写道：
>
> > +1
> >
> > It would be good to determine and document a compatibility policy to
> state
> > which versions are going to be supported and for how long.
> >
> > On Wed, Sep 28, 2022 at 8:31 PM 沈家琦  wrote:
> >
> > > +1
> > > pulsar-go-client from Go 1.15 to Go 1.16
> > >
> > > Thanks
> > >
> > > r...@apache.org  于2022年9月29日周四 11:11写道：
> > >
> > > > Hello everyone:
> > > >
> > > > In the current version of pulsar-go-client, we do not support
> embedding
> > > the
> > > > version information of pulsar-go-client into
> > > > Client Command to send to the Broker. Now the hard code is fixed to
> > > 0.1.0.
> > > > This makes it impossible for us to determine
> > > > which version of pulsar-go-client the current user is using through
> the
> > > > version information.
> > > >
> > > > To support this feature, the more clumsy method is described using
> > > > https://github.com/apache/pulsar-client-go/pull/820.
> > > > However, Go 1.16 officially supports the feature of embed (
> > > > https://pkg.go.dev/embed), which supports reading the
> > > > specified file directly, which seems to be a more elegant way, but
> this
> > > > requires Go 1.16 version. Yes, the current minimum
> > > > version supported by our pulsar-go-client is Go 1.15, which requires
> us
> > > to
> > > > change the minimum version of
> > > > pulsar-go-client from Go 1.15 to Go 1.16.
> > > >
> > > > So here we start a discussion within the community to see if we can
> > make
> > > > this change.
> > > >
> > > > --
> > > > Thanks
> > > > Xiaolong Ran
> > > >
> > >
> > --
> > --
> > Matteo Merli
> > 
> >
>


-- 
Powered by Pulsar
Engineering, Kesque