Re: [DISCUSS] PIP-239: Retry Mechanism per Message

2023-01-04 Thread Zixuan Liu
+1 Thanks, Zixuan Nitin Goyal 于2023年1月5日周四 13:50写道: > Hi all, > > I made a PIP to discuss: https://github.com/apache/pulsar/issues/19136 > > Thanks, > Nitin Goyal >

Re: [DISCUSS] PIP-236: Upload AUTO_CONSUME SchemaType to Broker

2023-01-04 Thread Yunze Xu
It's not related to the schema itself. When an AUTO_CONSUME consumer subscribes to a topic, the option tells the broker that it's an AUTO_CONSUME consumer so that the broker should not treat it as an active consumer when performing schema compatibility check. If there is a consumer that also wants

[DISCUSS] PIP-239: Retry Mechanism per Message

2023-01-04 Thread Nitin Goyal
Hi all, I made a PIP to discuss: https://github.com/apache/pulsar/issues/19136 Thanks, Nitin Goyal

Re: [DISCUSS] Retry logic per message

2023-01-04 Thread Nitin Goyal
I have created a PIP for the same please find the url below for it. https://github.com/apache/pulsar/issues/19136 On Thu, Jan 5, 2023 at 8:20 AM Haiting Jiang wrote: > The motivation looks good. > > But I think we should go through a PIP process to get it approved. > So that other languages can

Re: [DISCUSS] PIP-236: Upload AUTO_CONSUME SchemaType to Broker

2023-01-04 Thread SiNan Liu
I have modified pip issue and title last night. Yunze. You mean that in PulsarApi.proto, take `optional bool is_auto_consume_schema = 6 [default = false]; ` in CommandSubscribe instead of Schema? But shouldn't schema-related stuff be in Schema? Thanks, Sinan Yunze Xu 于 2023年1月5日周四 上午12:31写道: >

[ANNOUNCE] Apache Pulsar 2.10.3 released

2023-01-04 Thread Xiangying Meng
The Apache Pulsar team is proud to announce Apache Pulsar version 2.10.3. Pulsar is a highly scalable, low latency messaging platform running on commodity hardware. It provides simple pub-sub semantics over topics, guaranteed at-least-once delivery of messages, automatic cursor management for

Re: [DISCUSS] Retry logic per message

2023-01-04 Thread Haiting Jiang
The motivation looks good. But I think we should go through a PIP process to get it approved. So that other languages can get track of this new feature, especially it use a new system property to implement this feature. Haiting On Wed, Jan 4, 2023 at 6:09 PM Zixuan Liu wrote: > > Hi Nitin, > >

Re: Experimenting with Gradle Enterprise build scans for Pulsar CI

2023-01-04 Thread Lari Hotari
Here's an example of builds scans for all Maven builds that are run as part of Pulsar CI workflow: https://ge.apache.org/scans?search.names=Git%20commit%20id%20short&search.timeZoneId=Europe/Helsinki&search.values=db7ab997&selection.buildScanB=ohtx53k6ymsgc There are 69 maven builds that are run

Re: [DISCUSS] PIP-236: Upload AUTO_CONSUME SchemaType to Broker

2023-01-04 Thread Yunze Xu
I found a similar compatibility problem with my closed PR. We should not set the `Schema` field for AUTO_CONSUME schema. More explanations can be found here [1]. Instead, we can add an optional field into CommandSubscribe to indicate the schema compatibility check is skipped. ```protobuf optional

Re: [DISCUSS] PIP-236: Upload AUTO_CONSUME SchemaType to Broker

2023-01-04 Thread Yunze Xu
Could you also update the PIP issue? This solution is totally different from your original proposal. Since it still introduces changes to `PulsarApi.proto`, it also requires a PIP (we can reuse this one). BTW, I tested again about carrying the SchemaInfo in the CommandSubscribe request. It c

Re: Experimenting with Gradle Enterprise build scans for Pulsar CI

2023-01-04 Thread Enrico Olivelli
Good idea Let's see the results Enrico Il Mer 4 Gen 2023, 12:52 Lari Hotari ha scritto: > Hi all, > > ASF Infra team together with Gradle Inc. has setup a dedicated Gradle > Enterprise instance for Apache projects at https://ge.apache.org/ . > > Gradle Enterprise works for both Gradle builds

Re: [DISCUSS] PIP-236: Upload AUTO_CONSUME SchemaType to Broker

2023-01-04 Thread SiNan Liu
I just implemented add an optional field in the subscribe request and compatibility seems to be fine. You guys can have a look at my PR ( https://github.com/apache/pulsar/pull/17449). Yunze Xu 于2023年1月4日周三 21:31写道: > > Why can't we upload negative schema types? > > I want to avoid the changes to

Re: [DISCUSS] PIP-236: Upload AUTO_CONSUME SchemaType to Broker

2023-01-04 Thread Yunze Xu
> Why can't we upload negative schema types? I want to avoid the changes to existing methods like Commands#getSchemaType, which converts all negative schema types to NONE: ```java private static Schema.Type getSchemaType(SchemaType type) { if (type.getValue() < 0) { return Schema.Type

Re: [DISCUSS] PIP-236: Upload AUTO_CONSUME SchemaType to Broker

2023-01-04 Thread 丛搏
> It does not affect the public API so it can be cherry-picked into old > branches. The main difference with this proposal is that my solution > carries the identity info (i.e. `AUTO_CONSUME`) in the schema data, > which is a byte array. The negative schema types should not be exposed > to users. A

Re: [VOTE] Pulsar Release 2.11.0 Candidate-5

2023-01-04 Thread Nicolò Boschi
About Pulsar Shell, 1) zip file is distributed as a commodity for Windows users since the .tar.gz archive is more tricky to extract. If this is a problem, we can discuss it in another thread 2) The bug on Windows is not a blocker for the release since it's not a regression. We could fix it in 2.11.

Experimenting with Gradle Enterprise build scans for Pulsar CI

2023-01-04 Thread Lari Hotari
Hi all, ASF Infra team together with Gradle Inc. has setup a dedicated Gradle Enterprise instance for Apache projects at https://ge.apache.org/ . Gradle Enterprise works for both Gradle builds and Maven builds. There's a Maven extension which provides the integration for Maven builds, https://

Re: [VOTE] Pulsar Release 2.11.0 Candidate-5

2023-01-04 Thread 丛搏
+1 (non-binding) system: mac os 12.6, Apple M1 maven: 3.8.5 java: OpenJDK 17.0.3 - Checked the signature - Checked LICENSE - Start standalone with zookeeper stream storage - Publish and consume messages - Verified Function and State Function - Verified Cassandra connector - Build from the source

Re: [DISCUSS] Retry logic per message

2023-01-04 Thread Zixuan Liu
Hi Nitin, This discussion looks good to me. Thanks, Zixuan Nitin Goyal 于2023年1月4日周三 17:53写道: > Hello all, > > We can add an retry logic per message. This feature can allow developers to > add custom retries per message and defaults to consumer retries. > > Currently all client libs are taking

[DISCUSS] Retry logic per message

2023-01-04 Thread Nitin Goyal
Hello all, We can add an retry logic per message. This feature can allow developers to add custom retries per message and defaults to consumer retries. Currently all client libs are taking care of when to send a message to Retry or DLQ topic based on redelivery times defined while creating consu

Re: [DISCUSS] PIP-236: Upload AUTO_CONSUME SchemaType to Broker

2023-01-04 Thread Yunze Xu
I opened a PR to fix this issue: https://github.com/apache/pulsar/pull/19128 It does not affect the public API so it can be cherry-picked into old branches. The main difference with this proposal is that my solution carries the identity info (i.e. `AUTO_CONSUME`) in the schema data, which is a byt

Re: [VOTE] Pulsar Release 2.11.0 Candidate-5

2023-01-04 Thread Yunze Xu
Okay, I will give my +1 (binding). Thanks, Yunze On Wed, Jan 4, 2023 at 4:00 PM PengHui Li wrote: > > I think it's ok. > 2.11.0 is the first release version of pulsar-shell > So it's not a regression that was introduced in 2.11.0 and not > critical security issues or license issues which will bl

Re: [ATTENTION] The source of Pulsar document is moved to apache/pulsar-site

2023-01-04 Thread Yu Liu
Thanks, tison and Michael! I've added one workflow that needs to be updated as Michael talked about the PR template [1] https://github.com/apache/pulsar/issues/19111#issuecomment-1370596495 On Wed, Jan 4, 2023 at 1:34 PM Michael Marshall wrote: > > I'd prefer not to put too many things in the

Re: [VOTE] Pulsar Release 2.11.0 Candidate-5

2023-01-04 Thread PengHui Li
I think it's ok. 2.11.0 is the first release version of pulsar-shell So it's not a regression that was introduced in 2.11.0 and not critical security issues or license issues which will block users move to the new version. Thanks, Penghui On Wed, Jan 4, 2023 at 3:36 PM Yunze Xu wrote: > +0 (b