??????[Discuss][PIP-164] Support split bundle by flow or qps
When the broker is just started, the traffic or QPS may not be stable at this time. At this time, we can solve the problem by adjusting the time interval of the first load reporting task. The time interval for executing the report task is 5 seconds, and it is not configurable. I can add a configuration to control the delay time of the first report execution, such as: //Add the delay time configuration for the first execution long delayReportTime = config.getDelayReportTime(); this.loadReportTask = this.loadManagerExecutor.scheduleAtFixedRate(new LoadReportUpdaterTask(loadManager), delayReportTime, loadReportMinInterval, TimeUnit.MILLISECONDS);
Re: [VOTE] PIP-187 Add API to analyze a subscription backlog and provide a accurate value
Thank you all
I am closing this VOTE as Passed
Let's follow up on the PR for detailed feedback about the implementation
Enrico
Il giorno lun 25 lug 2022 alle ore 07:21 mattison chao
ha scritto:
>
> +1
>
> Best,
> Mattison
>
> On Sun, 24 Jul 2022 at 14:51, Haiting Jiang wrote:
> >
> > +1
> >
> > Thanks,
> > Haiting
> >
> > On 2022/07/23 02:00:32 PengHui Li wrote:
> > > +1
> > >
> > > Penghui
> > >
> > > On Wed, Jul 20, 2022 at 9:41 PM Asaf Mesika wrote:
> > >
> > > > Sorry to barge in the vote - I forgot to send my reply on the
> > > > discussion 2
> > > > days ago :)
> > > >
> > > >
> > > > On Tue, Jul 19, 2022 at 11:22 PM Nicolò Boschi
> > > > wrote:
> > > >
> > > > > +1, thanks
> > > > >
> > > > > Nicolò Boschi
> > > > >
> > > > > Il Mar 19 Lug 2022, 22:16 Christophe Bornet
> > > > > ha
> > > > > scritto:
> > > > >
> > > > > > +1
> > > > > >
> > > > > > Le mar. 19 juil. 2022 à 20:01, Andrey Yegorov <
> > > > > andrey.yego...@datastax.com
> > > > > > >
> > > > > > a écrit :
> > > > > >
> > > > > > > +1
> > > > > > >
> > > > > > > On Tue, Jul 19, 2022 at 10:51 AM Dave Fisher
> > > > wrote:
> > > > > > >
> > > > > > > > +1 (binding)
> > > > > > > >
> > > > > > > > I support this enhancement for when a user occasionally requires
> > > > > > accurate
> > > > > > > > backlog stats. Once we bring this into service we can see if
> > > > further
> > > > > > > > guardrails are required.
> > > > > > > >
> > > > > > > > Regards,
> > > > > > > > Dave
> > > > > > > >
> > > > > > > > > On Jul 19, 2022, at 10:02 AM, Enrico Olivelli <
> > > > eolive...@gmail.com
> > > > > >
> > > > > > > > wrote:
> > > > > > > > >
> > > > > > > > > This is the VOTE thread for PIP-187
> > > > > > > > >
> > > > > > > > > This is the GH issue:
> > > > > https://github.com/apache/pulsar/issues/16597
> > > > > > > > > This is the PR: https://github.com/apache/pulsar/pull/16545
> > > > > > > > >
> > > > > > > > > The vote is open for at least 48 hours
> > > > > > > > >
> > > > > > > > > Below you can find a copy of the text of the PIP
> > > > > > > > >
> > > > > > > > > Best regards
> > > > > > > > > Enrico
> > > > > > > > >
> > > > > > > > >
> > > > > > > > > Motivation
> > > > > > > > >
> > > > > > > > > Currently there is no way to have a accurate backlog for a
> > > > > > > subscription:
> > > > > > > > >
> > > > > > > > > you have only the number of "entries", not messages
> > > > > > > > > server side filters (PIP-105) may filter out some messages
> > > > > > > > >
> > > > > > > > > Having the number of entries is sometimes not enough because
> > > > > > > > > with
> > > > > > > > > batch messages the amount of work on the Consumers is
> > > > proportional
> > > > > to
> > > > > > > > > the number of messages, that may vary from entry to entry.
> > > > > > > > >
> > > > > > > > > Goal
> > > > > > > > >
> > > > > > > > > The idea of this patch is to provide a dedicate API (REST,
> > > > > > > > > pulsar-admin, and Java PulsarAdmin) to "analyze" a
> > > > > > > > > subscription
> > > > and
> > > > > > > > > provide detailed information about that is expected to be
> > > > delivered
> > > > > > to
> > > > > > > > > Consumers.
> > > > > > > > >
> > > > > > > > > The operation will be quite expensive because we have to load
> > > > > > > > > the
> > > > > > > > > messages from storage and pass them to the filters, but due to
> > > > the
> > > > > > > > > dynamic nature of Pulsar subscriptions there is no other way
> > > > > > > > > to
> > > > > have
> > > > > > > > > this value.
> > > > > > > > >
> > > > > > > > > One good strategy to do monitoring/alerting is to setup
> > > > > > > > > alerts on
> > > > > the
> > > > > > > > > usual "stats" and use this new API to inspect the subscription
> > > > > > deeper,
> > > > > > > > > typically be issuing a manual command.
> > > > > > > > >
> > > > > > > > > API Changes
> > > > > > > > >
> > > > > > > > > internal ManagedCursor API:
> > > > > > > > >
> > > > > > > > > CompletableFuture scan(Predicate
> > > > > > > > > condition,
> > > > > long
> > > > > > > > > maxEntries, long timeOutMs);
> > > > > > > > >
> > > > > > > > > This method scans the Cursor from the lastMarkDelete position
> > > > > > > > > to
> > > > > the
> > > > > > > > tail.
> > > > > > > > > There is a time limit and a maxEntries limit, these are
> > > > > > > > > needed in
> > > > > > > > > order to prevent huge (and useless) scans.
> > > > > > > > > The Predicate can stop the scan, if it doesn't want to
> > > > > > > > > continue
> > > > the
> > > > > > > > > processing for some reasons.
> > > > > > > > >
> > > > > > > > > New REST API:
> > > > > > > > >
> > > > > > > > >@GET
> > > > > > > > >
> > > > > > > >
> > > > > > >
> > > > > >
> > > > >
> > > > @Path("/{tenant}/{namespace}/{topic}/subscription/{subName}/analyzeBacklog
> > > > > > > > > Backlog")
> > > > > > > > >@ApiOperation(value = "Analyze a subscription, by scanning
> > > > > > > > > all
> > > > > the
> > > > > > > > > unprocessed messages"
Re: [DISCUSS] PIP 194 : Pulsar client: seek command add epoch
Hi, Qiang Huang.
This is a good proposal to solve the seek issue of readers. Overall
looks good to me. Left some comments here. Thanks.
> > - stage 1: Check the current cursor status when handling flowPermits from
> > the server side.
Could you explain more details on this step? It looks like there is
not much described above. What kind of status needs to be checked, and
what kind of behavior will the broker take?
> > 1. Consumer reconnect need reset epoch.
Why do we need to reset the epoch when the consumer reconnects?
Thanks!
Zike Yang
On Tue, Jul 26, 2022 at 11:51 AM Anon Hxy wrote:
>
> +1, Good work.
>
> Thanks,
> Xiaoyu Hou
>
> Qiang Huang 于2022年7月24日周日 22:25写道：
>
> > Hi Pulsar community:
> > I open a pip to discuss "Pulsar client: seek command add epoch"
> > Proposal Link:
> >
> >- issue link: https://github.com/apache/pulsar/issues/16757
> >
> > --
> > ## Motivation
> > `Reader` belongs to exclusive subscription type, and it uses `nonDurable`
> > cursor. After receiving messages, `Reader` will ack cumulatively
> > immediately.
> > The `flowPermits` are triggered in multiple scenarios from the client side
> > and it is isolated from `seek` of `Consumer`. Therefore, it is possibile
> > that `flowPermits` will execute after `seek` from the client side, like the
> > following flow chart.
> >
> > [image: image.png]
> >
> > When `handleSeek` processing is delay from the server side, the
> > `MarkDelete position` is modified in a wrong way.
> > The expected result is that `Reader`can re-consume messages from `mark
> > delete:(1,1)` after `seek`. But it doesn't work.
> >
> > Pulsar read message and seek position is not a synchronous operation, the
> > seek request can't prevent an in-process entry reading operation. The
> > client-side also has an opportunity to receive messages after the seek
> > position.
> >
> > Pulsar client make read messages operation and seek position operation
> > synchronized so add an epoch into server and client consumer. After client
> > reader consumer invoke `seek` , the epoch increase 1 and send `seek`
> > command carry the epoch and then server consumer will update the epoch.
> > When dispatcher messages to client will carry the epoch which the cursor
> > read at the time. Client consumer will filter the send messages command
> > which is smaller than current epoch.
> > In this way, after the client consumer send `seek` command successfully,
> > because it has passed the epoch filtering, the consumer will not receive a
> > message with a messageID greater than the user previously seek position.
> >
> >
> > ### Current implementation details
> > CommandSeek Protocal
> > ```proto
> > // Reset an existing consumer to a particular message id
> > message CommandSeek {
> > required uint64 consumer_id = 1;
> > required uint64 request_id = 2;
> >
> > optional MessageIdData message_id = 3;
> > optional uint64 message_publish_time = 4;
> > }
> > ```
> > ### CommandMessage
> > ```proto
> > message CommandMessage {
> > required uint64 consumer_id = 1;
> > required MessageIdData message_id = 2;
> > optional uint32 redelivery_count = 3 [default = 0];
> > repeated int64 ack_set = 4;
> > optional uint64 epoch = 5 [default = 0];
> > }
> > ```
> > `CommandMessage` already add epoch by [PIP-84](
> > https://github.com/apache/pulsar/wiki/PIP-84-:-Pulsar-client:-Redeliver-command-add-epoch)
> > , when client receive `CommandMessage` will compare the command epoch and
> > local epoch to handle this command.
> >
> > ## Goal
> > Add epoch into seek command.
> >
> > ## API Changes
> > ### Protocal change: CommandSeek
> > ```proto
> > // Reset an existing consumer to a particular message id
> > message CommandSeek {
> > required uint64 consumer_id = 1;
> > required uint64 request_id = 2;
> >
> > optional MessageIdData message_id = 3;
> > optional uint64 message_publish_time = 4;
> > optional uint64 consumer_epoch = 5;
> > }
> > ```
> > `CommandSeek` command add epoch field, when client send seek command to
> > server successfully, the server will change the server consumer epoch to
> > the command epoch. The epoch only can bigger than the old epoch in server.
> > Now the client can filter out the message which contains less consumer
> > epoch.
> >
> > ## Implementation
> > - stage 1: Check the current cursor status when handling flowPermits from
> > the server side.
> > - stage 2: Add epoch into seek command, and server update the consumer
> > epoch. It can prevent an in-process entry reading operation after the seek
> > request.
> >
> > ## Reject Alternatives
> > None yet.
> >
> > ## Note
> > 1. Consumer reconnect need reset epoch.
> >
> > --
> > BR,
> > Qiang Huang
> >
[GitHub] [pulsar-test-infra] Anonymitaet commented on pull request #55: Add copy of Mattraks/delete-workflow-runs@a6457c4 action
Anonymitaet commented on PR #55: URL: https://github.com/apache/pulsar-test-infra/pull/55#issuecomment-1195248390 This PR blocks https://github.com/apache/pulsar/pull/16539 @lhotari can you help review this PR? Thank you! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [DISCUSS] PIP 193 : Sink preprocessing Function
When the users registers a Sink, the validation code will check that the return type of the Function is a Record. The Function can then be used either individually or together with a Sink. IMO, adding a new API would only add complexity. Le lun. 25 juil. 2022 à 14:55, Baodi Shi a écrit : > > Can you explain more what you mean ? > This PIP doesn't change the API of a Function and it's already possible to > write a Function>. > And when declaring a Sink with a Function we'll check that it's the case. > > I mean: we should constrain the function interface, otherwise, the user > may return a structure that is not a record. > > Thanks, > Baodi Shi > > > On Jul 25, 2022, at 01:0233, Christophe Bornet > wrote: > > > > Thanks for the feedback Asaf > > > > > >>> - preprocess-function: the preprocess function applied before the > >>> Sink. Starts by builtin:// for built-in functions, function:// for > >>> package function, http:// or file:// > >>> > >>> 1. While this function is applied only before sink? I thought it > replaces > >> the identity function, so why a source can't have a function that reads > >> from the source (say S3), runs the function and only then writes to a > >> pulsar topic? > >> > > > > Yes that's totally possible to implement and will be done in future work > > like written in the PIP. > > > > > >> 2. Can you clarify more about built in and function for package > function? > >> Is this an existing functionality ? > >> > > Yes those are existing functionalities. > > Built-in functions are not documented (and we should do something about > > that). > > Package management of functions is described in > > > https://pulsar.apache.org/docs/functions-deploy#use-package-management-service > > > > > >> 3. Regarding http - Are you loading a class through that URL? Aren't we > >> exposed to same problem Log4Shell security issue had? If so, what > measures > >> are you taking to protect ? > >> > > Yes we are loading code via URL. This feature already exists for > > Sources/Sinks/Functions. > > I guess you need to have a huge trust of the source from where you > download. > > This PIP has the same security level as what already exists for this > > functionality. > > > > > >> > >> The field extraFunctionPackageLocation to the protobuf structure > >>> FunctionMetaData will be added. This field will be filled with the > >>> location of the extra function to apply when registering a sink and > used > >> in > >>> the Runtime to load the function code. > >> > >> Can you please expand on that? You mean the JAR location, which you will > >> search that class name and function specified in the 3 fields you've > added > >> to the config? > >> > > Not exactly. It's the location of where the JAR is stored. It can be > > BookKeeper, package management, built-in NAR, etc... > > In KubernetesRuntime, there are cases where the builtin or package > function > > you provide in the preprocess-function param could be copied to BK. > > That's the same as for a regular Sink/Source and if we need to copy to > BK, > > we append `__sink-function` to the storage path to prevent conflict with > > the sink code. > > The class name is indeed looked up in this JAR. > > > > > >> The parameters extraFunctionFile and originalExtraFunctionFileName will > be > >>> added to RuntimeFactory::createContainer > >> > >> 1. File and fileName containing what? How does this related to > >> extraFunctionPackageLocation? > >> > > That part mimicks what is already done for the main code of the > Source/Sink > > code (with respectively codeFile, originalCodeFileName and > packageLocation) > > Before starting the ThreadedRuntime, we download locally the JAR from the > > extraFunctionPackageLocation in the extraFunctionFile so we can load the > > code from it. > > > > > >> > >> In here you use the terminology Extra Function" and in fields of config > and > >> admin you used the term Pre-Process Function. I would stick to > Pro-Process > >> Function and stick with it all over. > >> > > This terminology need to be applicable to a Function that would be > applied > > after a Source so we can't use "preprocess" here. > > I haven't found better than "extra function". Don't hesitate to propose > > something ! > > > > > >> > >> > >>> The following parameters will be added to JavaInstanceStarter: > >>> > >>> - --extra_function_jar: the path to the extra function jar > >>> > >>> > >>> - --extra_function_id: the extra function UUID cache key > >>> > >>> These parameters are then used by the ThreadRuntime to load the > function > >>> from the FunctionCacheManager or create it there if needed. > >> > >> > >> Can you elaborate on that? JavaInstanceStarter is used to start a single > >> Function? It's used from command line? > > > > The JavaInstanceStarter is indeed a CLI to start a JavaInstance. > > The JavaInstance is the process that will execute the code to read from a > > Source, execute a Function and write to a Sink. > > Generally Pulsar users do
[GitHub] [pulsar-site] SignorMercurio commented on pull request #147: Generate config docs from source code
SignorMercurio commented on PR #147: URL: https://github.com/apache/pulsar-site/pull/147#issuecomment-1195262339 @urfreespace PTAL. Only `custom.css` is changed, but I also deleted `brodocs/documents/*.md` as they should be ignored in version control (see `.gitignore`). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [DISCUSS] PIP 193 : Sink preprocessing Function
Thanks for the feedback Jerry. We don't modify the way sources, sinks and functions are detected when it's based on their fields. The proposal is just to modify the classname of the function applied in the instance so the same detection rules apply. The only difference is when detecting if the sink or function is built-in. For this we add some code to do this detection also based on the ComponentType (either detected or explicit). You can check the implementation PR about it: https://github.com/apache/pulsar/pull/16740 IMO, making it separate implementation of what currently exist would make things more complex and this more error prone for no good reason. The proposal is "just" to replace the name of the already existing function (IdentityFunction) by another one and to provide the location of the function JAR. Best regards Christophe Le lun. 25 juil. 2022 à 23:31, Jerry Peng a écrit : > My feedback is to make this change as self contained as possible. Can we > just have a special implementation of a sink that will run the logic of the > "preprocess" function? There are many places in the code where we figure > out if it is a source, sink or a function based on the fields in the > Function metadata. Changing that may have unintended consequences. > > On Mon, Jul 25, 2022 at 5:55 AM Baodi Shi > wrote: > > > > Can you explain more what you mean ? > > This PIP doesn't change the API of a Function and it's already possible > to > > write a Function>. > > And when declaring a Sink with a Function we'll check that it's the case. > > > > I mean: we should constrain the function interface, otherwise, the user > > may return a structure that is not a record. > > > > Thanks, > > Baodi Shi > > > > > On Jul 25, 2022, at 01:0233, Christophe Bornet > > > wrote: > > > > > > Thanks for the feedback Asaf > > > > > > > > >>> - preprocess-function: the preprocess function applied before the > > >>> Sink. Starts by builtin:// for built-in functions, function:// for > > >>> package function, http:// or file:// > > >>> > > >>> 1. While this function is applied only before sink? I thought it > > replaces > > >> the identity function, so why a source can't have a function that > reads > > >> from the source (say S3), runs the function and only then writes to a > > >> pulsar topic? > > >> > > > > > > Yes that's totally possible to implement and will be done in future > work > > > like written in the PIP. > > > > > > > > >> 2. Can you clarify more about built in and function for package > > function? > > >> Is this an existing functionality ? > > >> > > > Yes those are existing functionalities. > > > Built-in functions are not documented (and we should do something about > > > that). > > > Package management of functions is described in > > > > > > https://pulsar.apache.org/docs/functions-deploy#use-package-management-service > > > > > > > > >> 3. Regarding http - Are you loading a class through that URL? Aren't > we > > >> exposed to same problem Log4Shell security issue had? If so, what > > measures > > >> are you taking to protect ? > > >> > > > Yes we are loading code via URL. This feature already exists for > > > Sources/Sinks/Functions. > > > I guess you need to have a huge trust of the source from where you > > download. > > > This PIP has the same security level as what already exists for this > > > functionality. > > > > > > > > >> > > >> The field extraFunctionPackageLocation to the protobuf structure > > >>> FunctionMetaData will be added. This field will be filled with the > > >>> location of the extra function to apply when registering a sink and > > used > > >> in > > >>> the Runtime to load the function code. > > >> > > >> Can you please expand on that? You mean the JAR location, which you > will > > >> search that class name and function specified in the 3 fields you've > > added > > >> to the config? > > >> > > > Not exactly. It's the location of where the JAR is stored. It can be > > > BookKeeper, package management, built-in NAR, etc... > > > In KubernetesRuntime, there are cases where the builtin or package > > function > > > you provide in the preprocess-function param could be copied to BK. > > > That's the same as for a regular Sink/Source and if we need to copy to > > BK, > > > we append `__sink-function` to the storage path to prevent conflict > with > > > the sink code. > > > The class name is indeed looked up in this JAR. > > > > > > > > >> The parameters extraFunctionFile and originalExtraFunctionFileName > will > > be > > >>> added to RuntimeFactory::createContainer > > >> > > >> 1. File and fileName containing what? How does this related to > > >> extraFunctionPackageLocation? > > >> > > > That part mimicks what is already done for the main code of the > > Source/Sink > > > code (with respectively codeFile, originalCodeFileName and > > packageLocation) > > > Before starting the ThreadedRuntime, we download locally the JAR from > the > > > extraFunctionPackageLocation in the extraFuncti
Re: [VOTE]PIP-189: No batching if only one message in batch.
+1(non-binding) BR, Qiang Huang mattison chao 于2022年7月25日周一 13:17写道： > +1(non-binding) > > Best, > Mattison > > On Mon, 25 Jul 2022 at 10:35, Haiting Jiang > wrote: > > > > +1 > > Thanks, > > Haiting > > > > On 2022/07/25 02:23:20 Anon Hxy wrote: > > > Dear Community, > > > > > > I would like to start a VOTE on "PIP-189: No batching if only one > message > > > in batch." > > > > > > The proposal can be read at [0] and the discussion thread is available > at > > > [1] and the PR link is available at [2] > > > > > > Voting will stay open for at least 48h. > > > > > > [0] https://github.com/apache/pulsar/issues/16619 > > > [1] https://lists.apache.org/thread/dbq1lrv03bhtk0lr5nwm5txo9ndjplv0 > > > [2] https://github.com/apache/pulsar/pull/16605 > > > > > > Thanks, > > > Xiaoyu > > > > -- BR, Qiang Huang
Re: [VOTE]PIP-189: No batching if only one message in batch.
Hi, Anon So are you going to add the configuration `batchingSingleMessage ` to the producer? I saw that it was still in the PIP: > So this may cause ((BatchMessageIdImpl) messageId) throw ClassCastException. > we need to add a switch for the producer to enable or disable this feature > ProducerBuilder batchingSingleMessage(boolean batchingSingleMessage); // > default value is true Zike Yang On Tue, Jul 26, 2022 at 6:05 PM Qiang Huang wrote: > > +1(non-binding) > > BR, > Qiang Huang > > mattison chao 于2022年7月25日周一 13:17写道： > > > +1(non-binding) > > > > Best, > > Mattison > > > > On Mon, 25 Jul 2022 at 10:35, Haiting Jiang > > wrote: > > > > > > +1 > > > Thanks, > > > Haiting > > > > > > On 2022/07/25 02:23:20 Anon Hxy wrote: > > > > Dear Community, > > > > > > > > I would like to start a VOTE on "PIP-189: No batching if only one > > message > > > > in batch." > > > > > > > > The proposal can be read at [0] and the discussion thread is available > > at > > > > [1] and the PR link is available at [2] > > > > > > > > Voting will stay open for at least 48h. > > > > > > > > [0] https://github.com/apache/pulsar/issues/16619 > > > > [1] https://lists.apache.org/thread/dbq1lrv03bhtk0lr5nwm5txo9ndjplv0 > > > > [2] https://github.com/apache/pulsar/pull/16605 > > > > > > > > Thanks, > > > > Xiaoyu > > > > > > > > > -- > BR, > Qiang Huang
[GitHub] [pulsar-test-infra] lhotari closed pull request #55: Add copy of Mattraks/delete-workflow-runs@a6457c4 action
lhotari closed pull request #55: Add copy of Mattraks/delete-workflow-runs@a6457c4 action URL: https://github.com/apache/pulsar-test-infra/pull/55 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar-test-infra] lhotari commented on pull request #55: Add copy of Mattraks/delete-workflow-runs@a6457c4 action
lhotari commented on PR #55: URL: https://github.com/apache/pulsar-test-infra/pull/55#issuecomment-1195315072 > This PR blocks https://github.com/apache/pulsar/pull/16539 > > @lhotari can you help review this PR? Thank you! We don't need to delete old workflows anymore. I have explained it in the references PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [VOTE]PIP-189: No batching if only one message in batch.
Hi Zike Thanks for your reminding. I just forgot to update the issue and have updated it now. I am not going to add the configuration `batchingSingleMessage ` to the producer. Thanks, Xiaoyu Hou Zike Yang 于2022年7月26日周二 18:16写道： > Hi, Anon > > So are you going to add the configuration `batchingSingleMessage ` to > the producer? I saw that it was still in the PIP: > > > So this may cause ((BatchMessageIdImpl) messageId) throw > ClassCastException. we need to add a switch for the producer to enable or > disable this feature > > ProducerBuilder batchingSingleMessage(boolean > batchingSingleMessage); // default value is true > > > > > Zike Yang > > On Tue, Jul 26, 2022 at 6:05 PM Qiang Huang > wrote: > > > > +1(non-binding) > > > > BR, > > Qiang Huang > > > > mattison chao 于2022年7月25日周一 13:17写道： > > > > > +1(non-binding) > > > > > > Best, > > > Mattison > > > > > > On Mon, 25 Jul 2022 at 10:35, Haiting Jiang > > > wrote: > > > > > > > > +1 > > > > Thanks, > > > > Haiting > > > > > > > > On 2022/07/25 02:23:20 Anon Hxy wrote: > > > > > Dear Community, > > > > > > > > > > I would like to start a VOTE on "PIP-189: No batching if only one > > > message > > > > > in batch." > > > > > > > > > > The proposal can be read at [0] and the discussion thread is > available > > > at > > > > > [1] and the PR link is available at [2] > > > > > > > > > > Voting will stay open for at least 48h. > > > > > > > > > > [0] https://github.com/apache/pulsar/issues/16619 > > > > > [1] > https://lists.apache.org/thread/dbq1lrv03bhtk0lr5nwm5txo9ndjplv0 > > > > > [2] https://github.com/apache/pulsar/pull/16605 > > > > > > > > > > Thanks, > > > > > Xiaoyu > > > > > > > > > > > > > > -- > > BR, > > Qiang Huang >
[DISCUSS] Restart Release Pulsar 2.7.5
Hello Pulsar Community, Previously in early February, we had a discussion about 2.7.5 release in [1]. But unfortunately, there seems to be a test regression in Candidate 1 and the release process is blocked since [2]. I would like to pick up the work and volunteer to be the release manager. Here [3] you can find the list of 52 commits to branch-2.7 since the 2.7.4 release. The commits number is 41 more than RC1. Since 2.7.5 should be the last release of branch 2.7, I plan to include all existing commit this time. After that branch 2.7 should not accept any more commits. There are 4 closed PRs targeting 2.7.5 that have not yet been cherry-picked [4]. I'll follow up on these PRs and solve the test regression issue and start preparing branch-2.7 RC2 for the release. BR, Haiting Jiang [1] https://lists.apache.org/thread/wxl5z67nnzv05r5vldjq3c6sm7fxnrcc [2] https://lists.apache.org/thread/yvrmwldtzbo9lbs0jqjfj31lxtd792jq [3] https://github.com/apache/pulsar/compare/v2.7.4...branch-2.7 [4] https://github.com/apache/pulsar/pulls?q=is%3Apr+label%3Arelease%2F2.7.5+-label%3Acherry-picked%2Fbranch-2.7
Re: [DISCUSS] Restart Release Pulsar 2.7.5
+1 thank you Enrico Il giorno mar 26 lug 2022 alle ore 13:19 Haiting Jiang ha scritto: > > Hello Pulsar Community, > > Previously in early February, we had a discussion about 2.7.5 release in [1]. > But unfortunately, there seems to be a test regression in Candidate 1 and the > release process is blocked since [2]. I would like to pick up the work and > volunteer to be the release manager. > > Here [3] you can find the list of 52 commits to branch-2.7 since the 2.7.4 > release. The commits number is 41 more than RC1. Since 2.7.5 should be the > last release of branch 2.7, I plan to include all existing commit this time. > After that branch 2.7 should not accept any more commits. > > There are 4 closed PRs targeting 2.7.5 that have not yet been cherry-picked > [4]. I'll follow up on these PRs and solve the test regression issue and start > preparing branch-2.7 RC2 for the release. > > BR, > Haiting Jiang > > [1] https://lists.apache.org/thread/wxl5z67nnzv05r5vldjq3c6sm7fxnrcc > [2] https://lists.apache.org/thread/yvrmwldtzbo9lbs0jqjfj31lxtd792jq > [3] https://github.com/apache/pulsar/compare/v2.7.4...branch-2.7 > [4] > https://github.com/apache/pulsar/pulls?q=is%3Apr+label%3Arelease%2F2.7.5+-label%3Acherry-picked%2Fbranch-2.7
Re: [VOTE]PIP-189: No batching if only one message in batch.
+1 (binding) This is a small improvement but a great impact !! Enrico Il giorno mar 26 lug 2022 alle ore 12:48 Anon Hxy ha scritto: > > Hi Zike > > Thanks for your reminding. I just forgot to update the issue and have > updated it now. I am not going to add the configuration > `batchingSingleMessage ` to the producer. > > Thanks, > Xiaoyu Hou > > Zike Yang 于2022年7月26日周二 18:16写道： > > > Hi, Anon > > > > So are you going to add the configuration `batchingSingleMessage ` to > > the producer? I saw that it was still in the PIP: > > > > > So this may cause ((BatchMessageIdImpl) messageId) throw > > ClassCastException. we need to add a switch for the producer to enable or > > disable this feature > > > ProducerBuilder batchingSingleMessage(boolean > > batchingSingleMessage); // default value is true > > > > > > > > > > Zike Yang > > > > On Tue, Jul 26, 2022 at 6:05 PM Qiang Huang > > wrote: > > > > > > +1(non-binding) > > > > > > BR, > > > Qiang Huang > > > > > > mattison chao 于2022年7月25日周一 13:17写道： > > > > > > > +1(non-binding) > > > > > > > > Best, > > > > Mattison > > > > > > > > On Mon, 25 Jul 2022 at 10:35, Haiting Jiang > > > > wrote: > > > > > > > > > > +1 > > > > > Thanks, > > > > > Haiting > > > > > > > > > > On 2022/07/25 02:23:20 Anon Hxy wrote: > > > > > > Dear Community, > > > > > > > > > > > > I would like to start a VOTE on "PIP-189: No batching if only one > > > > message > > > > > > in batch." > > > > > > > > > > > > The proposal can be read at [0] and the discussion thread is > > available > > > > at > > > > > > [1] and the PR link is available at [2] > > > > > > > > > > > > Voting will stay open for at least 48h. > > > > > > > > > > > > [0] https://github.com/apache/pulsar/issues/16619 > > > > > > [1] > > https://lists.apache.org/thread/dbq1lrv03bhtk0lr5nwm5txo9ndjplv0 > > > > > > [2] https://github.com/apache/pulsar/pull/16605 > > > > > > > > > > > > Thanks, > > > > > > Xiaoyu > > > > > > > > > > > > > > > > > > > -- > > > BR, > > > Qiang Huang > >
Re: [DISCUSS] Restart Release Pulsar 2.7.5
+1 Thanks, Hang Enrico Olivelli 于2022年7月26日周二 19:21写道： > > +1 > > thank you > > Enrico > > Il giorno mar 26 lug 2022 alle ore 13:19 Haiting Jiang > ha scritto: > > > > Hello Pulsar Community, > > > > Previously in early February, we had a discussion about 2.7.5 release in > > [1]. > > But unfortunately, there seems to be a test regression in Candidate 1 and > > the > > release process is blocked since [2]. I would like to pick up the work and > > volunteer to be the release manager. > > > > Here [3] you can find the list of 52 commits to branch-2.7 since the 2.7.4 > > release. The commits number is 41 more than RC1. Since 2.7.5 should be the > > last release of branch 2.7, I plan to include all existing commit this time. > > After that branch 2.7 should not accept any more commits. > > > > There are 4 closed PRs targeting 2.7.5 that have not yet been cherry-picked > > [4]. I'll follow up on these PRs and solve the test regression issue and > > start > > preparing branch-2.7 RC2 for the release. > > > > BR, > > Haiting Jiang > > > > [1] https://lists.apache.org/thread/wxl5z67nnzv05r5vldjq3c6sm7fxnrcc > > [2] https://lists.apache.org/thread/yvrmwldtzbo9lbs0jqjfj31lxtd792jq > > [3] https://github.com/apache/pulsar/compare/v2.7.4...branch-2.7 > > [4] > > https://github.com/apache/pulsar/pulls?q=is%3Apr+label%3Arelease%2F2.7.5+-label%3Acherry-picked%2Fbranch-2.7
[Discuss][PIP-164] Support split bundle by flow or qps
Hi Pulsar Community, This is a PIP discussion on how to support splitting bundles according to flow or qps. The issue can be found: https://github.com/apache/pulsar/issues/16782 I copy the content here for convenience, any suggestions are welcome and appreciated. ## Motivation As we all know, Bundle split has 3 algorithms: - range_equally_divide - topic_count_equally_divide - specified_positions_divide However, none of these algorithms can divide bundles according to flow or qps, which may cause bundles to be split multiple times. ## Goal Our goal is to split bundles according to flow or QPS, so we propose a PIP to introduce a split algorithm based on flow or QPS. The main idea is that we can get the flow or qps information of a topic contained in a bundle, and then split from the position where the flow or qps are evenly divided. For example, there is bundle with boundaries 0x to 0x0200, and six topics : t1 , t2 , t3 , t4, t5, t6. **Step 1: Get their hash position and corresponding flow or QPS:** > t1 with hashcode 10 msgRate 100/s throughput 1M/s > > t2 with hashcode 20 msgRate 200/s throughput 2M/s > > t3 with hashcode 80 msgRate 300/s throughput 3M/s > > t4 with hashcode 90 msgRate 400/s throughput 4M/s > > t5 with hashcode 100 msgRate 500/s throughput 5M/s > > t6 with hashcode 110 msgRate 2000/s throughput 190M/s **Step 2: Calculate the total flow and qps of the bundle:** > bundleMsgRate=3500 > bundleThroughput=205MB
Re:[Discuss][PIP-164] Support split bundle by flow or qps
When the broker is just started, the traffic or QPS may not be stable at this time. At this time, we can solve the problem by adjusting the time interval of the first load reporting task. At present, the time interval for executing the report task is 5 seconds, and it is not configurable. I can add a configuration to control the delay time of the first report execution, such as: public static final long LOAD_REPORT_UPDATE_MINIMUM_INTERVAL = TimeUnit.SECONDS.toMillis(5); long loadReportMinInterval = LoadManagerShared.LOAD_REPORT_UPDATE_MINIMUM_INTERVAL; //Add the delay time configuration for the first execution long delayReportTime = config.getDelayReportTime(); this.loadReportTask = this.loadManagerExecutor.scheduleAtFixedRate(new LoadReportUpdaterTask(loadManager), delayReportTime, loadReportMinInterval, TimeUnit.MILLISECONDS);
A note from your list moderator - please subscribe
Hi - Just a note that you will get your emails onto the list much faster if you subscribe to the list by sending an email to dev-suscr...@pulsar.apacche.org and responding to the confirmation email. If you don’t subscribe then you have to wait for a moderator to approve the email and that might take a few days. Best, Dave
Re: [VOTE]PIP-189: No batching if only one message in batch.
+1 (non-binding) Great work! Thanks Zike Yang On Tue, Jul 26, 2022 at 7:31 PM Enrico Olivelli wrote: > > +1 (binding) > > This is a small improvement but a great impact !! > > Enrico > > Il giorno mar 26 lug 2022 alle ore 12:48 Anon Hxy > ha scritto: > > > > Hi Zike > > > > Thanks for your reminding. I just forgot to update the issue and have > > updated it now. I am not going to add the configuration > > `batchingSingleMessage ` to the producer. > > > > Thanks, > > Xiaoyu Hou > > > > Zike Yang 于2022年7月26日周二 18:16写道： > > > > > Hi, Anon > > > > > > So are you going to add the configuration `batchingSingleMessage ` to > > > the producer? I saw that it was still in the PIP: > > > > > > > So this may cause ((BatchMessageIdImpl) messageId) throw > > > ClassCastException. we need to add a switch for the producer to enable or > > > disable this feature > > > > ProducerBuilder batchingSingleMessage(boolean > > > batchingSingleMessage); // default value is true > > > > > > > > > > > > > > > Zike Yang > > > > > > On Tue, Jul 26, 2022 at 6:05 PM Qiang Huang > > > wrote: > > > > > > > > +1(non-binding) > > > > > > > > BR, > > > > Qiang Huang > > > > > > > > mattison chao 于2022年7月25日周一 13:17写道： > > > > > > > > > +1(non-binding) > > > > > > > > > > Best, > > > > > Mattison > > > > > > > > > > On Mon, 25 Jul 2022 at 10:35, Haiting Jiang > > > > > wrote: > > > > > > > > > > > > +1 > > > > > > Thanks, > > > > > > Haiting > > > > > > > > > > > > On 2022/07/25 02:23:20 Anon Hxy wrote: > > > > > > > Dear Community, > > > > > > > > > > > > > > I would like to start a VOTE on "PIP-189: No batching if only one > > > > > message > > > > > > > in batch." > > > > > > > > > > > > > > The proposal can be read at [0] and the discussion thread is > > > available > > > > > at > > > > > > > [1] and the PR link is available at [2] > > > > > > > > > > > > > > Voting will stay open for at least 48h. > > > > > > > > > > > > > > [0] https://github.com/apache/pulsar/issues/16619 > > > > > > > [1] > > > https://lists.apache.org/thread/dbq1lrv03bhtk0lr5nwm5txo9ndjplv0 > > > > > > > [2] https://github.com/apache/pulsar/pull/16605 > > > > > > > > > > > > > > Thanks, > > > > > > > Xiaoyu > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > BR, > > > > Qiang Huang > > >
Re: [DISCUSS] Restart Release Pulsar 2.7.5
+1 Thanks Zike Yang On Tue, Jul 26, 2022 at 7:46 PM Hang Chen wrote: > > +1 > > Thanks, > Hang > > Enrico Olivelli 于2022年7月26日周二 19:21写道： > > > > +1 > > > > thank you > > > > Enrico > > > > Il giorno mar 26 lug 2022 alle ore 13:19 Haiting Jiang > > ha scritto: > > > > > > Hello Pulsar Community, > > > > > > Previously in early February, we had a discussion about 2.7.5 release in > > > [1]. > > > But unfortunately, there seems to be a test regression in Candidate 1 and > > > the > > > release process is blocked since [2]. I would like to pick up the work and > > > volunteer to be the release manager. > > > > > > Here [3] you can find the list of 52 commits to branch-2.7 since the 2.7.4 > > > release. The commits number is 41 more than RC1. Since 2.7.5 should be the > > > last release of branch 2.7, I plan to include all existing commit this > > > time. > > > After that branch 2.7 should not accept any more commits. > > > > > > There are 4 closed PRs targeting 2.7.5 that have not yet been > > > cherry-picked > > > [4]. I'll follow up on these PRs and solve the test regression issue and > > > start > > > preparing branch-2.7 RC2 for the release. > > > > > > BR, > > > Haiting Jiang > > > > > > [1] https://lists.apache.org/thread/wxl5z67nnzv05r5vldjq3c6sm7fxnrcc > > > [2] https://lists.apache.org/thread/yvrmwldtzbo9lbs0jqjfj31lxtd792jq > > > [3] https://github.com/apache/pulsar/compare/v2.7.4...branch-2.7 > > > [4] > > > https://github.com/apache/pulsar/pulls?q=is%3Apr+label%3Arelease%2F2.7.5+-label%3Acherry-picked%2Fbranch-2.7
Re: A note from your list moderator - please subscribe
Hi Dave, There are two typos in that email. The email to use when subscribing to this list is dev-subscr...@pulsar.apache.org. - Michael On Tue, Jul 26, 2022 at 9:19 AM Dave Fisher wrote: > > Hi - > > Just a note that you will get your emails onto the list much faster if you > subscribe to the list by sending an email to dev-suscr...@pulsar.apacche.org > and responding to the confirmation email. > > If you don’t subscribe then you have to wait for a moderator to approve the > email and that might take a few days. > > Best, > Dave
[ANNOUNCE] Micheal Marshall as a new PMC member in Pulsar
I am glad to announce that the Apache Pulsar PMC invited Micheal to join the PMC and he accepted. Micheal is doing a great job in stewarding our community Please join me and celebrate ! Enrico Olivelli
Re: [DISCUSS] Apache Pulsar 2.11.0 Release
I opened a PR https://github.com/apache/pulsar/pull/16803 that might be the blocker of the release of 2.11.0, PTAL. Thanks, Yunze > 2022年7月22日 18:21，Zixuan Liu 写道： > > +1 > > Thanks, > Zixuan > > Enrico Olivelli 于2022年7月22日周五 18:06写道： > >> Il giorno ven 22 lug 2022 alle ore 12:05 guo jiwei >> ha scritto: >>> >>> I will take the release >> >> Thanks ! >> >> Enrico >> >>> >>> Regards >>> Jiwei Guo (Tboy) >>> >>> >>> On Fri, Jul 22, 2022 at 12:37 AM Nicolò Boschi >> wrote: >>> I understand the need for the Pulsar Summit. In that case I have to step back because I will be offline for the >> next few weeks Sorry Nicolò Boschi Il Gio 21 Lug 2022, 06:32 PengHui Li ha scritto: > Thanks for volunteering Nicolò. > >> So a plan could be to try to merge the work in progress targeted >> for 2.11 > by the mid of August and then start the code freezing as described >> in the > PIP. > > So the target release date will be early September. One point is >> Pulsar > Summit > San Francisco will start on August 18, 2022. I think maybe we can >> start to > test > the master branch for now and continue the in-progress tasks. If we >> can > have a > major release before Pulsar Summit, it should be good news to the > Community. > > Thanks. > Penghui > > On Mon, Jul 18, 2022 at 4:06 PM Enrico Olivelli >> > wrote: > >> Nicolò, >> >> Il Lun 18 Lug 2022, 10:00 Nicolò Boschi ha > scritto: >> >>> Thanks Penghui for the reminder. >>> I'd like to also include PIP: 181 Pulsar shell if the time >> permits. >>> >>> I believe that is a good idea to start testing the code freeze proposed >> by >>> PIP-175 (https://github.com/apache/pulsar/issues/15966). Even >> if not >>> officially approved, we discussed it many times and agreed to the >>> usefulness of the code freezing. >>> >> >> Great idea! >> >> We should really try it >> >> So a plan could be to try to merge the work in progress targeted >> for 2.11 >>> by the mid of August and then start the code freezing as >> described in > the >>> PIP. >>> >>> Also, I volunteer for driving the release if nobody else is interested >>> >> >> >> Thanks for volunteering >> >> Enrico >> >> >>> Thanks, >>> Nicolò Boschi >>> >>> >>> Il giorno lun 18 lug 2022 alle ore 06:59 Yunze Xu >>> ha scritto: >>> In addition to #16202, there is a following PR to support the correct ACK implementation for chunked messages. It should depend on >> #16202 But I think I can submit an initial PR this week and change the tests after #16202 is merged. Thanks, Yunze > 2022年7月18日 11:22，PengHui Li 写道： > > Hi all, > > We released 2.10.0 three months ago. And there are many great > changes >>> in > the master branch, > including new features and performance improvements. > > - PIP 74: apply client memory to consumer > https://github.com/apache/pulsar/pull/15216 > - PIP 143: Support split bundles by specified boundaries > https://github.com/apache/pulsar/pull/13796 > - PIP 145: regex subscription improvements > https://github.com/apache/pulsar/pull/16062 > - PIP 160: transaction performance improvements (still in progress >> and > merged some PRs) > - PIP 161: new exclusive producer mode support > https://github.com/apache/pulsar/pull/15488 > - PIP 182: Provide new load balance placement strategy > implementation >>> for > ModularLoadManagerStrategy >> https://github.com/apache/pulsar/pull/16281 > Add Pulsar Auth support for the Pulsar SQL > https://github.com/apache/pulsar/pull/15571 > > And some features are blocked in the review stage, but they >> are >>> powerful > improvements for Pulsar > > PIP 37: Support chunking with Shared subscription > https://github.com/apache/pulsar/pull/16202 > PIP-166: Function add MANUAL delivery semantics > https://github.com/apache/pulsar/pull/16279 > > You can find the complete change list in 2.11.0 at > >>> >> > >> https://github.com/apache/pulsar/pulls?q=is%3Apr+milestone%3A2.11.0+-label%3Arelease%2F2.10.1+-label%3Arelease%2F2.10.2 > > And maybe I missed some important in-progress PRs, please >> let me > know >>> if it > should be a blocker of the 2.11.0 release. > > It's a good time to discuss the target time of the 2.11.0 rel
Re: [ANNOUNCE] Micheal Marshall as a new PMC member in Pulsar
Congratulations, Micheal. Thanks ZhangJian He On Tue, 26 Jul 2022 at 23:22, Enrico Olivelli wrote: > I am glad to announce that the Apache Pulsar PMC invited Micheal to > join the PMC and he accepted. > > Micheal is doing a great job in stewarding our community > > Please join me and celebrate ! > > Enrico Olivelli >
Re: [ANNOUNCE] Micheal Marshall as a new PMC member in Pulsar
Congratulations Michael ! Well deserved ! Le mar. 26 juil. 2022 à 17:22, Enrico Olivelli a écrit : > I am glad to announce that the Apache Pulsar PMC invited Micheal to > join the PMC and he accepted. > > Micheal is doing a great job in stewarding our community > > Please join me and celebrate ! > > Enrico Olivelli >
Re: [ANNOUNCE] Micheal Marshall as a new PMC member in Pulsar
Congrats! Very well deserved!! Nicolò Boschi Il Mar 26 Lug 2022, 18:26 Christophe Bornet ha scritto: > Congratulations Michael ! Well deserved ! > > Le mar. 26 juil. 2022 à 17:22, Enrico Olivelli a > écrit : > > > I am glad to announce that the Apache Pulsar PMC invited Micheal to > > join the PMC and he accepted. > > > > Micheal is doing a great job in stewarding our community > > > > Please join me and celebrate ! > > > > Enrico Olivelli > > >
Re: [ANNOUNCE] Micheal Marshall as a new PMC member in Pulsar
Congrats Michael! Thanks for your contribution to the doc and website! On Wed, Jul 27, 2022 at 12:46 AM Nicolò Boschi wrote: > Congrats! Very well deserved!! > > Nicolò Boschi > > Il Mar 26 Lug 2022, 18:26 Christophe Bornet ha > scritto: > > > Congratulations Michael ! Well deserved ! > > > > Le mar. 26 juil. 2022 à 17:22, Enrico Olivelli a > > écrit : > > > > > I am glad to announce that the Apache Pulsar PMC invited Micheal to > > > join the PMC and he accepted. > > > > > > Micheal is doing a great job in stewarding our community > > > > > > Please join me and celebrate ! > > > > > > Enrico Olivelli > > > > > >
Re: [ANNOUNCE] Micheal Marshall as a new PMC member in Pulsar
Congratulations, Michael! BR, Haiting On 2022/07/26 15:21:59 Enrico Olivelli wrote: > I am glad to announce that the Apache Pulsar PMC invited Micheal to > join the PMC and he accepted. > > Micheal is doing a great job in stewarding our community > > Please join me and celebrate ! > > Enrico Olivelli >
Re: [ANNOUNCE] Micheal Marshall as a new PMC member in Pulsar
Congrats Michael! Regards Jiwei Guo (Tboy) On Wed, Jul 27, 2022 at 9:56 AM Haiting Jiang wrote: > Congratulations, Michael! > > BR, > Haiting > > On 2022/07/26 15:21:59 Enrico Olivelli wrote: > > I am glad to announce that the Apache Pulsar PMC invited Micheal to > > join the PMC and he accepted. > > > > Micheal is doing a great job in stewarding our community > > > > Please join me and celebrate ! > > > > Enrico Olivelli > > >
Re: [ANNOUNCE] Micheal Marshall as a new PMC member in Pulsar
Congratulations, Michael! BR Zike Yang On Wed, Jul 27, 2022 at 10:04 AM guo jiwei wrote: > > Congrats Michael! > > > Regards > Jiwei Guo (Tboy) > > > On Wed, Jul 27, 2022 at 9:56 AM Haiting Jiang > wrote: > > > Congratulations, Michael! > > > > BR, > > Haiting > > > > On 2022/07/26 15:21:59 Enrico Olivelli wrote: > > > I am glad to announce that the Apache Pulsar PMC invited Micheal to > > > join the PMC and he accepted. > > > > > > Micheal is doing a great job in stewarding our community > > > > > > Please join me and celebrate ! > > > > > > Enrico Olivelli > > > > >
Re: [ANNOUNCE] Micheal Marshall as a new PMC member in Pulsar
Congratulations, Michael! BR, xiangying Meng On Wed, Jul 27, 2022 at 10:11 AM Zike Yang wrote: > Congratulations, Michael! > > BR > Zike Yang > > On Wed, Jul 27, 2022 at 10:04 AM guo jiwei wrote: > > > > Congrats Michael! > > > > > > Regards > > Jiwei Guo (Tboy) > > > > > > On Wed, Jul 27, 2022 at 9:56 AM Haiting Jiang > > wrote: > > > > > Congratulations, Michael! > > > > > > BR, > > > Haiting > > > > > > On 2022/07/26 15:21:59 Enrico Olivelli wrote: > > > > I am glad to announce that the Apache Pulsar PMC invited Micheal to > > > > join the PMC and he accepted. > > > > > > > > Micheal is doing a great job in stewarding our community > > > > > > > > Please join me and celebrate ! > > > > > > > > Enrico Olivelli > > > > > > > >
Re: [ANNOUNCE] Micheal Marshall as a new PMC member in Pulsar
Congratulations Michael! Thanks, Yunze > 2022年7月26日 23:21，Enrico Olivelli 写道： > > I am glad to announce that the Apache Pulsar PMC invited Micheal to > join the PMC and he accepted. > > Micheal is doing a great job in stewarding our community > > Please join me and celebrate ! > > Enrico Olivelli
Re: [ANNOUNCE] Micheal Marshall as a new PMC member in Pulsar
Congratulations, Micheal. Thanks, Xiaoyu Hou Enrico Olivelli 于2022年7月26日周二 23:22写道： > I am glad to announce that the Apache Pulsar PMC invited Micheal to > join the PMC and he accepted. > > Micheal is doing a great job in stewarding our community > > Please join me and celebrate ! > > Enrico Olivelli >
Re: [ANNOUNCE] Micheal Marshall as a new PMC member in Pulsar
Congrats, Micheal! Thanks, Zixuan Anon Hxy 于2022年7月27日周三 10:25写道： > Congratulations, Micheal. > > Thanks, > Xiaoyu Hou > > Enrico Olivelli 于2022年7月26日周二 23:22写道： > > > I am glad to announce that the Apache Pulsar PMC invited Micheal to > > join the PMC and he accepted. > > > > Micheal is doing a great job in stewarding our community > > > > Please join me and celebrate ! > > > > Enrico Olivelli > > >
Re: [DISCUSS] Apache Pulsar 2.11.0 Release
+1 Thanks, Zixuan Yunze Xu 于2022年7月26日周二 23:34写道： > I opened a PR https://github.com/apache/pulsar/pull/16803 that might be > the blocker > of the release of 2.11.0, PTAL. > > Thanks, > Yunze > > > > > > 2022年7月22日 18:21，Zixuan Liu 写道： > > > > +1 > > > > Thanks, > > Zixuan > > > > Enrico Olivelli 于2022年7月22日周五 18:06写道： > > > >> Il giorno ven 22 lug 2022 alle ore 12:05 guo jiwei > >> ha scritto: > >>> > >>> I will take the release > >> > >> Thanks ! > >> > >> Enrico > >> > >>> > >>> Regards > >>> Jiwei Guo (Tboy) > >>> > >>> > >>> On Fri, Jul 22, 2022 at 12:37 AM Nicolò Boschi > >> wrote: > >>> > I understand the need for the Pulsar Summit. > > In that case I have to step back because I will be offline for the > >> next few > weeks > Sorry > > Nicolò Boschi > > Il Gio 21 Lug 2022, 06:32 PengHui Li ha scritto: > > > Thanks for volunteering Nicolò. > > > >> So a plan could be to try to merge the work in progress targeted > >> for > 2.11 > > by the mid of August and then start the code freezing as described > >> in the > > PIP. > > > > So the target release date will be early September. One point is > >> Pulsar > > Summit > > San Francisco will start on August 18, 2022. I think maybe we can > >> start > to > > test > > the master branch for now and continue the in-progress tasks. If we > >> can > > have a > > major release before Pulsar Summit, it should be good news to the > > Community. > > > > Thanks. > > Penghui > > > > On Mon, Jul 18, 2022 at 4:06 PM Enrico Olivelli >>> > > wrote: > > > >> Nicolò, > >> > >> Il Lun 18 Lug 2022, 10:00 Nicolò Boschi ha > > scritto: > >> > >>> Thanks Penghui for the reminder. > >>> I'd like to also include PIP: 181 Pulsar shell if the time > >> permits. > >>> > >>> I believe that is a good idea to start testing the code freeze > proposed > >> by > >>> PIP-175 (https://github.com/apache/pulsar/issues/15966). Even > >> if not > >>> officially approved, we discussed it many times and agreed to the > >>> usefulness of the code freezing. > >>> > >> > >> Great idea! > >> > >> We should really try it > >> > >> So a plan could be to try to merge the work in progress targeted > >> for > 2.11 > >>> by the mid of August and then start the code freezing as > >> described in > > the > >>> PIP. > >>> > >>> Also, I volunteer for driving the release if nobody else is > interested > >>> > >> > >> > >> Thanks for volunteering > >> > >> Enrico > >> > >> > >>> Thanks, > >>> Nicolò Boschi > >>> > >>> > >>> Il giorno lun 18 lug 2022 alle ore 06:59 Yunze Xu > >>> ha scritto: > >>> > In addition to #16202, there is a following PR to support the > correct > ACK implementation for chunked messages. It should depend on > >> #16202 > But I think I can submit an initial PR this week and change the > tests > after #16202 is merged. > > Thanks, > Yunze > > > > > > 2022年7月18日 11:22，PengHui Li 写道： > > > > Hi all, > > > > We released 2.10.0 three months ago. And there are many great > > changes > >>> in > > the master branch, > > including new features and performance improvements. > > > > - PIP 74: apply client memory to consumer > > https://github.com/apache/pulsar/pull/15216 > > - PIP 143: Support split bundles by specified boundaries > > https://github.com/apache/pulsar/pull/13796 > > - PIP 145: regex subscription improvements > > https://github.com/apache/pulsar/pull/16062 > > - PIP 160: transaction performance improvements (still in > progress > >> and > > merged some PRs) > > - PIP 161: new exclusive producer mode support > > https://github.com/apache/pulsar/pull/15488 > > - PIP 182: Provide new load balance placement strategy > > implementation > >>> for > > ModularLoadManagerStrategy > >> https://github.com/apache/pulsar/pull/16281 > > Add Pulsar Auth support for the Pulsar SQL > > https://github.com/apache/pulsar/pull/15571 > > > > And some features are blocked in the review stage, but they > >> are > >>> powerful > > improvements for Pulsar > > > > PIP 37: Support chunking with Shared subscription > > https://github.com/apache/pulsar/pull/16202 > > PIP-166: Function add MANUAL delivery semantics > > https://github.com/apache/pulsar/pull/16279 > > > > You can find the complete change list in 2.11.0 at > > > > >>> > >> > > > > >> > https://github.com/
Re: [DISCUSS] Restart Release Pulsar 2.7.5
+1 Thanks, Zixuan Zike Yang 于2022年7月26日周二 22:24写道： > +1 > > Thanks > Zike Yang > > On Tue, Jul 26, 2022 at 7:46 PM Hang Chen wrote: > > > > +1 > > > > Thanks, > > Hang > > > > Enrico Olivelli 于2022年7月26日周二 19:21写道： > > > > > > +1 > > > > > > thank you > > > > > > Enrico > > > > > > Il giorno mar 26 lug 2022 alle ore 13:19 Haiting Jiang > > > ha scritto: > > > > > > > > Hello Pulsar Community, > > > > > > > > Previously in early February, we had a discussion about 2.7.5 > release in [1]. > > > > But unfortunately, there seems to be a test regression in Candidate > 1 and the > > > > release process is blocked since [2]. I would like to pick up the > work and > > > > volunteer to be the release manager. > > > > > > > > Here [3] you can find the list of 52 commits to branch-2.7 since the > 2.7.4 > > > > release. The commits number is 41 more than RC1. Since 2.7.5 should > be the > > > > last release of branch 2.7, I plan to include all existing commit > this time. > > > > After that branch 2.7 should not accept any more commits. > > > > > > > > There are 4 closed PRs targeting 2.7.5 that have not yet been > cherry-picked > > > > [4]. I'll follow up on these PRs and solve the test regression issue > and start > > > > preparing branch-2.7 RC2 for the release. > > > > > > > > BR, > > > > Haiting Jiang > > > > > > > > [1] https://lists.apache.org/thread/wxl5z67nnzv05r5vldjq3c6sm7fxnrcc > > > > [2] https://lists.apache.org/thread/yvrmwldtzbo9lbs0jqjfj31lxtd792jq > > > > [3] https://github.com/apache/pulsar/compare/v2.7.4...branch-2.7 > > > > [4] > https://github.com/apache/pulsar/pulls?q=is%3Apr+label%3Arelease%2F2.7.5+-label%3Acherry-picked%2Fbranch-2.7 >
Re: [DISCUSS] PIP-195: New bucket based delayed message tracker
+1 On 2022/07/25 02:22:00 Cong Zhao wrote: > Hello Pulsar Community, > > > I would like to open a discussion here about PIP-195 : New bucket based > delayed message tracker. I look forward to your feedback. > > > PIP: https://github.com/apache/pulsar/issues/16763 > > > Thanks, > > Cong Zhao >
Re: Disable reconsumerLaster method for non shared subscription
They can use `reconsumeLaterCumulative ` for non shared sub instead On 2022/07/26 02:06:05 Haiting Jiang wrote: > Hi Gavin, > > Do we have any plans for compatibility issue? As the exclusive subscription > type > is the default setting, user may use it even if they don't have any message > order > requirements. > > Thanks, > Haiting > > On 2022/07/25 06:50:12 Gavin gao wrote: > > As The official doc says "Currently, retry letter topic is enabled in > > Shared subscription types". But, actually We can use this method when using > > non shared sub.We should disable using this method for message order > > considering. > > > > Pr: https://github.com/apache/pulsar/pull/16745 > > >
Re: 回复: [Discuss] PIP-190: Simplify Pulsar documentation release and maintenance strategy
+1 for this proposal since it improves the efficiency of managing docs. We'll take it as we reach a lazy consensus and implement this plan if there is no objection until tomorrow. On 2022/07/25 09:13:50 M Jun wrote: > Hi Asaf, > > The doc files for all older versions are independent copies, stored at > https://github.com/apache/pulsar/tree/master/site2/website/versioned_docs. > > For example, the doc for 2.10.0 was a copy saved from the next (master) doc > at that milestone. > > 发件人: Asaf Mesika > 发送时间: 2022年7月24日 21:00 > 收件人: dev@pulsar.apache.org > 主题: Re: [Discuss] PIP-190: Simplify Pulsar documentation release and > maintenance strategy > > If I understand correctly, using tags, you can automatically create the > docs for the next minor version? > So for older minor versions, where those files will be at? using tags from > git? > > On Sat, Jul 23, 2022 at 7:04 AM Ma Jun wrote: > > > Hi, Pulsar community, > > > > Happy weekend! > > > > I'd like to open a discussion about PIP-190: Simplify Pulsar documentation > > release and maintenance strategy. > > > > Proposal link: https://github.com/apache/pulsar/issues/16637. > > > > > > - > > > > ## Motivation > > > > This proposal is focused on improving and simplifying the release and > > maintenance strategy of the existing Apache Pulsar documentation. > > > > In general, this proposal provides equal values and a better user > > experience without sacrificing unpredictable efforts in the release and > > maintenance process. The benefits include: > > * Improve user experience when they look for the documentation of a > > specific release. > > * Optimize the doc development and release process for bug-fix releases: > >* Turn doc development for bug-fix releases from post-release into > > just-in-time. > >* Save Release Manager’s effort in generating doc files for bug-fix > > releases. > > * Save a great amount of the community’s effort in syncing doc > > improvements and fixes to historical doc versions that are in the > > maintenance cycle. > > > > > > Since there are quite a few illustrations and links in this proposal, I > > didn't copy and paste the complete content into this email thread. You can > > access the GitHub link to review the proposal with a better view. > > > > > > Cheers! > > > > momo-jun > > > > >
Re: [DISCUSS] PIP-195: New bucket based delayed message tracker
+1, good job. Thanks, Dezhi On 2022/07/25 02:22:00 Cong Zhao wrote: > Hello Pulsar Community, > > > I would like to open a discussion here about PIP-195 : New bucket based > delayed message tracker. I look forward to your feedback. > > > PIP: https://github.com/apache/pulsar/issues/16763 > > > Thanks, > > Cong Zhao >
Re: [Discuss] PIP-190: Simplify Pulsar documentation release and maintenance strategy
+1 as it may save a lot of effort and time for both the release manager and TWs. BR//Huanli > On Jul 27, 2022, at 11:33 AM, Liu Yu wrote: > > +1 for this proposal since it improves the efficiency of managing docs. > > We'll take it as we reach a lazy consensus and implement this plan if there > is no objection until tomorrow. > > On 2022/07/25 09:13:50 M Jun wrote: >> Hi Asaf, >> >> The doc files for all older versions are independent copies, stored at >> https://github.com/apache/pulsar/tree/master/site2/website/versioned_docs. >> >> For example, the doc for 2.10.0 was a copy saved from the next (master) doc >> at that milestone. >> >> 发件人: Asaf Mesika >> 发送时间: 2022年7月24日 21:00 >> 收件人: dev@pulsar.apache.org >> 主题: Re: [Discuss] PIP-190: Simplify Pulsar documentation release and >> maintenance strategy >> >> If I understand correctly, using tags, you can automatically create the >> docs for the next minor version? >> So for older minor versions, where those files will be at? using tags from >> git? >> >> On Sat, Jul 23, 2022 at 7:04 AM Ma Jun wrote: >> >>> Hi, Pulsar community, >>> >>> Happy weekend! >>> >>> I'd like to open a discussion about PIP-190: Simplify Pulsar documentation >>> release and maintenance strategy. >>> >>> Proposal link: https://github.com/apache/pulsar/issues/16637. >>> >>> >>> - >>> >>> ## Motivation >>> >>> This proposal is focused on improving and simplifying the release and >>> maintenance strategy of the existing Apache Pulsar documentation. >>> >>> In general, this proposal provides equal values and a better user >>> experience without sacrificing unpredictable efforts in the release and >>> maintenance process. The benefits include: >>> * Improve user experience when they look for the documentation of a >>> specific release. >>> * Optimize the doc development and release process for bug-fix releases: >>> * Turn doc development for bug-fix releases from post-release into >>> just-in-time. >>> * Save Release Manager’s effort in generating doc files for bug-fix >>> releases. >>> * Save a great amount of the community’s effort in syncing doc >>> improvements and fixes to historical doc versions that are in the >>> maintenance cycle. >>> >>> >>> Since there are quite a few illustrations and links in this proposal, I >>> didn't copy and paste the complete content into this email thread. You can >>> access the GitHub link to review the proposal with a better view. >>> >>> >>> Cheers! >>> >>> momo-jun >>> >>> >>
Re: [ANNOUNCE] Micheal Marshall as a new PMC member in Pulsar
Congratulations Michael! Thanks, Kai
Re: [ANNOUNCE] Micheal Marshall as a new PMC member in Pulsar
Congratulations, Michael ! Best, Max Xu On Wed, Jul 27, 2022 at 2:04 PM Kai Wang wrote: > Congratulations Michael! > > Thanks, > Kai >
Re: [Discuss] PIP-190: Simplify Pulsar documentation release and maintenance strategy
+1 Documentation will definitely benefit from it. Best, Max Xu On Wed, Jul 27, 2022 at 12:19 PM Huanli Meng wrote: > +1 as it may save a lot of effort and time for both the release manager > and TWs. > > BR//Huanli > > > On Jul 27, 2022, at 11:33 AM, Liu Yu wrote: > > > > +1 for this proposal since it improves the efficiency of managing docs. > > > > We'll take it as we reach a lazy consensus and implement this plan if > there is no objection until tomorrow. > > > > On 2022/07/25 09:13:50 M Jun wrote: > >> Hi Asaf, > >> > >> The doc files for all older versions are independent copies, stored at > https://github.com/apache/pulsar/tree/master/site2/website/versioned_docs. > >> > >> For example, the doc for 2.10.0 was a copy saved from the next (master) > doc at that milestone. > >> > >> 发件人: Asaf Mesika > >> 发送时间: 2022年7月24日 21:00 > >> 收件人: dev@pulsar.apache.org > >> 主题: Re: [Discuss] PIP-190: Simplify Pulsar documentation release and > maintenance strategy > >> > >> If I understand correctly, using tags, you can automatically create the > >> docs for the next minor version? > >> So for older minor versions, where those files will be at? using tags > from > >> git? > >> > >> On Sat, Jul 23, 2022 at 7:04 AM Ma Jun wrote: > >> > >>> Hi, Pulsar community, > >>> > >>> Happy weekend! > >>> > >>> I'd like to open a discussion about PIP-190: Simplify Pulsar > documentation > >>> release and maintenance strategy. > >>> > >>> Proposal link: https://github.com/apache/pulsar/issues/16637. > >>> > >>> > >>> > - > >>> > >>> ## Motivation > >>> > >>> This proposal is focused on improving and simplifying the release and > >>> maintenance strategy of the existing Apache Pulsar documentation. > >>> > >>> In general, this proposal provides equal values and a better user > >>> experience without sacrificing unpredictable efforts in the release and > >>> maintenance process. The benefits include: > >>> * Improve user experience when they look for the documentation of a > >>> specific release. > >>> * Optimize the doc development and release process for bug-fix > releases: > >>> * Turn doc development for bug-fix releases from post-release into > >>> just-in-time. > >>> * Save Release Manager’s effort in generating doc files for bug-fix > >>> releases. > >>> * Save a great amount of the community’s effort in syncing doc > >>> improvements and fixes to historical doc versions that are in the > >>> maintenance cycle. > >>> > >>> > >>> Since there are quite a few illustrations and links in this proposal, I > >>> didn't copy and paste the complete content into this email thread. You > can > >>> access the GitHub link to review the proposal with a better view. > >>> > >>> > >>> Cheers! > >>> > >>> momo-jun > >>> > >>> > >> > >
Re: 回复: [Discuss] PIP-190: Simplify Pulsar documentation release and maintenance strategy
Yu Il Mer 27 Lug 2022, 05:33 Liu Yu ha scritto: > +1 for this proposal since it improves the efficiency of managing docs. > > We'll take it as we reach a lazy consensus and implement this plan if > there is no objection until tomorrow. > This is a PIP so we must wait for a explicit approval by running a VOTE. If you wanted to go with lazy consensus then we shouldn't have started this thread Enrico > On 2022/07/25 09:13:50 M Jun wrote: > > Hi Asaf, > > > > The doc files for all older versions are independent copies, stored at > https://github.com/apache/pulsar/tree/master/site2/website/versioned_docs. > > > > For example, the doc for 2.10.0 was a copy saved from the next (master) > doc at that milestone. > > > > 发件人: Asaf Mesika > > 发送时间: 2022年7月24日 21:00 > > 收件人: dev@pulsar.apache.org > > 主题: Re: [Discuss] PIP-190: Simplify Pulsar documentation release and > maintenance strategy > > > > If I understand correctly, using tags, you can automatically create the > > docs for the next minor version? > > So for older minor versions, where those files will be at? using tags > from > > git? > > > > On Sat, Jul 23, 2022 at 7:04 AM Ma Jun wrote: > > > > > Hi, Pulsar community, > > > > > > Happy weekend! > > > > > > I'd like to open a discussion about PIP-190: Simplify Pulsar > documentation > > > release and maintenance strategy. > > > > > > Proposal link: https://github.com/apache/pulsar/issues/16637. > > > > > > > > > > - > > > > > > ## Motivation > > > > > > This proposal is focused on improving and simplifying the release and > > > maintenance strategy of the existing Apache Pulsar documentation. > > > > > > In general, this proposal provides equal values and a better user > > > experience without sacrificing unpredictable efforts in the release and > > > maintenance process. The benefits include: > > > * Improve user experience when they look for the documentation of a > > > specific release. > > > * Optimize the doc development and release process for bug-fix > releases: > > >* Turn doc development for bug-fix releases from post-release into > > > just-in-time. > > >* Save Release Manager’s effort in generating doc files for bug-fix > > > releases. > > > * Save a great amount of the community’s effort in syncing doc > > > improvements and fixes to historical doc versions that are in the > > > maintenance cycle. > > > > > > > > > Since there are quite a few illustrations and links in this proposal, I > > > didn't copy and paste the complete content into this email thread. You > can > > > access the GitHub link to review the proposal with a better view. > > > > > > > > > Cheers! > > > > > > momo-jun > > > > > > > > >
Re: [DISCUSS] PIP-195: New bucket based delayed message tracker
Good proposal. How do you enable this feature? A flag in broker.conf? Is it possible to enable the feature by doing a rolling upgrade of the brokers? What happens if you have brokers that are still running the old tracker? Is it possible to rollback seamlessly? Enrico Il Mer 27 Lug 2022, 05:39 Dezhi Liu ha scritto: > +1, good job. > > Thanks, > Dezhi > > On 2022/07/25 02:22:00 Cong Zhao wrote: > > Hello Pulsar Community, > > > > > > I would like to open a discussion here about PIP-195 : New bucket based > > delayed message tracker. I look forward to your feedback. > > > > > > PIP: https://github.com/apache/pulsar/issues/16763 > > > > > > Thanks, > > > > Cong Zhao > > >
[GitHub] [pulsar-site] urfreespace merged pull request #147: Generate config docs from source code
urfreespace merged PR #147: URL: https://github.com/apache/pulsar-site/pull/147 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
