[GitHub] [pulsar-dotpulsar] blankensteiner merged pull request #95: Add token factory support, respond to server auth challenge on token refresh
blankensteiner merged pull request #95: URL: https://github.com/apache/pulsar-dotpulsar/pull/95 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar-dotpulsar] blankensteiner commented on pull request #95: Add token factory support, respond to server auth challenge on token refresh
blankensteiner commented on pull request #95: URL: https://github.com/apache/pulsar-dotpulsar/pull/95#issuecomment-1019310909 Hi @goldenccargill I really appreciate the PR, thanks! I'll just go ahead and merge it. Then I'll do some minor changes and next week we can create a new release :-) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar-helm-chart] michaeljmarshall merged pull request #113: Add multi volume support in bookkeeper.
michaeljmarshall merged pull request #113: URL: https://github.com/apache/pulsar-helm-chart/pull/113 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar-helm-chart] michaeljmarshall closed issue #112: Support multi volume in bookkeeper
michaeljmarshall closed issue #112: URL: https://github.com/apache/pulsar-helm-chart/issues/112 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar-helm-chart] michaeljmarshall commented on a change in pull request #155: Add grafana data persistence support.
michaeljmarshall commented on a change in pull request #155: URL: https://github.com/apache/pulsar-helm-chart/pull/155#discussion_r790222142 ## File path: charts/pulsar/values.yaml ## @@ -988,6 +988,24 @@ grafana: requests: memory: 250Mi cpu: 0.1 + volumes: +# use a persistent volume or emptyDir +persistence: true Review comment: I agree there is a chance for confusion. However, the official Grafana helm chart disables persistence by default, so I think we should too. https://github.com/grafana/helm-charts/blob/2a422f6c43c3db33d31fc12d23957d5b6ff5e8c6/charts/grafana/values.yaml#L275-L277 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar-manager] michaeljmarshall commented on issue #441: Support HerdDB in the standard docker image.
michaeljmarshall commented on issue #441: URL: https://github.com/apache/pulsar-manager/issues/441#issuecomment-1019416957 @mattisonchao - what do you think about making it configurable to skip the spawning of any database process in the docker image? Then, in the Pulsar Helm Chart, we could add configuration to make it possible to add the database as a container in the Pulsar Manager's pod. That allows for better visibility into processes that the pod is running and it would also make it the database a bit more configurable. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar-helm-chart] dragonls commented on a change in pull request #155: Add grafana data persistence support.
dragonls commented on a change in pull request #155: URL: https://github.com/apache/pulsar-helm-chart/pull/155#discussion_r790233237 ## File path: charts/pulsar/values.yaml ## @@ -988,6 +988,24 @@ grafana: requests: memory: 250Mi cpu: 0.1 + volumes: +# use a persistent volume or emptyDir +persistence: true Review comment: I agree, change default persistence value to false. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [DISCUSSION] PIP-124: Create init subscription before sending message to DLQ
> users will not touch the metadata of the producer of a DLQ This is true of the DLQ producer, but by using the metadata map, won't we be exposing this feature to end users implicitly because they could add the field to the metadata map for other producers? (I am okay with giving users this functionality, but I know that Matteo said in a community meeting that this feature should be limited to the DLQ producer.) > we also introduce some system message properties such as > __original_message_id > in retry letter topic. Thanks for this context. I didn't know we were already doing this. Thanks, Michael On Fri, Jan 21, 2022 at 3:35 AM Zike Yang wrote: > > +1 for adding init_subscription filed to the metadata of CommandProducer. > > > > On Thu, Jan 20, 2022 at 2:52 PM PengHui Li wrote: > > > > > What is the justification for avoiding the new protobuf field? If we > > add a structured field to a map of , we are still > > modifying the protocol, even if we aren't modifying the protobuf. > > > > Not all cases need a new field named init_subscription when creating > > producer, > > and users will not touch the metadata of the producer of a DLQ, so I think > > we can use the metadata to achieve the purpose more flexibly. > > > > Yes, it modifies the protocol, but in different ways. It like the message > > properties, > > we also introduce some system message properties such as > > __original_message_id > > in retry letter topic. > > > > Penghui > > > > On Thu, Jan 20, 2022 at 2:05 PM Michael Marshall > > wrote: > > > > > > I think that, good or bad, the impression that users have that the DLQ > > > > is not a "normal" topic > > > > > > Thanks for your perspective, Matteo. I still prefer my alternative > > > design that bypasses subscription creation, but it seems there > > > is insufficient interest in it, so we should move forward > > > discussing a DLQ specific feature and its implementation. > > > > > > > 1. Instead of modifying the current protocol, we can use producer > > > > metadata to carry the init subscription > > > > > > What is the justification for avoiding the new protobuf field? If we > > > add a structured field to a map of , we are still > > > modifying the protocol, even if we aren't modifying the protobuf. > > > > > > Thanks, > > > Michael > > > > > > > > > On Tue, Jan 18, 2022 at 8:38 AM PengHui Li wrote: > > > > > > > > +1 for adding the DLQ_init_sub to producer metadata so that we don't > > > > need to introduce a new field in CommandProducer, and the new field > > > > looks a little confusing > > > > > > > > Thanks, > > > > Penghui > > > > > > > > On Mon, Jan 17, 2022 at 10:19 PM Hang Chen wrote: > > > > > > > > > Thanks for creating this proposal Zike Yang. I have two ideas about > > > > > it. > > > > > 1. Instead of modifying the current protocol, we can use producer > > > > > metadata to carry the init subscription > > > > > 2. Please add auth for subscription creation when create topic by > > > > > producer, otherwise, it will be easily attacked. > > > > > > > > > > Thanks, > > > > > Hang > > > > > > > > > > Matteo Merli 于2022年1月12日周三 15:13写道： > > > > > > > > > > > > > If we want to hold that the DLQ is not a normal topic, then I can > > > see > > > > > > > why we would have a DLQ specific feature here. > > > > > > > > > > > > I think that, good or bad, the impression that users have that the > > > DLQ > > > > > > is not a "normal" topic comes from 2 factors: > > > > > > 1. The experience with traditional messaging systems JMS and others > > > > > > where the DLQ are handled in slightly different ways, compared to > > > > > > other topics > > > > > > 2. The name "DLQ" which in a way it's implying a "queue"... which > > > can > > > > > > be implemented on topic, using a subscription.. > > > > > > > > > > > > -- > Zike Yang
Re: [DISCUSS] [Transaction] Clear namespace backlog can't clear system topic and system sub backlog.
This is a really important thread. I think our system topic design
needs more definition, structure, and documentation. Thanks for
starting the thread, Bo Cong.
> If the user uses it, it means that the user has considered the result of the
> clear system sub backlog
I think we need to discuss authorization for operating on system
topics, especially when topic level policies are not enabled.
>From reading the PIP [0], it looks like we already have many different
ways to designate a system topic (prefixes, suffixes, and total string
equality). I am concerned that this design will not scale and that it
will be very expensive on installations with many topics.
Is it possible for us to pivot our implementation so that system
topics and only system topics start with two underscores? This could
simplify system topic authorization. It would also improve the speed
of operations on namespaces because filtering out system topics would
only require a quick check for `startsWith("__")`.
While it's possible that users already have topics that start with
"__", I think it is unlikely, and I think it is essential that we
design system topics so that they are highly scalable.
Thanks,
Michael
[0] https://github.com/apache/pulsar/issues/13794
On Thu, Jan 13, 2022 at 10:27 AM PengHui Li wrote:
>
> I agree with the point, we should avoid the `clearNamespaceBacklog(String
> namespace)` to apply to the internal topic or internal cursor.
> It will introduce abnormal behaviors, e.g. clear a replication cursor
> backlog, clear a dedup cursor backlog, clear a compaction cursor backlog.
>
> I would suggest letting the `clearNamespaceBacklog(String namespace)`
> method does not apply to the Pulsar internal topics and cursors.
>
> Thanks,
> Penghui
>
> On Thu, Jan 13, 2022 at 4:32 PM 丛搏 wrote:
>
> > Hi :
> >
> > issue link : https://github.com/streamnative/support-tickets/issues/408 <
> > https://github.com/streamnative/support-tickets/issues/408>
> >
> > # Background
> > Now we have many clear namespace backlog interfaces etc.
> > ```
> > clearNamespaceBacklog(String namespace)
> >
> > clearNamespaceBacklogAsync(String namespace)
> >
> > clearNamespaceBacklogForSubscription(String namespace, String subscription)
> >
> > clearNamespaceBacklogForSubscriptionAsync(String namespace, String
> > subscription)
> >
> > clearNamespaceBundleBacklog(String namespace, String bundle)
> >
> > clearNamespaceBundleBacklogAsync(String namespace, String bundle)
> >
> > clearNamespaceBundleBacklogForSubscription(String namespace, String
> > bundle, String subscription)
> >
> > clearNamespaceBundleBacklogForSubscriptionAsync(String namespace, String
> > bundle, String subscription)
> > ```
> > There are two types of interfaces:
> > 1. clear namespace backlog with subscriptionName
> > 2. clear namespace backlog no subscriptionName
> >
> > But there have a problem, users do not know that there are many system
> > topics in the namespace.
> >
> > ```
> >
> > /**
> > * Local topic name for the namespace events.
> > */
> > public static final String NAMESPACE_EVENTS_LOCAL_NAME =
> > "__change_events";
> >
> > /**
> > * Local topic name for the transaction buffer snapshot.
> > */
> > public static final String TRANSACTION_BUFFER_SNAPSHOT =
> > "__transaction_buffer_snapshot";
> >
> > public static final String SUBSCRIPTION_NAME =
> > "transaction-buffer-sub";
> > ```
> > User only want to clear the backlog witch they own created and sub. But
> > now we have clear the system topic sub backlog when user don't specify
> > subName and clear system sub with a topic witch user created. It will bring
> > many problems. etc.
> > 1. clear `NAMESPACE_EVENTS_LOCAL_NAME` will clear any topic policy in this
> > namespace, but user don't know and they don't want to be cleared. If the
> > user does not understand the problems caused by this interface, it will
> > delete many topic policy configurations and is difficult to recover, which
> > will lead to unpredictable and disastrous problems.
> > 2. clear `TRANSACTION_BUFFER_SNAPSHOT` topic backlog will cause
> > transaction buffer recover incompletely.
> > 3. clear `transaction-buffer-sub` sub also will cause transaction buffer
> > recover incompletely.
> > 4. if add new system topic and system sub, we should think about this
> > problem.
> >
> > # Motivation
> > We want to solve the above problems, but we can't make users unable to
> > delete the backlog of system topic and system sub.
> > So, we should solve types of interface
> > There are two types of interfaces:
> > 1. clear namespace backlog with subscriptionName
> > 2. clear namespace backlog no subscriptionName
> >
> > when clear namespace backlog no subscriptionName:
> > It represents user know the sub name is system subName and really want to
> > delete it, We should not block the user clear system sub backlog and do not
> > allow the user to create system sub. If the user uses it, it means that the
> > user has co
[GitHub] [pulsar-helm-chart] dragonls commented on pull request #113: Add multi volume support in bookkeeper.
dragonls commented on pull request #113: URL: https://github.com/apache/pulsar-helm-chart/pull/113#issuecomment-1019429492 It seems that this PR makes CI failed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar-helm-chart] lhotari merged pull request #204: Upgrade cert-manager to 1.5.4
lhotari merged pull request #204: URL: https://github.com/apache/pulsar-helm-chart/pull/204 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar-helm-chart] dragonls opened a new pull request #208: Fix ci error caused by wrong block of if clause
dragonls opened a new pull request #208: URL: https://github.com/apache/pulsar-helm-chart/pull/208 Fixes #112 #113 ### Motivation Fix ci error caused by wrong block of if clause. ### Modifications Fix the wrong block of if clause. ### Verifying this change - [ ] Make sure that the change passes the CI checks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
