Maven wrapper
Hi folks, Some time ago, I opened a PR to add the maven wrapper: https://github.com/apache/pulsar/pull/10487 This has some advantages as we can select a working version of Maven and avoid having to patch it like what we had to do in the past (see https://github.com/apache/pulsar/pull/10487#issuecomment-832757816). If it's of interest, can this be merged ? If not, please comment and close the PR. Christophe
[GitHub] [pulsar-dotpulsar] blankensteiner commented on pull request #95: Add token factory support, respond to server auth challenge on token refresh
blankensteiner commented on pull request #95: URL: https://github.com/apache/pulsar-dotpulsar/pull/95#issuecomment-1027901019 Hi @goldenccargill Would you be able to give the bits in Master a run and see if it works for your scenario? :-) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar-helm-chart] lhotari closed issue #160: topic lose: helm install --values examples/values-minikube.yaml --set initialize=true --namespace pulsar-st pulsar-minikube ./charts/pulsar
lhotari closed issue #160: URL: https://github.com/apache/pulsar-helm-chart/issues/160 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar-helm-chart] lhotari commented on issue #160: topic lose: helm install --values examples/values-minikube.yaml --set initialize=true --namespace pulsar-st pulsar-minikube ./charts/pulsa
lhotari commented on issue #160: URL: https://github.com/apache/pulsar-helm-chart/issues/160#issuecomment-1027928383 I think it's intentional that `examples/values-minikube.yaml` doesn't have persistence enabled. `--set volumes.persistence=true` should resolve this issue. You also need support for PVCs. It might require installing Rancher Local Path Provisioner: https://github.com/rancher/local-path-provisioner#installation to support that in test environments. Nowadays a good alternative for minikube and Docker desktop is Rancher Desktop, which is free and open source. It includes a k8s environment based on k3s. https://rancherdesktop.io/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar-dotpulsar] blankensteiner commented on issue #92: Client usage
blankensteiner commented on issue #92: URL: https://github.com/apache/pulsar-dotpulsar/issues/92#issuecomment-1027952975 Hi @VisualBean In Master, we now have support for on-the-fly authentication. Does that fit your needs? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar-helm-chart] frankjkelly opened a new pull request #224: Allow us to turn on PodSecurityPolicy for Pulsar clusters with the sa…
frankjkelly opened a new pull request #224: URL: https://github.com/apache/pulsar-helm-chart/pull/224 …me name deployed in different K8S namespaces in the same cluster. (PodSecurityPolicy is a cluster-level-resource so we are creating new PSPs by appending the pulsar namespace) Fixes # ### Motivation * We want to be able to deploy multiple Pulsar clusters in different K8S namespaces but having the same helm release name. ### Modifications * PodSecurityPolicies are cluster-wide so to avoid clashing we need to distinguish each Pulsar Cluster PSP somehow so we appended the `pulsar.namespace` value * For backwards compatiblity we keyed off of `rbac.limit_to_namespace` which by default is `false` https://github.com/apache/pulsar-helm-chart/blob/master/charts/pulsar/values.yaml#L93 ### Verifying this change - [ ] Make sure that the change passes the CI checks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [DISCUSS] PIP-136: Sync Pulsar policies across multiple clouds
Thanks for your responses. > I don't see a need of protobuf for this particular usecase If no one else feels strongly on this point, I am good with using a POJO. > It doesn't matter if it's system-topic or not because it's > configurable and the admin of the system can decide and configure it > according to the required persistent policy. On my first reading, it wasn't clear if there was only one topic required for this feature. I now see that the topic is not tied to a specific tenant or namespace. As such, we can avoid complicated authorization questions by putting the required event topic(s) into a "system" tenant and namespace, by default. The `pulsar/system` tenant and namespace seem appropriate to me. > I would keep the system topic > separate because this topic serves a specific purpose with specific schema, > replication policy and retention policy. I think we need a more formal definition for system topics. This topic is exactly the kind of topic I would call a system topic: its intended producers and consumers are Pulsar components. However, because this feature can live on a topic in a system namespace, we can avoid the classification discussion for this PIP. > Source region will have a broker which will create a failover consumer on > that topic and a broker with an active consumer will watch the metadata > changes and publish the changes to the event topic. How do we designate the host broker? Is it manual? How does it work when the host broker is removed from the cluster? If we collocate the active consumer with the broker hosting the event topic, can we skip creating the failover consumer? > PIP briefly talks about it but I will update the PIP with more > explanation. I look forward to seeing more about this design for conflict resolution. Thanks, Michael On Tue, Feb 1, 2022 at 3:01 AM Rajan Dhabalia wrote: > > Please find my response inline. > > On Mon, Jan 31, 2022 at 9:17 PM Michael Marshall > wrote: > > > I think this is a very appropriate direction to take Pulsar's > > geo-replication. Your proposal is essentially to make the > > inter-cluster configuration event driven. This increases fault > > tolerance and better decouples clusters. > > > > Thank you for your detailed proposal. After reading through it, I have > > some questions :) > > > > 1. What do you think about using protobuf to define the event > > protocol? I know we already have a topic policy event stream > > defined with Java POJOs, but since this feature is specifically > > designed for egressing cloud providers, ensuring compact data transfer > > would keep egress costs down. Additionally, protobuf can help make it > > clear that the schema is strict, should evolve thoughtfully, and > > should be designed to work between clusters of different versions. > > > > >>> I don't see a need of protobuf for this particular usecase because of > two reasons: > >> a. policy changes don't generate huge traffic which could be 1 rps b. > and it doesn't need performance optimization. > >> It should be similar as storing policy in text instead protobuf which > doesn't impact footprint size or performance due to limited number of > >> update operations and relatively less complexity. I agree that protobuf > could be another option but in this case it's not needed. Also, POJO > >> can also support schema and versioning. > > > > > > > 2. In your view, which tenant/namespace will host > > `metadataSyncEventTopic`? Will there be several of these topics or is > > it just hosted in a system tenant/namespace? This question gets back > > to my questions about system topics on this mailing list last week [0]. I > > view this topic as a system topic, so we'd need to make sure that it > > has the right authorization rules and that it won't be affected by calls > > like "clearNamespaceBacklog". > > > >> It doesn't matter if it's system-topic or not because it's > configurable and the admin of the system can decide and configure it > according to the required persistent policy. I would keep the system topic > separate because this topic serves a specific purpose with specific schema, > replication policy and retention policy. > > > > > > > 3. Which broker will host the metadata update publisher? I assume we > > want the producer to be collocated with the bundle that hosts the > > event topic. How will this be coordinated? > > > >> It's already explained into PIP in section: "Event publisher and handler" > >> Every isolated cluster deployed on a separate cloud platform will have a > source region and part of replicated clusters for the event topic. The > Source region will have a broker which will create a failover consumer on > that topic and a broker with an active consumer will watch the metadata > changes and publish the changes to the event topic. > > > > > > > 4. Why isn't a topic a `ResourceType`? Is this because the topic level > > policies already have this feature? If so, is there a way to integrate > > this feature w
[GitHub] [pulsar-dotpulsar] goldenccargill commented on pull request #95: Add token factory support, respond to server auth challenge on token refresh
goldenccargill commented on pull request #95: URL: https://github.com/apache/pulsar-dotpulsar/pull/95#issuecomment-1028698501 > Hi @goldenccargill Would you be able to give the bits in Master a run and see if it works for your scenario? :-) Hi Daniel. It looks pretty different to how I remember it. Regarding if it works I had tests there that would verify it automatically. As you've removed/changed them I can't easily say but I guess with the tests you've replaced them with you must have some confidence it works? Our scenario is no different to anyone having a token that expires, as long as it is refreshed then that will work for us -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
