Re: [VOTE] Pulsar Release 2.9.2 Candidate 2

[PengHui Li]

Now, there is a regression introduced in 2.9.2

I have pushed out the fix https://github.com/apache/pulsar/pull/14231, PTAL.

-1 from my side

Need to get the fix merged and roll out the new RC3 @Ran

Regards,
Penghui

On Thu, Feb 10, 2022 at 9:54 PM Nicolò Boschi  wrote:

> Penghui,
>
>
> I didn't know that there were so many known bugs around transactions
> scheduled for 2.9.3, my bad.
>
> However, as Enrico pointed out, the issue impacts Pulsar clients that are
> not using the transactions, so we can't just say - ok, just another bug
> about transactions, it's not critical since they're not production ready
> (btw, where we state that they aren't production ready on the
> documentation?).
>
>
> The workaround you mentioned is not always viable, since you can have
> clients of different tenants/customers that are not using transactions
> while, at the same time, a little portion that are experiencing with them.
>
> I agree that it is uncommon to have only one message produced. On the other
> hand, it's a very common case where other projects using Pulsar have
> unit/integration tests that write only one message and expect to be
> consumed (that's because they test the application logic and not Pulsar).
>
>
> Given that, it's fair to say that 2.9.2 is not worse than 2.9.1, so,
> finally, we can go ahead.
>
> Looking forward to see 2.9.3 soon
>
>
> I tested the artifacts, so I'll put my vote here:
>
>
> +1 (non binding)
>
>
> Checks:
>
> - Checksum and signatures
>
> - Apache Rat check passes
>
> - Compile from source w JDK11
>
> - Build docker image from source
>
> - Run Pulsar standalone and produce-consume from CLI
>
>
> BR,
>
> Nicolò
>
> Il giorno gio 10 feb 2022 alle ore 13:39 PengHui Li 
> ha
> scritto:
>
> > > Please go ahead with the release, I won't VOTE on this thread.
> > But I hope we can follow up soon with a new release, otherwise due to
> that
> > bug
> > you cannot enable transactions on your Pulsar cluster if you have to
> > support Pulsar client that do not enable transactions
> >
> >
> > Yes, agree. We will follow up the 2.9.3 soon. There are other
> > ongoing transaction fixes
> > we will complete them ASAP and provide a version with certain guarantees
> > for transaction stability.
> > We are doing lots of tests these days, 2.9.3 should be a good version for
> > transactions.
> >
> > Thanks,
> > Penghui
> >
> >
> > On Thu, Feb 10, 2022 at 7:37 PM Lin Lin  wrote:
> >
> > >
> > >
> > > +1(binding)
> > >
> > > 1. Checked the signature
> > > 2. Start standalone
> > > 3. Publish and consume successfully
> > > 4. Checked function
> > >
> >
>
>
> --
> Nicolò Boschi
>

[GitHub] [pulsar-helm-chart] eolivelli commented on a change in pull request #223: Improve Zookeeper "ruok" probes: use TLS port when TLS is enabled, specify "-q 1" for nc

[GitBox]

eolivelli commented on a change in pull request #223:
URL: https://github.com/apache/pulsar-helm-chart/pull/223#discussion_r804508083



##
File path: charts/pulsar/templates/zookeeper-statefulset.yaml
##
@@ -151,7 +157,9 @@ spec:
 command:
 - timeout
 - "{{ .Values.zookeeper.probe.readiness.timeoutSeconds }}"
-- bin/pulsar-zookeeper-ruok.sh
+- bash

Review comment:
   what about adding a comment about the fact that we are no more using 
"bin/pulsar-zookeeper-ruok.sh" ?
   
   




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [pulsar-helm-chart] lhotari commented on a change in pull request #223: Improve Zookeeper "ruok" probes: use TLS port when TLS is enabled, specify "-q 1" for nc

[GitBox]

lhotari commented on a change in pull request #223:
URL: https://github.com/apache/pulsar-helm-chart/pull/223#discussion_r804576961



##
File path: charts/pulsar/templates/zookeeper-statefulset.yaml
##
@@ -151,7 +157,9 @@ spec:
 command:
 - timeout
 - "{{ .Values.zookeeper.probe.readiness.timeoutSeconds }}"
-- bin/pulsar-zookeeper-ruok.sh
+- bash

Review comment:
   Where did you mean adding a comment?




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [pulsar-dotpulsar] GeroL opened a new issue #96: Expose Redelivery Count By Message Properties

[GitBox]

GeroL opened a new issue #96:
URL: https://github.com/apache/pulsar-dotpulsar/issues/96


   Basically same as here https://github.com/apache/pulsar/issues/3030 
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [pulsar-dotpulsar] GeroL closed issue #96: Expose Redelivery Count By Message Properties

[GitBox]

GeroL closed issue #96:
URL: https://github.com/apache/pulsar-dotpulsar/issues/96


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [pulsar-dotpulsar] GeroL commented on issue #96: Expose Redelivery Count By Message Properties

[GitBox]

GeroL commented on issue #96:
URL: 
https://github.com/apache/pulsar-dotpulsar/issues/96#issuecomment-1036144623


   Sorry. found it on the IMessage interface


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[VOTE] Pulsar Release 2.7.5 Candidate 1

[Lari Hotari]

This is the first release candidate for Apache Pulsar, version 2.7.5.

It contains the following commits after the previous release:
https://github.com/apache/pulsar/compare/v2.7.4...v2.7.5-candidate-1

*** Please download, test and vote on this release. This vote will stay open
for at least 72 hours ***

Note that we are voting upon the source (tag), binaries are provided for
convenience.

Source and binary files:
https://dist.apache.org/repos/dist/dev/pulsar/pulsar-2.7.5-candidate-1/

SHA-512 checksums:

b0d4e8c05870e0fe8cb50a80e15811b59972fc7f1d3fa93fa6cf2b61797176e80b434334aa67a5619de2a46a6ae9e51286dad524cac7af76239cda790d3574cc
  apache-pulsar-2.7.5-bin.tar.gz
e9a88d6847828fdb051fe2a4663e5b66beb0a066e9e170682df4e0330fe5d59d27c88908f493e2100f35a2469a8f77a97c94da752aac320bf4413aaed57570d6
  apache-pulsar-2.7.5-src.tar.gz

Maven staging repo:
https://repository.apache.org/content/repositories/orgapachepulsar-1138/

The tag to be voted upon:
v2.7.5-candidate-1 (db8761ebb370db1ae731a807afb583ac346378fe)
https://github.com/apache/pulsar/releases/tag/v2.7.5-candidate-1

Pulsar's KEYS file containing PGP keys we use to sign the release:
https://dist.apache.org/repos/dist/dev/pulsar/KEYS

Please download the source package, and follow the README to build
and run the Pulsar standalone service.

[GitHub] [pulsar-dotpulsar] GeroL commented on pull request #97: Add crude first version of DLQ

[GitBox]

GeroL commented on pull request #97:
URL: https://github.com/apache/pulsar-dotpulsar/pull/97#issuecomment-1036395632


   This is a first version as I do not have any more time this week. Will work 
on it later again.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [pulsar-dotpulsar] GeroL opened a new issue #98: DeadLetterQueue handling in consumer

[GitBox]

GeroL opened a new issue #98:
URL: https://github.com/apache/pulsar-dotpulsar/issues/98


   Let the consumer handle dead letters 
   
   First design: #97 
   
   Feel free to add comments. Will work on it again after next week.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

[GitHub] [pulsar-helm-chart] eolivelli commented on a change in pull request #223: Improve Zookeeper "ruok" probes: use TLS port when TLS is enabled, specify "-q 1" for nc

[GitBox]

eolivelli commented on a change in pull request #223:
URL: https://github.com/apache/pulsar-helm-chart/pull/223#discussion_r804843537



##
File path: charts/pulsar/templates/zookeeper-statefulset.yaml
##
@@ -151,7 +157,9 @@ spec:
 command:
 - timeout
 - "{{ .Values.zookeeper.probe.readiness.timeoutSeconds }}"
-- bin/pulsar-zookeeper-ruok.sh
+- bash

Review comment:
   Here in this line
   It is strange that we are not using the script we provide with Pulsar.
   
   I understand the reason but it won't be clear to people reading this code




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@pulsar.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Re: [VOTE] Pulsar Release 2.7.5 Candidate 1

[Nicolò Boschi]

+1 (non binding)


Checks:

- Checksum and signatures

- Apache Rat check passes

- Compile from source w JDK11

- Build docker image from source

- Run Pulsar standalone and produce-consume from CLI

Il giorno ven 11 feb 2022 alle ore 14:47 Lari Hotari 
ha scritto:

> This is the first release candidate for Apache Pulsar, version 2.7.5.
>
> It contains the following commits after the previous release:
> https://github.com/apache/pulsar/compare/v2.7.4...v2.7.5-candidate-1
>
> *** Please download, test and vote on this release. This vote will stay
> open
> for at least 72 hours ***
>
> Note that we are voting upon the source (tag), binaries are provided for
> convenience.
>
> Source and binary files:
> https://dist.apache.org/repos/dist/dev/pulsar/pulsar-2.7.5-candidate-1/
>
> SHA-512 checksums:
>
> b0d4e8c05870e0fe8cb50a80e15811b59972fc7f1d3fa93fa6cf2b61797176e80b434334aa67a5619de2a46a6ae9e51286dad524cac7af76239cda790d3574cc
> apache-pulsar-2.7.5-bin.tar.gz
> e9a88d6847828fdb051fe2a4663e5b66beb0a066e9e170682df4e0330fe5d59d27c88908f493e2100f35a2469a8f77a97c94da752aac320bf4413aaed57570d6
> apache-pulsar-2.7.5-src.tar.gz
>
> Maven staging repo:
> https://repository.apache.org/content/repositories/orgapachepulsar-1138/
>
> The tag to be voted upon:
> v2.7.5-candidate-1 (db8761ebb370db1ae731a807afb583ac346378fe)
> https://github.com/apache/pulsar/releases/tag/v2.7.5-candidate-1
>
> Pulsar's KEYS file containing PGP keys we use to sign the release:
> https://dist.apache.org/repos/dist/dev/pulsar/KEYS
>
> Please download the source package, and follow the README to build
> and run the Pulsar standalone service.
>


-- 
Nicolò Boschi

Re: Architecture of function authorization for process mode

[Devin Bost]

Thanks for all the feedback on this.
So, I'm looking into running functions in the Kubernetes runtime, and I'm
seeing another potential issue in terms of restricting function
authorization scope.

The function worker reads the token from the pulsar-admin call's
Authorization header when the function is created (
https://github.com/apache/pulsar/blob/8496afc58bdd27c47cde8a9ba3c76b80ab796320/pulsar-functions/worker/src/main/java/org/apache/pulsar/functions/worker/rest/api/FunctionsImpl.java#L207)
and saves it as the Kubernetes secret (
https://github.com/apache/pulsar/blob/1ea381d02bf2c817547b4759b0dbf57366fd1358/pul[…]e/pulsar/functions/auth/KubernetesSecretsTokenAuthProvider.java
).
When the function starts, it uses that secret for broker authentication.
The problem is that the pulsar-admin create action requires the token to
have a subject that matches an adminRole specified on that tenant (
https://github.com/apache/pulsar/blob/7576a6594233f3ac9e20028db12ec731bd485a68/pulsar-functions/worker/src/main/java/org/apache/pulsar/functions/worker/rest/api/ComponentImpl.java#L1472).
So, the role used to create the function must be an admin on the tenant,
but that role is then inherited and assigned to the function that's
created. So, every function in the Kubernetes runtime would at least have
admin privilege within its tenant.

Is my understanding correct? Is there a way around this?

Devin G. Bost


On Tue, Jan 25, 2022 at 4:42 PM Niclas Hedhman  wrote:

> On 2022-01-25 08:57, Matteo Merli wrote:
> > The only recommended way to run a multi-tenant Pulsar functions
> > clusters is to run it with Kubernetes runtime.
> >
> > In thread or process runtime, there is no reliable way to restrict the
> > access to the credentials of each function instance (since it needs to
> > be readable by the same unix user), or for what it matters, to
> > restrict the resources that this function has access to (eg: cpu,
> > memory, network, disk..).
>
> Thank you, that helps a lot.
>
> Niclas
>

Re: Architecture of function authorization for process mode

[Matteo Merli]

You don't need tenant admin access to create functions, you just need
to give "functions" access to a normal user:

pulsar-admin namespaces grant-permission --actions
produce,consume,functions --role $MY_PRINCIPAL



--
Matteo Merli

On Fri, Feb 11, 2022 at 5:46 PM Devin Bost  wrote:
>
> Thanks for all the feedback on this.
> So, I'm looking into running functions in the Kubernetes runtime, and I'm
> seeing another potential issue in terms of restricting function
> authorization scope.
>
> The function worker reads the token from the pulsar-admin call's
> Authorization header when the function is created (
> https://github.com/apache/pulsar/blob/8496afc58bdd27c47cde8a9ba3c76b80ab796320/pulsar-functions/worker/src/main/java/org/apache/pulsar/functions/worker/rest/api/FunctionsImpl.java#L207)
> and saves it as the Kubernetes secret (
> https://github.com/apache/pulsar/blob/1ea381d02bf2c817547b4759b0dbf57366fd1358/pul[…]e/pulsar/functions/auth/KubernetesSecretsTokenAuthProvider.java
> ).
> When the function starts, it uses that secret for broker authentication.
> The problem is that the pulsar-admin create action requires the token to
> have a subject that matches an adminRole specified on that tenant (
> https://github.com/apache/pulsar/blob/7576a6594233f3ac9e20028db12ec731bd485a68/pulsar-functions/worker/src/main/java/org/apache/pulsar/functions/worker/rest/api/ComponentImpl.java#L1472).
> So, the role used to create the function must be an admin on the tenant,
> but that role is then inherited and assigned to the function that's
> created. So, every function in the Kubernetes runtime would at least have
> admin privilege within its tenant.
>
> Is my understanding correct? Is there a way around this?
>
> Devin G. Bost
>
>
> On Tue, Jan 25, 2022 at 4:42 PM Niclas Hedhman  wrote:
>
> > On 2022-01-25 08:57, Matteo Merli wrote:
> > > The only recommended way to run a multi-tenant Pulsar functions
> > > clusters is to run it with Kubernetes runtime.
> > >
> > > In thread or process runtime, there is no reliable way to restrict the
> > > access to the credentials of each function instance (since it needs to
> > > be readable by the same unix user), or for what it matters, to
> > > restrict the resources that this function has access to (eg: cpu,
> > > memory, network, disk..).
> >
> > Thank you, that helps a lot.
> >
> > Niclas
> >