Re: [VOTE] PIP-412: Support setting messagePayloadProcessor in Pulsar Functions and Sinks

Hello all, The vote to PIP-412: Support setting messagePayloadProcessor in Pulsar Functions and Sinks is now closed. The vote PASSED with 4 binding "+1" votes @Penghui Li Would you please help to merge the PR? many thanks Sincerely, Pengcheng Jiang Enrico Olivelli 于2025年4月5日

[VOTE] PIP-412: Support setting messagePayloadProcessor in Pulsar Functions and Sinks

, Pengcheng Jiang -- This email and any attachments are intended solely for the recipient(s) named above and may contain confidential, privileged, or proprietary information. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution, or reproduction of

Re: [DISCUSS] PIP-412: Support setting messagePayloadProcessor in Pulsar Functions and Sinks

Thanks, I will start the vote thread. Sincerely, Pengcheng Jiang Lari Hotari 于2025年3月21日周五 23:48写道： > Thanks, LGTM. Good work on the proposal. > > -Lari > > On 2025/03/21 10:27:20 Pengcheng Jiang wrote: > > Hello Community, > > > > I opened the PIP-412 <h

[DISCUSS] PIP-412: Support setting messagePayloadProcessor in Pulsar Functions and Sinks

Hello Community, I opened the PIP-412 <https://github.com/apache/pulsar/pull/24105> to make users able to set `messagePayloadProcessor` in Pulsar Functions and Sinks, leave any comment you want, many thanks! Refers: - https://github.com/apache/pulsar/pull/24105 Sincerely, Pengcheng

dev@pulsar.apache.org

l/23793? thx~ Sincerely Pengcheng Jiang Apache Rui Fu 于2025年1月8日周三 12:51写道： > +1 (nonbinding) > > With best regards, > > Rui Fu > On Jan 7, 2025 at 11:53 +0800, dev@pulsar.apache.org, wrote: > > > > +1 nonbinding >

dev@pulsar.apache.org

ours Sincerely Pengcheng Jiang

dev@pulsar.apache.org

> pass a function that accepts a ProducerBuilder as an argument for configuration This sounds more reasonable And I agree that this should be handled in a separate PIP. Sincerely Pengcheng Jiang Lari Hotari 于2025年1月3日周五 03:45写道： > It seems that your suggestion could be one viable so

dev@pulsar.apache.org

o use distinct batching configurations, I think it's better to introduce a new newOutputMessage method in Context that accepts a producerConfig argument. This would allow users to override all producer settings, including batching configurations, HDYT? Sincerely Pengcheng Jiang Lari Hotari 于2025

dev@pulsar.apache.org

Hello community, I opened the PIP-401 <https://github.com/apache/pulsar/pull/23793> to make users able to customize the batching settings for Pulsar Functions & Sources, leave any comments you want, many thanks. Refers: - https://github.com/apache/pulsar/pull/23793 Sincerely, Pengcheng Jiang

Re: [VOTE] PIP-339: Introducing the --log-topic Option for Pulsar Sinks and Sources

@Jiwei Guo would you please help to merge the PR? many thanks Regards, Pengcheng Jiang Pengcheng Jiang 于2024年2月28日周三 09:02写道： > Close this vote with 4 +1(binding) and 2 +1(non-binding). > > mattison chao 于2024年2月27日周二 19:14写道： > >> +1 (binding) >> >> Best, >

Re: [VOTE] PIP-339: Introducing the --log-topic Option for Pulsar Sinks and Sources

; > > > > > > > > > > > > BR, > > > > > > > > Zike Yang > > > > > > > > > > > > > > > > On Tue, Feb 27, 2024 at 8:56 AM PengHui Li > wrote: > > > > > > > > > > > > &g

Re: [DISCUSS] PIP-338: Introducing the --log-topic Option for Pulsar Sinks and Sources

ty(FQN, fqn) .sendAsync(); } ``` Regards, Pengcheng Jiang PengHui Li 于2024年2月26日周一 09:48写道： > Do we need to take the authorization into account? > Users might need to take one more step to grant permission to access the > log topic. > It is better to mention it in the proposal. > >

[VOTE] PIP-339: Introducing the --log-topic Option for Pulsar Sinks and Sources

Hi, community I'm starting the vote for PIP-339: Introducing the --log-topic Option for Pulsar Sinks and Sources PIP link: https://github.com/apache/pulsar/pull/22071 Thanks, Pengcheng Jiang

Re: [DISCUSS] PIP-338: Introducing the --log-topic Option for Pulsar Sinks and Sources

Sorry, the PIP number should be PIP-339 Pengcheng Jiang 于2024年2月19日周一 17:30写道： > Dear community, > > I created a PIP to support `--log-topic` for Pulsar Sinks and Sources: > https://github.com/apache/pulsar/pull/22071 > > It will make Pulsar Functions and Connectors have the

[DISCUSS] PIP-338: Introducing the --log-topic Option for Pulsar Sinks and Sources

Dear community, I created a PIP to support `--log-topic` for Pulsar Sinks and Sources: https://github.com/apache/pulsar/pull/22071 It will make Pulsar Functions and Connectors have the same way to manage their logs. Any feedback and suggestions are welcome. Sincerely Pengcheng Jiang

Re: [VERIFY] Pulsar Release 3.2.0 Candidate 2

> 1. The querystate exit with `Reason: key 'hello' doesn't exist`, which the old version will not exit there is a bug for it, a 500 error is returned instead of a 404 error, there is the fix: https://github.com/apache/pulsar/pull/21921 > The output missed the `version` field In PIP-312

Re: [VOTE] PIP-312 Use StateStoreProvider to manage state in Pulsar Functions endpoints

typo, should be: Close this vote with 3 +1(binding) and 5 +1(non-binding). Pengcheng Jiang 于2023年11月20日周一 09:13写道： > Close this vote with 3 +1(binding) and 1 +5(non-binding). > > guo jiwei 于2023年11月20日周一 09:07写道： > >> +1 binding >> >> >> Regards >>

Re: [VOTE] PIP-312 Use StateStoreProvider to manage state in Pulsar Functions endpoints

@Jiwei Guo would you please help to merge the PR? many thanks Pengcheng Jiang 于2023年11月20日周一 09:13写道： > Close this vote with 3 +1(binding) and 1 +5(non-binding). > > guo jiwei 于2023年11月20日周一 09:07写道： > >> +1 binding >> >> >> Regards >> Jiwei Guo

Re: [VOTE] PIP-312 Use StateStoreProvider to manage state in Pulsar Functions endpoints

56写道： > > > > > +1 binding > > > > > > Thanks for your proposal. > > > > > > On 2023/11/15 03:39:42 Pengcheng Jiang wrote: > > > > Hi Pulsar Community, > > > > > > > > This thread is to start a vote for PIP-312:

[VOTE] PIP-312 Use StateStoreProvider to manage state in Pulsar Functions endpoints

://lists.apache.org/thread/0rz29wotonmdck76pdscwbqo19t3rbds Sincerely, Pengcheng Jiang

[DISCUSS] PIP-312 Use StateStoreProvider to manage state in Pulsar Functions endpoints

7;s state store from BookKeeper and enable it to use other state store backends. Any feedback and suggestions are welcome. Sincerely Pengcheng Jiang

Re: Has anyone EVER gotten a Python function to work with Avro??

\ --className avro_schema_test_function.AvroSchemaTestFunction \ --schema-type avro \ --input-type-class-name avro_schema_test_function.AvroTestObject \ --output-type-class-name avro_schema_test_function.AvroTestObject ``` Sincerely Pengcheng Jiang Devin Bost 于2023年6月21日周三 08:35写道： > After many of my

[VOTE] PIP-272

Hello, community: This thread is to start a vote for PIP-272: Add stateStorageConfig to WorkerConfig. Discussion thread: https://lists.apache.org/thread/pwfv7nj64frfnbw7jfydzx8my15b3lj6 PR: https://github.com/apache/pulsar/pull/20455 Sincerely Pengcheng Jiang

Re: [DISCUSS] PIP-272 Add a `StateStoreConfig` to the `WorkerConfig`

pr is created: https://github.com/apache/pulsar/pull/20455 Pengcheng Jiang 于2023年6月1日周四 15:49写道： > Sure, I will create a PR and update the issue when development is done > > Asaf Mesika 于2023年5月31日周三 16:39写道： > >> Pengcheng, would you be willing to be the inaugural PIP in our

Re: [DISCUSS] PIP-272 Add a `StateStoreConfig` to the `WorkerConfig`

hub.com/apache/pulsar/blob/master/pip/README.md > > Thanks! > > Asaf > > > On Wed, May 31, 2023 at 12:55 AM Neng Lu wrote: > > > thanks for the improvements, +1 > > > > On Tue, May 30, 2023 at 2:20 AM Pengcheng Jiang > > wrote: > > >

Re: [DISCUSS] PIP-272 Add a `StateStoreConfig` to the `WorkerConfig`

in pip 8. the `stateStorageServiceUrl` won't be touched Sincerely Pengcheng Jiang Asaf Mesika 于2023年5月29日周一 19:53写道： > Hi Pengcheng, > > Looks like a solid improvement, definitely helping people using their own > state store. > > I have a few comments: > > 1. Backgr

[DISCUSS] PIP-272 Add a `StateStoreConfig` to the `WorkerConfig`

Dear Pulsar community, I created a pip to make pulsar functions' `StateStoreProvider` configurable with custom configurations: https://github.com/apache/pulsar/issues/20419 Any feedback and suggestions are welcome Sincerely Pengcheng Jiang

Re: [DISCUSS] Improve Pulsar Function Source Primitive Schema Mapping

Hello Neng, IMO, we should update code[2] to follow the doc, and for existing functions, if they are in running status, they won't touch code[2]; and for a new run, functions will fail to start, and this will remind users to update their function Regards, Pengcheng Jiang Neng Lu 于2023年4月

Re: [DISCUSS] Separate the Python Functions from the installation of the Python client

> I'm not familiar with the Pulsar Functions, so feel free to show your > > > suggestions if any of you have any concerns. > > > > > > [1] https://github.com/apache/pulsar/pull/17881 > > > [2] http://github.com/apache/pulsar-client-python > > > [3] > https://github.com/apache/pulsar/tree/master/pulsar-functions/instance/src/main/python > > > [4] https://pulsar.apache.org/docs/2.11.x/client-libraries-python/ > > > [5] > https://github.com/apache/pulsar-client-python/blob/a6476d9c45508f55a7af4b25001038a8e3a27489/setup.py#L80-L88 > > > > > > Thanks, > > > Yunze > > > > -- <https://streamnative.io/> Pengcheng Jiang Software Engineer e: pengcheng.ji...@streamnative.io p: 13540631948 streamnative.io <http://github.com/streamnative> <https://www.linkedin.com/company/streamnative/> <https://twitter.com/streamnativeio/>

Re: [DISCUSS] Support custom compressionType for pulsar functions

backward compatibility. > > Best, > > Rui Fu > On Mar 1, 2023 at 10:18 +0800, Neng Lu , wrote: > > +1 for the change. > > > > On 2023/02/28 01:06:51 Pengcheng Jiang wrote: > > > Hello, community: > > > > > > ### Motivation > > > &g

[DISCUSS] Support custom compressionType for pulsar functions

pulsar function instances will use `LZ4` to keep the same behavior with before. PTAL when you have time and feel free to leave any comments. Best Regards, Pengcheng Jiang [0] https://github.com/apache/pulsar/pull/19470 -- <https://streamnative.io/> Pengcheng Jiang Software Engi