[GitHub] [pulsar] anand-ftc added a comment to the discussion: raspberry 3 - pulsar-client python

2022-11-11 Thread GitBox
GitHub user anand-ftc added a comment to the discussion: raspberry 3 - pulsar-client python > @anand-ftc There is no support for arm32 (`armv7l`) in PyPI and manylinux, so > it would make it hard to distribute these wheel files. @merlimat Is it possible to help build a arm32 wheel files so th

[GitHub] [pulsar] anand-ftc added a comment to the discussion: raspberry 3 - pulsar-client python

2022-11-11 Thread GitBox
GitHub user anand-ftc added a comment to the discussion: raspberry 3 - pulsar-client python @yaalsn We have several rpi3 out in the field in several places and we cannot upgrade them now. Thanks @Demogorgon314 Our rpi runs softwares from multiple teams and changing the OS at this point migh

[GitHub] [pulsar] merlimat added a comment to the discussion: raspberry 3 - pulsar-client python

2022-11-11 Thread GitBox
GitHub user merlimat added a comment to the discussion: raspberry 3 - pulsar-client python @anand-ftc There is no support for arm32 (`armv7l`) in PyPI and manylinux, so it would make it hard to distribute these wheel files. GitHub link: https://github.com/apache/pulsar/discussions/18423#disc

Re: Data quality problem

2022-11-11 Thread Matteo Merli
Kafka does not have a schema registry to begin with. Confluent does have it. On Fri, Nov 11, 2022 at 10:32 AM Dave Fisher wrote: > > > > On Nov 11, 2022, at 6:56 AM, Elliot West > > > wrote: > > > > Hey Devin, > > > > *"Kafka conforms to the JSON Schema specification"* > > Only when using Conf

Re: [DISCUSSION] Redesign the MessageId interface

2022-11-11 Thread Michael Marshall
Great discussion. I generally agree with the conclusions. I'll add two points. Several endpoints in the topics admin api require that message ids are consistently serialized and deserialized over HTTP. For example, the reset cursor call requires a message id in a correct format or "latest" or "ea

[GitHub] [pulsar] michaeljmarshall added a comment to the discussion: struggling with pulsar functions in kubernetes

2022-11-11 Thread GitBox
GitHub user michaeljmarshall added a comment to the discussion: struggling with pulsar functions in kubernetes An actual stack trace would be very helpful. Can you share what you're observing? GitHub link: https://github.com/apache/pulsar/discussions/18332#discussioncomment-4120601 Thi

Re: [DISCUSS] PIP-211: Introduce offload throttling

2022-11-11 Thread Michael Marshall
First, I am +1 on the design proposed in this PIP. > Could you please explain that why the design will leads to unnecessary load > on the broker? > IMO, I don't think it will bring huge expenses. I did not mean the throttling design proposed by this PIP was expensive. I meant that running the of

Re: Data quality problem

2022-11-11 Thread Dave Fisher
> On Nov 11, 2022, at 6:56 AM, Elliot West > wrote: > > Hey Devin, > > *"Kafka conforms to the JSON Schema specification"* > Only when using Confluent's Schema Registry. If that is true then Apache Kafka does NOT conform while Confluent does. Can you point to some documentation? > > *"if

Re: Data quality problem

2022-11-11 Thread Elliot West
Hey Devin, *"Kafka conforms to the JSON Schema specification"* Only when using Confluent's Schema Registry. *"if a producer makes a change or omission, such as in a value used for tracking, it might not surface until way down the line"* So let me understand this: Although the producer has a schem

[GitHub] [pulsar] phijohns-tibco added a comment to the discussion: struggling with pulsar functions in kubernetes

2022-11-11 Thread GitBox
GitHub user phijohns-tibco added a comment to the discussion: struggling with pulsar functions in kubernetes Just in case this helps you out. The solution for my error was to update my version of Java 8. For me it was an invalid certificate. Notice that I said invalid not bad. The build of Ja

Re: [DISCUSS] PIP-211: Introduce offload throttling

2022-11-11 Thread Jiuming Tao
Hi Michael, > The main trade off with throttling is that we might > leave performance on the table if the cluster is not heavily utilized. It is indeed possible. When a broker don’t publish/consume many messages and offloaders are in the throttling, the performance of the broker cannot be relea

[GitHub] [pulsar] Demogorgon314 added a comment to the discussion: raspberry 3 - pulsar-client python

2022-11-11 Thread GitBox
GitHub user Demogorgon314 added a comment to the discussion: raspberry 3 - pulsar-client python @anand-ftc The raspberry 3 supports 64-bit OS, See https://www.raspberrypi.com/software/operating-systems/#raspberry-pi-os-64-bit , can you switch the system to 64-bit? GitHub link: https://gith

[GitHub] [pulsar] yaalsn added a comment to the discussion: raspberry 3 - pulsar-client python

2022-11-11 Thread GitBox
GitHub user yaalsn added a comment to the discussion: raspberry 3 - pulsar-client python @anand-ftc Or you can switch to raspberry pi 4B, it's ARM64. GitHub link: https://github.com/apache/pulsar/discussions/18423#discussioncomment-4116409 This is an automatically sent email for dev@pul

[GitHub] [pulsar] github-actions[bot] deleted a comment on the discussion: Pulsar 2.10.1 will not run on m1 macos due to java.net.MalformedURLException

2022-11-11 Thread GitBox
GitHub user github-actions[bot] deleted a comment on the discussion: Pulsar 2.10.1 will not run on m1 macos due to java.net.MalformedURLException The issue had no activity for 30 days, mark with Stale label. GitHub link: https://github.com/apache/pulsar/discussions/18433#discussioncomment-411

[GitHub] [pulsar] tisonkun added a comment to the discussion: Pulsar 2.10.1 will not run on m1 macos due to java.net.MalformedURLException

2022-11-11 Thread GitBox
GitHub user tisonkun added a comment to the discussion: Pulsar 2.10.1 will not run on m1 macos due to java.net.MalformedURLException Work for me. So transferred to Q&A that it should be an env issue. GitHub link: https://github.com/apache/pulsar/discussions/18433#discussioncomment-4115892 --

[GitHub] [pulsar] github-actions[bot] added a comment to the discussion: Pulsar 2.10.1 will not run on m1 macos due to java.net.MalformedURLException

2022-11-11 Thread GitBox
GitHub user github-actions[bot] added a comment to the discussion: Pulsar 2.10.1 will not run on m1 macos due to java.net.MalformedURLException The issue had no activity for 30 days, mark with Stale label. GitHub link: https://github.com/apache/pulsar/discussions/18433#discussioncomment-41158

[GitHub] [pulsar] yaalsn added a comment to the discussion: Pulsar 2.10.1 will not run on m1 macos due to java.net.MalformedURLException

2022-11-11 Thread GitBox
GitHub user yaalsn added a comment to the discussion: Pulsar 2.10.1 will not run on m1 macos due to java.net.MalformedURLException @LasseJacobs How about running with command `docker run --platfrom linux/amd64 -it -p 6650:6650 -p 8080:8080 --mount source=pulsardata,target=/pulsar/data --mount

[GitHub] [pulsar] tisonkun added a comment to the discussion: Add BC testing for functions CLI and function worker

2022-11-11 Thread GitBox
GitHub user tisonkun added a comment to the discussion: Add BC testing for functions CLI and function worker A wishlist without a proposal is converted to open-ended Discussions. GitHub link: https://github.com/apache/pulsar/discussions/18429#discussioncomment-4115503 This is an automat

[GitHub] [pulsar] sijie created a discussion: Add BC testing for functions CLI and function worker

2022-11-11 Thread GitBox
GitHub user sijie created a discussion: Add BC testing for functions CLI and function worker *Motivation* Functions CLI does a lot of "magic" in submitting functions. so changes between releases can potentially break functionalities. e.g. #2501 breaks the trigger function. so we need a BC te

[GitHub] [pulsar] tisonkun added a comment to the discussion: Hive Pulsar Integration

2022-11-11 Thread GitBox
GitHub user tisonkun added a comment to the discussion: Hive Pulsar Integration Wishlist without a proposal is converted to open-ended Discussions. GitHub link: https://github.com/apache/pulsar/discussions/18428#discussioncomment-4115417 This is an automatically sent email for dev@pulsar

[GitHub] [pulsar] tisonkun edited a comment on the discussion: Hive Pulsar Integration

2022-11-11 Thread GitBox
GitHub user tisonkun edited a comment on the discussion: Hive Pulsar Integration A wishlist without a proposal is converted to open-ended Discussions. GitHub link: https://github.com/apache/pulsar/discussions/18428#discussioncomment-4115417 This is an automatically sent email for dev@pul

[GitHub] [pulsar] sijie added a comment to the discussion: Hive Pulsar Integration

2022-11-11 Thread GitBox
GitHub user sijie added a comment to the discussion: Hive Pulsar Integration @jerrypeng will start with a sub module of pulsar, and get the work done first and then talk to Hive team to push the work there. GitHub link: https://github.com/apache/pulsar/discussions/18428#discussioncomment-4115

[GitHub] [pulsar] tiankangkan added a comment to the discussion: Hive Pulsar Integration

2022-11-11 Thread GitBox
GitHub user tiankangkan added a comment to the discussion: Hive Pulsar Integration We need this feature ... GitHub link: https://github.com/apache/pulsar/discussions/18428#discussioncomment-4115409 This is an automatically sent email for dev@pulsar.apache.org. To unsubscribe, please sen

[GitHub] [pulsar] ambition119 added a comment to the discussion: Hive Pulsar Integration

2022-11-11 Thread GitBox
GitHub user ambition119 added a comment to the discussion: Hive Pulsar Integration Issue #3399 also involve support Apache Hive, it is similar to [kafka-handler](https://github.com/apache/hive/tree/master/kafka-handler) in Hive. So it's Hive team or Pulsar team that decides to do this wor

[GitHub] [pulsar] sijie added a comment to the discussion: Hive Pulsar Integration

2022-11-11 Thread GitBox
GitHub user sijie added a comment to the discussion: Hive Pulsar Integration @ambition119 - ah I see. I am working with @VinceShieh on this feature. I forgot that @VinceShieh already created an issue for that. I will probably mark that issue as a duplicate of this one. GitHub link: https:

[GitHub] [pulsar] jerrypeng added a comment to the discussion: Hive Pulsar Integration

2022-11-11 Thread GitBox
GitHub user jerrypeng added a comment to the discussion: Hive Pulsar Integration @sijie it would be great if we can host this integration in the Hive repo as it would give Pulsar more visibility, though I am open to developing it first in the Apache Pulsar repo or another repo under Apache Pul

[GitHub] [pulsar] sijie created a discussion: Hive Pulsar Integration

2022-11-11 Thread GitBox
GitHub user sijie created a discussion: Hive Pulsar Integration **Is your feature request related to a problem? Please describe.** Hive is a data warehouse software that built on top of Hadoop. It would be great that Pulsar topics can be registered as external tables in Hive, so people can us

[GitHub] [pulsar] ghost added a comment to the discussion: State change feed

2022-11-11 Thread GitBox
GitHub user ghost added a comment to the discussion: State change feed yes like DynamoDB Streams GitHub link: https://github.com/apache/pulsar/discussions/18427#discussioncomment-4115382 This is an automatically sent email for dev@pulsar.apache.org. To unsubscribe, please send an email t

[GitHub] [pulsar] aahmed-se created a discussion: State change feed

2022-11-11 Thread GitBox
GitHub user aahmed-se created a discussion: State change feed They are certain use cases that need to monitor changes to the state , we should be able to publish write and update operations in our state store to a change feed pulsar topic that other functions can listen to and act upon. GitH