Re: [DISCUSS] PIP-180: Shadow Topic, an alternative way to support readonly topic ownership.
Hi Asafa, On 2022/07/13 08:15:59 Asaf Mesika wrote: > Thanks for the detailed answer. I have a few follow-up questions: > > 1. Can you clarify how data is flowing exactly. Specifically, from source > topic broker to shadow topic broker? I tried to follow the PIP again to > understand that, especially the overview but couldn't figure it out. > Also if you can while you're at it, how do you do flow control? I mean, you > stream all messages from source topic to shadow topic broker - there will > be some cases it will not have any more room in memory. > Once data arrives at the shadow topic, where does it go exactly? A simple answer to these questions would be it's the same as geo-replication except that shadow topic don't actually write entry to BK. The normal data flow goes by this: - User producer client - Source topic broker cache - Bookkeeper (Normal topic finishes here) - Shadow replicator in source topic broker (As a consumer and re-producer) - Shadow topic broker cache. The flow control settings for replicator can be configured by `replicatorDispatchRate` in `Policies` and `TopicPolicies`. We don't need to worry about the impact of memory management here, since the EntryCache implementation has its cache eviction policy, by time and space. > > 2. You wrote it was the easiest way to populate the entry cache, but there > is a very big penalty to pay here: contaminating the protocol. > The way I see it, the protocol should be sacred. > Most importantly, the internal details of a broker-to-broker should never > leak to the client. > In this case, the Command Send which is what the producer's clients are > using now has an extra field, called shadow_message_id. This field is an > internal detail of broker-to-broker communication and thus should never > leak outside. > > I'm not that fluent in the Pulsar codebase to give a decent alternative of > implementation. I can only point out that this IMO is too high a price to > pay. > > One possible solution comes to mind for brokers to broker RPC without > interfering with the protocol: open a consumer in shadow topic broker and > consume the messages and from there fill up the cache of managed ledger? > > Maybe it's a bit of over exaggerated here. Software development is all about trade-off and resources are constantly limited. I totally agree that we should minimize the change to the protocol and keep it clean and simple. IMO, there would be no meaning talking about the price is too high without considering the cost of the alternatives. In this case, the alternative would be open a consumer in shadow topic. It's quite a classic case of choosing "push vs pull" model to solve message syncing problems. And I believe both could solve it with about the same effort if we are building this from scratch. But the currently situation is that the "push" model is already well implemented and verified under a lot of production usages. If we ignore the code reusability here, then we need to manage the whole package of the consumers, like the life cycle, the configurations, running-threads, buffer size limitations, flow controls, metrics, stats, etc. I think this price would be too high to implement another similar "pull" model for broker-to-broker communication. For an extra field in Command Send, as a matter of fact, there is already an extra field called marker introduced in Command Send to improved logic for pausing replicated subscription snapshots when no traffic (PR #11922). I believe it's introduced by similar reason. > 3. I understand that you want to minimize consumption latency, hence you're > copying produced messages from the source topic broker topic cache (memory) > to the shadow topic broker cache (memory). > Since those produced messages are fairly new, if you are reading from BK, > won't those messages appear in the BK cache? Yes, these messages is most likely in the BK cache. > How bad percentage-wise is > reading from BK cache relative to reading from Pulsar cache - both on a > remote machine? The latency issue of reading messages directly from BK here is because of the piggyback lac advance mechanism. It's always in the latency of one message or maximum of `bookkeeperExplicitLacIntervalInMills` if it's set in broker.conf. You can find more detailed info from PR #5822 #3828 #4976. > > Geo-replication, from my understanding, is a fairly simple idea: the source > topic is in charge to push out messages (replicate) to a remote cluster > topic, using the normal client (producer) for this. > If I understand correctly, here, we re-use the replicator, which pushes out > messages to a topic, but those messages won't really be written to BK. > I'm afraid that creates a complicated code that will be hard to read later > and ties up one area with another which will be very hard later to untangle. Not really, the replicator (producer) part is in source topic, and the "not really write to BK" part is in shadow topic. For the replicator, nothing
Re: [DISCUSS] PIP-180: Shadow Topic, an alternative way to support readonly topic ownership.
Hi Penghui, > I had a conversation with Asaf, looks like we can use the RawReader for a > shadow topic internally to consume > messages from the original topic. So that we don't need to introduce > `shadow_message_id`, and don't need to introduce > changes to the replicator. For this point, like I wrote in the reply to Asaf, I still think the cost of maintaining the consumers would be a little too high, especially the lifecycle management part. It's quite complicated in replicator already. > > And using a persistent replicator might cause the very old messages to add > to the entry cache of the shadow topic? > Maybe there are some backlogs of the replicator. Yes, we should skip old backlogs for shadow replicator. But it seems that there are not much difference to reset the cursor in persistent replicator or in the raw reader. BR, Haiting On 2022/07/13 08:29:34 PengHui Li wrote: > Hi Haiting, > > I had a conversation with Asaf, looks like we can use the RawReader for a > shadow topic internally to consume > messages from the original topic. So that we don't need to introduce > `shadow_message_id`, and don't need to introduce > changes to the replicator. > > And using a persistent replicator might cause the very old messages to add > to the entry cache of the shadow topic? > Maybe there are some backlogs of the replicator. > > Thanks, > Penghui > > On Wed, Jul 13, 2022 at 4:16 PM Asaf Mesika wrote: > > > Thanks for the detailed answer. I have a few follow-up questions: > > > > 1. Can you clarify how data is flowing exactly. Specifically, from source > > topic broker to shadow topic broker? I tried to follow the PIP again to > > understand that, especially the overview but couldn't figure it out. > > Also if you can while you're at it, how do you do flow control? I mean, you > > stream all messages from source topic to shadow topic broker - there will > > be some cases it will not have any more room in memory. > > Once data arrives at the shadow topic, where does it go exactly? > > > > 2. You wrote it was the easiest way to populate the entry cache, but there > > is a very big penalty to pay here: contaminating the protocol. > > The way I see it, the protocol should be sacred. > > Most importantly, the internal details of a broker-to-broker should never > > leak to the client. > > In this case, the Command Send which is what the producer's clients are > > using now has an extra field, called shadow_message_id. This field is an > > internal detail of broker-to-broker communication and thus should never > > leak outside. > > > > I'm not that fluent in the Pulsar codebase to give a decent alternative of > > implementation. I can only point out that this IMO is too high a price to > > pay. > > > > One possible solution comes to mind for brokers to broker RPC without > > interfering with the protocol: open a consumer in shadow topic broker and > > consume the messages and from there fill up the cache of managed ledger? > > > > > > 3. I understand that you want to minimize consumption latency, hence you're > > copying produced messages from the source topic broker topic cache (memory) > > to the shadow topic broker cache (memory). > > Since those produced messages are fairly new, if you are reading from BK, > > won't those messages appear in the BK cache? How bad percentage-wise is > > reading from BK cache relative to reading from Pulsar cache - both on a > > remote machine? > > > > Geo-replication, from my understanding, is a fairly simple idea: the source > > topic is in charge to push out messages (replicate) to a remote cluster > > topic, using the normal client (producer) for this. > > If I understand correctly, here, we re-use the replicator, which pushes out > > messages to a topic, but those messages won't really be written to BK. > > I'm afraid that creates a complicated code that will be hard to read later > > and ties up one area with another which will be very hard later to > > untangle. > > > > > > Thanks! > > > > > > On Tue, Jul 12, 2022 at 5:35 PM Haiting Jiang > > wrote: > > > > > Hi Asaf, > > > > > > On 2022/07/11 13:08:52 Asaf Mesika wrote: > > > > On Thu, Jun 23, 2022 at 6:26 AM Haiting Jiang > > > > > > wrote: > > > > > > > > > Hi Asaf, > > > > > > > > > > > I did a quick reading and I couldn't understand the gist of this > > > change: > > > > > > The shadow topic doesn't really have it's own messages, or it's own > > > > > ledgers > > > > > > right? When it reads messages, it reads from the original topic > > > ledgers. > > > > > So > > > > > > the only thing you need to do is sync the "metadata" - ledgers > > list? > > > > > > > > > > Yes, mostly ledger id list and LAC of the last ledger. > > > > > > > > > > > > > > One question comes to mind here: Why not simply read the ledger > > > > > information > > > > > > from original topic, without copy? > > > > > > > > > > Yes, old ledger information will be read from metadata store when > > > > > ShadowManagedLedger initializes. Th
Re: [DISCUSS] PIP-180: Shadow Topic, an alternative way to support readonly topic ownership.
Hi Haiting, > For this point, like I wrote in the reply to Asaf, I still think the cost of maintaining the consumers would be a little too high, especially the lifecycle management part. It's quite complicated in replicator already. Yes, I see it. Makes sense. Without the replicator, it looks like we are trying to implement a very similar function to replicate data. The only difference is which broker to maintain the consumer. Can we use `message_id` directly? It's a message ID from the source cluster, it can be used by the shadow topic feature but can also be used by other features in the feature. > there are not much difference to reset the cursor in persistent replicator or in the raw reader. Do you want to control it in the shadow topic? or by users? Maybe we can make some changes to the replicator to support skip backlogs. Thanks, Penghui On Sun, Jul 17, 2022 at 10:57 PM Haiting Jiang wrote: > Hi Penghui, > > > I had a conversation with Asaf, looks like we can use the RawReader for a > > shadow topic internally to consume > > messages from the original topic. So that we don't need to introduce > > `shadow_message_id`, and don't need to introduce > > changes to the replicator. > > For this point, like I wrote in the reply to Asaf, I still think the cost > of > maintaining the consumers would be a little too high, especially the > lifecycle > management part. It's quite complicated in replicator already. > > > > > > And using a persistent replicator might cause the very old messages to > add > > to the entry cache of the shadow topic? > > Maybe there are some backlogs of the replicator. > > Yes, we should skip old backlogs for shadow replicator. But it seems that > there are not much difference to reset the cursor in persistent replicator > or > in the raw reader. > > BR, > Haiting > > > On 2022/07/13 08:29:34 PengHui Li wrote: > > Hi Haiting, > > > > I had a conversation with Asaf, looks like we can use the RawReader for a > > shadow topic internally to consume > > messages from the original topic. So that we don't need to introduce > > `shadow_message_id`, and don't need to introduce > > changes to the replicator. > > > > And using a persistent replicator might cause the very old messages to > add > > to the entry cache of the shadow topic? > > Maybe there are some backlogs of the replicator. > > > > Thanks, > > Penghui > > > > On Wed, Jul 13, 2022 at 4:16 PM Asaf Mesika > wrote: > > > > > Thanks for the detailed answer. I have a few follow-up questions: > > > > > > 1. Can you clarify how data is flowing exactly. Specifically, from > source > > > topic broker to shadow topic broker? I tried to follow the PIP again to > > > understand that, especially the overview but couldn't figure it out. > > > Also if you can while you're at it, how do you do flow control? I > mean, you > > > stream all messages from source topic to shadow topic broker - there > will > > > be some cases it will not have any more room in memory. > > > Once data arrives at the shadow topic, where does it go exactly? > > > > > > 2. You wrote it was the easiest way to populate the entry cache, but > there > > > is a very big penalty to pay here: contaminating the protocol. > > > The way I see it, the protocol should be sacred. > > > Most importantly, the internal details of a broker-to-broker should > never > > > leak to the client. > > > In this case, the Command Send which is what the producer's clients are > > > using now has an extra field, called shadow_message_id. This field is > an > > > internal detail of broker-to-broker communication and thus should never > > > leak outside. > > > > > > I'm not that fluent in the Pulsar codebase to give a decent > alternative of > > > implementation. I can only point out that this IMO is too high a price > to > > > pay. > > > > > > One possible solution comes to mind for brokers to broker RPC without > > > interfering with the protocol: open a consumer in shadow topic broker > and > > > consume the messages and from there fill up the cache of managed > ledger? > > > > > > > > > 3. I understand that you want to minimize consumption latency, hence > you're > > > copying produced messages from the source topic broker topic cache > (memory) > > > to the shadow topic broker cache (memory). > > > Since those produced messages are fairly new, if you are reading from > BK, > > > won't those messages appear in the BK cache? How bad percentage-wise is > > > reading from BK cache relative to reading from Pulsar cache - both on a > > > remote machine? > > > > > > Geo-replication, from my understanding, is a fairly simple idea: the > source > > > topic is in charge to push out messages (replicate) to a remote cluster > > > topic, using the normal client (producer) for this. > > > If I understand correctly, here, we re-use the replicator, which > pushes out > > > messages to a topic, but those messages won't really be written to BK. > > > I'm afraid that creates a complicated code that will be ha
Re: [VOTE] PIP-184: Topic specific consumer priorityLevel
+1 Thanks On Fri, Jul 15, 2022 at 3:58 PM Dave Maughan wrote: > > Hi Pulsar Community > > I would like to start a VOTE on "PIP-184: Topic specific consumer > priorityLevel". > > The proposal can be read at https://github.com/apache/pulsar/issues/16481 > > and the discussion thread is available at > https://lists.apache.org/thread/5zs4gd3r0rtzz16nv62o8ntygg01qjhq > > Voting will stay open for at least 48h. > > Thanks, > Dave
Re: [DISCUSS]PIP-189: No batching if only one message in batch
Hi Enrico
> it is always a good idea to not create a batch message
I agree this. So maybe I need a little work to fix the internal tests.:)
Thanks
Xiaoyu Hou
Enrico Olivelli 于2022年7月15日周五 17:29写道：
> Thanks for driving this initiative.
>
> I think that it is better to not add a configuration flag for this feature.
> Because:
> - it is always a good idea to not create a batch message
> - everybody will have to turn this feature explicitly on, otherwise
> there is no benefit
> - it is very hard to explain why you should not enable this feature
> and we will have to tell to everybody to turn it on
> - BatchMessageIdImpl is a implementation detail, that is not part of
> the public API, applications must never rely on internal Impl classes
>
> If you really would like to have a way to turn off this behaviour I
> would at most use a system property, but this is something that we
> never did in the Pulsar API.
>
> I know that it will be a pain to fix internal Pulsar tests about
> batching, because we won't be guaranteed anymore to create batch
> messages.
>
>
> Enrico
>
> Il giorno ven 15 lug 2022 alle ore 11:21 Anon Hxy
> ha scritto:
> >
> > Hi Pulsar community:
> >
> > I open a pip to discuss "No batching if only one message in batch"
> >
> > Proposal Link: https://github.com/apache/pulsar/issues/16619
> >
> > ---
> >
> > ## Motivation
> >
> > The original discussion mail :
> > https://lists.apache.org/thread/5svpl5qp3bfoztf5fvtojh51zbklcht2
> >
> > linked issue: https://github.com/apache/pulsar/issues/16547
> >
> > Introduce the ability for producers to publish a non-batched message if
> > there is only one message in the batch.
> >
> > It is useful to save the `SingleMessageMetadata` space in entry and
> reduce
> > workload of consumers to deserialize the `SingleMessageMetadata`,
> > especially when sometimes there is an amount of batched messages with
> > only one real message.
> >
> > ## API Changes
> >
> > When this feature is applied, the returned type of `MessageId` may not be
> > `BatchMessageIdImpl`, even if we have set the `enableBatching` as true.
> It
> > is because the producer will publish a single message as a non-batched
> > message. Also, the consumer will deserialize the entry as a non-batched
> > message, which will receive a message with normal `MessageIdImpl` but
> not
> > `BatchMessageIdImpl`.
> >
> > So this may cause `((BatchMessageIdImpl) messageId)` throw
> > `ClassCastException`. we need to add a switch for the producer to enable
> > or disable this feature
> >
> > ```
> > ProducerBuilder batchingSingleMessage(boolean batchingSingleMessage);
> > // default value is true
> >
> > ```
> >
> > ## Implementation
> >
> > For `BatchMessageContainerImpl` :
> > ```
> > public OpSendMsg createOpSendMsg() throws IOException {
> > if (!producer.conf.isBatchingSingleMessage() && messages.size()
> ==
> > 1) {
> > // If only one message, create OpSendMsg as non-batched
> > publish.
> > }
> >
> >//
> > }
> > ```
> >
> > For `BatchMessageKeyBasedContainer`, there is no need to change, because
> > it uses `BatchMessageContainerImpl` to create `OpSendMsg`
> >
> >
> > ## Reject Alternatives
> >
> > - Always return `BatchMessageIdImpl` when `enableBatching` is set as
> true,
> > even if publish single message with a non-batched message.
> > Rejection reason: Consumer have to deserialize to check if there is
> > `SingleMessageMetadata` from the payload
>
Re: [VOTE] PIP-184: Topic specific consumer priorityLevel
+1 Penghui On Mon, Jul 18, 2022 at 10:15 AM Zike Yang wrote: > +1 > > Thanks > > > On Fri, Jul 15, 2022 at 3:58 PM Dave Maughan > wrote: > > > > Hi Pulsar Community > > > > I would like to start a VOTE on "PIP-184: Topic specific consumer > > priorityLevel". > > > > The proposal can be read at > https://github.com/apache/pulsar/issues/16481 > > > > and the discussion thread is available at > > https://lists.apache.org/thread/5zs4gd3r0rtzz16nv62o8ntygg01qjhq > > > > Voting will stay open for at least 48h. > > > > Thanks, > > Dave >
[DISCUSS] Apache Pulsar 2.11.0 Release
Hi all, We released 2.10.0 three months ago. And there are many great changes in the master branch, including new features and performance improvements. - PIP 74: apply client memory to consumer https://github.com/apache/pulsar/pull/15216 - PIP 143: Support split bundles by specified boundaries https://github.com/apache/pulsar/pull/13796 - PIP 145: regex subscription improvements https://github.com/apache/pulsar/pull/16062 - PIP 160: transaction performance improvements (still in progress and merged some PRs) - PIP 161: new exclusive producer mode support https://github.com/apache/pulsar/pull/15488 - PIP 182: Provide new load balance placement strategy implementation for ModularLoadManagerStrategy https://github.com/apache/pulsar/pull/16281 Add Pulsar Auth support for the Pulsar SQL https://github.com/apache/pulsar/pull/15571 And some features are blocked in the review stage, but they are powerful improvements for Pulsar PIP 37: Support chunking with Shared subscription https://github.com/apache/pulsar/pull/16202 PIP-166: Function add MANUAL delivery semantics https://github.com/apache/pulsar/pull/16279 You can find the complete change list in 2.11.0 at https://github.com/apache/pulsar/pulls?q=is%3Apr+milestone%3A2.11.0+-label%3Arelease%2F2.10.1+-label%3Arelease%2F2.10.2 And maybe I missed some important in-progress PRs, please let me know if it should be a blocker of the 2.11.0 release. It's a good time to discuss the target time of the 2.11.0 release. I think we can leave 2 weeks to complete the in-progress PRs and 2 weeks to accept bug fixes. And target the 2.11.0 release in mid-August. Please let me know what you think. Thanks, Penghui
[GitHub] [pulsar] tisonkun added a comment to the discussion: What's the best practice to build java-test-image?
GitHub user tisonkun added a comment to the discussion: What's the best practice to build java-test-image? Each time it failed in different step but all about cannot download from a remote place. Perhaps it's about network issues. GitHub link: https://github.com/apache/pulsar/discussions/16633#discussioncomment-3168345 This is an automatically sent email for dev@pulsar.apache.org. To unsubscribe, please send an email to: dev-unsubscr...@pulsar.apache.org
Re: [DISCUSS] Apache Pulsar 2.11.0 Release
In addition to #16202, there is a following PR to support the correct ACK implementation for chunked messages. It should depend on #16202 But I think I can submit an initial PR this week and change the tests after #16202 is merged. Thanks, Yunze > 2022年7月18日 11:22，PengHui Li 写道： > > Hi all, > > We released 2.10.0 three months ago. And there are many great changes in > the master branch, > including new features and performance improvements. > > - PIP 74: apply client memory to consumer > https://github.com/apache/pulsar/pull/15216 > - PIP 143: Support split bundles by specified boundaries > https://github.com/apache/pulsar/pull/13796 > - PIP 145: regex subscription improvements > https://github.com/apache/pulsar/pull/16062 > - PIP 160: transaction performance improvements (still in progress and > merged some PRs) > - PIP 161: new exclusive producer mode support > https://github.com/apache/pulsar/pull/15488 > - PIP 182: Provide new load balance placement strategy implementation for > ModularLoadManagerStrategy https://github.com/apache/pulsar/pull/16281 > Add Pulsar Auth support for the Pulsar SQL > https://github.com/apache/pulsar/pull/15571 > > And some features are blocked in the review stage, but they are powerful > improvements for Pulsar > > PIP 37: Support chunking with Shared subscription > https://github.com/apache/pulsar/pull/16202 > PIP-166: Function add MANUAL delivery semantics > https://github.com/apache/pulsar/pull/16279 > > You can find the complete change list in 2.11.0 at > https://github.com/apache/pulsar/pulls?q=is%3Apr+milestone%3A2.11.0+-label%3Arelease%2F2.10.1+-label%3Arelease%2F2.10.2 > > And maybe I missed some important in-progress PRs, please let me know if it > should be a blocker of the 2.11.0 release. > > It's a good time to discuss the target time of the 2.11.0 release. > I think we can leave 2 weeks to complete the in-progress PRs and 2 weeks to > accept bug fixes. > And target the 2.11.0 release in mid-August. > > Please let me know what you think. > > Thanks, > Penghui
[GitHub] [pulsar-helm-chart] mattisonchao opened a new pull request, #277: Bump Apache Pulsar version to 2.9.3
mattisonchao opened a new pull request, #277: URL: https://github.com/apache/pulsar-helm-chart/pull/277 ### Motivation Bump Apache Pulsar version to 2.9.3 ### Modifications - Bump Apache Pulsar version to 2.9.3 ### Verifying this change - [x] Make sure that the change passes the CI checks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: dev-unsubscr...@pulsar.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [pulsar] tisonkun created a discussion: How to build java-test-image on Apple M1?
GitHub user tisonkun created a discussion: How to build java-test-image on Apple M1? cc @lhotari Today I'm trying to build java-test-image on Apple M1. Run `build/build_java_test_image.sh` and get: ``` [ERROR] Failed to execute goal com.spotify:dockerfile-maven-plugin:1.4.13:build (default) on project pulsar-docker-image: Could not build image: java.util.concurrent.ExecutionException: com.spotify.docker.client.shaded.javax.ws.rs.ProcessingException: java.lang.UnsatisfiedLinkError: could not load FFI provider com.spotify.docker.client.shaded.jnr.ffi.provider.jffi.Provider: ExceptionInInitializerError: Can't overwrite cause with java.lang.UnsatisfiedLinkError: java.lang.UnsatisfiedLinkError: Can't load library: /var/folders/n2/rnpcrm6x3mb2c1jn1dkdb33cgn/T/jffi13338179526049207391.dylib [ERROR] at java.base/java.lang.ClassLoader.loadLibrary(ClassLoader.java:2393) [ERROR] at java.base/java.lang.Runtime.load0(Runtime.java:755) [ERROR] at java.base/java.lang.System.load(System.java:1953) [ERROR] at com.kenai.jffi.internal.StubLoader.loadFromJar(StubLoader.java:371) [ERROR] at com.kenai.jffi.internal.StubLoader.load(StubLoader.java:258) [ERROR] at com.kenai.jffi.internal.StubLoader.(StubLoader.java:444) [ERROR] at java.base/java.lang.Class.forName0(Native Method) [ERROR] at java.base/java.lang.Class.forName(Class.java:467) [ERROR] at com.kenai.jffi.Init.load(Init.java:68) [ERROR] at com.kenai.jffi.Foreign$InstanceHolder.getInstanceHolder(Foreign.java:49) [ERROR] at com.kenai.jffi.Foreign$InstanceHolder.(Foreign.java:45) [ERROR] at com.kenai.jffi.Foreign.getInstance(Foreign.java:103) [ERROR] at com.kenai.jffi.Type$Builtin.lookupTypeInfo(Type.java:242) [ERROR] at com.kenai.jffi.Type$Builtin.getTypeInfo(Type.java:237) [ERROR] at com.kenai.jffi.Type.resolveSize(Type.java:155) [ERROR] at com.kenai.jffi.Type.size(Type.java:138) [ERROR] at com.spotify.docker.client.shaded.jnr.ffi.provider.jffi.NativeRuntime$TypeDelegate.size(NativeRuntime.java:187) [ERROR] at com.spotify.docker.client.shaded.jnr.ffi.provider.AbstractRuntime.(AbstractRuntime.java:48) [ERROR] at com.spotify.docker.client.shaded.jnr.ffi.provider.jffi.NativeRuntime.(NativeRuntime.java:66) [ERROR] at com.spotify.docker.client.shaded.jnr.ffi.provider.jffi.NativeRuntime.(NativeRuntime.java:41) [ERROR] at com.spotify.docker.client.shaded.jnr.ffi.provider.jffi.NativeRuntime$SingletonHolder.(NativeRuntime.java:62) [ERROR] at com.spotify.docker.client.shaded.jnr.ffi.provider.jffi.NativeRuntime.getInstance(NativeRuntime.java:58) [ERROR] at com.spotify.docker.client.shaded.jnr.ffi.provider.jffi.Provider.(Provider.java:29) [ERROR] at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) [ERROR] at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:77) [ERROR] at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) [ERROR] at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:499) [ERROR] at java.base/java.lang.reflect.ReflectAccess.newInstance(ReflectAccess.java:128) [ERROR] at java.base/jdk.internal.reflect.ReflectionFactory.newInstance(ReflectionFactory.java:347) [ERROR] at java.base/java.lang.Class.newInstance(Class.java:645) [ERROR] at com.spotify.docker.client.shaded.jnr.ffi.provider.FFIProvider$SystemProviderSingletonHolder.getInstance(FFIProvider.java:68) [ERROR] at com.spotify.docker.client.shaded.jnr.ffi.provider.FFIProvider$SystemProviderSingletonHolder.(FFIProvider.java:57) [ERROR] at com.spotify.docker.client.shaded.jnr.ffi.provider.FFIProvider.getSystemProvider(FFIProvider.java:35) [ERROR] at com.spotify.docker.client.shaded.jnr.ffi.LibraryLoader.create(LibraryLoader.java:73) [ERROR] at com.spotify.docker.client.shaded.jnr.unixsocket.Native.(Native.java:76) [ERROR] at com.spotify.docker.client.shaded.jnr.unixsocket.UnixSocketChannel.(UnixSocketChannel.java:101) [ERROR] at com.spotify.docker.client.shaded.jnr.unixsocket.UnixSocketChannel.open(UnixSocketChannel.java:60) [ERROR] at com.spotify.docker.client.UnixConnectionSocketFactory.createSocket(UnixConnectionSocketFactory.java:69) [ERROR] at com.spotify.docker.client.UnixConnectionSocketFactory.createSocket(UnixConnectionSocketFactory.java:44) [ERROR] at com.spotify.docker.client.shaded.org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:118) [ERROR] at com.spotify.docker.client.shaded.org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.jav
[GitHub] [pulsar] lhotari added a comment to the discussion: What's the best practice to build java-test-image?
GitHub user lhotari added a comment to the discussion: What's the best practice to build java-test-image? There seems to be some network issues. In the output, you can see: > [INFO] Err:6 http://mirrors.ubuntu.com/mirrors.txt Mirrorlist > [INFO] Could not connect to mirrors.ubuntu.com:80 (198.19.194.95), > connection timed out By default, the docker image uses http://mirrors.ubuntu.com/mirrors.txt as the source for Ubuntu mirrors. You can manually choose the mirror by setting UBUNTU_MIRROR environment variable. You can find the listing of mirrors by country in http://mirrors.ubuntu.com . For example, http://mirrors.ubuntu.com/US.txt, http://mirrors.ubuntu.com/CN.txt ```shell # example of using the Ubuntu default mirror UBUNTU_MIRROR=http://archive.ubuntu.com/ubuntu/ ./build/build_java_test_image.sh ``` ```shell # example of using a specific mirror from http://mirrors.ubuntu.com/CN.txt UBUNTU_MIRROR=http://mirrors.huaweicloud.com/repository/ubuntu/ ./build/build_java_test_image.sh ``` I hope this helps. GitHub link: https://github.com/apache/pulsar/discussions/16633#discussioncomment-3168787 This is an automatically sent email for dev@pulsar.apache.org. To unsubscribe, please send an email to: dev-unsubscr...@pulsar.apache.org
