[jira] [Created] (KAFKA-10726) How to detect heartbeat failure between broker/zookeeper leader
Keiichiro Wakasa created KAFKA-10726: Summary: How to detect heartbeat failure between broker/zookeeper leader Key: KAFKA-10726 URL: https://issues.apache.org/jira/browse/KAFKA-10726 Project: Kafka Issue Type: Bug Components: controller, logging Affects Versions: 2.1.1 Reporter: Keiichiro Wakasa Hello experts, I'm not sure this is proper place to ask but I'd appreciate if you could help us with the following question... We've continuously suffered from broker exclusion caused by heartbeat timeout between broker and zookeeper leader. This issue can easily detected by checking ephemeral nodes via zkcli.sh but we'd like to detect this with logs like server.log/controller.log since we have an existing system to forward these logs to our system. Looking at server.log/controller.log, we couldn't find any logs that indicates the heartbeat timeout. Is there any other logs to check for heartbeat health? -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: [DISCUSS] KIP-660: Pluggable ReplicaAssignor
Hi all, If I don't see additional feedback in the next few days, I'll start a vote. Thanks On Thu, Nov 5, 2020 at 6:29 PM Mickael Maison wrote: > > Hi all, > > I've updated the KIP to reflect the latest discussions. > > Tom, > 2) Updated > 4) I decided against switching to a "batch interface" and added the > reasons in the Rejected Alternatives section > > Please take a look and let me know if you have any feedback. > Thanks > > On Tue, Oct 6, 2020 at 9:43 AM Mickael Maison > wrote: > > > > Hi Efe, > > > > Thanks for the feedback. > > We also need to assign replicas when adding partitions to an existing > > topic. This is why I choose to use a list of partition ids. Otherwise > > we'd need the number of partitions and the starting partition id. > > > > Let me know if you have more questions > > > > On Tue, Oct 6, 2020 at 2:16 AM Efe Gencer > > wrote: > > > > > > Hi Mickael, > > > > > > Thanks for the KIP! > > > A call to an external system, e.g. Cruise Control, in the implementation > > > of the provided interface can indeed help with the initial assignment of > > > partitions. > > > > > > I am curious why the proposed `ReplicaAssignor#assignReplicasToBrokers` > > > receives a list of partition ids as opposed to the number of partitions > > > to create the topic with? > > > > > > Would you clarify if this API is expected to be used (1) only for new > > > topics or (2) also for existing topics? > > > > > > Best, > > > Efe > > > > > > From: Mickael Maison > > > Sent: Thursday, October 1, 2020 9:43 AM > > > To: dev > > > Subject: Re: [DISCUSS] KIP-660: Pluggable ReplicaAssignor > > > > > > Thanks Tom for the feedback! > > > > > > 1. If the data returned by the ReplicaAssignor implementation does not > > > match that was requested, we'll also throw a ReplicaAssignorException > > > > > > 2. Good point, I'll update the KIP > > > > > > 3. The KIP mentions an error code associated with > > > ReplicaAssignorException: REPLICA_ASSIGNOR_FAILED > > > > > > 4. (I'm naming your last question 4.) I spent some time looking at it. > > > Initially I wanted to follow the model from the topic policies. But as > > > you said, computing assignments for the whole batch may be more > > > desirable and also avoids incrementally updating the cluster state. > > > The logic in AdminManager is very much centered around doing 1 topic > > > at a time but as far as I can tell we should be able to update it to > > > compute assignments for the whole batch. > > > > > > I'll play a bit more with 4. and I'll update the KIP in the next few days > > > > > > On Mon, Sep 21, 2020 at 10:29 AM Tom Bentley wrote: > > > > > > > > Hi Mickael, > > > > > > > > A few thoughts about the ReplicaAssignor contract: > > > > > > > > 1. What happens if a ReplicaAssignor impl returns a Map where some > > > > assignments don't meet the given replication factor? > > > > 2. Fixing the signature of assignReplicasToBrokers() as you have would > > > > make > > > > it hard to pass extra information in the future (e.g. maybe someone > > > > comes > > > > up with a use case where passing the clientId would be needed) because > > > > it > > > > would require the interface be changed. If you factored all the > > > > parameters > > > > into some new type then the signature could be > > > > assignReplicasToBrokers(RequiredReplicaAssignment) and adding any new > > > > properties to RequiredReplicaAssignment wouldn't break the contract. > > > > 3. When an assignor throws RepliacAssignorException what error code > > > > will be > > > > returned to the client? > > > > > > > > Also, this sentence got me thinking: > > > > > > > > > If multiple topics are present in the request, AdminManager will > > > > > update > > > > the Cluster object so the ReplicaAssignor class has access to the up to > > > > date cluster metadata. > > > > > > > > Previously I've looked at how we can improve Kafka's pluggable policy > > > > support to pass the more of the cluster state to policy > > > > implementations. A > > > > similar problem exists there, but the more cluster state you pass the > > > > harder it is to incrementally change it as you iterate through the > > > > topics > > > > to be created/modified. This likely isn't a problem here and now, but it > > > > could limit any future changes to the pluggable assignors. Did you > > > > consider > > > > the alternative of the assignor just being passed a Set of assignments? > > > > That means you can just pass the cluster state as it exists at the > > > > time. It > > > > also gives the implementation more information to work with to find more > > > > optimal assignments. For example, it could perform a bin packing type > > > > assignment which found a better optimum for the whole collection of > > > > topics > > > > than one which was only told about all the topics in the request > > > > sequentially. > > > > > > > > Otherwise this looks like a valuable feature to me. > > > > > > > > Kin
[VOTE] KIP-681: Rename master key in delegation token feature
Hi, I'd like to start a vote on KIP-681 which proposes to rename a broker config to use a more racially neutral term: https://cwiki.apache.org/confluence/display/KAFKA/KIP-681 %3A+Rename+master+key+in+delegation+token+feature Kind regards, Tom
[VOTE] KIP-684 - Support mutual TLS authentication on SASL_SSL listeners
Hi all, I would like to start vote on KIP-684 to support TLS client authentication (mTLS) on SASL_SSL listeners: - https://cwiki.apache.org/confluence/display/KAFKA/KIP-684+-+Support+mutual+TLS+authentication+on+SASL_SSL+listeners Thank you... Regards, Rajini
[jira] [Created] (KAFKA-10727) Kafka clients throw AuthenticationException during Kerberos re-login
Rajini Sivaram created KAFKA-10727: -- Summary: Kafka clients throw AuthenticationException during Kerberos re-login Key: KAFKA-10727 URL: https://issues.apache.org/jira/browse/KAFKA-10727 Project: Kafka Issue Type: Bug Reporter: Rajini Sivaram Assignee: Rajini Sivaram During Kerberos re-login, we log out and login again. There is a timing issue where the principal in the Subject has been cleared, but a new one hasn't been populated yet. We need to ensure that we don't throw AuthenticationException in this case to avoid Kafka clients (consumer/producer etc.) failing instead of retrying. -- This message was sent by Atlassian Jira (v8.3.4#803005)
Re: [DISCUSS] KIP-660: Pluggable ReplicaAssignor
Hi Mickael, Thanks for the KIP. It is an interesting one. I imagine that custom assignors may use a rather complex model of the cluster in order to be able to allocate partitions in smarter ways. For instance, one may use the distribution of leaders in the cluster to allocate the new leaders. With the current interface, that means computing the distribution based on the Cluster received for every assignment request. That could become pretty inefficient in clusters with a large number of nodes and/or partitions. That could become even worse if the model is more complicated. I wonder if you have thought about this or experienced this with your prototype? Have you considered going with an approach à la ClientQuotaCallback where the plugin is updated when the Cluster has changed? That would allow to keep an internal model ready. Another way would be to use batching as suggested as it would allow to amortize the cost of building a model for the current request/user. I also wonder if using Cluster is a good idea here. With KIP-500, I can imagine that this plugin will run in the controller directly instead of running in the AdminManager as today. In this case, we could obviously continue to build that Cluster object but we may have better ways. Therefore, I wonder if having an interface to represent the cluster may be better from an evolution perspective. Have you considered this? Best, David On Mon, Nov 16, 2020 at 12:10 PM Mickael Maison wrote: > Hi all, > > If I don't see additional feedback in the next few days, I'll start a vote. > Thanks > > On Thu, Nov 5, 2020 at 6:29 PM Mickael Maison > wrote: > > > > Hi all, > > > > I've updated the KIP to reflect the latest discussions. > > > > Tom, > > 2) Updated > > 4) I decided against switching to a "batch interface" and added the > > reasons in the Rejected Alternatives section > > > > Please take a look and let me know if you have any feedback. > > Thanks > > > > On Tue, Oct 6, 2020 at 9:43 AM Mickael Maison > wrote: > > > > > > Hi Efe, > > > > > > Thanks for the feedback. > > > We also need to assign replicas when adding partitions to an existing > > > topic. This is why I choose to use a list of partition ids. Otherwise > > > we'd need the number of partitions and the starting partition id. > > > > > > Let me know if you have more questions > > > > > > On Tue, Oct 6, 2020 at 2:16 AM Efe Gencer > wrote: > > > > > > > > Hi Mickael, > > > > > > > > Thanks for the KIP! > > > > A call to an external system, e.g. Cruise Control, in the > implementation of the provided interface can indeed help with the initial > assignment of partitions. > > > > > > > > I am curious why the proposed > `ReplicaAssignor#assignReplicasToBrokers` receives a list of partition ids > as opposed to the number of partitions to create the topic with? > > > > > > > > Would you clarify if this API is expected to be used (1) only for > new topics or (2) also for existing topics? > > > > > > > > Best, > > > > Efe > > > > > > > > From: Mickael Maison > > > > Sent: Thursday, October 1, 2020 9:43 AM > > > > To: dev > > > > Subject: Re: [DISCUSS] KIP-660: Pluggable ReplicaAssignor > > > > > > > > Thanks Tom for the feedback! > > > > > > > > 1. If the data returned by the ReplicaAssignor implementation does > not > > > > match that was requested, we'll also throw a ReplicaAssignorException > > > > > > > > 2. Good point, I'll update the KIP > > > > > > > > 3. The KIP mentions an error code associated with > > > > ReplicaAssignorException: REPLICA_ASSIGNOR_FAILED > > > > > > > > 4. (I'm naming your last question 4.) I spent some time looking at > it. > > > > Initially I wanted to follow the model from the topic policies. But > as > > > > you said, computing assignments for the whole batch may be more > > > > desirable and also avoids incrementally updating the cluster state. > > > > The logic in AdminManager is very much centered around doing 1 topic > > > > at a time but as far as I can tell we should be able to update it to > > > > compute assignments for the whole batch. > > > > > > > > I'll play a bit more with 4. and I'll update the KIP in the next few > days > > > > > > > > On Mon, Sep 21, 2020 at 10:29 AM Tom Bentley > wrote: > > > > > > > > > > Hi Mickael, > > > > > > > > > > A few thoughts about the ReplicaAssignor contract: > > > > > > > > > > 1. What happens if a ReplicaAssignor impl returns a Map where some > > > > > assignments don't meet the given replication factor? > > > > > 2. Fixing the signature of assignReplicasToBrokers() as you have > would make > > > > > it hard to pass extra information in the future (e.g. maybe > someone comes > > > > > up with a use case where passing the clientId would be needed) > because it > > > > > would require the interface be changed. If you factored all the > parameters > > > > > into some new type then the signature could be > > > > > assignReplicasToBrokers(RequiredReplicaAssignment) and adding any > new
[jira] [Created] (KAFKA-10728) Mirroring data without decompressing with MirrorMaker 2.0
Eazhilan Nagarajan created KAFKA-10728:
--
Summary: Mirroring data without decompressing with MirrorMaker 2.0
Key: KAFKA-10728
URL: https://issues.apache.org/jira/browse/KAFKA-10728
Project: Kafka
Issue Type: Improvement
Components: mirrormaker
Reporter: Eazhilan Nagarajan
Hello,
I use MirrorMaker 2.0 to copy data across two Kafka clusters and it's all
working fine. Recently we enabled compressing while producing data into any
topic which had a very positive impact on the storage and other resources but
while mirroring, the data seems to be decompressed at the target Kafka cluster.
I tried enabling compression using the below config in MM2, the data at the
target cluster is compressed now, the decompress and re-compress continues to
happen and it eats up a lot of resources unnecessarily.
{noformat}
- alias: my-passive-cluster
authentication:
passwordSecret:
password: password
secretName: passive-cluster-secret
type: scram-sha-512
username: user-1
bootstrapServers: my-passive-cluster.com:443
config:
config.storage.replication.factor: 3
offset.storage.replication.factor: 3
status.storage.replication.factor: 3
producer.compression.type: gzip{noformat}
I found couple of Jira issues talking about it but I don't know if the shallow
iterator option is available now.
https://issues.apache.org/jira/browse/KAFKA-732,
https://issues.apache.org/jira/browse/KAFKA-845
Kindly let me if this is currently available or if it'll be available in the
future.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
Re: [VOTE] KIP-684 - Support mutual TLS authentication on SASL_SSL listeners
+1 (non-binding). Thanks, Rajini. Ron > On Nov 16, 2020, at 6:31 AM, Rajini Sivaram wrote: > > Hi all, > > I would like to start vote on KIP-684 to support TLS client authentication > (mTLS) on SASL_SSL listeners: > > - > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-684+-+Support+mutual+TLS+authentication+on+SASL_SSL+listeners > > > Thank you... > > Regards, > > Rajini
Re: [VOTE] KIP-684 - Support mutual TLS authentication on SASL_SSL listeners
+1 (binding). Thanks for the KIP, Rajini! On Mon, Nov 16, 2020 at 3:03 PM Ron Dagostino wrote: > +1 (non-binding). Thanks, Rajini. > > Ron > > > On Nov 16, 2020, at 6:31 AM, Rajini Sivaram > wrote: > > > > Hi all, > > > > I would like to start vote on KIP-684 to support TLS client > authentication > > (mTLS) on SASL_SSL listeners: > > > > - > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-684+-+Support+mutual+TLS+authentication+on+SASL_SSL+listeners > > > > > > Thank you... > > > > Regards, > > > > Rajini >
Re: [VOTE] KIP-684 - Support mutual TLS authentication on SASL_SSL listeners
Thanks for the KIP, +1 (binding). Ismael On Mon, Nov 16, 2020 at 3:33 AM Rajini Sivaram wrote: > Hi all, > > I would like to start vote on KIP-684 to support TLS client authentication > (mTLS) on SASL_SSL listeners: > >- > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-684+-+Support+mutual+TLS+authentication+on+SASL_SSL+listeners > > > Thank you... > > Regards, > > Rajini >
Re: [DISCUSS] KIP-660: Pluggable ReplicaAssignor
Why would the replica placement logic run in the controller rather than in the AdminManager? My understanding, and correct me if I got it wrong, is that we are aiming at better separation of concerns. The controller job will be managing consensus and consistency of metadata, but creating the metadata itself will be in the AdminManager. On Mon, Nov 16, 2020, 5:31 AM David Jacot wrote: > Hi Mickael, > > Thanks for the KIP. It is an interesting one. > > I imagine that custom assignors may use a rather complex model of the > cluster in order > to be able to allocate partitions in smarter ways. For instance, one may > use the distribution > of leaders in the cluster to allocate the new leaders. With the current > interface, that > means computing the distribution based on the Cluster received for every > assignment > request. That could become pretty inefficient in clusters with a large > number of nodes > and/or partitions. That could become even worse if the model is more > complicated. > > I wonder if you have thought about this or experienced this with your > prototype? > > Have you considered going with an approach à la ClientQuotaCallback where > the plugin > is updated when the Cluster has changed? That would allow to keep an > internal model > ready. Another way would be to use batching as suggested as it would allow > to amortize > the cost of building a model for the current request/user. > > I also wonder if using Cluster is a good idea here. With KIP-500, I can > imagine that this > plugin will run in the controller directly instead of running in the > AdminManager as today. > In this case, we could obviously continue to build that Cluster object but > we may have > better ways. Therefore, I wonder if having an interface to represent the > cluster may be > better from an evolution perspective. Have you considered this? > > Best, > David > > On Mon, Nov 16, 2020 at 12:10 PM Mickael Maison > wrote: > > > Hi all, > > > > If I don't see additional feedback in the next few days, I'll start a > vote. > > Thanks > > > > On Thu, Nov 5, 2020 at 6:29 PM Mickael Maison > > wrote: > > > > > > Hi all, > > > > > > I've updated the KIP to reflect the latest discussions. > > > > > > Tom, > > > 2) Updated > > > 4) I decided against switching to a "batch interface" and added the > > > reasons in the Rejected Alternatives section > > > > > > Please take a look and let me know if you have any feedback. > > > Thanks > > > > > > On Tue, Oct 6, 2020 at 9:43 AM Mickael Maison < > mickael.mai...@gmail.com> > > wrote: > > > > > > > > Hi Efe, > > > > > > > > Thanks for the feedback. > > > > We also need to assign replicas when adding partitions to an existing > > > > topic. This is why I choose to use a list of partition ids. Otherwise > > > > we'd need the number of partitions and the starting partition id. > > > > > > > > Let me know if you have more questions > > > > > > > > On Tue, Oct 6, 2020 at 2:16 AM Efe Gencer > > > wrote: > > > > > > > > > > Hi Mickael, > > > > > > > > > > Thanks for the KIP! > > > > > A call to an external system, e.g. Cruise Control, in the > > implementation of the provided interface can indeed help with the initial > > assignment of partitions. > > > > > > > > > > I am curious why the proposed > > `ReplicaAssignor#assignReplicasToBrokers` receives a list of partition > ids > > as opposed to the number of partitions to create the topic with? > > > > > > > > > > Would you clarify if this API is expected to be used (1) only for > > new topics or (2) also for existing topics? > > > > > > > > > > Best, > > > > > Efe > > > > > > > > > > From: Mickael Maison > > > > > Sent: Thursday, October 1, 2020 9:43 AM > > > > > To: dev > > > > > Subject: Re: [DISCUSS] KIP-660: Pluggable ReplicaAssignor > > > > > > > > > > Thanks Tom for the feedback! > > > > > > > > > > 1. If the data returned by the ReplicaAssignor implementation does > > not > > > > > match that was requested, we'll also throw a > ReplicaAssignorException > > > > > > > > > > 2. Good point, I'll update the KIP > > > > > > > > > > 3. The KIP mentions an error code associated with > > > > > ReplicaAssignorException: REPLICA_ASSIGNOR_FAILED > > > > > > > > > > 4. (I'm naming your last question 4.) I spent some time looking at > > it. > > > > > Initially I wanted to follow the model from the topic policies. But > > as > > > > > you said, computing assignments for the whole batch may be more > > > > > desirable and also avoids incrementally updating the cluster state. > > > > > The logic in AdminManager is very much centered around doing 1 > topic > > > > > at a time but as far as I can tell we should be able to update it > to > > > > > compute assignments for the whole batch. > > > > > > > > > > I'll play a bit more with 4. and I'll update the KIP in the next > few > > days > > > > > > > > > > On Mon, Sep 21, 2020 at 10:29 AM Tom Bentley > > wrote: > > > > > > > > > > > > Hi Mickael, > > > > > > > > > > > >
Re: [VOTE] KIP-681: Rename master key in delegation token feature
+1 Thank you for catching this. On Mon, Nov 16, 2020, 3:22 AM Tom Bentley wrote: > Hi, > > I'd like to start a vote on KIP-681 which proposes to rename a broker > config to use a more racially neutral term: > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-681 > %3A+Rename+master+key+in+delegation+token+feature > > Kind regards, > > Tom >
New contributor
Hello, could you please add me as a contributor for Apache Kafka? github: dorwi (https://github.com/dorwi) jira: dorwi Thanks, Vidor
[jira] [Resolved] (KAFKA-10704) Mirror maker with TLS at target
[
https://issues.apache.org/jira/browse/KAFKA-10704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tushar Bhasme resolved KAFKA-10704.
---
Resolution: Not A Problem
> Mirror maker with TLS at target
> ---
>
> Key: KAFKA-10704
> URL: https://issues.apache.org/jira/browse/KAFKA-10704
> Project: Kafka
> Issue Type: Bug
> Components: mirrormaker
>Affects Versions: 2.6.0
>Reporter: Tushar Bhasme
>Priority: Critical
> Fix For: 2.7.0
>
>
> We need to setup mirror maker from a single node kafka cluster to a three
> node Strimzi cluster. There is no SSL setup at source, however the target
> cluster is configured with MTLS.
> With below config, commands from source like listing topics etc are working:
> {code:java}
> cat client-ssl.properties
> security.protocol=SSL
> ssl.truststore.location=my.truststore
> ssl.truststore.password=123456
> ssl.keystore.location=my.keystore
> ssl.keystore.password=123456
> ssl.key.password=password{code}
> However, we are not able to get mirror maker working with the similar configs:
> {code:java}
> source.security.protocol=PLAINTEXT
> target.security.protocol=SSL
> target.ssl.truststore.location=my.truststore
> target.ssl.truststore.password=123456
> target.ssl.keystore.location=my.keystore
> target.ssl.keystore.password=123456
> target.ssl.key.password=password{code}
> Errors while running mirror maker:
> {code:java}
> org.apache.kafka.common.errors.TimeoutException: Call(callName=fetchMetadata,
> deadlineMs=1605011994642, tries=1, nextAllowedTryMs=1605011994743) timed out
> at 1605011994643 after 1 attempt(s)
> Caused by: org.apache.kafka.common.errors.TimeoutException: Timed out waiting
> for a node assignment. Call: fetchMetadata
> [2020-11-10 12:40:24,642] INFO App info kafka.admin.client for adminclient-8
> unregistered (org.apache.kafka.common.utils.AppInfoParser:83)
> [2020-11-10 12:40:24,643] INFO [AdminClient clientId=adminclient-8] Metadata
> update failed
> (org.apache.kafka.clients.admin.internals.AdminMetadataManager:235)
> org.apache.kafka.common.errors.TimeoutException: Call(callName=fetchMetadata,
> deadlineMs=1605012024643, tries=1, nextAllowedTryMs=-9223372036854775709)
> timed out at 9223372036854775807 after 1attempt(s)
> Caused by: org.apache.kafka.common.errors.TimeoutException: The AdminClient
> thread has exited. Call: fetchMetadata
> [2020-11-10 12:40:24,644] INFO Metrics scheduler closed
> (org.apache.kafka.common.metrics.Metrics:668)
> [2020-11-10 12:40:24,644] INFO Closing reporter
> org.apache.kafka.common.metrics.JmxReporter
> (org.apache.kafka.common.metrics.Metrics:672)
> [2020-11-10 12:40:24,644] INFO Metrics reporters closed
> (org.apache.kafka.common.metrics.Metrics:678)
> [2020-11-10 12:40:24,645] ERROR Stopping due to error
> (org.apache.kafka.connect.mirror.MirrorMaker:304)
> org.apache.kafka.connect.errors.ConnectException: Failed to connect to and
> describe Kafka cluster. Check worker's broker connection and security
> properties.
> at
> org.apache.kafka.connect.util.ConnectUtils.lookupKafkaClusterId(ConnectUtils.java:70)
> at
> org.apache.kafka.connect.util.ConnectUtils.lookupKafkaClusterId(ConnectUtils.java:51)
> at
> org.apache.kafka.connect.mirror.MirrorMaker.addHerder(MirrorMaker.java:235)
> at
> org.apache.kafka.connect.mirror.MirrorMaker.lambda$new$1(MirrorMaker.java:136)
> at java.lang.Iterable.forEach(Iterable.java:75)
> at
> org.apache.kafka.connect.mirror.MirrorMaker.(MirrorMaker.java:136)
> at
> org.apache.kafka.connect.mirror.MirrorMaker.(MirrorMaker.java:148)
> at
> org.apache.kafka.connect.mirror.MirrorMaker.main(MirrorMaker.java:291)
> Caused by: java.util.concurrent.ExecutionException:
> org.apache.kafka.common.errors.TimeoutException: Call(callName=listNodes,
> deadlineMs=1605012024641, tries=1, nextAllowedTryMs=1605012024742)timed out
> at 1605012024642 after 1 attempt(s)
> at
> org.apache.kafka.common.internals.KafkaFutureImpl.wrapAndThrow(KafkaFutureImpl.java:45)
> at
> org.apache.kafka.common.internals.KafkaFutureImpl.access$000(KafkaFutureImpl.java:32)
> at
> org.apache.kafka.common.internals.KafkaFutureImpl$SingleWaiter.await(KafkaFutureImpl.java:89)
> at
> org.apache.kafka.common.internals.KafkaFutureImpl.get(KafkaFutureImpl.java:260)
> at
> org.apache.kafka.connect.util.ConnectUtils.lookupKafkaClusterId(ConnectUtils.java:64)
> ... 7 more
> Caused by: org.apache.kafka.common.errors.TimeoutException:
> Call(callName=listNodes, deadlineMs=1605012024641, tries=1,
> nextAllowedTryMs=1605012024742) timed out at 1605012024642 after 1 attempt(s)
> Caused by: org.apache.kafka.common.errors.TimeoutException: Timed out waiting
> for a node assignment. Call: listNodes
> {code}
--
This
Re: [VOTE] KIP-681: Rename master key in delegation token feature
+1 (binding) Thanks Tom On Mon, Nov 16, 2020 at 3:58 PM Gwen Shapira wrote: > > +1 > Thank you for catching this. > > > On Mon, Nov 16, 2020, 3:22 AM Tom Bentley wrote: > > > Hi, > > > > I'd like to start a vote on KIP-681 which proposes to rename a broker > > config to use a more racially neutral term: > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-681 > > %3A+Rename+master+key+in+delegation+token+feature > > > > Kind regards, > > > > Tom > >
Re: New contributor
Hi Vidor, I've added you to the contributor list. Thanks On Mon, Nov 16, 2020 at 5:23 PM Vidor Kanalas wrote: > > Hello, > > could you please add me as a contributor for Apache Kafka? > github: dorwi (https://github.com/dorwi) > jira: dorwi > > Thanks, > Vidor
[jira] [Created] (KAFKA-10729) KIP-482: Bump remaining RPC's to use flexible versions
Gardner Vickers created KAFKA-10729:
---
Summary: KIP-482: Bump remaining RPC's to use flexible versions
Key: KAFKA-10729
URL: https://issues.apache.org/jira/browse/KAFKA-10729
Project: Kafka
Issue Type: Improvement
Reporter: Gardner Vickers
Assignee: Gardner Vickers
With
[KIP-482|https://cwiki.apache.org/confluence/display/KAFKA/KIP-482%3A+The+Kafka+Protocol+should+Support+Optional+Tagged+Fields],
the Kafka protocol gained support for tagged fields.
Not all RPC's were bumped to use flexible versioning and tagged fields. We
should bump the remaining RPC's and provide a new IBP to take advantage of
tagged fields via the flexible versioning mechanism.
The RPC's which need to be bumped are:
{code:java}
AddOffsetsToTxnRequest
AddOffsetsToTxnResponse
AddPartitionsToTxnRequest
AddPartitionsToTxnResponse
AlterClientQuotasRequest
AlterClientQuotasResponse
AlterConfigsRequest
AlterConfigsResponse
AlterReplicaLogDirsRequest
AlterReplicaLogDirsResponse
DescribeClientQuotasRequest
DescribeClientQuotasResponse
DescribeConfigsRequest
DescribeConfigsResponse
EndTxnRequest
EndTxnResponse
ListOffsetRequest
ListOffsetResponse
OffsetForLeaderEpochRequest
OffsetForLeaderEpochResponse
ProduceRequest
ProduceResponse
WriteTxnMarkersRequest
WriteTxnMarkersResponse
{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
New Contributor
Dear team, could you please add me as a contributor for Apache Kafka? github: dorwi (https://github.com/lqjack/kafka) jira: username : Jack-Lee, full name : lqjacklee
Looking for a PR review for FinalizedFeatureCache cleanup
Hi, I'm looking for a PR review for a small cleanup/refactor to use string interpolation in FinalizedFeatureCache class. Please could one of you help review this change? Here is a link to the PR: https://github.com/apache/kafka/pull/9602 Cheers, Kowshik
[jira] [Resolved] (KAFKA-10709) Sender#sendProduceRequest should use auto-generated protocol directly
[ https://issues.apache.org/jira/browse/KAFKA-10709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chia-Ping Tsai resolved KAFKA-10709. Resolution: Won't Fix This fix will be included by https://github.com/apache/kafka/pull/9401 > Sender#sendProduceRequest should use auto-generated protocol directly > - > > Key: KAFKA-10709 > URL: https://issues.apache.org/jira/browse/KAFKA-10709 > Project: Kafka > Issue Type: Improvement >Reporter: Chia-Ping Tsai >Assignee: Chia-Ping Tsai >Priority: Minor > > That can avoid extra conversion to improve the performance. > related discussion: > https://github.com/apache/kafka/pull/9401#discussion_r521902936 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (KAFKA-10730) KafkaApis#handleProduceRequest should use auto-generated protocol
Chia-Ping Tsai created KAFKA-10730: -- Summary: KafkaApis#handleProduceRequest should use auto-generated protocol Key: KAFKA-10730 URL: https://issues.apache.org/jira/browse/KAFKA-10730 Project: Kafka Issue Type: Improvement Reporter: Chia-Ping Tsai Assignee: Chia-Ping Tsai the construction of ProduceResponse is able to accept auto-generated protocol data so KafkaApis#handleProduceRequest should apply auto-generated protocol to avoid extra conversion. -- This message was sent by Atlassian Jira (v8.3.4#803005)
