ableegoldman opened a new pull request, #13134:
URL: https://github.com/apache/kafka/pull/13134
This test is supposed to be a sanity check that rebalancing with a large
number of partitions/consumers won't start to take obscenely long or approach
the `max.poll.interval.ms` -- bumping up the
divijvaidya opened a new pull request, #13135:
URL: https://github.com/apache/kafka/pull/13135
This covers two JIRAs https://issues.apache.org/jira/browse/KAFKA-14632 and
https://issues.apache.org/jira/browse/KAFKA-14633
## Background
> Compression optimization: Use zstd dictionary based (de)compression
clolov commented on PR #12821:
URL: https://github.com/apache/kafka/pull/12821#issuecomment-1398172190
Thank you very much for the review and merge!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go t
[
https://issues.apache.org/jira/browse/KAFKA-14642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17679105#comment-17679105
]
Satish Duggana commented on KAFKA-14642:
Thanks [~hzh0425@apache] for the JIRA.
divijvaidya commented on code in PR #13135:
URL: https://github.com/apache/kafka/pull/13135#discussion_r1082366873
##
clients/src/main/java/org/apache/kafka/common/compress/ZstdFactory.java:
##
@@ -26,21 +26,25 @@
import org.apache.kafka.common.utils.ByteBufferInputStream;
imp
[
https://issues.apache.org/jira/browse/KAFKA-14642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17679106#comment-17679106
]
Alexandre Dupriez commented on KAFKA-14642:
---
Thanks [~hzh0425@apache] for the
[
https://issues.apache.org/jira/browse/KAFKA-14642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17679124#comment-17679124
]
hzh0425 commented on KAFKA-14642:
-
Thanks for your reply! [~satish.duggana] [~Hangleton]
[
https://issues.apache.org/jira/browse/KAFKA-14642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17679131#comment-17679131
]
hzh0425 commented on KAFKA-14642:
-
BTW, can I implement a Rocksdb based metadata storage
[
https://issues.apache.org/jira/browse/KAFKA-14642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17679131#comment-17679131
]
hzh0425 edited comment on KAFKA-14642 at 1/20/23 12:31 PM:
---
BT
[
https://issues.apache.org/jira/browse/KAFKA-14579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17679147#comment-17679147
]
Federico Valeri commented on KAFKA-14579:
-
This tool has dependency on core modu
[
https://issues.apache.org/jira/browse/KAFKA-14579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17679147#comment-17679147
]
Federico Valeri edited comment on KAFKA-14579 at 1/20/23 1:28 PM:
[
https://issues.apache.org/jira/browse/KAFKA-14579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Federico Valeri reassigned KAFKA-14579:
---
Assignee: (was: Federico Valeri)
> Move DumpLogSegments to tools
>
divijvaidya commented on PR #13135:
URL: https://github.com/apache/kafka/pull/13135#issuecomment-1398428117
@ijuma please review when you get a chance since you already have context
about this code change.
--
This is an automated message from the Apache Git Service.
To respond to the mess
[
https://issues.apache.org/jira/browse/KAFKA-14565?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Terry Beard updated KAFKA-14565:
Description:
The Consumer and Producer interceptor interfaces and their corresponding Kafka
Consu
[
https://issues.apache.org/jira/browse/KAFKA-14642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17679158#comment-17679158
]
Alexandre Dupriez commented on KAFKA-14642:
---
In addition to here, I would reco
ijuma commented on code in PR #13135:
URL: https://github.com/apache/kafka/pull/13135#discussion_r1082588469
##
clients/src/main/java/org/apache/kafka/common/compress/ZstdFactory.java:
##
@@ -62,10 +68,11 @@ public void release(ByteBuffer buffer) {
}
ijuma commented on PR #13135:
URL: https://github.com/apache/kafka/pull/13135#issuecomment-1398459548
One more thing: when it comes to the testing, can we include the case where
the batches have a single 10 byte message?
--
This is an automated message from the Apache Git Service.
To resp
divijvaidya commented on code in PR #13135:
URL: https://github.com/apache/kafka/pull/13135#discussion_r1082615267
##
clients/src/main/java/org/apache/kafka/common/record/CompressionType.java:
##
@@ -126,6 +144,11 @@ public OutputStream wrapForOutput(ByteBufferOutputStream
buff
ijuma commented on code in PR #13135:
URL: https://github.com/apache/kafka/pull/13135#discussion_r1082622801
##
clients/src/main/java/org/apache/kafka/common/record/CompressionType.java:
##
@@ -126,6 +144,11 @@ public OutputStream wrapForOutput(ByteBufferOutputStream
buffer, by
ijuma commented on code in PR #13135:
URL: https://github.com/apache/kafka/pull/13135#discussion_r1082623877
##
clients/src/main/java/org/apache/kafka/common/record/CompressionType.java:
##
@@ -126,6 +144,11 @@ public OutputStream wrapForOutput(ByteBufferOutputStream
buffer, by
clolov commented on code in PR #13131:
URL: https://github.com/apache/kafka/pull/13131#discussion_r1082663006
##
server-common/src/main/java/org/apache/kafka/server/util/CommandLineUtils.java:
##
@@ -0,0 +1,201 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under on
fvaleri commented on code in PR #13131:
URL: https://github.com/apache/kafka/pull/13131#discussion_r1082712140
##
server-common/src/main/java/org/apache/kafka/server/util/CommandLineUtils.java:
##
@@ -0,0 +1,201 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under o
fvaleri opened a new pull request, #13136:
URL: https://github.com/apache/kafka/pull/13136
This PR is based on https://github.com/apache/kafka/pull/13131.
This class is also used by the system tests, so I need to check if the
replacement works fine there too.
--
This is an automate
kamalcph commented on PR #13059:
URL: https://github.com/apache/kafka/pull/13059#issuecomment-1398670571
@ijuma @showuon
can you please merge the patch?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
U
kamalcph commented on PR #13060:
URL: https://github.com/apache/kafka/pull/13060#issuecomment-1398673057
@ijuma @showuon
Please take a look when you get chance.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
U
=BrokerTopicMetrics,name=MessagesInPerSec --attributes
FifteenMinuteRate,FiveMinuteRate --date-format "MMdd-hh:mm:ss"
--reporting-interval 1000 --report-format tsv
Trying to connect to JMX url: service:jmx:rmi:///jndi/rmi://:/jmxrmi
time 20230120-06:23:14
kafka.s
C0urante opened a new pull request, #13137:
URL: https://github.com/apache/kafka/pull/13137
[Jira](https://issues.apache.org/jira/browse/KAFKA-10586)
Implements the internal REST API changes described in
[KIP-710](https://cwiki.apache.org/confluence/display/KAFKA/KIP-710%3A+Full+suppo
C0urante commented on PR #13137:
URL: https://github.com/apache/kafka/pull/13137#issuecomment-1398797302
@gharris1727 @viktorsomogyi @mimaison would you mind taking a look at this
when you have a moment?
--
This is an automated message from the Apache Git Service.
To respond to the messag
hachikuji opened a new pull request, #13138:
URL: https://github.com/apache/kafka/pull/13138
This patch contains a few cleanups in the new refactored consumer logic:
- Use `CompletableFuture` instead of `RequestFuture` in
`NetworkClientDelegate`. This is a much more extensible API and
philipnee commented on code in PR #13138:
URL: https://github.com/apache/kafka/pull/13138#discussion_r1082970121
##
clients/src/main/java/org/apache/kafka/clients/consumer/internals/NetworkClientDelegate.java:
##
@@ -97,8 +97,8 @@ private void trySend(final long currentTimeMs) {
kirktrue commented on code in PR #12813:
URL: https://github.com/apache/kafka/pull/12813#discussion_r1082997576
##
clients/src/main/java/org/apache/kafka/clients/producer/internals/Sender.java:
##
@@ -641,7 +643,7 @@ private void completeBatch(ProducerBatch batch,
ProduceRespon
kirktrue commented on code in PR #12813:
URL: https://github.com/apache/kafka/pull/12813#discussion_r1082997268
##
clients/src/main/java/org/apache/kafka/clients/producer/internals/Sender.java:
##
@@ -552,9 +552,11 @@ private void handleProduceResponse(ClientResponse
response,
kirktrue commented on code in PR #12813:
URL: https://github.com/apache/kafka/pull/12813#discussion_r1082998126
##
clients/src/test/java/org/apache/kafka/clients/producer/internals/SenderTest.java:
##
@@ -2519,7 +2519,7 @@ public void testInflightBatchesExpireOnDeliveryTimeout()
hachikuji commented on code in PR #13138:
URL: https://github.com/apache/kafka/pull/13138#discussion_r1083069990
##
clients/src/main/java/org/apache/kafka/clients/consumer/internals/NetworkClientDelegate.java:
##
@@ -97,8 +97,8 @@ private void trySend(final long currentTimeMs) {
ableegoldman commented on PR #13134:
URL: https://github.com/apache/kafka/pull/13134#issuecomment-1398999167
It's just timing out, there's no error beyond that
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL ab
ableegoldman merged PR #13134:
URL: https://github.com/apache/kafka/pull/13134
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscr...@kafka.ap
ableegoldman commented on PR #13134:
URL: https://github.com/apache/kafka/pull/13134#issuecomment-1399019480
Merged to trunk and cherrypicked to 3.4
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go t
ableegoldman merged PR #13132:
URL: https://github.com/apache/kafka/pull/13132
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscr...@kafka.ap
philipnee opened a new pull request, #13139:
URL: https://github.com/apache/kafka/pull/13139
The current documentation indicates two positions are tracked, but these
positions were removed a few years ago. Now we use a single position to track
the last consumed record. Updated the documenta
Jason Gustafson created KAFKA-14644:
---
Summary: Process should stop after failure in raft IO thread
Key: KAFKA-14644
URL: https://issues.apache.org/jira/browse/KAFKA-14644
Project: Kafka
Iss
hachikuji opened a new pull request, #13140:
URL: https://github.com/apache/kafka/pull/13140
Unexpected errors caught in the Raft IO thread should cause the process to
stop. This is similar to the handling of exceptions in the controller.
### Committer Checklist (excluded from commit
[
https://issues.apache.org/jira/browse/KAFKA-14382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Guozhang Wang updated KAFKA-14382:
--
Labels: new-consumer-threading-should-fix rebalancing (was: rebalancing)
> StreamThreads can
[
https://issues.apache.org/jira/browse/KAFKA-14382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17679346#comment-17679346
]
Guozhang Wang commented on KAFKA-14382:
---
Thanks for catching this bug [~ableegoldm
guozhangwang merged PR #13139:
URL: https://github.com/apache/kafka/pull/13139
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: jira-unsubscr...@kafka.ap
cmccabe commented on code in PR #13140:
URL: https://github.com/apache/kafka/pull/13140#discussion_r1083120749
##
core/src/main/scala/kafka/raft/RaftManager.scala:
##
@@ -47,19 +47,27 @@ import org.apache.kafka.raft.RaftConfig.{AddressSpec,
InetAddressSpec, NON_ROUTA
import or
cmccabe commented on code in PR #13140:
URL: https://github.com/apache/kafka/pull/13140#discussion_r1083122187
##
core/src/test/scala/unit/kafka/raft/RaftManagerTest.scala:
##
@@ -221,16 +225,23 @@ class RaftManagerTest {
@Test
def testUncaughtExceptionInIoThread(): Unit =
guozhangwang commented on code in PR #13138:
URL: https://github.com/apache/kafka/pull/13138#discussion_r1083125061
##
clients/src/main/java/org/apache/kafka/clients/consumer/internals/CoordinatorRequestManager.java:
##
@@ -149,11 +156,14 @@ private void onSuccessfulResponse(fin
guozhangwang commented on code in PR #12813:
URL: https://github.com/apache/kafka/pull/12813#discussion_r1083134655
##
clients/src/main/java/org/apache/kafka/clients/producer/internals/Sender.java:
##
@@ -549,7 +549,13 @@ private boolean awaitNodeReady(Node node,
FindCoordinato
kirktrue closed pull request #12945: KAFKA-14365: Refactor Fetcher to allow
different implementations
URL: https://github.com/apache/kafka/pull/12945
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to
philipnee commented on PR #13139:
URL: https://github.com/apache/kafka/pull/13139#issuecomment-1399093410
Thanks!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubsc
ableegoldman opened a new pull request, #13141:
URL: https://github.com/apache/kafka/pull/13141
The timeouts used for starting up Streams and waiting for the RUNNING state
are all over the place across our integration tests, with some as low as 15s
(which are unsurprisingly rather flaky). W
vcrfxia opened a new pull request, #13142:
URL: https://github.com/apache/kafka/pull/13142
This PR refactors how the list of open iterators for RocksDB stores is
managed. Prior to this PR, the `openIterators` list was passed into the
constructor for the iterators themselves, allowing `Rocks
vcrfxia opened a new pull request, #13143:
URL: https://github.com/apache/kafka/pull/13143
(This PR is stacked on https://github.com/apache/kafka/pull/13142. The first
commit does not need to be reviewed separately.)
Today's KeyValueSegments create a new RocksDB instance for each
Key
hachikuji commented on code in PR #13138:
URL: https://github.com/apache/kafka/pull/13138#discussion_r1083199766
##
clients/src/main/java/org/apache/kafka/clients/consumer/internals/CoordinatorRequestManager.java:
##
@@ -149,11 +156,14 @@ private void onSuccessfulResponse(final
mjsax commented on code in PR #13126:
URL: https://github.com/apache/kafka/pull/13126#discussion_r1083223501
##
streams/src/main/java/org/apache/kafka/streams/state/internals/RocksDBVersionedStoreSegmentValueFormatter.java:
##
@@ -0,0 +1,523 @@
+/*
+ * Licensed to the Apache Sof
mjsax commented on code in PR #13126:
URL: https://github.com/apache/kafka/pull/13126#discussion_r1083224137
##
streams/src/main/java/org/apache/kafka/streams/state/internals/RocksDBVersionedStoreSegmentValueFormatter.java:
##
@@ -0,0 +1,523 @@
+/*
+ * Licensed to the Apache Sof
mjsax commented on code in PR #13126:
URL: https://github.com/apache/kafka/pull/13126#discussion_r1081924210
##
streams/src/main/java/org/apache/kafka/streams/state/internals/RocksDBVersionedStoreSegmentValueFormatter.java:
##
@@ -0,0 +1,523 @@
+/*
+ * Licensed to the Apache Sof
mjsax commented on code in PR #13126:
URL: https://github.com/apache/kafka/pull/13126#discussion_r1083231834
##
streams/src/main/java/org/apache/kafka/streams/state/internals/RocksDBVersionedStoreSegmentValueFormatter.java:
##
@@ -0,0 +1,523 @@
+/*
+ * Licensed to the Apache Sof
[
https://issues.apache.org/jira/browse/KAFKA-14642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17679393#comment-17679393
]
hzh0425 commented on KAFKA-14642:
-
Sure, I will do that
> TopicBased RemoteLogMetadataM
61 matches
Mail list logo
