Build failed in Jenkins: kafka-1.1-jdk7 #97

See Changes: [ismael] KAFKA-6683; Ensure producer state not mutated prior to append (#4755) -- [...truncated 421.90 KB...] kafka.zk.KafkaZkClientTest > testGetDataAndVersion

Build failed in Jenkins: kafka-trunk-jdk7 #3282

See Changes: [ismael] KAFKA-6683; Ensure producer state not mutated prior to append (#4755) [github] MINOR: Java 9/10 fixes, gradle and minor deps update (#4725) -- Start

Build failed in Jenkins: kafka-trunk-jdk7 #3281

See Changes: [github] KAFKA-6611: PART I, Use JMXTool in SimpleBenchmark (#4650) -- [...truncated 1.69 MB...] org.apache.kafka.streams.integration.TableTableJoinIntegratio

Re: Gradle strategy for exposing and using public test-utils modules

+1 > 在 2018年3月23日，下午12:20，Ted Yu 写道： > > +1 > Original message From: "Matthias J. Sax" > Date: 3/22/18 9:07 PM (GMT-08:00) To: > dev@kafka.apache.org Subject: Re: Gradle strategy for exposing and using > public test-utils modules > +1 from my side. > > -Matthias > > On 3

[jira] [Resolved] (KAFKA-6683) ReplicaFetcher crashes with "Attempted to complete a transaction which was not started"

[ https://issues.apache.org/jira/browse/KAFKA-6683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ismael Juma resolved KAFKA-6683. Resolution: Fixed Fix Version/s: 1.1.0 > ReplicaFetcher crashes with "Attempted to complete a

Re: Kafka Connect task re-balance repeatedly

Thanks Ewen. Will take a look at the config and if there are any findings, will come back here later. -- Sent from my iPhone On Mar 22, 2018, at 7:39 PM, Ewen Cheslack-Postava wrote: The log is showing that the Connect worker is trying to make sure it has read the entire log and gets to offset

Re: Gradle strategy for exposing and using public test-utils modules

+1 Original message From: "Matthias J. Sax" Date: 3/22/18 9:07 PM (GMT-08:00) To: dev@kafka.apache.org Subject: Re: Gradle strategy for exposing and using public test-utils modules +1 from my side. -Matthias On 3/22/18 5:12 PM, John Roesler wrote: > Yep, I'm super happy wi

Re: [DISCUSS] KIP-262 Metadata should include the number of state stores for task

Hi Richard, with KIP-268 in place (should be accepted soon) the upgrade path is covered. Thus, you can update your KIP accordingly, referring to KIP-268. Can you also update your KIP similar to KIP-268 to cover the old and new metadata format? Thanks! -Matthias On 2/24/18 4:07 PM, Richard Yu

Re: [DISCUSS] KIP-268: Simplify Kafka Streams Rebalance Metadata Upgrade

Yes, absolutely. I'll reply to the other thread. -Matthias On 3/21/18 8:45 PM, Richard Yu wrote: > Hi Matthias, > > Just wondering, once this KIP goes through. Could I restart my older KIP > to update SubscriptionInfo? > > Thanks > Richard > > On Wed, Mar 21, 2018 at 11:18 AM, Matthias J. Sa

Re: Gradle strategy for exposing and using public test-utils modules

+1 from my side. -Matthias On 3/22/18 5:12 PM, John Roesler wrote: > Yep, I'm super happy with this approach vs. a third module just for the > tests. > > For clairty, here's a PR demonstrating the model we're proposing: > https://github.com/apache/kafka/pull/4760 > > Thanks, > -John > > On Thu

Re: Kafka Connect task re-balance repeatedly

The log is showing that the Connect worker is trying to make sure it has read the entire log and gets to offset 119, but some other worker says it has read to offset 169. The two are in inconsistent states, so the one that seems to be behind will not start work with potentially outdated configurati

Build failed in Jenkins: kafka-trunk-jdk8 #2496

See Changes: [github] MINOR: Fix AdminClient.describeConfigs() of listener configs (#4747) -- [...truncated 3.67 MB...] org.apache.kafka.streams.state.internals.RocksDBKe

Jenkins build is back to normal : kafka-trunk-jdk7 #3280

See

[jira] [Created] (KAFKA-6705) producer.send() should be non-blocking

Dong Lin created KAFKA-6705: --- Summary: producer.send() should be non-blocking Key: KAFKA-6705 URL: https://issues.apache.org/jira/browse/KAFKA-6705 Project: Kafka Issue Type: Improvement

Build failed in Jenkins: kafka-1.1-jdk7 #96

See Changes: [rajinisivaram] MINOR: Fix AdminClient.describeConfigs() of listener configs (#4747) -- [...truncated 1.53 MB...] org.apache.kafka.streams.integration.TableTable

Re: [DISCUSS] KIP-253: Support in-order message delivery with partition expansion

Hi, Jan, Thanks for the reply. Ok. So your idea is to have a special consumer reshuffle the data to a new set of partitions. Each consumer group will not increase its tasks immediately. We just need to make sure that each consumer instance is assigned the set of new partitions covering the same s

Re: Gradle strategy for exposing and using public test-utils modules

Yep, I'm super happy with this approach vs. a third module just for the tests. For clairty, here's a PR demonstrating the model we're proposing: https://github.com/apache/kafka/pull/4760 Thanks, -John On Thu, Mar 22, 2018 at 6:21 PM, Guozhang Wang wrote: > I'm +1 to the approach as well across

Re: Gradle strategy for exposing and using public test-utils modules

I'm +1 to the approach as well across modules that are going to have test utils artifacts in the future. To me this seems to be a much smaller change we can make to break the circular dependencies than creating a new package for our own testing code. Guozhang On Thu, Mar 22, 2018 at 1:26 PM, Bill

Re: [VOTE] KIP-268: Simplify Kafka Streams Rebalance Metadata Upgrade

+1 | | zhenya Sun 邮箱：toke...@126.com | 签名由 网易邮箱大师 定制 On 03/23/2018 03:34, James Cheng wrote: +1 (non-binding) -James > On Mar 21, 2018, at 2:28 AM, Damian Guy wrote: > > +1 > > On Wed, 21 Mar 2018 at 01:44 abel-sun wrote: > >> >> Thanks you of your offer ,agree with you! >> >> On 2018/03/

Re: [VOTE] KIP-268: Simplify Kafka Streams Rebalance Metadata Upgrade

Thanks Matthias for the detailed plan of implementation. As people seem to agree on keeping the upgrade.from config to avoid enforcing to upgrades via 1.2, I'm +1 on the proposal. Guozhang On Tue, Mar 20, 2018 at 3:42 PM, Matthias J. Sax wrote: > Hi, > > I would like to start the vote for KI

Build failed in Jenkins: kafka-trunk-jdk7 #3279

See Changes: [github] MINOR: Fix AdminClient.describeConfigs() of listener configs (#4747) -- [...truncated 422.58 KB...] kafka.security.auth.SimpleAclAuthorizerTest > te

Re: [VOTE] KIP-257 - Configurable Quota Management

+1 Original message From: Rajini Sivaram Date: 3/22/18 2:56 PM (GMT-08:00) To: dev Subject: [VOTE] KIP-257 - Configurable Quota Management Hi all, I would like to start vote on KIP-257 to enable customisation of client quota computation: https://cwiki.apache.org/confluence

[VOTE] KIP-257 - Configurable Quota Management

Hi all, I would like to start vote on KIP-257 to enable customisation of client quota computation: https://cwiki.apache.org/confluence/display/KAFKA/KIP-257+-+Configurable+Quota+Management The KIP proposes to make quota management pluggable to enable group-based and partition-based quotas for cl

Re: [DISCUSS] KIP-257 - Configurable Quota Management

Since there all the comments so far have been addressed, I will start vote for this KIP. Regards, Rajini On Thu, Mar 15, 2018 at 6:37 PM, Rajini Sivaram wrote: > Thanks, Jun. > > 11. updatePartitionMetadata() provides all partitions with their leaders > so that callbacks that scale down quotas

Re: [VOTE] KIP-268: Simplify Kafka Streams Rebalance Metadata Upgrade

+1 (non-binding) On Thu, Mar 22, 2018 at 2:34 PM, James Cheng wrote: > +1 (non-binding) > > -James > > > On Mar 21, 2018, at 2:28 AM, Damian Guy wrote: > > > > +1 > > > > On Wed, 21 Mar 2018 at 01:44 abel-sun > wrote: > > > >> > >> Thanks you of your offer ,agree with you! > >> > >> On 2018/

[jira] [Created] (KAFKA-6704) Checking hasNext from SegementIterator could throw InvalidStateStoreException

Bill Bejeck created KAFKA-6704: -- Summary: Checking hasNext from SegementIterator could throw InvalidStateStoreException Key: KAFKA-6704 URL: https://issues.apache.org/jira/browse/KAFKA-6704 Project: Kaf

Re: Gradle strategy for exposing and using public test-utils modules

John, Thanks for the clear, detailed explanation. I'm +1 on what you have proposed. While I agree with you manually pulling in transitive test dependencies is not ideal, in this case, I think it's worth it to get over the circular dependency hurdle and use streams:test-utils ourselves. -Bill On

Gradle strategy for exposing and using public test-utils modules

Hey everyone, In 1.1, kafka-streams adds an artifact called 'kafka-streams-test-utils' (see https://kafka.apache.org/11/documentation/streams/developer-guide/testing.html ). The basic idea is to provide first-class support for testing Kafka Streams applications. Without that, users were forced to

Re: [VOTE] KIP-268: Simplify Kafka Streams Rebalance Metadata Upgrade

+1 (non-binding) -James > On Mar 21, 2018, at 2:28 AM, Damian Guy wrote: > > +1 > > On Wed, 21 Mar 2018 at 01:44 abel-sun wrote: > >> >> Thanks you of your offer ,agree with you! >> >> On 2018/03/21 00:56:11, Richard Yu wrote: >>> Hi Matthias, >>> Thanks for setting up the upgrade path.

Re: [DISCUSS] KIP-268: Simplify Kafka Streams Rebalance Metadata Upgrade

Sure. Glad you like this KIP. (KIP-258 will be more difficult... It seems that is was a good decision into two). -Matthias On 3/22/18 11:35 AM, James Cheng wrote: > > >> On Mar 21, 2018, at 11:45 PM, Matthias J. Sax wrote: >> >> Yes, it only affects the metadata. KIP-268 targets metadata upg

Re: [DISCUSS] KIP-272: Add API version tag to broker's RequestsPerSec metric

> > This kind of change will be problematic to us as the total RequestsPerSec > will be double counted in our metric system as we do automatic aggregation. > It has been a headache for us as we always have to do some special handling > when querying such metrics. > I agree with Allen, most of the

Re: [DISCUSS] KIP-268: Simplify Kafka Streams Rebalance Metadata Upgrade

> On Mar 21, 2018, at 11:45 PM, Matthias J. Sax wrote: > > Yes, it only affects the metadata. KIP-268 targets metadata upgrade > without store upgrade. > > We can discuss store upgrade further in KIP-258: I think in general, the > upgrade/downgrade behavior might be an issue for upgrading stor

[jira] [Created] (KAFKA-6703) MirrorMaker cannot make progress when any matched topic from a whitelist regexp has -1 leader

Attila Sasvari created KAFKA-6703: - Summary: MirrorMaker cannot make progress when any matched topic from a whitelist regexp has -1 leader Key: KAFKA-6703 URL: https://issues.apache.org/jira/browse/KAFKA-6703

Re: [DISCUSS] KIP-272: Add API version tag to broker's RequestsPerSec metric

Hi James, I updated the KIP to reflect the exact change to the metric as you suggested. > kafka.network:type=RequestMetrics,name=RequestsPerSec,request=Produce > > And the KIP is proposing to change it to: > > kafka.network:type=RequestMetrics,name=RequestsPerSec,request= Produce,version=1.0.0 >

Re: [DISCUSS] KIP-270 A Scala wrapper library for Kafka Streams

Submitted the PR .. https://github.com/apache/kafka/pull/4756 and updated the KIP https://cwiki.apache.org/confluence/display/KAFKA/KIP-270+-+A+Scala+Wrapper+Library+for+Kafka+Streams regards. On Tue, Mar 20, 2018 at 4:06 AM, Ismael Juma wrote: > Matthias, > > You have to choose between the pac

Kafka Connect task re-balance repeatedly

Hi, I have 2 Kafka Connect instances runs in 2 boxes which forms a Kafka Connect Cluster. One of the instance seems doing the re-balance repeatedly in a dead loop without running the actual Sink task, the other works fine. The following is the output message in the console. May I ask if you have

Re: [DISCUSSION] KIP-266: Add TimeoutException to KafkaConsumer#position()

I do have one question though: in the current KIP, throwing TimeoutException to mark that time limit is exceeded is applied to all new methods introduced in this proposal. However, how would users respond when a TimeoutException (since it is considered a RuntimeException)? Thanks, Richard On Mo