Build failed in Jenkins: kafka-trunk-jdk11 #247

See Changes: [junrao] KAFKA-7837: Ensure offline partitions are picked up as soon as possible [vahid.hashemian] Added quotes around the class path (#4469) -- [...truncate

Re: [DISCUSS] KIP-415: Incremental Cooperative Rebalancing in Kafka Connect

Thanks for the KIP Konstantine. Quick question: introducing a new serialization format (ie flatbuffers) has major implications. Have we considered json? If so, why did we reject it? Ismael On Fri, Jan 11, 2019, 3:44 PM Konstantine Karantasis < konstant...@confluent.io wrote: > Hi all, > > I just

Build failed in Jenkins: kafka-trunk-jdk8 #3349

See Changes: [colin] MINOR: In the MetadataResponse schema, ignorable should be a boolean -- [...truncated 4.54 MB...] org.apache.kafka.streams.scala.TopologyTest > shou

Build failed in Jenkins: kafka-trunk-jdk11 #246

See Changes: [colin] MINOR: In the MetadataResponse schema, ignorable should be a boolean -- [...truncated 2.28 MB...] org.apache.kafka.connect.runtime.WorkerSinkTaskThrea

[jira] [Resolved] (KAFKA-7837) maybeShrinkIsr may not reflect OfflinePartitions immediately

[ https://issues.apache.org/jira/browse/KAFKA-7837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao resolved KAFKA-7837. Resolution: Fixed Fix Version/s: 2.2.0 Merged the PR to trunk. > maybeShrinkIsr may not reflect Off

Build failed in Jenkins: kafka-trunk-jdk8 #3348

See Changes: [wangguoz] MINOR: clarify why suppress can sometimes drop tombstones (#6195) [junrao] KAFKA-5692: Change PreferredReplicaLeaderElectionCommand to use Admin… [junrao] KAFKA-7838: Log leader and foll

Build failed in Jenkins: kafka-trunk-jdk11 #245

See Changes: [wangguoz] MINOR: clarify why suppress can sometimes drop tombstones (#6195) [junrao] KAFKA-5692: Change PreferredReplicaLeaderElectionCommand to use Admin… [junrao] KAFKA-7838: Log leader and foll

[jira] [Created] (KAFKA-7873) KafkaBasedLog's consumer should always seek to beginning when starting

Randall Hauch created KAFKA-7873: Summary: KafkaBasedLog's consumer should always seek to beginning when starting Key: KAFKA-7873 URL: https://issues.apache.org/jira/browse/KAFKA-7873 Project: Kafka

Build failed in Jenkins: kafka-2.1-jdk8 #117

See Changes: [github] MINOR: bump Jackson version to 2.9.8 (#6199) -- [...truncated 520.02 KB...] kafka.api.AdminClientIntegrationTest > testDelayedClose PASSED kafka.api.Ad

[VOTE] 2.1.1 RC0

Hi all, This is the first candidate for release of Apache Kafka 2.1.1. This release includes many bug fixes for Apache Kafka 2.1. Check out the release notes here: http://home.apache.org/~cmccabe/kafka-2.1.1-rc0/RELEASE_NOTES.html The vote will go until Wednesday, January 30th. * Release arti

Build failed in Jenkins: kafka-trunk-jdk11 #244

See Changes: [colin] MINOR: Upgrade ducktape to 0.7.5 (#6197) -- [...truncated 2.27 MB...] org.apache.kafka.trogdor.rest.RestExceptionMapperTest > testToResponseClassNot

[jira] [Resolved] (KAFKA-7838) improve logging in Partition.maybeShrinkIsr()

[ https://issues.apache.org/jira/browse/KAFKA-7838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jun Rao resolved KAFKA-7838. Resolution: Fixed Fix Version/s: 2.2.0 Merged the PR to trunk. > improve logging in Partition.maybe

Re: [DISCUSS] KIP-415: Incremental Cooperative Rebalancing in Kafka Connect

Yeah, my expectation was that the use of flatbuffers in v1 was giving us another option to evolve the protocol, so I was hoping it would be some time before we needed v2. Maybe we could delay the introduction of `connect.protocol` until we're convinced it's necessary? The cost of supporting both v0

Build failed in Jenkins: kafka-trunk-jdk8 #3347

See Changes: [colin] MINOR: Upgrade ducktape to 0.7.5 (#6197) -- [...truncated 2.27 MB...] org.apache.kafka.streams.test.OutputVerifierTest > shouldFailIfValueIsDifferen

RE: [VOTE] [REMINDER] KIP-383 Pluggable interface for SSL Factory

Like I said, it is extremely unlikely to have custom configs in your pluggable interface now because Kafka does not support custom configs at the moment. Custom configs are desirable. I will change KIP-383 one more time assuming custom configs will be implemented one day. I will reuse your ssl ali

Re: Kafka official docker image

On Wed, Jan 23, 2019, at 20:16, Matt Farmer wrote: > Would it be possible to get official images that aren't Confluent branded > and only include the things that are maintained by under the scope of the > Apache PMC? Hi Matt, Apache doesn't officially distribute Docker images. The topic was disc

Re: [VOTE] [REMINDER] KIP-383 Pluggable interface for SSL Factory

Hi Clement, I think my note was probably not clear. Having SslEngineFactory separate from SslFactory is good. It simplifies reconfiguration since at least some of the logic from SslFactory can be retained. My question was around what custom configs `SslEngineFactory` implementations may have in

Re: [VOTE] KIP-421: Support resolving externalized secrets in AbstractConfig

On Fri, Jan 25, 2019, at 09:12, Andy Coates wrote: > > Further, if we're worried about confusion about how to) > load the two files, we could have a constructor that does that default > pattern for you. > > Yeah, I don't really see the need for this two step / two file approach. I > think the conf

Re: [DISCUSS] KIP-415: Incremental Cooperative Rebalancing in Kafka Connect

I was going to make a related comment about connect.protocol. Even if we have the config, we should default it to compatible given v0 state is small and we believe v1 is better and people should migrate to it. While I like getting rid of configs, not sure whether we should remove it here. If we th

[jira] [Created] (KAFKA-7872) Consider allowing transformations on GlobalKTable before materialization

Guozhang Wang created KAFKA-7872: Summary: Consider allowing transformations on GlobalKTable before materialization Key: KAFKA-7872 URL: https://issues.apache.org/jira/browse/KAFKA-7872 Project: Kafka

Re: [DISCUSS] KIP-307: Allow to define custom processor names with KStreams DSL

I was reading the KIP again, and there are still some open question and inconsistencies: For example for `KGroupedStream#count(Named)` the KIP says, that only the processor will be named, while the state store name will be `PREFIX + COUNT` (ie, an auto-generated name). Additionally, for `KGroupedS

Re: [DISCUSSION] KIP-430: Add support for user/client configuration in the Kafka Admin Client

This email thread is not needed any more, as I just updated the kip id. The new id is KIP 422. On Wed, Jan 23, 2019 at 3:30 PM Yaodong Yang wrote: > Hi folks, > > I've published KIP-430 which is about adding support for user/client > configurations in the Kafka Admin Client. > > Basically the st

Re: [VOTE] KIP-307: Allow to define custom processor names with KStreams DSL

Fair enough; could you summarize the open questions on the DISCUSS thread then? On Fri, Jan 25, 2019 at 9:46 AM Matthias J. Sax wrote: > Guozhang, > > KIP deadline passed yesterday. The last comments on the DISCUSS thread > are not addressed yet. We need to move this to 2.3. > > -Matthias > > On

[DISCUSSION] KIP-422: Add support for user/client configuration in the Kafka Admin Client

Hi folks, I've published KIP-422 which is about adding support for user/client configurations in the Kafka Admin Client. Basically the story here is to allow KafkaAdminClient to configure the user or client configurations for users, instead of requiring users to directly talk to ZK. The link for

Re: [VOTE] KIP-307: Allow to define custom processor names with KStreams DSL

Guozhang, KIP deadline passed yesterday. The last comments on the DISCUSS thread are not addressed yet. We need to move this to 2.3. -Matthias On 1/24/19 4:23 PM, Guozhang Wang wrote: > Hello folks, > > I'd like to call out for another time on voting for this KIP, given the > deadline is approa

Re: [DISCUSS] KIP-415: Incremental Cooperative Rebalancing in Kafka Connect

Hey Konstantine, Thanks for the reply. Just one response below: In 'compatible' mode, the worker sends both protocols to the broker > coordinator during the Join request. The field is already a list of > ProtocolMetadata. The broker, after gathering all the requests follows a > process of selecti

RE: [VOTE] [REMINDER] KIP-383 Pluggable interface for SSL Factory

That's what my previous solution was and it got vetoed. I was told to refactor my solution to keep the reconfiguration internal to Kafka. Indeed, the reconfiguration checks are not trivial and it is best if all pluggable implementations reuse it. Therefore it became an implicit goal not to let th

Re: [VOTE] KIP-421: Support resolving externalized secrets in AbstractConfig

> Further, if we're worried about confusion about how to load the two files, we could have a constructor that does that default pattern for you. Yeah, I don't really see the need for this two step / two file approach. I think the config providers should be listed in the main property file, not som

Re: [VOTE] KIP-369 Alternative Partitioner to Support "Always Round-Robin" Selection

Hello, I am trying to revive this thread. I only got 1 binding vote so far. Please feel free to revisit and comment here. Thanks, On Thu, 25 Oct 2018 at 00:15, M. Manna wrote: > Hey IJ, > > Thanks for your interest in the KIP. > > My point was simply that the round-robin should happen even if

Re: [VOTE] [REMINDER] KIP-383 Pluggable interface for SSL Factory

Not yet a vote, but almost there. Just wanted to clarify reconfiguration before voting. So the advantage of making `SslEngineFactory` reconfigurable is that it can define custom configs which may be reconfigured. Basically SslEngineFactory can define a set of custom reconfigurable configs and it w

RE: [VOTE] [REMINDER] KIP-383 Pluggable interface for SSL Factory

Is this a +1 binding vote? The KIP says: Since SslFactory will handle reconfiguration, the idea is to make the configuration immutable in the pluggable factory. SslFactory would create a new pluggable factory every time the configuration changes. The pluggable factory creates its SSLContext whe

Jenkins build is back to normal : kafka-2.1-jdk8 #116

See

Re: Kafka official docker image

Will certainly be useful. I would like to know what effort is needed for it. I can probably help. Regards, Amitav On Fri, Jan 25, 2019 at 6:15 PM Ishwor wrote: > Matt Farmer wrote: > > Would it be possible to get official images that aren't Confluent branded > > and only include the things that

Re: Kafka official docker image

Matt Farmer wrote: Would it be possible to get official images that aren't Confluent branded and only include the things that are maintained by under the scope of the Apache PMC? IIRC, the Confluent images typically include extras that are [...] Agreed. Would love to see lean `official` docker

Build failed in Jenkins: kafka-trunk-jdk8 #3346

See Changes: [vahid.hashemian] MINOR: Improve IntegrationTestUtils documentation (#5664) -- [...truncated 4.54 MB...] org.apache.kafka.streams.test.OutputVerifierTest >

Build failed in Jenkins: kafka-trunk-jdk11 #243

See Changes: [vahid.hashemian] MINOR: Improve IntegrationTestUtils documentation (#5664) -- [...truncated 2.27 MB...] org.apache.kafka.streams.test.ConsumerRecordFactoryTe

Re: [VOTE] KIP-349 Priorities for Source Topics

Hi Nick, a bit late to the party, sorry. I recently spent some time looking into this / a similar issue [1]. After some investigation and playing around with settings I think that the benefit that could be gained from this is somewhat limited and probably outweighed by the implementation effort.

[jira] [Created] (KAFKA-7871) Getting TimeoutException in Kafka Producer when using batch size 0

Pradeep Bansal created KAFKA-7871: - Summary: Getting TimeoutException in Kafka Producer when using batch size 0 Key: KAFKA-7871 URL: https://issues.apache.org/jira/browse/KAFKA-7871 Project: Kafka

Re: [VOTE] [REMINDER] KIP-383 Pluggable interface for SSL Factory

Hi Clement, Thanks for the KIP, it is looking good. Do you by any chance have a PR that you can attach to the JIRA? It wasn't clear to me how the SslEngineFactory got the new keystore/truststore if they were reconfigured. I am assuming the factory is creating the SSLContext and configuring the con

Build failed in Jenkins: kafka-2.1-jdk8 #115

See Changes: [jason] KAFKA-7692; Fix ProducerStateManager SequenceNumber overflow (#5990) [jason] KAFKA-7693; Fix SequenceNumber overflow in producer (#5989) -- [...truncate

Re: [VOTE] KIP-421: Support resolving externalized secrets in AbstractConfig

*Regarding brokers, I think if we want to avoid exposing secrets over DescribeConfigs responses, we'd probably need changes similar to those we needed to make for the Connect REST API. * Password configs are not returned in DescribeConfigs response in the broker. The response indicates that the co

[jira] [Created] (KAFKA-7870) Error sending fetch request (sessionId=1578860481, epoch=INITIAL) to node 2: java.io.IOException: Connection to 2 was disconnected before the response was read.

Chakhsu Lau created KAFKA-7870: -- Summary: Error sending fetch request (sessionId=1578860481, epoch=INITIAL) to node 2: java.io.IOException: Connection to 2 was disconnected before the response was read. Key: KAFKA-7870

Jenkins build is back to normal : kafka-1.1-jdk7 #241

See

Build failed in Jenkins: kafka-trunk-jdk8 #3345

See Changes: [me] MINOR: upgrade to jdk8 8u202 -- [...truncated 2.27 MB...] org.apache.kafka.streams.test.OutputVerifierTest > shouldFailIfValueIsDifferentWithNullForComp

Build failed in Jenkins: kafka-trunk-jdk11 #242

See Changes: [me] MINOR: upgrade to jdk8 8u202 -- [...truncated 2.27 MB...] org.apache.kafka.connect.header.ConnectHeadersTest > shouldNotValidateNullValuesWithBuiltInTy