Build failed in Jenkins: kafka-trunk-jdk11 #74

[Apache Jenkins Server]

See 


Changes:

[github] MINOR: Add try/finally blocks to close adminclient in

--
[...truncated 2.33 MB...]

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldFailIfValueIsDifferentWithNullForCompareKeyValueTimestampWithProducerRecord
 PASSED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldFailIfKeyIsDifferentWithNullReversForCompareKeyValueWithProducerRecord 
STARTED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldFailIfKeyIsDifferentWithNullReversForCompareKeyValueWithProducerRecord 
PASSED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldPassIfKeyAndValueIsEqualWithNullForCompareKeyValueWithProducerRecord 
STARTED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldPassIfKeyAndValueIsEqualWithNullForCompareKeyValueWithProducerRecord 
PASSED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldFailIfKeyIsDifferentWithNullForCompareKeyValueTimestampWithProducerRecord 
STARTED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldFailIfKeyIsDifferentWithNullForCompareKeyValueTimestampWithProducerRecord 
PASSED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldFailIfValueIsDifferentForCompareKeyValueTimestamp STARTED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldFailIfValueIsDifferentForCompareKeyValueTimestamp PASSED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldPassIfValueAndTimestampIsEqualForCompareValueTimestampWithProducerRecord 
STARTED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldPassIfValueAndTimestampIsEqualForCompareValueTimestampWithProducerRecord 
PASSED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldFailIfValueIsDifferentWithNullReversForCompareKeyValueTimestamp STARTED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldFailIfValueIsDifferentWithNullReversForCompareKeyValueTimestamp PASSED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldFailIfValueIsDifferentWithNullReverseForCompareValueTimestampWithProducerRecord
 STARTED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldFailIfValueIsDifferentWithNullReverseForCompareValueTimestampWithProducerRecord
 PASSED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldCreateConsumerRecordsFromKeyValuePairs STARTED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldCreateConsumerRecordsFromKeyValuePairs PASSED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldNotAllowToCreateTopicWithNullTopicNameWithNullKeyAndDefaultTimestamp 
STARTED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldNotAllowToCreateTopicWithNullTopicNameWithNullKeyAndDefaultTimestamp 
PASSED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldRequireCustomTopicNameIfNotDefaultFactoryTopicNameWithDefaultTimestamp 
STARTED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldRequireCustomTopicNameIfNotDefaultFactoryTopicNameWithDefaultTimestamp 
PASSED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldNotAllowToCreateTopicWithNullTopicNameWithNullKey STARTED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldNotAllowToCreateTopicWithNullTopicNameWithNullKey PASSED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldCreateNullKeyConsumerRecordWithOtherTopicNameAndTimestampWithTimetamp 
STARTED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldCreateNullKeyConsumerRecordWithOtherTopicNameAndTimestampWithTimetamp 
PASSED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldCreateConsumerRecordWithTimestamp STARTED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldCreateConsumerRecordWithTimestamp PASSED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldNotAllowToCreateTopicWithNullHeaders STARTED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldNotAllowToCreateTopicWithNullHeaders PASSED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldNotAllowToCreateTopicWithNullTopicNameWithDefaultTimestamp STARTED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldNotAllowToCreateTopicWithNullTopicNameWithDefaultTimestamp PASSED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldRequireCustomTopicNameIfNotDefaultFactoryTopicName STARTED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldRequireCustomTopicNameIfNotDefaultFactoryTopicName PASSED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldRequireCustomTopicNameIfNotDefaultFactoryTopicNameWithNullKey STARTED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldRequireCustomTopicNameIfNotDefaultFactoryTopicNameWithNullKey PASSED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
shouldCreateConsumerRecord STARTED

org.apache.kafka.streams.test.ConsumerRecordFactoryTest > 
should

Build failed in Jenkins: kafka-trunk-jdk8 #3177

[Apache Jenkins Server]

See 


Changes:

[github] MINOR: Add try/finally blocks to close adminclient in

--
[...truncated 2.73 MB...]

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldFailIfKeyIsDifferentForCompareKeyValueWithProducerRecord PASSED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldFailIfValueIsDifferentWithNullForCompareKeyValueTimestamp STARTED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldFailIfValueIsDifferentWithNullForCompareKeyValueTimestamp PASSED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldFailIfTimestampIsDifferentForCompareValueTimestampWithProducerRecord 
STARTED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldFailIfTimestampIsDifferentForCompareValueTimestampWithProducerRecord 
PASSED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldPassIfKeyAndValueIsEqualForCompareKeyValue STARTED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldPassIfKeyAndValueIsEqualForCompareKeyValue PASSED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldNotAllowNullProducerRecordForCompareKeyValueTimestamp STARTED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldNotAllowNullProducerRecordForCompareKeyValueTimestamp PASSED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldPassIfKeyAndValueIsEqualWithNullForCompareKeyValueTimestamp STARTED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldPassIfKeyAndValueIsEqualWithNullForCompareKeyValueTimestamp PASSED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldPassIfValueAndTimestampIsEqualForCompareValueTimestamp STARTED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldPassIfValueAndTimestampIsEqualForCompareValueTimestamp PASSED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldNotAllowNullExpectedRecordForCompareKeyValue STARTED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldNotAllowNullExpectedRecordForCompareKeyValue PASSED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldFailIfValueIsDifferentForCompareValueTimestampWithProducerRecord STARTED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldFailIfValueIsDifferentForCompareValueTimestampWithProducerRecord PASSED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldFailIfValueIsDifferentWithNullReversForCompareKeyValue STARTED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldFailIfValueIsDifferentWithNullReversForCompareKeyValue PASSED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldNotAllowNullExpectedRecordForCompareKeyValueTimestamp STARTED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldNotAllowNullExpectedRecordForCompareKeyValueTimestamp PASSED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldFailIfValueIsDifferentForCompareValueWithProducerRecord STARTED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldFailIfValueIsDifferentForCompareValueWithProducerRecord PASSED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldPassIfValueIsEqualForCompareValue STARTED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldPassIfValueIsEqualForCompareValue PASSED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldNotAllowNullProducerRecordWithExpectedRecordForCompareKeyValueTimestamp 
STARTED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldNotAllowNullProducerRecordWithExpectedRecordForCompareKeyValueTimestamp 
PASSED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldFailIfKeyIsDifferentForCompareKeyValue STARTED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldFailIfKeyIsDifferentForCompareKeyValue PASSED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldNotAllowNullProducerRecordWithExpectedRecordForCompareKeyValue STARTED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldNotAllowNullProducerRecordWithExpectedRecordForCompareKeyValue PASSED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldFailIfKeyIsDifferentWithNullForCompareKeyValueWithProducerRecord STARTED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldFailIfKeyIsDifferentWithNullForCompareKeyValueWithProducerRecord PASSED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldPassIfKeyAndValueAndTimestampIsEqualForCompareKeyValueTimestampWithProducerRecord
 STARTED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldPassIfKeyAndValueAndTimestampIsEqualForCompareKeyValueTimestampWithProducerRecord
 PASSED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldPassIfValueIsEqualWithNullForCompareValue STARTED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldPassIfValueIsEqualWithNullForCompareValue PASSED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldPassIfValueAndTimestampIsEqualWithNullForCompareValueTimestamp STARTED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldPassIfValueAndTimestampIsEqualWi

[jira] [Created] (KAFKA-7580) Unit Test 'shouldThrowProcessorStateExceptionOnOpeningReadOnlyDir' fails when run as root user

[Sarvesh Tamba (JIRA)]

Sarvesh Tamba created KAFKA-7580:


 Summary: Unit Test 
'shouldThrowProcessorStateExceptionOnOpeningReadOnlyDir' fails when run as root 
user
 Key: KAFKA-7580
 URL: https://issues.apache.org/jira/browse/KAFKA-7580
 Project: Kafka
  Issue Type: Bug
  Components: unit tests
Affects Versions: 2.0.0, 2.0.1, 2.1.0, 2.2.0, 2.1.1, 2.0.2
 Environment: Ubuntu 16.04.3 LTS
Reporter: Sarvesh Tamba


Created a non-root user and ran the following command to execute the failiing 
unit test:-
./gradlew streams:unitTest --tests 
org.apache.kafka.streams.state.internals.RocksDBStoreTest.shouldThrowProcessorStateExceptionOnOpeningReadOnlyDir

For a root user, the test case fails:-
=
> Task :streams:testClasses UP-TO-DATE

> Task :streams:unitTest

org.apache.kafka.streams.state.internals.RocksDBStoreTest > 
shouldThrowProcessorStateExceptionOnOpeningReadOnlyDir FAILED
 java.lang.AssertionError: Expected exception: 
org.apache.kafka.streams.errors.ProcessorStateException

1 test completed, 1 failed

> Task :streams:unitTest FAILED

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':streams:unitTest'.
> There were failing tests. See the report at: 
> file:///root/sarvesh/kafka-gradle/kafka-2.0.0/streams/build/reports/tests/unitTest/index.html

* Try:
Run with --stacktrace option to get the stack trace. Run with --info or --debug 
option to get more log output. Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 20s
26 actionable tasks: 2 executed, 24 up-to-date
=

However, for a non-root user the test cass passes as success:-
=
> Task :streams:testClasses

> Task :streams:unitTest

org.apache.kafka.streams.state.internals.RocksDBStoreTest > 
shouldThrowProcessorStateExceptionOnOpeningReadOnlyDir PASSED

BUILD SUCCESSFUL in 45s
26 actionable tasks: 4 executed, 22 up-to-date

=

The failing unit test - 
"shouldThrowProcessorStateExceptionOnOpeningReadOnlyDir" creates a temporary 
file directory and sets it as readOnly. The unit test is intended to throw an 
exception - "ProcessorStateException", when the readOnly temporary file 
directory is opened/accessed.

By default, non-root users opening/accessing readOnly file directory is not 
allowed and it rightly throws up an error/exception in the unit test(which is 
the intention of the unit test and it passes for non-root users).

sar@p006vm18:~/kafka-gradle/kafka-2.0.0$ mkdir /tmp/readOnlyDir/parent
 mkdir: cannot create directory â/tmp/readOnlyDir/parentâ: Permission denied
 
 sar@p006vm18:~/kafka-gradle/kafka-2.0.0$ ll /tmp/readOnlyDir/
 ls: cannot access '/tmp/readOnlyDir/..': Permission denied
 ls: cannot access '/tmp/readOnlyDir/.': Permission denied
 ls: cannot access '/tmp/readOnlyDir/kid': Permission denied
 ls: cannot access '/tmp/readOnlyDir/child': Permission denied
 total 0
 d? ? ? ? ? ? ./
 d? ? ? ? ? ? ../
 d? ? ? ? ? ? child/
 d? ? ? ? ? ? kid/

However, by default, root user can access any file in the system.:-
 root@p006vm18:/tmp# ll /tmp/readOnlyDir/
 total 112
 dr--rw-rw- 4 root root 4096 Nov 1 03:47 ./
 drwxrwxrwt 24 root root 98304 Nov 1 04:02 ../
 drwxr-xr-x 2 root root 4096 Nov 1 03:44 child/
 drwxrwxr-x 2 sar sar 4096 Nov 1 03:47 kid/
 
 root@p006vm18:/tmp# mkdir /tmp/readOnlyDir/parent
 
 root@p006vm18:/tmp# ll /tmp/readOnlyDir/
 total 116
 dr--rw-rw- 5 root root 4096 Nov 1 04:03 ./
 drwxrwxrwt 24 root root 98304 Nov 1 04:02 ../
 drwxr-xr-x 2 root root 4096 Nov 1 03:44 child/
 drwxrwxr-x 2 sar sar 4096 Nov 1 03:47 kid/
 drwxr-xr-x 2 root root 4096 Nov 1 04:03 parent/

Hence the unit test does not throw an exception - "ProcessorStateException" 
when the readOnly temporary file directory is opened, and the unit test rightly 
fails for a root user.

Two approaches for resolving this failing unit test case:-
1.) Run the unit tests as non-root users(simplest).
2.) If running the unit test as root user, make the temporary file directory as 
immutable in the unit test code and then test for exception(needs code changes 
in the unit tests):-

root@p006vm18:/tmp# chattr +i /tmp/readOnlyDir/
root@p006vm18:/tmp# mkdir /tmp/readOnlyDir/grandparent
mkdir: cannot create directory â/tmp/readOnlyDir/grandparentâ: Permission denied

A file with an immutable attribute can not be:
- Modified
- Deleted
- Renamed
- No soft or hard link created by anyone including root user.

 

 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (KAFKA-7581) Issues in building kafka using gradle on a Ubuntu based docker container

[Sarvesh Tamba (JIRA)]

Sarvesh Tamba created KAFKA-7581:


 Summary: Issues in building kafka using gradle on a Ubuntu based 
docker container
 Key: KAFKA-7581
 URL: https://issues.apache.org/jira/browse/KAFKA-7581
 Project: Kafka
  Issue Type: Bug
  Components: build
Affects Versions: 2.0.0, 2.0.1, 2.1.0, 2.2.0, 2.1.1, 2.0.2
 Environment: Ubuntu 16.04.3 LTS
Reporter: Sarvesh Tamba


The following issues are seen when kafka is built using gradle on a Ubuntu 
based docker container:-

/kafka-gradle/kafka-2.0.0/core/src/main/scala/kafka/coordinator/transaction/TransactionStateManager.scala:177:
 File name too long
 This can happen on some encrypted or legacy file systems. Please see SI-3623 
for more details.
 .foreach { txnMetadataCacheEntry =>
 ^
 56 warnings found
 one error found

> Task :core:compileScala FAILED

FAILURE: Build failed with an exception.

* What went wrong:
 Execution failed for task ':core:compileScala'.
 > Compilation failed



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (KAFKA-7582) MetadataRequest don't support autoCreateTopic below version 4.

[Yanjie Wang (JIRA)]

Yanjie Wang created KAFKA-7582:
--

 Summary: MetadataRequest don't support autoCreateTopic below 
version 4.
 Key: KAFKA-7582
 URL: https://issues.apache.org/jira/browse/KAFKA-7582
 Project: Kafka
  Issue Type: Bug
  Components: clients
Reporter: Yanjie Wang


This line of MetadataRequest.

''if (!allowAutoTopicCreation && version < 4)" should remove "!".

Location: 
[https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/common/requests/MetadataRequest.java#L110]

I hope that I am not mistaken. 

As a Java beginner



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: [VOTE] 2.0.1 RC0

[Jakub Scholz]

+1 (non-binding) ... I used the staged binaries and run tests with
different clients.

On Fri, Oct 26, 2018 at 4:29 AM Manikumar  wrote:

> Hello Kafka users, developers and client-developers,
>
> This is the first candidate for release of Apache Kafka 2.0.1.
>
> This is a bug fix release closing 49 tickets:
> https://cwiki.apache.org/confluence/display/KAFKA/Release+Plan+2.0.1
>
> Release notes for the 2.0.1 release:
> http://home.apache.org/~manikumar/kafka-2.0.1-rc0/RELEASE_NOTES.html
>
> *** Please download, test and vote by  Tuesday, October 30, end of day
>
> Kafka's KEYS file containing PGP keys we use to sign the release:
> http://kafka.apache.org/KEYS
>
> * Release artifacts to be voted upon (source and binary):
> http://home.apache.org/~manikumar/kafka-2.0.1-rc0/
>
> * Maven artifacts to be voted upon:
> https://repository.apache.org/content/groups/staging/
>
> * Javadoc:
> http://home.apache.org/~manikumar/kafka-2.0.1-rc0/javadoc/
>
> * Tag to be voted upon (off 2.0 branch) is the 2.0.1 tag:
> https://github.com/apache/kafka/releases/tag/2.0.1-rc0
>
> * Documentation:
> http://kafka.apache.org/20/documentation.html
>
> * Protocol:
> http://kafka.apache.org/20/protocol.html
>
> * Successful Jenkins builds for the 2.0 branch:
> Unit/integration tests: https://builds.apache.org/job/kafka-2.0-jdk8/177/
>
> /**
>
> Thanks,
> Manikumar
>

Re: [VOTE] 2.1.0 RC0

[Jakub Scholz]

+1 (non-binding) ... I used the staged binaries and checked it with
different clients.

On Wed, Oct 24, 2018 at 10:17 AM Dong Lin  wrote:

> Hello Kafka users, developers and client-developers,
>
> This is the first candidate for feature release of Apache Kafka 2.1.0.
>
> This is a major version release of Apache Kafka. It includes 28 new KIPs
> and
>
> critical bug fixes. Please see the Kafka 2.1.0 release plan for more
> details:
>
> *
> https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=91554044*
>  >
>
> Here are a few notable highlights:
>
> - Java 11 support
> - Support for Zstandard, which achieves compression comparable to gzip with
> higher compression and especially decompression speeds(KIP-110)
> - Avoid expiring committed offsets for active consumer group (KIP-211)
> - Provide Intuitive User Timeouts in The Producer (KIP-91)
> - Kafka's replication protocol now supports improved fencing of zombies.
> Previously, under certain rare conditions, if a broker became partitioned
> from Zookeeper but not the rest of the cluster, then the logs of replicated
> partitions could diverge and cause data loss in the worst case (KIP-320)
> - Streams API improvements (KIP-319, KIP-321, KIP-330, KIP-353, KIP-356)
> - Admin script and admin client API improvements to simplify admin
> operation (KIP-231, KIP-308, KIP-322, KIP-324, KIP-338, KIP-340)
> - DNS handling improvements (KIP-235, KIP-302)
>
> Release notes for the 2.1.0 release:
> http://home.apache.org/~lindong/kafka-2.1.0-rc0/RELEASE_NOTES.html
>
> *** Please download, test and vote ***
>
> * Kafka's KEYS file containing PGP keys we use to sign the release:
> http://kafka.apache.org/KEYS
>
> * Release artifacts to be voted upon (source and binary):
> http://home.apache.org/~lindong/kafka-2.1.0-rc0/
>
> * Maven artifacts to be voted upon:
> https://repository.apache.org/content/groups/staging/
>
> * Javadoc:
> http://home.apache.org/~lindong/kafka-2.1.0-rc0/javadoc/
>
> * Tag to be voted upon (off 2.1 branch) is the 2.1.0-rc0 tag:
> https://github.com/apache/kafka/tree/2.1.0-rc0
>
> * Documentation:
> *http://kafka.apache.org/21/documentation.html*
> 
>
> * Protocol:
> http://kafka.apache.org/21/protocol.html
>
> * Successful Jenkins builds for the 2.1 branch:
> Unit/integration tests: *https://builds.apache.org/job/kafka-2.1-jdk8/38/
> *
>
> Please test and verify the release artifacts and submit a vote for this RC,
> or report any issues so we can fix them and get a new RC out ASAP. Although
> this release vote requires PMC votes to pass, testing, votes, and bug
> reports are valuable and appreciated from everyone.
>
> Cheers,
> Dong
>

[jira] [Resolved] (KAFKA-7582) MetadataRequest don't support autoCreateTopic below version 4.

[Yanjie Wang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/KAFKA-7582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Yanjie Wang resolved KAFKA-7582.

Resolution: Not A Bug

> MetadataRequest don't support autoCreateTopic below version 4.
> --
>
> Key: KAFKA-7582
> URL: https://issues.apache.org/jira/browse/KAFKA-7582
> Project: Kafka
>  Issue Type: Bug
>  Components: clients
>Reporter: Yanjie Wang
>Priority: Minor
>
> This line of MetadataRequest.
> ''if (\!allowAutoTopicCreation && version < 4)" should remove "!".
> Location: 
> [https://github.com/apache/kafka/blob/trunk/clients/src/main/java/org/apache/kafka/common/requests/MetadataRequest.java#L110]
> I hope that I am not mistaken. 
> As a Java beginner



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: [VOTE] 2.0.1 RC0

[Eno Thereska]

Anything else holding this up?

Thanks
Eno

On Thu, Nov 1, 2018 at 10:27 AM Jakub Scholz  wrote:

> +1 (non-binding) ... I used the staged binaries and run tests with
> different clients.
>
> On Fri, Oct 26, 2018 at 4:29 AM Manikumar 
> wrote:
>
> > Hello Kafka users, developers and client-developers,
> >
> > This is the first candidate for release of Apache Kafka 2.0.1.
> >
> > This is a bug fix release closing 49 tickets:
> > https://cwiki.apache.org/confluence/display/KAFKA/Release+Plan+2.0.1
> >
> > Release notes for the 2.0.1 release:
> > http://home.apache.org/~manikumar/kafka-2.0.1-rc0/RELEASE_NOTES.html
> >
> > *** Please download, test and vote by  Tuesday, October 30, end of day
> >
> > Kafka's KEYS file containing PGP keys we use to sign the release:
> > http://kafka.apache.org/KEYS
> >
> > * Release artifacts to be voted upon (source and binary):
> > http://home.apache.org/~manikumar/kafka-2.0.1-rc0/
> >
> > * Maven artifacts to be voted upon:
> > https://repository.apache.org/content/groups/staging/
> >
> > * Javadoc:
> > http://home.apache.org/~manikumar/kafka-2.0.1-rc0/javadoc/
> >
> > * Tag to be voted upon (off 2.0 branch) is the 2.0.1 tag:
> > https://github.com/apache/kafka/releases/tag/2.0.1-rc0
> >
> > * Documentation:
> > http://kafka.apache.org/20/documentation.html
> >
> > * Protocol:
> > http://kafka.apache.org/20/protocol.html
> >
> > * Successful Jenkins builds for the 2.0 branch:
> > Unit/integration tests:
> https://builds.apache.org/job/kafka-2.0-jdk8/177/
> >
> > /**
> >
> > Thanks,
> > Manikumar
> >
>

Re: [VOTE] 2.0.1 RC0

[Harsha Chintalapani]

+1.
Ran a 3 node cluster with few simple tests.

Thanks,
Harsha
On Nov 1, 2018, 9:50 AM -0700, Eno Thereska , wrote:
> Anything else holding this up?
>
> Thanks
> Eno
>
> On Thu, Nov 1, 2018 at 10:27 AM Jakub Scholz  wrote:
>
> > +1 (non-binding) ... I used the staged binaries and run tests with
> > different clients.
> >
> > On Fri, Oct 26, 2018 at 4:29 AM Manikumar 
> > wrote:
> >
> > > Hello Kafka users, developers and client-developers,
> > >
> > > This is the first candidate for release of Apache Kafka 2.0.1.
> > >
> > > This is a bug fix release closing 49 tickets:
> > > https://cwiki.apache.org/confluence/display/KAFKA/Release+Plan+2.0.1
> > >
> > > Release notes for the 2.0.1 release:
> > > http://home.apache.org/~manikumar/kafka-2.0.1-rc0/RELEASE_NOTES.html
> > >
> > > *** Please download, test and vote by Tuesday, October 30, end of day
> > >
> > > Kafka's KEYS file containing PGP keys we use to sign the release:
> > > http://kafka.apache.org/KEYS
> > >
> > > * Release artifacts to be voted upon (source and binary):
> > > http://home.apache.org/~manikumar/kafka-2.0.1-rc0/
> > >
> > > * Maven artifacts to be voted upon:
> > > https://repository.apache.org/content/groups/staging/
> > >
> > > * Javadoc:
> > > http://home.apache.org/~manikumar/kafka-2.0.1-rc0/javadoc/
> > >
> > > * Tag to be voted upon (off 2.0 branch) is the 2.0.1 tag:
> > > https://github.com/apache/kafka/releases/tag/2.0.1-rc0
> > >
> > > * Documentation:
> > > http://kafka.apache.org/20/documentation.html
> > >
> > > * Protocol:
> > > http://kafka.apache.org/20/protocol.html
> > >
> > > * Successful Jenkins builds for the 2.0 branch:
> > > Unit/integration tests:
> > https://builds.apache.org/job/kafka-2.0-jdk8/177/
> > >
> > > /**
> > >
> > > Thanks,
> > > Manikumar
> > >
> >

Re: [VOTE] 2.0.1 RC0

[Manikumar]

We were waiting for the system test results. There were few failures:
KAFKA-7579,  KAFKA-7559, KAFKA-7561
they are not blockers for 2.0.1 release. We need more votes from
PMC/committers :)

Thanks Stanislav! for the system test results.

Thanks,
Manikumar

On Thu, Nov 1, 2018 at 10:20 PM Eno Thereska  wrote:

> Anything else holding this up?
>
> Thanks
> Eno
>
> On Thu, Nov 1, 2018 at 10:27 AM Jakub Scholz  wrote:
>
> > +1 (non-binding) ... I used the staged binaries and run tests with
> > different clients.
> >
> > On Fri, Oct 26, 2018 at 4:29 AM Manikumar 
> > wrote:
> >
> > > Hello Kafka users, developers and client-developers,
> > >
> > > This is the first candidate for release of Apache Kafka 2.0.1.
> > >
> > > This is a bug fix release closing 49 tickets:
> > > https://cwiki.apache.org/confluence/display/KAFKA/Release+Plan+2.0.1
> > >
> > > Release notes for the 2.0.1 release:
> > > http://home.apache.org/~manikumar/kafka-2.0.1-rc0/RELEASE_NOTES.html
> > >
> > > *** Please download, test and vote by  Tuesday, October 30, end of day
> > >
> > > Kafka's KEYS file containing PGP keys we use to sign the release:
> > > http://kafka.apache.org/KEYS
> > >
> > > * Release artifacts to be voted upon (source and binary):
> > > http://home.apache.org/~manikumar/kafka-2.0.1-rc0/
> > >
> > > * Maven artifacts to be voted upon:
> > > https://repository.apache.org/content/groups/staging/
> > >
> > > * Javadoc:
> > > http://home.apache.org/~manikumar/kafka-2.0.1-rc0/javadoc/
> > >
> > > * Tag to be voted upon (off 2.0 branch) is the 2.0.1 tag:
> > > https://github.com/apache/kafka/releases/tag/2.0.1-rc0
> > >
> > > * Documentation:
> > > http://kafka.apache.org/20/documentation.html
> > >
> > > * Protocol:
> > > http://kafka.apache.org/20/protocol.html
> > >
> > > * Successful Jenkins builds for the 2.0 branch:
> > > Unit/integration tests:
> > https://builds.apache.org/job/kafka-2.0-jdk8/177/
> > >
> > > /**
> > >
> > > Thanks,
> > > Manikumar
> > >
> >
>

KSQL with changelog topic

[Boyang Chen]

Hey there,


a dumb question that I haven't found the answer anywhere: does KSQL support 
query towards Stream applications'

changelog topic? Especially window store changelog, since it has a more complex 
key schema.


Thanks!

Re: KSQL with changelog topic

[Matthias J. Sax]

I am not sure if I understand the question. Can you elaborate?

-Matthias


On 11/1/18 10:38 AM, Boyang Chen wrote:
> Hey there,
> 
> 
> a dumb question that I haven't found the answer anywhere: does KSQL support 
> query towards Stream applications'
> 
> changelog topic? Especially window store changelog, since it has a more 
> complex key schema.
> 
> 
> Thanks!
> 



signature.asc
Description: OpenPGP digital signature

Re: request for wiki permission to create KIP

[Matthias J. Sax]

Done.

On 10/31/18 9:49 AM, ChienHsing Wu wrote:
> Hi, my ID is chienhsw.
> 
> Please enable me to add KIP pages using this URL: 
> https://cwiki.apache.org/confluence?templateId=54329345&spaceKey=KAFKA&newSpaceKey=KAFKA&title=KIP-NEXT%3A+Insert+Title+Here.
>  I would like to add a KIP to discuss Kafak-3932 design.
> 
> Thanks, CH
> 
> 
> 



signature.asc
Description: OpenPGP digital signature

[jira] [Resolved] (KAFKA-7568) Return leader epoch in ListOffsets responses

[Jason Gustafson (JIRA)]

 [ 
https://issues.apache.org/jira/browse/KAFKA-7568?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jason Gustafson resolved KAFKA-7568.

   Resolution: Fixed
Fix Version/s: 2.2.0

> Return leader epoch in ListOffsets responses
> 
>
> Key: KAFKA-7568
> URL: https://issues.apache.org/jira/browse/KAFKA-7568
> Project: Kafka
>  Issue Type: Improvement
>Reporter: Jason Gustafson
>Assignee: Jason Gustafson
>Priority: Major
> Fix For: 2.2.0
>
>
> This is part of KIP-320. The changes to the API have already been made, but 
> currently we return unknown epoch. We need to update the logic to search for 
> the epoch corresponding to a fetched offset in the leader epoch cache.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (KAFKA-7583) Producer force shutdown doesn't work when all brokers is down.

[Vitalina Horyukova (JIRA)]

Vitalina Horyukova created KAFKA-7583:
-

 Summary: Producer force shutdown doesn't work when all brokers is 
down.
 Key: KAFKA-7583
 URL: https://issues.apache.org/jira/browse/KAFKA-7583
 Project: Kafka
  Issue Type: Bug
  Components: producer 
Affects Versions: 0.11.0.0
 Environment: {code}
uname -a
Linux kassa 3.13.0-139-generic #188-Ubuntu SMP Tue Jan 9 14:43:09 UTC 2018 
x86_64 x86_64 x86_64 GNU/Linux
java --version
java 1.8.0_152
{code}
Reporter: Vitalina Horyukova


Hi!
When all Kafka brokers are down, thread which called {{KafkaProducer.close}} 
the infinity stucks in second join to {{KafkaProducer.ioThread}}, because 
{{KafkaProducer.ioThread}} infinity spins over {{while}} cycle in 
{{Sender.maybeWaitForProducerId}}. The root cause of this is that 
{{Sender.awaitLeastLoadedNodeReady}} -> {{NetworkClientUtils.awaitReady}} 
throws {{IOException}} every iteration.
In logs you can see infinity repeation of this part every `retry.backoff.ms`:
{code:java}
[2018-11-01T16:19:47.583+03:00] TRACE [kafka-producer-network-thread | 
producer-1] [] [] [] [NetworkClient] Found least loaded node kafka:9093 (id: -1 
rack: null)
[2018-11-01T16:19:47.583+03:00] TRACE [kafka-producer-network-thread | 
producer-1] [] [] [] [NetworkClient] Found least loaded node kafka:9093 (id: -1 
rack: null)
[2018-11-01T16:19:47.583+03:00] DEBUG [kafka-producer-network-thread | 
producer-1] [] [] [] [NetworkClient] Initialize connection to node -1 for 
sending metadata request
[2018-11-01T16:19:47.583+03:00] DEBUG [kafka-producer-network-thread | 
producer-1] [] [] [] [NetworkClient] Initiating connection to node -1 at 
kafka:9093.
[2018-11-01T16:19:47.585+03:00] DEBUG [kafka-producer-network-thread | 
producer-1] [] [] [] [Selector] Connection with kafka/xxx.xxx.xxx.xxx 
disconnected
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at 
sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
at 
org.apache.kafka.common.network.SslTransportLayer.finishConnect(SslTransportLayer.java:109)
at 
org.apache.kafka.common.network.KafkaChannel.finishConnect(KafkaChannel.java:95)
at 
org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:359)
at org.apache.kafka.common.network.Selector.poll(Selector.java:326)
at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:432)
at 
org.apache.kafka.clients.NetworkClientUtils.isReady(NetworkClientUtils.java:39)
at 
org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:62)
at 
org.apache.kafka.clients.producer.internals.Sender.awaitLeastLoadedNodeReady(Sender.java:409)
at 
org.apache.kafka.clients.producer.internals.Sender.maybeWaitForProducerId(Sender.java:418)
at 
org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:203)
at 
org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:162)
at java.lang.Thread.run(Thread.java:748)
[2018-11-01T16:19:47.585+03:00] DEBUG [kafka-producer-network-thread | 
producer-1] [] [] [] [NetworkClient] Node -1 disconnected.
[2018-11-01T16:19:47.585+03:00] WARN  [kafka-producer-network-thread | 
producer-1] [] [] [] [NetworkClient] Connection to node -1 could not be 
established. Broker may not be available.
[2018-11-01T16:19:47.585+03:00] DEBUG [kafka-producer-network-thread | 
producer-1] [] [] [] [Sender] Broker {} disconnected while awaiting 
InitProducerId response
java.io.IOException: Connection to kafka:9093 (id: -1 rack: null) failed.
at 
org.apache.kafka.clients.NetworkClientUtils.awaitReady(NetworkClientUtils.java:68)
at 
org.apache.kafka.clients.producer.internals.Sender.awaitLeastLoadedNodeReady(Sender.java:409)
at 
org.apache.kafka.clients.producer.internals.Sender.maybeWaitForProducerId(Sender.java:418)
at 
org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:203)
at 
org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:162)
at java.lang.Thread.run(Thread.java:748)
[2018-11-01T16:19:47.585+03:00] TRACE [kafka-producer-network-thread | 
producer-1] [] [] [] [Sender] Retry InitProducerIdRequest in 100ms.
{code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Jenkins build is back to normal : kafka-trunk-jdk11 #75

[Apache Jenkins Server]

See 

Build failed in Jenkins: kafka-trunk-jdk8 #3178

[Apache Jenkins Server]

See 


Changes:

[github] KAFKA-7568; Return leader epoch in ListOffsets response (#5855)

--
[...truncated 2.73 MB...]

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldFailIfKeyIsDifferentForCompareKeyValueWithProducerRecord PASSED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldFailIfValueIsDifferentWithNullForCompareKeyValueTimestamp STARTED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldFailIfValueIsDifferentWithNullForCompareKeyValueTimestamp PASSED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldFailIfTimestampIsDifferentForCompareValueTimestampWithProducerRecord 
STARTED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldFailIfTimestampIsDifferentForCompareValueTimestampWithProducerRecord 
PASSED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldPassIfKeyAndValueIsEqualForCompareKeyValue STARTED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldPassIfKeyAndValueIsEqualForCompareKeyValue PASSED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldNotAllowNullProducerRecordForCompareKeyValueTimestamp STARTED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldNotAllowNullProducerRecordForCompareKeyValueTimestamp PASSED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldPassIfKeyAndValueIsEqualWithNullForCompareKeyValueTimestamp STARTED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldPassIfKeyAndValueIsEqualWithNullForCompareKeyValueTimestamp PASSED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldPassIfValueAndTimestampIsEqualForCompareValueTimestamp STARTED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldPassIfValueAndTimestampIsEqualForCompareValueTimestamp PASSED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldNotAllowNullExpectedRecordForCompareKeyValue STARTED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldNotAllowNullExpectedRecordForCompareKeyValue PASSED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldFailIfValueIsDifferentForCompareValueTimestampWithProducerRecord STARTED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldFailIfValueIsDifferentForCompareValueTimestampWithProducerRecord PASSED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldFailIfValueIsDifferentWithNullReversForCompareKeyValue STARTED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldFailIfValueIsDifferentWithNullReversForCompareKeyValue PASSED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldNotAllowNullExpectedRecordForCompareKeyValueTimestamp STARTED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldNotAllowNullExpectedRecordForCompareKeyValueTimestamp PASSED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldFailIfValueIsDifferentForCompareValueWithProducerRecord STARTED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldFailIfValueIsDifferentForCompareValueWithProducerRecord PASSED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldPassIfValueIsEqualForCompareValue STARTED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldPassIfValueIsEqualForCompareValue PASSED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldNotAllowNullProducerRecordWithExpectedRecordForCompareKeyValueTimestamp 
STARTED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldNotAllowNullProducerRecordWithExpectedRecordForCompareKeyValueTimestamp 
PASSED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldFailIfKeyIsDifferentForCompareKeyValue STARTED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldFailIfKeyIsDifferentForCompareKeyValue PASSED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldNotAllowNullProducerRecordWithExpectedRecordForCompareKeyValue STARTED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldNotAllowNullProducerRecordWithExpectedRecordForCompareKeyValue PASSED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldFailIfKeyIsDifferentWithNullForCompareKeyValueWithProducerRecord STARTED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldFailIfKeyIsDifferentWithNullForCompareKeyValueWithProducerRecord PASSED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldPassIfKeyAndValueAndTimestampIsEqualForCompareKeyValueTimestampWithProducerRecord
 STARTED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldPassIfKeyAndValueAndTimestampIsEqualForCompareKeyValueTimestampWithProducerRecord
 PASSED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldPassIfValueIsEqualWithNullForCompareValue STARTED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldPassIfValueIsEqualWithNullForCompareValue PASSED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldPassIfValueAndTimestampIsEqualWithNullForCompareValueTimestamp STARTED

org.apache.kafka.streams.test.OutputVerifierTest > 
shouldPassIfValueAndTimestam

Re: KSQL with changelog topic

[Boyang Chen]

For example, I have a stream windowed aggregation changlog topic 
KStream--XXX--changelog-26, does KSQL support a simple query like

```

select * from

changelog_topic

limit 10

```

I'm not sure because the serde for window store changelog is compound 
Windowed.


Hope this makes sense. Thank you!



From: Matthias J. Sax 
Sent: Friday, November 2, 2018 5:06 AM
To: dev@kafka.apache.org
Subject: Re: KSQL with changelog topic

I am not sure if I understand the question. Can you elaborate?

-Matthias


On 11/1/18 10:38 AM, Boyang Chen wrote:
> Hey there,
>
>
> a dumb question that I haven't found the answer anywhere: does KSQL support 
> query towards Stream applications'
>
> changelog topic? Especially window store changelog, since it has a more 
> complex key schema.
>
>
> Thanks!
>