Hi Vahid, There are a few known issues when running Kafka on Windows. A PR with some fixes is: https://github.com/apache/kafka/pull/3283. The fact that the index cannot be accessed indicates that it may be a similar issue. I suggest we move this discussion to the relevant JIRAs instead of the release thread.
Ismael On Mon, Jun 26, 2017 at 11:25 PM, Vahid S Hashemian < vahidhashem...@us.ibm.com> wrote: > Hi Ismael, > > This is the output of core tests from the start until the first failed > test. > > kafka.admin.AdminRackAwareTest > testAssignmentWithRackAwareWithUnevenRacks > PASSED > > kafka.admin.AdminRackAwareTest > testAssignmentWith2ReplicasRackAware > PASSED > > kafka.admin.AdminRackAwareTest > testAssignmentWithRackAwareWithUnevenReplicas > PASSED > > kafka.admin.AdminRackAwareTest > testSkipBrokerWithReplicaAlreadyAssigned > PASSED > > kafka.admin.AdminRackAwareTest > testAssignmentWithRackAware PASSED > > kafka.admin.AdminRackAwareTest > testRackAwareExpansion PASSED > > kafka.admin.AdminRackAwareTest > > testAssignmentWith2ReplicasRackAwareWith6Partitions > PASSED > > kafka.admin.AdminRackAwareTest > testAssignmentWith2ReplicasRac > kAwareWith6PartitionsAnd3Brokers PASSED > > kafka.admin.AdminRackAwareTest > > testGetRackAlternatedBrokerListAndAssignReplicasToBrokers > PASSED > > kafka.admin.AdminRackAwareTest > testMoreReplicasThanRacks PASSED > > kafka.admin.AdminRackAwareTest > testSingleRack PASSED > > kafka.admin.AdminRackAwareTest > > testAssignmentWithRackAwareWithRandomStartIndex > PASSED > > kafka.admin.AdminRackAwareTest > testLargeNumberPartitionsAssignment > PASSED > > kafka.admin.AdminRackAwareTest > testLessReplicasThanRacks PASSED > > kafka.admin.AclCommandTest > testInvalidAuthorizerProperty PASSED > > kafka.admin.ConfigCommandTest > testScramCredentials PASSED > > kafka.admin.ConfigCommandTest > shouldParseArgumentsForTopicsEntityType > PASSED > > kafka.admin.ConfigCommandTest > testUserClientQuotaOpts PASSED > > kafka.admin.ConfigCommandTest > shouldAddTopicConfig PASSED > > kafka.admin.ConfigCommandTest > shouldAddClientConfig PASSED > > kafka.admin.ConfigCommandTest > shouldDeleteBrokerConfig PASSED > > kafka.admin.DeleteConsumerGroupTest > > testGroupWideDeleteInZKDoesNothingForActiveConsumerGroup > PASSED > > kafka.admin.ConfigCommandTest > testQuotaConfigEntity PASSED > > kafka.admin.ConfigCommandTest > > shouldNotUpdateBrokerConfigIfMalformedBracketConfig > PASSED > > kafka.admin.ConfigCommandTest > shouldFailIfUnrecognisedEntityType PASSED > > kafka.admin.AdminTest > testBasicPreferredReplicaElection PASSED > > kafka.admin.ConfigCommandTest > > shouldNotUpdateBrokerConfigIfNonExistingConfigIsDeleted > PASSED > > kafka.admin.AdminTest > testPreferredReplicaJsonData PASSED > > kafka.admin.BrokerApiVersionsCommandTest > checkBrokerApiVersionCommandOutput > PASSED > > kafka.admin.ReassignPartitionsCommandTest > > shouldRemoveThrottleReplicaListBasedOnProposedAssignment > PASSED > > kafka.admin.ReassignPartitionsCommandTest > > shouldFindMovingReplicasMultipleTopics > PASSED > > kafka.admin.ReassignPartitionsCommandTest > > shouldNotOverwriteExistingPropertiesWhenLimitIsAdded > PASSED > > kafka.admin.ReassignPartitionsCommandTest > > shouldFindMovingReplicasMultipleTopicsAndPartitions > PASSED > > kafka.admin.ReassignPartitionsCommandTest > > shouldRemoveThrottleLimitFromAllBrokers > PASSED > > kafka.admin.ReassignPartitionsCommandTest > shouldFindMovingReplicas > PASSED > > kafka.admin.ReassignPartitionsCommandTest > > shouldFindMovingReplicasMultiplePartitions > PASSED > > kafka.admin.ReassignPartitionsCommandTest > shouldSetQuotaLimit PASSED > > kafka.admin.ReassignPartitionsCommandTest > > shouldFindMovingReplicasWhenProposedIsSubsetOfExisting > PASSED > > kafka.admin.ReassignPartitionsCommandTest > shouldUpdateQuotaLimit PASSED > > kafka.admin.ReassignPartitionsCommandTest > > shouldFindTwoMovingReplicasInSamePartition > PASSED > > kafka.admin.ReassignPartitionsCommandTest > > shouldNotOverwriteEntityConfigsWhenUpdatingThrottledReplicas > PASSED > > kafka.admin.ConfigCommandTest > > shouldNotUpdateBrokerConfigIfMalformedEntityName > PASSED > > kafka.admin.ConfigCommandTest > shouldSupportCommaSeparatedValues PASSED > > kafka.admin.ConfigCommandTest > shouldNotUpdateBrokerConfigIfMalformedConfig > PASSED > > kafka.admin.DeleteConsumerGroupTest > testGroupTopicWideDeleteInZKDo > esNothingForActiveGroupConsumingMultipleTopics PASSED > > kafka.admin.AddPartitionsTest > testReplicaPlacementAllServers PASSED > > kafka.admin.ConfigCommandTest > shouldParseArgumentsForBrokersEntityType > PASSED > > kafka.admin.ConfigCommandTest > shouldAddBrokerConfig PASSED > > kafka.admin.AdminTest > testReassigningNonExistingPartition PASSED > > kafka.admin.ConfigCommandTest > testQuotaDescribeEntities PASSED > > kafka.admin.AdminTest > testGetBrokerMetadatas PASSED > > kafka.admin.ConfigCommandTest > shouldParseArgumentsForClientsEntityType > PASSED > > kafka.admin.AclCommandTest > testAclCli PASSED > > kafka.admin.ReassignPartitionsIntegrationTest > testRackAwareReassign > PASSED > > kafka.admin.AdminTest > testBootstrapClientIdConfig PASSED > > kafka.admin.ReassignPartitionsClusterTest > shouldExecuteThrottledReassignment > FAILED > java.nio.file.FileSystemException: C:\Users\IBM_AD~1\AppData\ > Local\Temp\kafka-719085320148197500\my-topic-0\00000000000000000000.index: > The process cannot access the file because it is being used by another > process. > > > From the error message, it sounds like one of the prior tests does not do > a proper clean-up?! > > Thanks. > --Vahid > > > > > From: Ismael Juma <ism...@juma.me.uk> > To: dev@kafka.apache.org > Cc: kafka-clients <kafka-clie...@googlegroups.com>, Kafka Users < > us...@kafka.apache.org> > Date: 06/26/2017 01:54 PM > Subject: Re: [VOTE] 0.11.0.0 RC2 > Sent by: isma...@gmail.com > ------------------------------ > > > > Hi Vahid, > > Can you please check which test fails first? The errors you mentioned can > happen if a test fails and doesn't clean-up properly. > > Ismael > > On Mon, Jun 26, 2017 at 8:41 PM, Vahid S Hashemian < > vahidhashem...@us.ibm.com> wrote: > > > Hi Ismael, > > > > To answer your questions: > > > > 1. Yes, the issues exists in trunk too. > > > > 2. I haven't checked with Cygwin, but I can give it a try. > > > > And thanks for addressing this issue. I can confirm with your PR I no > > longer see it. > > But now that the tests progress I see quite a few errors like this in > > core: > > > > kafka.server.ReplicaFetchTest > classMethod FAILED > > java.lang.AssertionError: Found unexpected threads, > > allThreads=Set(ZkClient-EventThread-268-127.0.0.1:56565, > > ProcessThread(sid:0 cport:56565):, metrics-mete > > r-tick-thread-2, SessionTracker, Signal Dispatcher, main, Reference > > Handler, ForkJoinPool-1-worker-1, Attach Listener, ProcessThread(sid:0 > > cport:59720):, ZkClie > > nt-EventThread-1347-127.0.0.1:59720, kafka-producer-network-thread | > > producer-1, Test worker-SendThread(127.0.0.1:56565), /127.0.0.1:54942 to > > /127.0.0.1:54926 w > > orkers Thread 2, Test worker, SyncThread:0, > > NIOServerCxn.Factory:/127.0.0.1:0, Test worker-EventThread, Test > > worker-SendThread(127.0.0.1:59720), /127.0.0.1:5494 > > 2 to /127.0.0.1:54926 workers Thread 3, > > ZkClient-EventThread-22-127.0.0.1:54976, ProcessThread(sid:0 > > cport:54976):, Test worker-SendThread(127.0.0.1:54976), Fin > > alizer, metrics-meter-tick-thread-1) > > > > I tested on a VM and a physical machine, and both give me a lot of errors > > like this. > > > > Thanks. > > --Vahid > > > > > > > > > > From: Ismael Juma <isma...@gmail.com> > > To: Vahid S Hashemian <vahidhashem...@us.ibm.com> > > Cc: dev@kafka.apache.org, kafka-clients > > <kafka-clie...@googlegroups.com>, Kafka Users <us...@kafka.apache.org> > > Date: 06/26/2017 03:53 AM > > Subject: Re: [VOTE] 0.11.0.0 RC2 > > > > > > > > Hi Vahid, > > > > Sorry for not replying to the previous email, I had missed it. A couple > of > > questions: > > > > 1. Is this also happening in trunk? Seems like it should be the case for > > months and seemingly no-one reported it until the RC stage. > > 2. Is it correct that this only happens when compiling on Windows without > > Cygwin? > > > > I believe the following PR should fix it, please verify: > > > > https://github.com/apache/kafka/pull/3431 > > > > Ismael > > > > On Fri, Jun 23, 2017 at 8:25 PM, Vahid S Hashemian < > > vahidhashem...@us.ibm.com> wrote: > > > > > Hi Ismael, > > > > > > Not sure if my response on RC1 was lost or this issue is not a > > > show-stopper: > > > > > > I checked again and with RC2, tests still fail in my Windown 64 bit > > > environment. > > > > > > :clients:checkstyleMain > > > [ant:checkstyle] [ERROR] C:\Users\User\Downloads\kafka- > > > > > 0.11.0.0-src\clients\src\main\java\org\apache\kafka\common\ > > protocol\Errors.java:89:1: > > > Class Data Abstraction Coupling is 57 (max allowed is 20) classes > > > [ApiExceptionBuilder, BrokerNotAvailableException, > > > ClusterAuthorizationException, ConcurrentTransactionsException, > > > ControllerMovedException, CoordinatorLoadInProgressException, > > > CoordinatorNotAvailableException, CorruptRecordException, > > > DuplicateSequenceNumberException, GroupAuthorizationException, > > > IllegalGenerationException, IllegalSaslStateException, > > > InconsistentGroupProtocolException, InvalidCommitOffsetSizeException, > > > InvalidConfigurationException, InvalidFetchSizeException, > > > InvalidGroupIdException, InvalidPartitionsException, > > > InvalidPidMappingException, InvalidReplicaAssignmentException, > > > InvalidReplicationFactorException, InvalidRequestException, > > > InvalidRequiredAcksException, InvalidSessionTimeoutException, > > > InvalidTimestampException, InvalidTopicException, > > InvalidTxnStateException, > > > InvalidTxnTimeoutException, LeaderNotAvailableException, > > NetworkException, > > > NotControllerException, NotCoordinatorException, > > > NotEnoughReplicasAfterAppendException, NotEnoughReplicasException, > > > NotLeaderForPartitionException, OffsetMetadataTooLarge, > > > OffsetOutOfRangeException, OperationNotAttemptedException, > > > OutOfOrderSequenceException, PolicyViolationException, > > > ProducerFencedException, RebalanceInProgressException, > > > RecordBatchTooLargeException, RecordTooLargeException, > > > ReplicaNotAvailableException, SecurityDisabledException, > > TimeoutException, > > > TopicAuthorizationException, TopicExistsException, > > > TransactionCoordinatorFencedException, > > TransactionalIdAuthorizationException, > > > UnknownMemberIdException, UnknownServerException, > > > UnknownTopicOrPartitionException, UnsupportedForMessageFormatExc > eption, > > > UnsupportedSaslMechanismException, UnsupportedVersionException]. > > > [ClassDataAbstractionCoupling] > > > [ant:checkstyle] [ERROR] C:\Users\User\Downloads\kafka- > > > > > 0.11.0.0-src\clients\src\main\java\org\apache\kafka\common\ > > protocol\Errors.java:89:1: > > > Class Fan-Out Complexity is 60 (max allowed is 40). > > [ClassFanOutComplexity] > > > [ant:checkstyle] [ERROR] C:\Users\User\Downloads\kafka- > > > 0.11.0.0-src\clients\src\main\java\org\apache\kafka\common\ > > > requests\AbstractRequest.java:26:1: Class Fan-Out Complexity is 43 > (max > > > allowed is 40). [ClassFanOutComplexity] > > > [ant:checkstyle] [ERROR] C:\Users\User\Downloads\kafka- > > > 0.11.0.0-src\clients\src\main\java\org\apache\kafka\common\ > > > requests\AbstractResponse.java:26:1: Class Fan-Out Complexity is 42 > (max > > > allowed is 40). [ClassFanOutComplexity] > > > :clients:checkstyleMain FAILED > > > > > > FAILURE: Build failed with an exception. > > > > > > Thanks. > > > --Vahid > > > > > > > > > > > > From: Ismael Juma <ism...@juma.me.uk> > > > To: dev@kafka.apache.org, Kafka Users <us...@kafka.apache.org>, > > > kafka-clients <kafka-clie...@googlegroups.com> > > > Date: 06/22/2017 06:16 PM > > > Subject: [VOTE] 0.11.0.0 RC2 > > > Sent by: isma...@gmail.com > > > ------------------------------ > > > > > > > > > > > > Hello Kafka users, developers and client-developers, > > > > > > This is the third candidate for release of Apache Kafka 0.11.0.0. > > > > > > This is a major version release of Apache Kafka. It includes 32 new > > KIPs. > > > See the release notes and release plan ( > > > https://cwiki.apache.org/confluence/display/KAFKA/ > Release+Plan+0.11.0.0) > > > for more details. A few feature highlights: > > > > > > * Exactly-once delivery and transactional messaging > > > * Streams exactly-once semantics > > > * Admin client with support for topic, ACLs and config management > > > * Record headers > > > * Request rate quotas > > > * Improved resiliency: replication protocol improvement and > > single-threaded > > > controller > > > * Richer and more efficient message format > > > > > > Release notes for the 0.11.0.0 release: > > > http://home.apache.org/~ijuma/kafka-0.11.0.0-rc2/RELEASE_NOTES.html > > > > > > *** Please download, test and vote by Tuesday, June 27, 6pm PT > > > > > > Kafka's KEYS file containing PGP keys we use to sign the release: > > > http://kafka.apache.org/KEYS > > > > > > * Release artifacts to be voted upon (source and binary): > > > http://home.apache.org/~ijuma/kafka-0.11.0.0-rc2/ > > > > > > * Maven artifacts to be voted upon: > > > https://repository.apache.org/content/groups/staging/org/apache/kafka/ > > > > > > * Javadoc: > > > http://home.apache.org/~ijuma/kafka-0.11.0.0-rc2/javadoc/ > > > > > > * Tag to be voted upon (off 0.11.0 branch) is the 0.11.0.0 tag: > > > https://git-wip-us.apache.org/repos/asf?p=kafka.git;a=tag;h= > > > 8698fa1f41102f1664b05baa4d6953fc9564d91e > > > > > > * Documentation: > > > http://kafka.apache.org/0110/documentation.html > > > > > > * Protocol: > > > http://kafka.apache.org/0110/protocol.html > > > > > > * Successful Jenkins builds for the 0.11.0 branch: > > > Unit/integration tests: https://builds.apache.org/job/ > > > kafka-0.11.0-jdk7/187/ > > > System tests: pending (will send an update tomorrow) > > > > > > /************************************** > > > > > > Thanks, > > > Ismael > > > > > > > > > > > > > > > > > > > > > > > > > >
