gharris1727 commented on PR #13575: URL: https://github.com/apache/kafka/pull/13575#issuecomment-1513825281
Okay, upon further investigation it appears that this is technically a flakey test failure, it just nearly always fails in CI due to CPU load. The core reason for the failure is that the FakeForwardingAdminWithLocalMetadata has a fixed 1-second timeout for each operation. This has the effect that each of the operations that the FakeForwardingAdminWithLocalMetadata intercepts can succeed in Kafka, and the connector will know it has succeeded, but the FakeLocalMetadataStore will not be updated, which can cause assertions to fail despite the connector satisfying the condition described by the test name. The reason for multiple distinct tests all regressing at once, and all passing locally appears to be due to the _tests not distinguishing between the different admin client operations, and conflating one operation for another_. This meant that before the regression, the tests which created the topics and created partitions actually were relying on the config sync to pass. After the regression, the tests which created partitions and synced configs passed locally because the topic creation had a good success rate with low CPU load. With this knowledge, I think that this implementation of the FakeLocalMetadataStore is fundamentally flawed, and these tests are ultimately not performing the assertions that they should be. They can not reliably detect regressions and aren't really providing any value as-is. I think in this PR i'll apply a tactical fix to get the tests to pass (either raise or remove the timeout, or mark the tests as ignored) and follow-up after the upcoming release to rewrite these tests to actually test what they say they do. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: jira-unsubscr...@kafka.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
