[
https://issues.apache.org/jira/browse/FLINK-15611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17019890#comment-17019890
]
Jiangjie Qin commented on FLINK-15611:
--------------------------------------
Just a quick update. The log actually shows some problems.
# In the first snapshot, the {{ValidatingExactlyOnceSink}} has seen 543
records, but the state only shows the offsets from partition 0 - partition 3.
Partition 4 was missing.
# There are 200 records consumed before the first checkpoint, and all of them
are from partition 3, so the checkpointed offset was 199.
Combining 1 and 2 that means partition 4 was consumed and its records have been
processed by the {{ValidatingExactlyOnceSink}}, but due to some bug the offset
was not checkpointed by the mapper consuming from it, therefore when the job
restarts the validating sink sees duplicates.
> KafkaITCase.testOneToOneSources fails on Travis
> -----------------------------------------------
>
> Key: FLINK-15611
> URL: https://issues.apache.org/jira/browse/FLINK-15611
> Project: Flink
> Issue Type: Bug
> Components: Connectors / Kafka, Tests
> Reporter: Yangze Guo
> Assignee: Jiangjie Qin
> Priority: Blocker
> Fix For: 1.10.0
>
>
> {{The test KafkaITCase.testOneToOneSources failed on Travis.}}
> {code:java}
> 03:15:02,019 INFO
> org.apache.flink.streaming.connectors.kafka.KafkaTestEnvironmentImpl -
> Deleting topic scale-down-before-first-checkpoint
> 03:15:02,037 INFO
> org.apache.flink.streaming.connectors.kafka.FlinkKafkaProducerITCase -
> --------------------------------------------------------------------------------
> Test
> testScaleDownBeforeFirstCheckpoint(org.apache.flink.streaming.connectors.kafka.FlinkKafkaProducerITCase)
> successfully run.
> ================================================================================
> 03:15:02,038 INFO org.apache.flink.streaming.connectors.kafka.KafkaTestBase
> - -------------------------------------------------------------------------
> 03:15:02,038 INFO org.apache.flink.streaming.connectors.kafka.KafkaTestBase
> - Shut down KafkaTestBase
> 03:15:02,038 INFO org.apache.flink.streaming.connectors.kafka.KafkaTestBase
> - -------------------------------------------------------------------------
> 03:15:25,728 INFO org.apache.flink.streaming.connectors.kafka.KafkaTestBase
> - -------------------------------------------------------------------------
> 03:15:25,728 INFO org.apache.flink.streaming.connectors.kafka.KafkaTestBase
> - KafkaTestBase finished
> 03:15:25,728 INFO org.apache.flink.streaming.connectors.kafka.KafkaTestBase
> - -------------------------------------------------------------------------
> 03:15:25.731 [INFO] Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time
> elapsed: 245.845 s - in
> org.apache.flink.streaming.connectors.kafka.FlinkKafkaProducerITCase
> 03:15:26.099 [INFO]
> 03:15:26.099 [INFO] Results:
> 03:15:26.099 [INFO]
> 03:15:26.099 [ERROR] Failures:
> 03:15:26.099 [ERROR]
> KafkaITCase.testOneToOneSources:97->KafkaConsumerTestBase.runOneToOneExactlyOnceTest:862
> Test failed: Job execution failed.
> {code}
> https://api.travis-ci.com/v3/job/276124537/log.txt
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
