[
https://issues.apache.org/jira/browse/FLINK-29295?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jingsong Lee closed FLINK-29295.
--------------------------------
Resolution: Fixed
master: 6a1b1bca4da131e47ee17cbf5d10472762a66da6
release-0.2: 75a732b40c868bab026f45478ce33530861407d8
> Clear RecordWriter slower to avoid causing frequent compaction conflicts
> ------------------------------------------------------------------------
>
> Key: FLINK-29295
> URL: https://issues.apache.org/jira/browse/FLINK-29295
> Project: Flink
> Issue Type: Bug
> Components: Table Store
> Reporter: Jingsong Lee
> Assignee: Jingsong Lee
> Priority: Major
> Labels: pull-request-available
> Fix For: table-store-0.3.0, table-store-0.2.1
>
>
> In AbstractTableWrite, the Writer is cleaned up as soon as no new files are
> generated, which may lead to the changes generated after the compaction have
> not been committed, but the new data from the next checkpoint comes to create
> a new writer, which conflicts with the changes generated in the next round of
> checkpoint and the previous round, resulting in an exception:
> {code:java}
> Caused by: java.lang.IllegalStateException: Trying to delete file
> {org.apache.flink.table.data.binary.BinaryRowData@5759f99e, 0, 0,
> data-7bf2498e-d0a1-42a5-97b7-b3860f10b076-0.orc} which is not previously
> added. Manifest might be corrupted.
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
