[
https://issues.apache.org/jira/browse/FLINK-19540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xinhao Li closed FLINK-19540.
-----------------------------
Resolution: Resolved
> How to reduce or disable checkpoint logs in Flink
> -------------------------------------------------
>
> Key: FLINK-19540
> URL: https://issues.apache.org/jira/browse/FLINK-19540
> Project: Flink
> Issue Type: Improvement
> Affects Versions: 1.11.0
> Reporter: Xinhao Li
> Priority: Major
>
> I am using Flink 1.11.0 and run it on Kubernetes in standalone mode with hdfs
> for storage and HA. These days, I try to enable the Flink checkpoint feature.
> But I notice both jobmanger and taskmanagers are logging too many logs relate
> to the checkpoint which is annoying. Examples are as following:
> *Jobmanager*
> {code:java}
> 2020-10-08 19:54:23,237 [INFO]
> org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Triggering
> checkpoint 1 (type=CHECKPOINT) @ 1602186863226 for job
> fbf26a33a6d5d235085d10e7a10c1cab.
> 2020-10-08 19:54:42,818 [INFO]
> org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Completed
> checkpoint 1 for job fbf26a33a6d5d235085d10e7a10c1cab (702534 bytes in 19488
> ms).
> 2020-10-08 19:54:42,825 [INFO]
> org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Triggering
> checkpoint 2 (type=CHECKPOINT) @ 1602186882820 for job
> fbf26a33a6d5d235085d10e7a10c1cab.
> 2020-10-08 19:54:43,384 [INFO]
> org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Completed
> checkpoint 2 for job fbf26a33a6d5d235085d10e7a10c1cab (729357 bytes in 494
> ms).
> 2020-10-08 19:54:43,392 [INFO]
> org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Triggering
> checkpoint 3 (type=CHECKPOINT) @ 1602186883388 for job
> fbf26a33a6d5d235085d10e7a10c1cab.
> 2020-10-08 19:54:44,295 [INFO]
> org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Completed
> checkpoint 3 for job fbf26a33a6d5d235085d10e7a10c1cab (736969 bytes in 836
> ms).
> 2020-10-08 19:54:44,302 [INFO]
> org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Triggering
> checkpoint 4 (type=CHECKPOINT) @ 1602186884298 for job
> fbf26a33a6d5d235085d10e7a10c1cab.
> 2020-10-08 19:54:44,794 [INFO]
> org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Completed
> checkpoint 4 for job fbf26a33a6d5d235085d10e7a10c1cab (748787 bytes in 431
> ms).
> 2020-10-08 19:54:44,800 [INFO]
> org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Triggering
> checkpoint 5 (type=CHECKPOINT) @ 1602186884796 for job
> fbf26a33a6d5d235085d10e7a10c1cab.
> 2020-10-08 19:54:45,198 [INFO]
> org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Completed
> checkpoint 5 for job fbf26a33a6d5d235085d10e7a10c1cab (755308 bytes in 327
> ms).
> 2020-10-08 19:54:45,703 [INFO]
> org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Triggering
> checkpoint 6 (type=CHECKPOINT) @ 1602186885698 for job
> fbf26a33a6d5d235085d10e7a10c1cab.
> 2020-10-08 19:54:45,897 [INFO]
> org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Completed
> checkpoint 6 for job fbf26a33a6d5d235085d10e7a10c1cab (757353 bytes in 163
> ms).
> 2020-10-08 19:54:45,903 [INFO]
> org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Triggering
> checkpoint 7 (type=CHECKPOINT) @ 1602186885899 for job
> fbf26a33a6d5d235085d10e7a10c1cab.
> {code}
> *Taskmanager*
> {code:java}
> 2020-10-08 20:04:02,090 [INFO]
> org.apache.flink.streaming.api.functions.sink.TwoPhaseCommitSinkFunction -
> FlinkKafkaProducer 1/2 - checkpoint 571 complete, committing transaction
> TransactionHolder{handle=KafkaTransactionState [transactionalId=null,
> producerId=-1, epoch=-1], transactionStartTime=1602187440992} from checkpoint
> 571
> 2020-10-08 20:04:03,086 [INFO]
> org.apache.flink.streaming.api.functions.sink.TwoPhaseCommitSinkFunction -
> FlinkKafkaProducer 1/2 - checkpoint 572 complete, committing transaction
> TransactionHolder{handle=KafkaTransactionState [transactionalId=null,
> producerId=-1, epoch=-1], transactionStartTime=1602187441992} from checkpoint
> 572
> 2020-10-08 20:04:03,086 [INFO]
> org.apache.flink.streaming.api.functions.sink.TwoPhaseCommitSinkFunction -
> FlinkKafkaProducer 1/2 - checkpoint 572 complete, committing transaction
> TransactionHolder{handle=KafkaTransactionState [transactionalId=null,
> producerId=-1, epoch=-1], transactionStartTime=1602187441992} from checkpoint
> 572
> 2020-10-08 20:04:04,099 [INFO]
> org.apache.flink.streaming.api.functions.sink.TwoPhaseCommitSinkFunction -
> FlinkKafkaProducer 1/2 - checkpoint 573 complete, committing transaction
> TransactionHolder{handle=KafkaTransactionState [transactionalId=null,
> producerId=-1, epoch=-1], transactionStartTime=1602187442992} from checkpoint
> 573
> 2020-10-08 20:04:04,099 [INFO]
> org.apache.flink.streaming.api.functions.sink.TwoPhaseCommitSinkFunction -
> FlinkKafkaProducer 1/2 - checkpoint 573 complete, committing transaction
> TransactionHolder{handle=KafkaTransactionState [transactionalId=null,
> producerId=-1, epoch=-1], transactionStartTime=1602187442992} from checkpoint
> 573
> 2020-10-08 20:04:05,130 [INFO]
> org.apache.flink.streaming.api.functions.sink.TwoPhaseCommitSinkFunction -
> FlinkKafkaProducer 1/2 - checkpoint 574 complete, committing transaction
> TransactionHolder{handle=KafkaTransactionState [transactionalId=null,
> producerId=-1, epoch=-1], transactionStartTime=1602187443999} from checkpoint
> 574
> 2020-10-08 20:04:05,130 [INFO]
> org.apache.flink.streaming.api.functions.sink.TwoPhaseCommitSinkFunction -
> FlinkKafkaProducer 1/2 - checkpoint 574 complete, committing transaction
> TransactionHolder{handle=KafkaTransactionState [transactionalId=null,
> producerId=-1, epoch=-1], transactionStartTime=1602187443999} from checkpoint
> 574
> 2020-10-08 20:04:06,096 [INFO]
> org.apache.flink.streaming.api.functions.sink.TwoPhaseCommitSinkFunction -
> FlinkKafkaProducer 1/2 - checkpoint 575 complete, committing transaction
> TransactionHolder{handle=KafkaTransactionState [transactionalId=null,
> producerId=-1, epoch=-1], transactionStartTime=1602187444995} from checkpoint
> 575
> 2020-10-08 20:04:06,096 [INFO]
> org.apache.flink.streaming.api.functions.sink.TwoPhaseCommitSinkFunction -
> FlinkKafkaProducer 1/2 - checkpoint 575 complete, committing transaction
> TransactionHolder{handle=KafkaTransactionState [transactionalId=null,
> producerId=-1, epoch=-1], transactionStartTime=1602187444992} from checkpoint
> 575
> {code}
> Is there any way we can disable or reduce the logs from checkpointing? Any
> help will be appreciated!
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
