Large backpressure and slow checkpoints in StateFun

2022-05-29 Thread Christopher Gustafson
Hi, I am running some benchmarks using StateFun and have encountered a problem with backpressure and slow checkpoints that I can't figure out the reason for, and was hoping that someone might have an idea of what is causing it. My setup is the following: I am running the Shopping Cart applic

Re: Custom restart strategy

2022-05-29 Thread Shengkai Fang
Hi. Maybe the metric reporter[1] is suitabe for your case. [1] https://nightlies.apache.org/flink/flink-docs-master/docs/deployment/metric_reporters/ unknown unknown 于2022年5月28日周六 12:49写道: > Thanks Shengkai! Unfortunately, this would require querying status for > each job continuously. Given v

Re: GlobalCommitter in Flink's two-phase commit

2022-05-29 Thread Jing Ge
Hi, 1. What are the general usage scenarios of GlobalCommitter? - GlobalCommitter is used for creating and committing an aggregated committable. It is part of a 2-phase-commit protocol. One use case is the compaction of small files. 2. Why should GlobalCommitter be removed in the new version of t