I looked into the disk issues and found that Fabian was on the right path.
The checkpoints that were lingering were in-fact in use.
Thanks for the help!
Clay
On Thu, Sep 26, 2019 at 8:09 PM Clay Teeter wrote:
> I see, I'll try turning off incremental checkpoints to see if that helps.
>
> re:
I see, I'll try turning off incremental checkpoints to see if that helps.
re: Diskspace, i could see a scenario with my application where i could get
10,000+ checkpoints, if the checkpoints are additive. I'll let you know
what i see.
Thanks!
Clay
On Wed, Sep 25, 2019 at 5:40 PM Fabian Hueske
Hi,
You enabled incremental checkpoints.
This means that parts of older checkpoints that did not change since the
last checkpoint are not removed because they are still referenced by the
incremental checkpoints.
Flink will automatically remove them once they are not needed anymore.
Are you sure t
Oh geez, checkmarks = checkpoints... sorry.
What i mean by stale "checkpoints" are checkpoints that should be reaped
by: "state.checkpoints.num-retained: 3".
What is happening is that directories:
- state.checkpoints.dir: file:///opt/ha/49/checkpoints
- high-availability.storageDir: file://
Hi Clay,
Sorry I don't get your point. I'm not sure what the "stale checkmarks"
exactly means. The HA storage and checkpoint directory left after shutting
down cluster?
Thanks,
Biao /'bɪ.aʊ/
On Tue, 24 Sep 2019 at 03:12, Clay Teeter wrote:
> I'm trying to get my standalone cluster to remove
I'm trying to get my standalone cluster to remove stale checkmarks.
The cluster is composed of a single job and task manager backed by rocksdb
with high availability.
The configuration on both the job and task manager are:
state.backend: rocksdb
state.checkpoints.dir: file:///opt/ha/49/checkpoin
