I assume you are using "*bin/flink run-application*" to submit a Flink
application to K8s cluster. Then you could simply
update your local log4j-console.properties, it will be shipped and mounted
to JobManager/TaskManager pods via ConfigMap.
Best,
Yang
Vladislav Keda 于2023年6月20日周二
22:15写道：
> Hi
Hi patricia,
The TTL compaction filter in RocksDB has been enabled in 1.10 by
default and it is always enabled in 1.11+[1], I think there is no
need to explicitly enable the ttl compaction filter.
[1]
https://nightlies.apache.org/flink/flink-docs-release-1.17/release-notes/flink-1.11/#removal-of-
Hi Dongwoo,
State changelogs are continuously uploaded to the durable storage when
Changelog state backend is enabled. In other words, it will also
persist data **outside the checkpoint phase**, and the exception at
this time will directly cause the job to fail. And only exceptions in
the checkpo
Hi Pritam,
Slots in TM focus more on logical management for cores, this means that TM
could not assign physical cores to each slot. RM will manage the core usage
of each TM, such as TM1 using 2 cores and TM2 using 3 cores. According to
the resource usage required by the task (also logical and esti
Hi Team,
I have a confusion about how TM's core will be used for task slots.
I have TMs with 3 cores and 4 task slots . Each task slot will be using one
core to perform a task and one task slot has to wait to get resources ? If
I increase the cores to 4 , same as task slots per TM will it help to
Thanks, Shammon and Alex, for the pointers. The Rocsdb state backend is
being used but without an incremental checkpoint. I will enable incremental
checkpoints and see if it works. Thanks.
On Tue, Jun 20, 2023 at 5:25 PM Shammon FY wrote:
> Hi Prabhu,
>
> I found that the size of `Full Checkpoi
Hi all again!
Please tell me if you can answer my question, thanks.
---
Best Regards,
Vladislav Keda
пт, 16 июн. 2023 г. в 16:12, Vladislav Keda <
vladislav.k...@glowbyteconsulting.com>:
> Hi all!
>
> Is it possible to change Flink* log4j-console.properties* in Native
> Kubernetes (for example
Hi Galen,
Yes, I'll be more than happy to help with Statefun releases.
Best regards,
Martijn
On Tue, Jun 20, 2023 at 2:21 PM Galen Warren
wrote:
> Thanks.
>
> Martijn, to answer your question, I'd need to do a small amount of work to
> get a PR ready, but not much. Happy to do it if we're dec
Thanks.
Martijn, to answer your question, I'd need to do a small amount of work to
get a PR ready, but not much. Happy to do it if we're deciding to restart
Statefun releases -- are we?
-- Galen
On Sat, Jun 17, 2023 at 9:47 AM Tzu-Li (Gordon) Tai
wrote:
> > Perhaps he could weigh in on whether
Hi Prabhu,
I found that the size of `Full Checkpoint Data Size` is equal to
`Checkpointed Data Size`. So what's the state backend you are using? I
recommend you to use rocksdb state backed for your job, and if so, you can
turn on incremental checkpoint [1] which will reduce the state size for the
Hello all, I have a question about changelog persist failure.
When changelog persist process fails due to an S3 timeout, it seems to lead
to the job failure regardless of our
"execution.checkpointing.tolerable-failed-checkpoints" configuration being
set to 5 with this log.
Caused by: java.io.IOExce
Hello Prabhu,
On your place I would check:
1. That there is no "state leak" in your job, because it seems that state
only accumulates for the job and is never cleaned, e.g. probably some timer
which cleans the state for some key is not configured correctly.
2. Probably you accumulate the state i
Hi,
Flink Checkpoint times out with checkpointed data size doubles every
checkpoint. Any ideas on what could be wrong in the application or how to
debug this?
[image: checkpoint_issue.png]
Hi,
>From 1.8 to 1.17 flink, enableTtlCompactionFilter() has been removed.
I have seen some examples to do a factory of options to pass as argumets
for settings, is this the right approach? If not what is the best way to
enable the compaction filter in rocksdbstatebackend?
Thanks
Regards
15 matches
Mail list logo
