imes while other times there's a more or
less contant flow of data, is there any specific network buffer settings that I
might try to tweak and observe the changes?
From: Jake.zhang
Sent: Tuesday, October 22, 2024 2:40 PM
To: Raihan Sunny ; user
Subject:
Hello,
I have an aggregator job that experiences backpressure after running for a
while and completely stops processing. It doesn't take any further input from
the source. Here's a bit of context:
- There are 3 producer jobs, all of which write data to a common Kafka topic
- The aggregator job r
rce, deployConfig);
}
After this modification, everything works perfectly fine as we expected in our
environment.
On 2024/05/07 05:48:02 Raihan Sunny wrote:
> Hi everyone,
>
> I need some help with deploying multiple jobs from a single main function in
> Application mode using Flink Ku
Hi everyone,
I need some help with deploying multiple jobs from a single main function in
Application mode using Flink Kubernetes Operator. As per the documentation [1]
it should be possible to use multiple "executeAsync()" to deploy multiple jobs
from the same file. This indeed is the case whe
Hi,
I have a few Flink jobs running on Kubernetes using the Flink Kubernetes
Operator. By following the documentation [1] I was able to set up
monitoring for the Operator itself. As for the jobs themselves, I'm a bit
confused about how to properly set it up. Here's my FlinkDeployment
configuration
decorators. It's true by default, but by setting it to false
> Flink won't add this mount.
>
> [1]
> https://nightlies.apache.org/flink/flink-docs-release-1.17/docs/deployment/config/#kubernetes-decorator-hadoop-conf-mount-enabled
>
> Regards,
> Mate
>
> Raihan
Hi,
I've been using HDFS with Flink for checkpoint and savepoint storage which
works perfectly fine. Now I have another use case where I want to read and
write to HDFS from the application code as well. For this, I'm using the
"pyarrow" library which is already installed with PyFlink as a dependen
nt.
> After that you can check the autoscaler logs for more info.
>
> Gyula
>
> On Tue, Aug 1, 2023 at 10:33 AM Raihan Sunny via user <
> user@flink.apache.org> wrote:
>
>> Hi,
>>
>> I have a workload that depends on the GPU. I have only 1 GPU card. As per
Hi,
I have a workload that depends on the GPU. I have only 1 GPU card. As per
the documentation I have added the necessary configurations and can run the
GPU workload in standalone REACTIVE mode with as many taskmanager instances
as required.
I have set the number of task slots to 1 so that a rai
doop-dependencies
>
> Best regards,
>
> Martijn
>
> On Mon, Jan 9, 2023 at 10:40 AM Raihan Sunny
> wrote:
>
>> Hi,
>>
>> I'm trying to integrate HDFS in Flink for checkpoint and savepoint
>> storage. I have an HDFS cluster running on D
Hi,
I'm trying to integrate HDFS in Flink for checkpoint and savepoint storage.
I have an HDFS cluster running on Docker. I have made the following changes
to the flink configuration:
state.backend: filesystem
state.savepoints.dir: hdfs://namenode:9000/user/root/savepoints
state.checkpoints.dir:
any pipelined regions are there?
> Since all edges are forward edges, A, B, C and D must have the same
> parallelism, and there will be the same pipelined regions as the
> parallelism.
>
> On Wed, Dec 21, 2022 at 5:14 PM Raihan Sunny
> wrote:
>
>> Hello Gen,
>>
&g
;>
>>
>>
>> Sincere greetings from the supposed close-by Zurich 😊
>>
>>
>>
>> Thias
>>
>>
>>
>>
>>
>> [1]
>> https://nightlies.apache.org/flink/flink-docs-master/docs/dev/datastream/fault-tolerance/checkpo
Hi,
I'm quite new to the world of stream and batch processing. I've been
reading about pipelined regions in Flink and am quite confused by what it
means. My specific problem involves a streaming job that looks like the
following:
1. There is a Kafka source that takes in an input data that sets of
14 matches
Mail list logo
