Hi, John
Could you tell us you application scenario? Is it a flink session cluster with
a lot of jobs?
Maybe you can try to dump the memory with jmap and use tools such as MAT to
analyze whether there are abnormal classes and classloaders
> 2022年3月30日 上午6:09，John Smith 写道：
>
> Hi running 1.
Hi,
Thank you for the quick responses. We are using the datastream api for
pyflink. We are trying to implement a wrapper in python for the same as we
speak. Hopefully it will work out. 😊
On Wed, 30 Mar, 2022, 8:02 am Xingbo Huang, wrote:
> Hi,
>
> Are you using datastream api or table api？If yo
Hi,
Are you using datastream api or table api？If you are using the table api,
you can use the connector by executing sql[1]. If you are using the
datastream api, there is really no es connector api provided, you need to
write python wrapper code, but the wrapper code is very simple. The
underlying
Hi running 1.14.4
My tasks manager still fails with java.lang.OutOfMemoryError: Metaspace.
The metaspace out-of-memory error has occurred. This can mean two things:
either the job requires a larger size of JVM metaspace to load classes or
there is a class loading leak.
I have 2GB of metaspace con
Thank you very much for your answer.
I was able to reduce the number of sinks as you described. That helped a
lot, thank you.
I think you must be right with regards to (2) - opening a new transaction
being the culprit. It's unlikely to be (1) since this behaviour occurs even
when there are 0 mess
In this situation, changing your configuration [1] to include
cluster.evenly-spread-out-slots: true
should change the scheduling behavior to what you are looking for.
[1]
https://nightlies.apache.org/flink/flink-docs-release-1.14/docs/deployment/config/#cluster-evenly-spread-out-slots
Regar
I got it working now: It needs to be specified both for the key and value
thanks
Am Mo., 28. März 2022 um 13:33 Uhr schrieb Ingo Bürk :
> Hi Georg,
>
> which Flink version are you using? The missing property is for the
> avro-confluent format, and if I recall correctly, how these are passed
>
Are you referring to creating Flink cluster on Kubernetes by yaml file?
How did you submit the job to Flink cluster? Not via the command line (flink
run xxx)?
> 2022年3月29日 下午10:38，Jin Yi 写道：
>
> no they are not. b/c we are using k8s, we use kubectl apply commands with a
> yaml file to specif
no they are not. b/c we are using k8s, we use kubectl apply commands with
a yaml file to specify the startup.
On Tue, Mar 29, 2022 at 7:37 AM 胡伟华 wrote:
> I see, can you provide the startup command for 1.12.3 and 1.14.4?
> Are these startup commands running on the same node?
>
> 2022年3月29日 下午10
I see, can you provide the startup command for 1.12.3 and 1.14.4?
Are these startup commands running on the same node?
> 2022年3月29日 下午10:32，Jin Yi 写道：
>
> it's running in k8s. we're not running in app mode b/c we have many jobs
> running in the same flink cluster.
>
> On Tue, Mar 29, 2022 at
it's running in k8s. we're not running in app mode b/c we have many jobs
running in the same flink cluster.
On Tue, Mar 29, 2022 at 4:29 AM huweihua wrote:
> Hi, Jin
>
> Can you provide more information about Flink cluster deployment modes? Is
> it running in Kubernetes/YARN or standalone mode?
Hello Matthias,
I am still using ProcessingTimeSessionWindow.
But it turns out I was wrong.
I tested a couple of times and it did not seem to work.
But now it does with both watermarkstrategies removed.
My apologies.'
Regards Hans-Peter
This is the code:
StreamExecutionEnvironment env =
Hello Hans-Peter,
I’m a little confused which version of your code you are testing against:
* ProcessingTimeSessionWindows or EventTimeSessionWindows?
* did you keep the withIdleness() ??
As said before:
* for ProcessingTimeSessionWindows, watermarks play no role
* if you keep w
Hello Everyone,
I have been working on a streaming application using elasticsearch as the
sink. I had achieved it using the java api quite easily. But due to a
recent policy change we are moving towards the python api for flink,
however we were unable to find any python elastic search connectors f
Hi,
I have a 2 node cluster just for testing.
When I start the cluster and the job I see that the parallelism is 2 as
expected.
But only they are both on the same node.
When I stop the taskmanager on that node it switches to the other one.
But I expected both nodes to have a subtask.
See below.
Hi, Jin
Can you provide more information about Flink cluster deployment modes? Is
it running in Kubernetes/YARN or standalone mode?
Maybe you can use application mode to keeps the environment (network
accessibility) always keep same. Application mode will run the user-main
method in the JobManager
Hello Matthias,
When I remove all the watermark strategies it does not process anything .
For example when I use WatermarkStrategy.noWatermarks() instead of the one
I build nothing seems to happen at all.
Also when I skip the part where I add wmStrategy to create tuple4dswm:
DataStream> tuple4
Hi Filip,
two things will impact sync time for Kafka:
1. Flushing all old data [1], in particular flushing all in-flight
partitions [2]. However, that shouldn't cause a stacking effect except when
the brokers are overloaded on checkpoint.
2. Opening a new transaction [3]. Since all transactions ar
cc Kevin
On Tue, Mar 29, 2022 at 9:15 AM David Morávek wrote:
> Hi Kevin,
>
> -dev@f.a.o +user@f.a.o
>
> Thanks for the report! I've run some experiments and unfortunately I'm not
> able to reproduce the behavior you're describing. The bash "$@" expansion
> seems to work as expected (always rece
Hi Kevin,
-dev@f.a.o +user@f.a.o
Thanks for the report! I've run some experiments and unfortunately I'm not
able to reproduce the behavior you're describing. The bash "$@" expansion
seems to work as expected (always receiving correctly expanded unquoted
strings in the main class). Can you maybe e
20 matches
Mail list logo
