Hi,
Are you sure that your growing memory came from DirectByteBuffer? What about
metaspace?
Flink 1.9 may have some metaspace leak after a full restart or fine-grained
restart, see [1] and [2] for more details. And if you didn't set a max
metaspace by -XX:MaxMetaspaceSize, it will grow indefin
Hi,
I’m reusing the same StreamTableEnvironment to submit multiple table/sql jobs
to a session cluster,
but I couldn’t find a proper way to specify job resources for each job (like
parallelism and total process
memory), and they all uses the cluster default.
I have considered overriding reso
We would like a counter of exceptions so we can alert if there's an
anomalous increase in them. I realize a counter in the JobManager would not
capture anywhere close to all exceptions but even capturing a count of a
subset that we're able to track would be helpful.
On Thu, Jul 15, 2021 at 3:47 PM
This is currently not possible. What metric are you interested in?
On 15/07/2021 21:16, Jeff Charles wrote:
Hi,
I would like to register a custom metric on the JobManager as opposed
to a TaskManager. I cannot seem to locate any documentation that
indicates how to do this or even if it's curre
Hi,
I would like to register a custom metric on the JobManager as opposed to a
TaskManager. I cannot seem to locate any documentation that indicates how
to do this or even if it's currently possible or not.
Does anyone have any guidance on how to register a custom metric on the
JobManager?
Jeff
Thanks so much Dawid for your time and explanations. Mans
On Thursday, July 15, 2021, 10:09:23 AM EDT, Dawid Wysakowicz
wrote:
Ad 1/2/4 Please refer to Avro's documentation on how do the reader and writer
schemas work e.g. here[1] and here[2]. That's purely Avro's thing.
Ad.3 Theo
I am not using the Kafka SSL port.
On Thu, Jul 15, 2021 at 9:48 PM Alexey Trenikhun wrote:
> Just in case, make sure that you are not using Kafka SSL port without
> setting security protocol, see [1]
>
> [1] https://issues.apache.org/jira/plugins/servlet/mobile#issue/KAFKA-4090
> ---
Just in case, make sure that you are not using Kafka SSL port without setting
security protocol, see [1]
[1] https://issues.apache.org/jira/plugins/servlet/mobile#issue/KAFKA-4090
From: bat man
Sent: Wednesday, July 14, 2021 10:55:54 AM
To: Timo Walther
Cc: user
Thanks for the feedback Piotrek.
We have observed the issue again today. As we are using Flink 1.11.1, I
tried to check the backpressure of Kafka source tasks from the
Jobmanager UI.
The backpressure request was canceled due to Timeout and "No Data" was
displayed in UI. Here are the respective log
Ad 1/2/4 Please refer to Avro's documentation on how do the reader and
writer schemas work e.g. here[1] and here[2]. That's purely Avro's thing.
Ad.3 Theoretically yes. The problem is the communication between
TaskManagers. We need to serialize the Avro records somehow. The chosen
approach is to u
Hello Dawid:
Thanks for your answers and references.
I do have a few questions:
1. Is there any scenario where the reader and writer schema should differ ?
2. How is the mismatch b/w the two schemas (one passed as argument and other
retrieved from schema registry) resolved at run time ?
3. As me
Hello Guys,
I'm sorry for asking you this question, it does not have any link with
Apache Flink, but if someone can help I would be so grateful. I want to
build a big data architecture for batch processing, we have a lot of data
that is generated everyday and we receive it upon a lot of sources
es
Hi Ragini,
AsyncDataStream.unorderedWait() or AsyncDataStream.orderedWait() takes a
timeout as a parameter and the TimeUnit for the timeout specified as
another parameter.
The timeout() method above is called when an Async I/O operation is timed
out. The default operation is to raise an exception
13 matches
Mail list logo
