Dear Team,
I am new to pyflink and request for your support in issue I am facing with
Pyflink. I am using Pyflink version 1.14.4 & using reference code from
pyflink getting started pages.
I am getting following error .
py4j.protocol.Py4JJavaError: An error occurred while calling o10.exec
Hi, Dan
1. Do you mean put the option into savepoint command? If so, I think it
will not work well. This option describe that how checkpoints will be
cleaned up in different job status. e.g. FAILED/CANCELED. It cannot be
covered in savepoint command.
2. Which flink version you use? I work on 1.14 a
Hi.
1. Why isn’t –externalizedCheckpointCleanup an option on savepoint (instead
of being needed at the start of a job run)?
2. Can we get a confirmation dialog when someone clicks "cancel job" in the
UI? Just in case people click on accident.
3. Can we get a way to have Flink clean up the previ
Hi David,
I don't find it troublesome per se, I was rather trying to understand what
should be expected (and documented) for my application. Before I restarted the
job and changed some configurations, it ran for around 10 days and ended up
with a state size of about 1.8GB, so I'm still not sure
Hi,
I am writing (functional) unit test for RichAsyncFunction; following thread
- https://lists.apache.org/thread/12n9fcj9v50sdpnrqsmxgx2ktc44c32c
I am getting following exception. How would I resolve this?
Test -
@ClassRule
val miniCluster = MiniClusterWithClientResource(
MiniClusterResour
Happy to hear that
(back-posted to usr list)
Thias
-Original Message-
From: Κωνσταντίνος Αγαπίδης
Sent: Friday, April 22, 2022 3:50 PM
To: Schwalbe Matthias
Subject: Re: Restore Job from CheckPoint in IntelliJ IDE - MiniCluster
*** EXTERNAL MESSAGE – CAUTION: Think Before You Click
Hi Quynh,
I have added an example on how to use AvroRowDeserializationSchema in
Python DataStream API in [1]. Please take a look at if that helps for you~
Regards,
Dian
[1]
https://github.com/apache/flink/blob/release-1.15/flink-python/pyflink/examples/datastream/formats/avro_format.py
On Fri,
Thanks & Regards,
Samir Vasani
Hi Quynh,
Could you show some sample code on how you use it?
Regards,
Dian
On Fri, Apr 22, 2022 at 1:42 PM lan tran wrote:
> Wonder if this is a bug or not but if I use
> *AvroRowDeserializationSchema,*
>
> In PyFlink the error still occure ?
>
> py4j.protocol.Py4JError: An error occurred whil
I created a JIRA ticket https://issues.apache.org/jira/browse/FLINK-27350 to
track this issue.
BRs,
Chenyu
From: "Zheng, Chenyu"
Date: Friday, April 22, 2022 at 6:26 PM
To: Yang Wang
Cc: "user@flink.apache.org" , "user...@flink.apache.org"
Subject: Re: JobManager doesn't bring up new TaskMan
Thank you, Yang!
In fact I have a fine-grained dashboard for Kubernetes cluster health (like
apiserver qps/latency etc.), and I didn't find anything unusual… Also, the
JobManager container cpu/memory usage is low.
Besides, I have a deep dive in these logs and Flink resource manager code, and
f
FYI: I created FLINK-27354 [1] to cover the issue of retrying to connect to
the RM while shutting down the JobMaster.
This doesn't explain your issue though, Peter. It's still unclear why the
JobMaster is still around as stated in my previous email.
Matthias
[1] https://issues.apache.org/jira/br
Just by looking through the code, it appears that these logs could be
produced while stopping the job. The ResourceManager sends a confirmation
of the JobMaster being disconnected at the end back to the JobMaster. If
the JobMaster is still around to process the request, it would try to
reconnect (I
Hi Shreekant,
Since you're mentioning Iceberg, I'm assuming this means you've added the
Iceberg runtime to Flink. I'm therefore wondering if this is a Flink issue
or an Iceberg issue. I would lean towards the latter. Have you asked them?
Best regards,
Martijn Visser
https://twitter.com/MartijnVi
The root cause might be you APIServer is overloaded or not running
normally. And then all the pods events of
taskmanager-1-9 and taskmanager-1-10 are not delivered to the watch in
FlinkResourceManager.
So the two taskmanagers are not recognized by ResourceManager and then
registration are rejected.
Hi Chesnay,
I had a look in my logs, there are not WARNINGS regarding metrics and
registering metrics when starting this job.
I ran the example jobs
- ./examples/table/ChangelogSocketExample.jar (table streaming)
- ./examples/streaming/StateMachineExample.jar (streaming)
When running those jobs
...if possible it would be good to get debug rather than only info logs.
Did you encounter anything odd in the TaskManager logs as well. Sharing
those might be of value as well.
On Fri, Apr 22, 2022 at 8:57 AM Matthias Pohl
wrote:
> Hi Peter,
> thanks for sharing. That doesn't sound right. May y
Hi Paul,
It's on purpose, we are building eco on Flink, it's good to share with
Flink users and the community.
Paul Lam 于2022年4月22日周五 14:57写道：
> Hi Fu,
>
> I think the mail might have been sent to Flink user mail list by mistake?
>
> Best,
> Paul Lam
>
> 2022年4月22日 11:00，Fu Chen 写道：
>
> Hi all
Alexis,
Compaction isn't an all-at-once procedure. RocksDB is organized as a series
of levels, each 10x larger than the one below. There are a few different
compaction algorithms available, and they are tunable, but what's typically
happening during compaction is that one SST file at level n is be
The DataStream API's BATCH execution mode first sorts by key, and within
each key, it sorts by timestamp. By operating this way, it only needs to
keep state for one key at a time, so this keeps the runtime simple and
efficient.
Regards,
David
P.S. I see you also asked this question on stack overf
Unsubscribe
21 matches
Mail list logo
