Hi folks!
I have a couple of questions on Flink's behaviour when writing to more than
one sink and overall things to look out when operating 500+ sinks. I am
starting my Flink journey so want to get inputs to two questions from the
community.
Challenge: Need to route incoming events to 500 differ
No, there is no need after the job has been submitted. It's only that the
web ui based submission is a two step process where you 1) upload the jar
and 2) submit it. If you should access between 1) and 2) a different rest
server, then the new rest server won't know about the uploaded jar.
Cheers,
The query which I'm testing now(trying to avoid the deduplication query
because of tombstones) is *almost* correct but there are two questions
which I can find an answer to:
1. Some of the *id*'s are just stopping to be produced.
2. Does the Tuble window select only the records whose upd_ts is new
Hi,
yes, we can confirm that your program has the behavior you mentioned.
Since we don't use any type of time operation or windowing, your query
has updating semantics. State is used for keeping the LAST_VALUEs as
well as the full input tables of the JOIN.
You can achieve the same with a Key
Improvements to the documentation are always welcome.
In this particular case we actually need to be really careful, as it is
not always the expected behavior. As you are registering your own kryo
serializer it is expected in your case.
However more often the case is, you don't want to use the Ge
Hi Jiazhi,
Could you elaborate what exactly do you want to achieve? What have you
tried so far?
Best,
Dawid
On 15/02/2021 11:11, ?g???U?[ wrote:
> Hi all
> ?0?2 ?0?2 ?0?2 ?0?2Using DataStream, How to implement a message and the same
> message appears again 10 minutes later?
> Thanks,
> Jiaz
Thanks Till, that sounds fantastic.
Is there any need for all Job Managers to see the jar after a job is running?
I plan to sync the leader address from the config map and might always end up
at the leader.
ThanksChirag
On Monday, 15 February, 2021, 03:16:50 pm IST, Till Rohrmann
wrote:
Hi all
Using DataStream, How to implement a message and the
same message appears again 10 minutes later?
Thanks,
Jiazhi
Hey Yaroslav,
Unfortunately I don't have enough knowledge to give you an educated
reply. The first part certainly does make sense to me, but I am not sure
how to mitigate the issue. I am ccing Yun Tang who worked more on the
RocksDB state backend (It might take him a while to answer though, as he
Hi Chirag,
when starting standby JobManagers, then Flink will already start a web
server for each process for serving REST requests. These servers will,
however, not necessarily ask the JobManager they have been started with but
always forward requests to the current leading JobManager. That way a
Hey,
Why do you say the way you did it, does not work? The logs you posted
say the classes cannot be handled by Flink's built-in mechanism for
serializing POJOs and it falls back to a GenericType which is serialized
with Kryo and should go through your registered serializer.
Best,
Dawid
On 14/
Thank you guys for the interest, feedback and advies,
Just to clarify further on the why we used tables with grouping,
Form each DataStream we only interested in the last updated or new Event,
Also, we need to have ALL the previous Events stored in order to identify
if the incoming event is a new
Hi,
We configured Job Manager HA with Kubernetes strategy and found that the Web UI
for all 3 Job Managers is accessible on their configured rpc addresses. There's
no information on the Web UI that suggests which Job Manager is the leader or
task managers are registered to. However, from the log
I'm adding the Flink user ML to the conversation again.
On Mon, Feb 15, 2021 at 8:18 AM Matthias Pohl
wrote:
> Hi Omer,
> thanks for sharing the configuration. You're right: Using NFS for HA's
> storageDir is fine.
>
> About the error message you're referring to: I haven't worked with the HA
> k
14 matches
Mail list logo
