Hi all,
we just started using the flink k8s operator to deploy our flink cluster.
>From what we understand we are only able to start a flink cluster per job.
So in our case when we have 2 jobs we have to create 2 different clusters.
obviously we would prefer to deploy these 2 job which relate to th
Hi Qing,
Thanks for the input. I think having a stateful function to accumulate the tree
after source is a reasonable solution to me. Under your design a split is
mapping to a znode so the state persisted in the source reader would be
per-node information, and it’s hard to accumulate them unde
Hi, Tom.
I don't reproduce the exception in the master. I am not sure whether the
problem is fixed or I missing something.
The only difference is my test udf extends ScalarFunction rather than
DPScalarFunction and I use String[] as the input type.
```
public static class ListToString extends Sc
Tanks Tymur, for the pointers.
I followed the GitHub link and got the understanding on how to define and
configure Remote Functions with Datastream Api
However, I need help in understanding the following:
1. I didn't find Stateful function definition and server code.
2. How should we deploy St
Hi all,
Thank you for the help.
It seems an exception thrown when Flink try to deserialize the object
> outputed by your udf. So is the obejct produced by your udf serializable?
> Does it contain any lambda function in the object/class?
The output object of the UDF is the string "foo" which sho
Hi,
Thank you for your help. Here's the requested info:
Could you also tell us which Flink version you are using, the schema of the
> source table and some test data? With these info, we can debug in our local
> environment.
>
Flink version: 1.11.6.
Schema of the source table:
name | type | null
Hi Zain,
You can view the list of flink applications on the yarn web ui and choose
to jump to the specified Flink web ui.
Best,
Weihua
On Mon, May 23, 2022 at 7:07 PM Zain Haider Nemati
wrote:
> Hi David,
> Thanks for your response.
> When submitting a job in application mode it gives a url at
Hi,
I am running a TensorFlow Extended (TFX) pipeline which uses Apache Beam
for data processing which in turn has a Flink Runner (Basically a batch job
on a Flink Session Cluster on Kubernetes) version 1.13.6, but the job (for
gathering stats) gets stuck.
There is nothing significant in the Job
Thanks Qingsheng ! This answers my doubt.
Regards,
Meghajit
On Tue, May 31, 2022 at 3:03 PM Qingsheng Ren wrote:
> Hi Meghajit,
>
> Good question! To make a short answer: splits won’t be returned back to
> enumerator by reader once they are assigned and *checkpointed*.
>
> As described by the J
Hi,
Afaik, there are still a lot of unit tests depending on it. I don't think
we can drop it before dropping all of these unit tests.
Best regards,
Jing
On Tue, May 31, 2022 at 8:10 AM Yun Gao wrote:
> Hi Jun,
>
> I think the release notes should only include the issues that cause changes
> vi
Hi Meghajit,
Good question! To make a short answer: splits won’t be returned back to
enumerator by reader once they are assigned and *checkpointed*.
As described by the JavaDoc of SplitEnumerator#addSplitsBack [1]:
> Add a split back to the split enumerator. It will only happen when a
> Sourc
Hello,
I had a question with regards to the behaviour of FileSource and
SourceReader in cases of failures. Let me know if I missed something
conceptually.
We are running a Parquet File Source. Let's say, we supply the source with
a directory path containing 5 files and the Flink job is configured
Hi Qingsheng, thanks for getting back.
I manage to find a workaround, but if you can provide other suggestions it'd be
great too.
I followed the documentation here:
https://nightlies.apache.org/flink/flink-docs-master/docs/dev/datastream/sources/
I implemented a Custom Source that emits all ch
13 matches
Mail list logo
