Hi
I did not understand why you are using table when we are working on a
program?
On Mon, Jul 26, 2021, 7:20 AM Caizhi Weng wrote:
> Hi!
>
> For the UDF solution, you can add a "file name" column to your csv file
> like this:
> id,value,filename
> 1,100,
> 2,200,
> 3,300,test.csv
>
> Only the f
Hi!
For the UDF solution, you can add a "file name" column to your csv file
like this:
id,value,filename
1,100,
2,200,
3,300,test.csv
Only the filename of the last record of the csv file is filled, so that
this indicates the end of file.
Then write a UDF like this:
public class MyUDF extends Sc
Hi,
Let me know if you have any idea as this is very critical for my project.
Thanks & Regards,
Samir Vasani
On Fri, Jul 23, 2021 at 1:26 PM Samir Vasani wrote:
> Hi,
>
> Can you elaborate more on UDF as I did not understand it.
>
> Thanks & Regards,
> Samir Vasani
>
>
>
> On Fri, Jul 23, 202
Hi,
Can you elaborate more on UDF as I did not understand it.
Thanks & Regards,
Samir Vasani
On Fri, Jul 23, 2021 at 1:22 PM Caizhi Weng wrote:
> Hi!
>
> In this case it won't work, as JobListener#onJobExecuted will only be
> called when the job finishes, successfully or unsuccessfully.
>
>
Hi!
In this case it won't work, as JobListener#onJobExecuted will only be
called when the job finishes, successfully or unsuccessfully.
For a forever-running job I would suggest adding a UDF right after the
source and adding a special "EOF" record in each of the csv file. This UDF
monitors the da
Hi Caizhi Weng,
Thanks for your input.
I would explain the requirement in little more detail.
Flink pipeline will be running forever (until some issue happens and we
would need to restart) so It will continuously monitor if a new file comes
to the *input *folder or not.
In this case will your sugg
Hi!
JobListener#onJobExecuted might help, if your job is not a forever-running
streaming job. See
https://ci.apache.org/projects/flink/flink-docs-master/api/java/org/apache/flink/core/execution/JobListener.html
Samir Vasani 于2021年7月23日周五 下午3:22写道：
> Hi,
>
> I am a new bee to flink and facing so
Hi,
I am a new bee to flink and facing some challenges to solve below use case
Use Case description:
I will receive a csv file with a timestamp on every single day in some
folder say *input*.The file format would be
*file_name_dd-mm-yy-hh-mm-ss.csv*.
Now my flink pipeline will read this csv fil
