Re: Unable to read a (text)file in FileProcessing.PROCESS_CONTINUOS mode

2021-09-06 Thread David Morávek
Hi Samir, I'd second Roman here. Can you please verify that the directory you're trying to read from is accessible in all of your docker containers (TMs / JMs)? Ideally if you could list the directory inside the docker running container and post the output in this thread. Also please check that t

Re: Unable to read a (text)file in FileProcessing.PROCESS_CONTINUOS mode

2021-08-30 Thread Samir Vasani
HI, Accessing file is not the problem. If i put the file before starting the job then this job reads it correctly but with if i add any file at runtime then it does not read this newly added files. Let me know if you need more information. Thanks & Regards, Samir Vasani On Mon, Aug 30, 2021 at

Re: Unable to read a (text)file in FileProcessing.PROCESS_CONTINUOS mode

2021-08-30 Thread Roman Khachatryan
Hi, If I understand correctly, the problem is accessing local files from Flink running in docker. Have you tried mounting the local directory into the container, for example as a bind mount [1]? [1] https://docs.docker.com/storage/bind-mounts/ Regards, Roman On Mon, Aug 30, 2021 at 3:33 PM Sami

Unable to read a (text)file in FileProcessing.PROCESS_CONTINUOS mode

2021-08-30 Thread Samir Vasani
I have a requirement to read a file continously from a specific path. Means flink job should continously poll the specified location and read a file will arrive at this location at certains intervals . Example: my location on windows machine is C:/inputfiles get a file file_1.txt at 2:00PM, file_