Hi Mike, please do not forget to include the mailing list address/reply to all, otherwise, this becomes a private conversation. How do you start your cluster?
Best, Alex On Fri, Jul 8, 2022 at 10:45 AM <pod...@gmx.com> wrote: > Thanks for support. > As you suggested, I did run it in IDE: > > WARNING: An illegal reflective access operation has occurred > WARNING: Illegal reflective access by > org.apache.flink.api.java.ClosureCleaner > (file:/C:/Flink/flink-1.15.0/lib/flink-dist-1.15.0.jar) to field > java.lang.String.value > WARNING: Please consider reporting this to the maintainers of > org.apache.flink.api.java.ClosureCleaner > WARNING: Use --illegal-access=warn to enable warnings of further illegal > reflective access operations > WARNING: All illegal access operations will be denied in a future release > +----------------------+ > | some_value | > +----------------------+ > | 3 | > +----------------------+ > 1 row in set > > So it works (path is 'file:///c:/Temp/test4.txt' - when path was wrong I > got error... unfortunately Flink do not display any error in such case) > > You say 'Check in the UI that after you start your cluster you have > TaskManagers registered successfully'. > If I go to 'Task Managers' managers menu ( > http://localhost:8081/#/task-manager) I do not see any - list is empty. > > No idea what it should be there or how to make one. > > *Sent:* Thursday, July 07, 2022 at 4:27 PM > *From:* "Alexander Fedulov" <alexan...@ververica.com> > *To:* pod...@gmx.com > *Subject:* Re: Is Flink able to read a CSV file or just like in Blink > this function does not work? > Did you try running in an IDE? > Check in the UI that after you start your cluster you have TaskManagers > registered successfully and they have enough slots to fulfill your job > parallelism requirements. > > Best, > Alexander Fedulov > > On Thu, Jul 7, 2022 at 2:53 PM <pod...@gmx.com> wrote: > >> >> Thank you very much Alex for your help. >> My environment is Windows and Github CLI. >> >> I tried different directory syntax: >> 'file:///c:/temp/test4.txt' >> 'file:///c://temp//test4.txt' >> '//c:/temp/test4.txt' >> 'c:/temp/test4.txt' >> 'c:\\temp\\test4.txt' >> ... >> dozens of others >> Nothing helps >> >> I run this job with command: >> $ $FLINK_HOME/bin/flink run c:/Temp/flinkTest2.jar >> >> $ $FLINK_HOME/bin/flink run c:/Temp/flinkTest2.jar >> WARNING: An illegal reflective access operation has occurred >> WARNING: Illegal reflective access by >> org.apache.flink.api.java.ClosureCleaner >> (file:/C:/Flink/flink-1.15.0/lib/flink-dist-1.15.0.jar) to field >> java.lang.String.value >> WARNING: Please consider reporting this to the maintainers of >> org.apache.flink.api.java.ClosureCleaner >> WARNING: Use --illegal-access=warn to enable warnings of further illegal >> reflective access operations >> WARNING: All illegal access operations will be denied in a future release >> Job has been submitted with JobID a9eaf86337ca14d0d80e6d6ef70dab6e >> >> >> >> But maybe problem is in another place - even if i put non existing path - >> there is no error; job is subited to Flink, has status RUNNING and it fails >> after 5 minuts or so. >> >> Source: Custom File source: FAILED >> CsvTableSource(read fields: ) -> SourceConversion[1] -> >> HashAggregate[2]: CANCELED >> HashAggregate[4] -> ConstraintEnforcer[5] -> Sink: Collect table sink: >> CANCELED >> >> In console I see: >> NoResourceAvailableException: Slot request bulk is not fulfillable! Could >> not allocate the required slot within slot request timeout >> >> >> >> *Sent:* Thursday, July 07, 2022 at 1:10 AM >> *From:* "Alexander Fedulov" <alexan...@ververica.com> >> *To:* pod...@gmx.com >> *Cc:* "user" <user@flink.apache.org> >> *Subject:* Re: Is Flink able to read a CSV file or just like in Blink >> this function does not work? >> Hi Mike, >> >> I do not see any issues with your code. With a sample csv file like this >> a,1.0 >> b,2.0 >> c,3.0 >> >> it produces the expected result >> >> +----------------------+ >> | some_value | >> +----------------------+ >> | 3 | >> +----------------------+ >> 1 row in set >> >> Process finished with exit code 0 >> >> -- >> >> I do not use Windows, so I am not 100% sure what is the correct way to >> specify the file path there, however, some observations: >> - You are using a forward slash notation, I assume you are running in WSL >> or using cygwin or something like that? Notice that in that notation an >> additional forward slash is missing 'file:///foo/bar' >> - Omitting the schema works with the local filesystem by default. On *nix >> systems this works fine 'connector.path' = '/home/alex/tmp/test.csv' . >> Maybe you could just try a standard windows backward slash notation instead. >> - That said, when a non-existent file is specified, the job fails >> immediately, so I would actually expect that behavior if the issue was >> indeed with the file path. >> Which version of Flink are you running? >> >> Best, >> Alexander Fedulov >> >> >> On Wed, Jul 6, 2022 at 10:39 PM <pod...@gmx.com> wrote: >> >>> If I'm reading Flink manul correctly (and this is not simple - no >>> examples), this code should read CSV file: >>> >>> >>> *package flinkTest2;* >>> >>> >>> *import org.apache.flink.table.api.EnvironmentSettings; import >>> org.apache.flink.table.api.TableEnvironment;* >>> >>> *public class flinkTest2 {* >>> >>> * public static void main(String[] args) throws Exception {* >>> >>> >>> >>> >>> >>> * EnvironmentSettings settings = EnvironmentSettings >>> .newInstance() //.inStreamingMode() .inBatchMode() >>> .build();* >>> * final TableEnvironment tEnv = TableEnvironment.create(settings);* >>> >>> >>> >>> >>> >>> * tEnv.executeSql("CREATE TABLE test_table (column_name1 STRING, >>> column_name2 DOUBLE) WITH ('connector.type' = 'filesystem', >>> 'connector.path' = 'file://C/temp/test4.txt', 'format.type' = 'csv')"); >>> tEnv.sqlQuery("SELECT COUNT(*) AS some_value FROM test_table") >>> .execute() .print();* >>> >>> * } }* >>> >>> Job is running and running and running... >>> I think Flink is not able to open file. Does not matter what is here: >>> 'connector.path' = 'file://C/temp/test4.txt' Flink does not display any >>> error or something. 'connector.path' = 'file:blah/blah/blah' could be fine >>> for flink as well. >>> >>> Anyone could help me with that? >>> Thanks >>> Mike >>> >>> >>
