I'm trying to run a Flink job as a standalone program and getting the
following error.
Caused by: org.apache.flink.table.api.ValidationException: Could not find
any factory for identifier 'filesystem' that implements
'org.apache.flink.table.factories.DynamicTableFactory' in the classpath.
Availab
I have a table that reads a Kafka topic and effective parallelism is equal
to the number of Kafka partitions. Is there a way to reshuffle the data
like with DataStream API to increase effective parallelism?
I can't manage to run an application on YARN because of classpath issues.
Flink distribution is unpacked in $HOME/flink-1.14.4
$HOME/flink1.14.4/usrlib contains all the dependency jars excluding the
main application jar as flat file structure.
The application is started with
./bin/flink run-applic
27;24' HOURS))
> )
> WHERE rownum = 1
>
> On Thu, Nov 4, 2021 at 11:18 AM Pavel Penkov
> wrote:
>
>> I'm trying to express a supposedly simple query with Flink SQL - log the
>> first visit a day for each user. Source table is defined like
>>
>
I'm trying to express a supposedly simple query with Flink SQL - log the
first visit a day for each user. Source table is defined like
CREATE TABLE visits (user_id int, ts timestamp(3), WATERMARK FOR ts AS ts)
WITH ('connector' = 'filesystem',
'path' = 'file:///visits.csv',
'format' = 'csv')
The
>
>
> From my understanding of the doc[1], you need to set it in>
> flink-conf.yaml instead of your job.>
>
> [1]
https://ci.apache.org/projects/flink/flink-docs-master/docs/deployment/filesystems/s3/#hadooppresto-s3-file-systems-plugins>
>
> Best,>
> Yangze Guo>
Apparently Flink 1.14.0 doesn't correctly translate S3 options when they
are set programmatically. I'm creating a local environment like this to
connect to local MinIO instance:
val flinkConf = new Configuration()
flinkConf.setString("s3.endpoint", "http://127.0.0.1:9000";)
flinkConf.setStri
