Re: Parquet reading/writing

Thank you, looks like shuffle() works -- Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

Re: Parquet reading/writing

Hi Fabian, Thank you for your answer. I've updated the flink version to 1.12.4 but unfortunately the problem still persists. I'm running this job in local mode, so I have only following log: WARNING: An illegal reflective access operation has occurred WARNING: Illegal reflective access by org.a

Re: Parquet reading/writing

Update: The job is working correctly if add an additional identity mapping step: env.createInput(parquetInputFormat) .map(record => record) .sinkTo(FileSink.forBulkFormat...) -- Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

Parquet reading/writing

Hi, I'm trying to read parquet file with Flink 1.12.0 Scala API and save it as another parquet file. Now it's working correctly with ParquetRowInputFormat: val inputPath: String = ... val messageType: MessageType = ... val parquetInputFormat = new ParquetRowInputFormat(new Path(inputPath), mess

Re: Flink reads data from JDBC table only on startup

Hi Danny, I use regular join and it looks like: SELECT ... FROM *dynamic_kafka_table* k JOIN *jdbc_table* j ON k.id = j.k_id Should I set some additional conditions for this join? -- Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

Flink reads data from JDBC table only on startup

Hi everyone! I'm using Flink 1.12.0 with SQL API. I'm developing a streaming job with join and insertion into postgreSQL. There is two tables in join: 1. Dynamic table based on kafka topic 2. Small lookup JDBC table >From what I can see Flink job reads data from JDBC table only on startup and mar

Re: Flink SQL continuous join checkpointing

Hi Leonard, Thank you for answer, in fact I used regular join because my interval condition was based on wrong column. I extended my join with attribute column condition and it solved the problem: ... FROM table_fx fx LEFT JOIN table_v v ON v.active = fx.instrument_active_id

Flink SQL continuous join checkpointing

Hi everyone! I'm using Flink *1.12.0* with SQL API. My streaming job is basically a join of two dynamic tables (from kafka topics) and insertion the result into PostgreSQL table. I have enabled watermarking based on kafka topic timestamp column for each table in join: CREATE TABLE table1 (