Flink SQL suport tfrecord format

2022-09-20 Thread
hi,when I write a sql like this: String sqlCreate = "CREATE TABLE fs_table (\n" + " `examplestr` bytes\n" + ") WITH (\n" + " 'connector'='filesystem',\n" + " 'format'='raw',\n" + " 'path'='/tmp/zhangyin

Table program cannot be compiled. This is a bug. Please file an issue

2021-09-13 Thread
I write a long sql,but when I explain my plan,it make a mistake: org.apache.flink.util.FlinkRuntimeException: org.apache.flink.api.common.InvalidProgramException: Table program cannot be compiled. This is a bug. Please file an issue. at org.apache.flink.table.runtime.generated.CompileUtils.com

Re: [ANNOUNCE] RocksDB Version Upgrade and Performance

2021-08-06 Thread
I am a heavy user of rocksdb, and my application works well on flink-1.10,but I am interst in the new features.I want to know if I can increase the manage memory to eliminate the impact of the performance regression In a certain degree。 Can you show me the params about rocksdb of your task?

when scalarfunction return treemap,the next operator is out of order

2021-07-29 Thread
I extends scalarfunction like this,and I invoke tableEnv.createTemporarySystemFunction("tree_map", new TreeMapFunction()); public class TreeMapFunction extends ScalarFunction { private static Logger profileLog = Logger.getLogger("zhangying480"); @Override public boolean isDeterministic() { return

Read Hive table in Stream Mode use distinct cause heap OOM

2021-04-25 Thread
hi,I met an appearance like this: this is my sql: SELECT distinct header,label,reqsig,dense_feat,item_id_feat,user_id_feat FROM app.app_ranking_feature_table_clk_ord_hp_new_all_tree_orc where dt='2021-04-01' When I useBlinkPlanner inBatchMode, It works well; But if I set inStreamMode, It cau

Does it support rate-limiting in flink 1.12?

2021-04-12 Thread
When I run a sql job with blink planner in my cluster,the task is almost preemption the whole resources in the cluster, and this is a bad effect to the stream task.As it is not necessary on speed,so is there any way to control the rate in my batch task? this is the machine performance in ru