Can flink1.15.2 use flink sql to create a broadcast table? I didn't find anything related in https://flink.apache.org/

2023-05-06 Thread yxy
Hello, we have a business scenario. We have a real-time process to calculate how much red envelopes should be given to them for each transaction. For example, if a customer pays $100, we will give him a rebate of one thousandth. We currently use flinksql to Realize this function, but we found

Re: Elastic scaling with HA (zookeeper)

2023-05-06 Thread Shubham Bansal
Thanks for the response Weihua. I am using flink v.15.2. I am unable to understand how I should be starting the jobmanager without the job so that it can poll HA store and start the job. Since the current method of running the script standalone-job.sh requires a job at start. If I provide the same

Re: Can flink1.15.2 use flink sql to create a broadcast table? I didn't find anything related in https://flink.apache.org/

2023-05-06 Thread Hang Ruan
Hi, yxy, I think this scenario could be resolved by a lookup join or a UDF. We can store the ratio in the mysql table. Then we could read it by a lookup join or implement a UDF to read the ratio. Best, Hang yxy 于2023年5月6日周六 15:14写道: > Hello, we have a business scenario. We have a real-time pr

Re: Elastic scaling with HA (zookeeper)

2023-05-06 Thread Weihua Hu
Hi, If I provide the same jobId, will it restore from the HA first, or just > start anew? it will restore from HA first, and will not start a new one. I tried with flink v1.15.2 with the flowing example, but can't reproduce. Could you provide the full start command? 1. ./bin/standalone-job.sh

Re: Can flink1.15.2 use flink sql to create a broadcast table? Ididn't find anything related in https://flink.apache.org/

2023-05-06 Thread Hang Ruan
Hi, yangxueyong, If the ratio will not be modified frequently, we could set a long TTL for the cache in the lookup table to reduce the frequency of reading, or put a cache in the UDF. If you need the exact ratio when the data arrives, we have to read it from the remote storage each time we get new

Flink SQL Async UDF

2023-05-06 Thread Giannis Polyzos
I can't really find any examples / docs for Flink's AsyncTableFunction and I have a hard time getting it to work. Is there any example you can share that just takes as input a String key and outputs lets say a record (returned by the lookup?) Also is not clear to me how the async happens internally