Re: HBase Connectors(sink)

Hi Chesnay, I saw the code about hbase here: https://github.com/apache/flink/tree/master/flink-connectors/flink-hbase/src/main/java/org/apache/flink/addons/hbase, it seems including both input and output. On Sep 6, 2019, at 17:45, Chesnay Schepler mailto:ches...@apache.org>> wrote: Where d

Re: Checkpointing is not performing well

Ravi, have you looked at the io operation(iops) rate of the disk? You can monitoring the iops performance and tune it accordingly with your work load. This helped us in our project when we hit the wall tuning prototype much all the parameters. Rohan From: Ravi

Re: Flink SQL client support for running in Flink cluster

Hi Dipanjan, Sorry that I didn’t make it clear. The ’standalone cluster’ could be fully distributed. It refers to a static cluster deployed without any cluster managers (or other related tools), such as YARN and Kubernetes. For more information, please check the documentation [1]. Best, Xingca

Re: Checkpointing is not performing well

Hi Rafi, Thank you for your quick response. I have tested with rocksdb state backend. Rocksdb required significantly more taskmanager to perform as compare to filesystem state backend. The problem here is that checkpoint process is not fast enough to complete. Our requirement is to do checkout a

Re: Checkpointing is not performing well

Hi Ravi, Consider moving to RocksDB state backend, where you can enable incremental checkpointing. This will make you checkpoints size stay pretty much constant even when your state becomes larger. https://ci.apache.org/projects/flink/flink-docs-release-1.9/ops/state/state_backends.html#the-rocks

Checkpointing is not performing well

Hi All, I am writing a streaming application using Flink 1.9. This application consumes data from kinesis stream which is basically avro payload. Application is using KeyedProcessFunction to execute business logic on the basis of correlation id using event time characteristics with below configura

Re: Flink SQL client support for running in Flink cluster

Hi Xingcan, Thanks a lot for this info and this verifies my suspect , so if i want to reuse the SQL client to implement an API based framework , it will not work in Flink cluster running in  cluster mode, is there any plan in near future to support that, if its planned is there any expected

Re: Flink SQL client support for running in Flink cluster

Hi Dipanjan, Here the ’standalone’ is short for ’standalone cluster mode’, which means you can setup a standalone Flink cluster and submit your SQL queries (via SQL client) to the cluster. As far as I know, the current SQL client can only connect to a Flink cluster deployed in standalone mode.