Hi Jing Ge: this is the example: table ddl : CREATE TABLE dim_db.dim_table ( column_one string, column_two string, column_three string, primary key(column_one) NOT ENFORCED ) WITH ( 'table-name' = 'xxx', 'connector' = 'hbase-1.4', 'zookeeper.quorum' = '127.0.0.1:2181' );
CREATE TABLE rtdw_db.kafka_table ( column_two string, column_three string, `PROCTIME` AS `PROCTIME`() ) WITH ( 'topic' = 'xxxx', 'properties.group.id' = 'group_id', 'connector' = 'kafka', 'format' = 'json', 'scan.startup.mode' = 'group-offsets', 'properties.bootstrap.servers' = '127.0.0.1:9092' ); flink sql : select * from rtdw_db.kafka_table kt left join dim_db.dim_table FOR SYSTEM_TIME AS OF kt.PROCTIME AS dt on kt.column_two = dt.column_two ; I want to implement such this logic which using flink sql . --- Best, WuKong From: Jing Ge Date: 2022-03-19 01:37 To: WuKong CC: user@flink.apache.org Subject: Re: Flink SQL 1.12 How to implement query Hbase table on secondary index HI WuKong, Afaiu, technically, you are not using Hbase secondary index(coprocessor). What you are trying to do is to store the synced dim table in elasticsearch and query from there to get the rowkeys and then use the rowkeys to get dim table rows from Hbase. In this way, a (full) table scan in Hbase will be avoided. Please correct me if I am wrong. Would you like to share more information about your requirements in detail? like DDLs of Kafka fact table, Hbase and ES dim table and the join logic you want to achieve. So we could give you more feasible suggestions. Thanks. Best regards, Jing On Thu, Mar 17, 2022 at 3:24 AM WuKong <wukon...@foxmail.com> wrote: Hi, now my data store hbase and I want use flink to implement kafka table temproal join hbase table , but condtion is not rowkey , I realize hbase secondary index, how can I implement this function what can use flink sql first query secondary index(such as es) and then use rowkey query hbase table (must implement own hbase secondary connectors ?) --- Best, WuKong