Carl created FLINK-23719:
----------------------------
Summary: Support switch WAL in Flink SQL DDL options
Key: FLINK-23719
URL: https://issues.apache.org/jira/browse/FLINK-23719
Project: Flink
Issue Type: New Feature
Components: Connectors / HBase
Affects Versions: 1.13.2
Reporter: Carl
My application scenario is like this:
Use flick SQL to write the batch data of hive table to a new HBase table every
day.
In this case, I think it is appropriate to disable WAL for the following
reasons:
1. If an error occurs, i can re-run it. No high security is required.
2. The write speed will be greatly improved. No additional writing to the wal
disk log
3. Save disk IO resources of the cluster
However, the HBase official website is described below,
h4. 70.6.10. Disabling the WAL
It is possible to disable the WAL, to improve performance in certain specific
situations. However, disabling the WAL puts your data at risk. The only
situation where this is recommended is during a bulk load. This is because, in
the event of a problem, the bulk load can be re-run with no risk of data loss.
The WAL is disabled by calling the HBase client field
{{Mutation.writeToWAL(false)}}. Use the
{{Mutation.setDurability(Durability.SKIP_WAL)}} and Mutation.getDurability()
methods to set and get the field’s value. There is no way to disable the WAL
for only a specific table.
[http://hbase.apache.org/2.3/book.html#hbase_default_configurations]
So i think it needs API level support, not properties support.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
