spaces-X opened a new issue #7179: URL: https://github.com/apache/incubator-doris/issues/7179
### Search before asking - [X] I had searched in the [issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and found no similar issues. ### Version version 0.14 or older ### What's Wrong? The config `default_max_filter_ratio` does not work in the INSERT statement. If not set `ENABLE_INSERT_STRICT = true`, most of the data may be filtered without notice, which does not match the expectation of `default_max_filter_ratio` ### What You Expected? Insert load should have the same behavior with stream load. If `default_max_filter_ratio` is set, the number of filtered rows should not be more than `default_max_filter_ratio` * `total_rows` in the insert load. ### How to Reproduce? DDL: 1. ``` CREATE TABLE `unique_null_bitmap` ( `dt` int(11) NULL COMMENT "日期分区", `city_id_bitmap` bitmap BITMAP_UNION COMMENT "city_id_bitmap" ) ENGINE=OLAP AGGREGATE KEY(`dt`) COMMENT "交易轻度聚合表" PARTITION BY RANGE(`dt`) (PARTITION p20211012 VALUES [("20211011"), ("20211012")), PARTITION p20211109 VALUES [("20211108"), ("20211109"))) DISTRIBUTED BY HASH(`dt`) BUCKETS 1 PROPERTIES ( "replication_num" = "1", "in_memory" = "false", "storage_format" = "V2" ); ``` 2. ``` CREATE TABLE `agg_bug_test` ( `dt` bigint(20) NULL COMMENT "", `id1` bigint(20) NULL COMMENT "", `id2` bigint(20) NULL COMMENT "", `id3` bigint(20) NULL COMMENT "", `first_time` varchar(20) MIN NULL COMMENT "", `last_time` varchar(20) MAX NULL COMMENT "" ) ENGINE=OLAP AGGREGATE KEY(`dt`, `id1`, `id2`, `id3`) COMMENT "OLAP" DISTRIBUTED BY HASH(`id1`, `id2`, `id3`) BUCKETS 10 PROPERTIES ( "replication_num" = "1", "in_memory" = "false", "storage_format" = "V2" ); ``` ``` insert into agg_bug_test values (20211108,10,10,10,null,null); insert into agg_bug_test values (20210111,10,10,10,null,null); ``` Insert SQL: 1 row inserted and 8 rows filtered without notice. Insert transaction should be aborted when `default_max_filter_ratio` is set. ``` insert into unique_null_bitmap (dt,city_id_bitmap) ( (select dt, bitmap_empty() as city_id_bitmap from agg_bug_test where dt = 20211108 limit 1) union all (select dt, null as city_id_bitmap from agg_bug_test where dt = 20211011 limit 1) union all (select dt, null as city_id_bitmap from agg_bug_test where dt = 20211011 limit 1) union all (select dt, null as city_id_bitmap from agg_bug_test where dt = 20211011 limit 1) union all (select dt, null as city_id_bitmap from agg_bug_test where dt = 20211011 limit 1) union all (select dt, null as city_id_bitmap from agg_bug_test where dt = 20211011 limit 1) union all (select dt, null as city_id_bitmap from agg_bug_test where dt = 20211011 limit 1) union all (select dt, null as city_id_bitmap from agg_bug_test where dt = 20211011 limit 1) union all (select dt, null as city_id_bitmap from agg_bug_test where dt = 20211011 limit 1) ); ``` Query: only returned one row with dt = 20211108 ``` select * from unique_null_bitmap ``` ### Anything Else? _No response_ ### Are you willing to submit PR? - [X] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
