Apache Doris(incubating) 0.11.0 is released since September. Lots of good features and enhancement are imported. I think it's time to release 0.12.0 to make them visible to our users.
New Features ## Segment V2 [#1305] We have re-implemented the underlying data storage format, using `Page-Based` logical concepts to organize different data types or index types. We call this storage format `Segment V2`. Based on the new storage format, we have introduced a series of new features, including dictionary compression and bitmap indexing, to improve Doris's read and write performance. ## Bitmap Index On the basis of Segment V2, Doris implements the function of adding bitmap indexes to the specified columns. This feature significantly enhances Doris's query performance for any combination of multi-dimensional scenarios. [#2487] [#2050] [#2319] ## Spark on Doris With spark-doris-connector, users can now query data stored in Doris directly through Spark. Better meet users' federal query needs. [#2228] ## Support Aggregated Materialized View Framework Supports the creation of Materialized Views tables based on the Duplicate data model, which not only meets the needs of users directly using detailed table analysis, but also meets the efficient query of some specific dimension analysis. [#2101] ## Support INTERSECT and EXCEPT operators [#2834] ## Support Loading ORC format file Supports reading and loading ORC format files stored on HDFS. [#2554] ## Create partition automatically [Dynamic Partition] Support for automatically adding partitions Doris can now automatically create partitions through scheduled tasks, reducing user maintenance costs for partition operations. [#2262] ## Grouping Set SQL syntax Supports `Grouping Set` SQL syntax. Can further reduce the complexity of writing SQL for data analysts. [#2293] # Enhancement ## Doris on ES Performance Enhancement Significantly enhanced query performance for Doris on Es. Doris on ES is probably the best open source "SQL on ES" solution. [#2237] [#2575] ## Bitmap aggregate type support BIGINT The integer type supported by the Bitmap aggregation method has been extended from 32 bits to 64 bits. Further strengthen the scope of application of Doris's precise deduplication function. [#2772] ## Supports conversion of more column types Now Doris support conversion of more column types by altering table. Float -> Double Datetime <-> Date Int -> Date Varchar -> TINTINT/SMALLINT/INT/BIGINT/LARGEINT/FLOAT/DOUBLE/DATE [#2310] [#2393] [#2481] [#2489] [#2501] ## NIO-based MySQL protocol connection layer A new MySQL protocol connection layer based on NIO is implemented, which can support highly concurrent connection requests. [Experimental, enable it by setting FE config `mysql_service_nio_enabled` to true] [#2603] ## More convenient information viewing Now we support adding filter conditions for following stmts: * SHOW ALTER TABLE [#2380] * SHOW PARTITIONS [#2553] * SHOW FUNCTIONS [#2880] # New Built-in Functions * `bitmap_or` [#2649] * `bitmap_and` [#2649] * `bitamp_to_string` [#2731] * `bitmap_from_string` [#2731] * `bitmap_contains` [#2752] * `bitmap_has_any` [#2752] * `bitmap_union_count` [#2425] * `bitmap_hash` [#2439] * `intersect_count` [#2418] * `ends_with` [#2746] * `curdate` [#2521] # Other * Support aggregation type of REPLACE_IF_NOT_NULL [#2127] * Introduce GSON library to serialize the meta data in FE [#2343] * Support correlated non-scalar subquery [#2468] * Support `ALTER VIEW` stmt [#2522] * Support `replication_num` setting for table level [#2737] * SupportBroker reading config from hdfs-site.xml [#2148] * Support `date_add` function for partition pruning [#2154] * Optimize compaction strategy of tablet on BE [#2473] * Support decompressing csv file with deflate format in hdfs broker load [#2583] * Support sql mode [2083] * Modify the authorization checking logic [#2372] * Improve SkipList memory usage tracking [#2359] * Create table with nullable column for default [#2256] * Enable decimalV2 cast to different precision [#2135] ## Thirdparty * Bump Apache Maven version to 3.6.3 in Docker dev image [#2395] * Bump JMockit version to 1.48 [#2423] * Bump Apache Arrow version to 0.15.1 [#2657] [#2769] # API Change ## Default Table Storage Model Change default storage model from `AGGREGATE KEY` to `DUPLICATE KEY` [#2318] ## Forbid Cluster Related Operations Forbid cluster related operations: * create/drop cluster * add free backend/add backend to cluster/decommission cluster balance * change the backends num of cluster * link/migration db [#2429] # Credits Thanks to everyone who contributed to this release! @EmmyMiao87 @HangyuanLiu @HappenLee @IanWatsons @JustinTsui @Lishi-hub @MLikeWater @Seaven @WingsGo @Wsyzxxn @Youngwb @blackfox1983 @caiconghui @chaoyli @firetree01 @frwrdt @gaodayue @imay @infearOnTheWay @kangkaisen @kangpinghuang @landon-dai @lingbin @liutang123 @lshmouse @marising @morningman @shengyunyao @vagetablechicken @vinson0526 @wangbo @wkhappy1 @wubiaoi @wutiangan @wuyunfeng @xhg-tech @xy720 @yangzhg @yuanfeng0905 @zhouhaibing089
