This is an automated email from the ASF dual-hosted git repository.
luzhijing pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris-website.git
The following commit(s) were added to refs/heads/master by this push:
new c3290b50e3e Update release note 2.1.4 (#795)
c3290b50e3e is described below
commit c3290b50e3e9373149721ad42901e4dff1fc185e
Author: lishiqi_amy <amylee9...@163.com>
AuthorDate: Fri Jun 28 10:19:34 2024 +0800
Update release note 2.1.4 (#795)
---
docs/releasenotes/release-2.1.4.md | 74 +++++++++++---------
.../current/releasenotes/release-2.1.4.md | 80 +++++++++++++---------
.../version-2.1/releasenotes/release-2.1.4.md | 80 +++++++++++++---------
.../version-2.1/releasenotes/release-2.1.4.md | 74 +++++++++++---------
4 files changed, 174 insertions(+), 134 deletions(-)
diff --git a/docs/releasenotes/release-2.1.4.md
b/docs/releasenotes/release-2.1.4.md
index bff23d629b5..d04a8be0cf6 100644
--- a/docs/releasenotes/release-2.1.4.md
+++ b/docs/releasenotes/release-2.1.4.md
@@ -24,7 +24,7 @@ specific language governing permissions and limitations
under the License.
-->
-Apache Doris version 2.1.4 was officially released on June 26, 2024. In this
update, we have optimized various functional experiences for data lakehouse
scenarios, with a focus on resolving the abnormal memory usage issue in the
previous version. Additionally, we have implemented several improvemnents and
bug fixes to enhance the stability. Welcome to download and use it.
+**Apache Doris version 2.1.4 was officially released on June 26, 2024.** In
this update, we have optimized various functional experiences for data
lakehouse scenarios, with a focus on resolving the abnormal memory usage issue
in the previous version. Additionally, we have implemented several
improvemnents and bug fixes to enhance the stability. Welcome to download and
use it.
**Quick Download:** https://doris.apache.org/download/
@@ -32,7 +32,7 @@ Apache Doris version 2.1.4 was officially released on June
26, 2024. In this upd
**GitHub Release:** https://github.com/apache/doris/releases
-## Behavior Changed
+## Behavior changes
- Non-existent files will be ignored when querying external tables such as
Hive. [#35319](https://github.com/apache/doris/pull/35319)
@@ -48,21 +48,21 @@ Apache Doris version 2.1.4 was officially released on June
26, 2024. In this upd
For more infomation, please see the documentations:
- - [Log Management - FE
Log](https://doris.apache.org/docs/admin-manual/log-management/fe-log)
+ - [Log Management - FE Log](../admin-manual/log-management/fe-log.md)
- - [Log Management - FE
Log](https://doris.apache.org/docs/admin-manual/log-management/be-log)
+ - [Log Management - FE Log](../admin-manual/log-management/be-log.md)
- If no table comment is provided when creating a table, the default comment
will be empty instead of using the table type as the default comment.
[#36025](https://github.com/apache/doris/pull/36025)
- The default precision of DECIMALV3 has been adjusted from (9, 0) to (38, 9)
to maintain compatibility with the version in which this feature was initially
released. [#36316](https://github.com/apache/doris/pull/36316)
-## New Features
+## New features
-### 01 Query Optimizer
+### Query optimizer
- Support FE flame graph tool
- For more information, see the
[documentation](https://doris.apache.org/community/developer-guide/fe-profiler)
+ For more information, see the
[documentation](/community/developer-guide/fe-profiler.md)
- Support `SELECT DISTINCT` to be used with aggregation.
@@ -70,7 +70,7 @@ Apache Doris version 2.1.4 was officially released on June
26, 2024. In this upd
- The new optimizer fully supports point query functionality
[#36205](https://github.com/apache/doris/pull/36205).
-### 02 Lakehouse
+### Lakehouse
- Support native reader of Apache Paimon deletion vector
[#35241](https://github.com/apache/doris/pull/35241)
@@ -78,33 +78,29 @@ Apache Doris version 2.1.4 was officially released on June
26, 2024. In this upd
- Access controller with Hive Ranger plugin supports Data Mask
-### 03 Asynchronous Materialized Views
+### Asynchronous materialized views
-- Support partition roll-up during construction.
[#31812](https://github.com/apache/doris/pull/31812)
+- Build support for internal table triggered updates, where if a materialized
view uses an internal table and the data in the internal table changes, it can
trigger a refresh of the materialized view, specifying REFRESH ON COMMIT when
creating the materialized view.
-- Support triggered updates during construction.
[#34548](https://github.com/apache/doris/pull/34548)
+- Support transparent rewriting for single tables. For more information, see
[Querying Async Materialized
View](../query/view-materialized-view/query-async-materialized-view.md).
-- Support specifying the `store_row_column` and `storage_medium` attribute
during construction. [#35860](https://github.com/apache/doris/pull/35860)
+- Transparent rewriting supports aggregation roll-up for agg_state, agg_union
types; materialized views can be defined as agg_state or agg_union, queries can
use specific aggregation functions, or use agg_merge. For more information, see
[AGG_STATE](../sql-manual/sql-types/Data-Types/AGG_STATE.md).
-- Transparent rewrite supports single table asynchronous materialized views.
[#34646](https://github.com/apache/doris/pull/34646)
-
-- Transparent rewrite supports `AGG_STATE` type aggregation roll-up.
[#35026](https://github.com/apache/doris/pull/35026)
-
-### 04 Others
+### Others
- Added function `replace_empty`.
- For more information, see
[documentation](https://doris.apache.org/docs/sql-manual/sql-functions/string-functions/replace_empty).
+ For more information, see
[documentation]../sql-manual/sql-functions/string-functions/replace_empty).
- Support `show storage policy using` statement.
- For more information, see
[documentation](https://doris.apache.org/docs/sql-manual/sql-statements/Show-Statements/SHOW-STORAGE-POLICY-USING/).
+ For more information, see
[documentation](../sql-manual/sql-statements/Show-Statements/SHOW-STORAGE-POLICY-USING.md).
- Support JVM metrics on the BE side.
By setting `enable_jvm_monitor=true` in `be.conf` to enable this feature.
-## Optimization
+## Improvements
- Supported creating inverted indexes for columns with Chinese names.
[#36321](https://github.com/apache/doris/pull/36321)
@@ -124,11 +120,17 @@ Apache Doris version 2.1.4 was officially released on
June 26, 2024. In this upd
- Optimize the `INSERT` operation that when the source is empty, the BE will
not execute. [#34418](https://github.com/apache/doris/pull/34418)
-- Support fetching file lists of Hive/Hudi tables in batches.
[#35107](https://github.com/apache/doris/pull/35107)
+- Support fetching file lists of Hive/Hudi tables in batches. In a senario
with 1.2 million files, the time taken to obtain the list of files has been
reduced from 390 seconds to 46 seconds.
[#35107](https://github.com/apache/doris/pull/35107)
+
+- Forbid dynamic partitioning when creating asynchronous materialized views.
+
+- Support detecting whether the partition data of external data of external
tables in Hive is synchronized with asynchronous materialized views.
-## Bugfix
+- Allow to create index for asynchronous materialized views.
-### 01 Query Optimizer
+## Bug fixes
+
+### Query optimizer
- Fixed the issue where SQL cache returns old results after truncating a
partition. [#34698](https://github.com/apache/doris/pull/34698)
@@ -162,30 +164,36 @@ Apache Doris version 2.1.4 was officially released on
June 26, 2024. In this upd
- Fixed the occasional issue where `<=>` was incorrectly converted to `=`.
[#36521](https://github.com/apache/doris/pull/36521)
-### 02 Query Execution
+### Query execution
- Fixed the issue where the query hangs if the limited rows are reached on the
pipeline engine and memory is not released.
[#35746](https://github.com/apache/doris/pull/35746)
- Fixed the BE coredump when `enable_decimal256` is true but falls back to the
old planner. [#35731](https://github.com/apache/doris/pull/35731)
-### 03 Asynchronous Materialized Views
+### Asynchronous materialized views
+
+- Fixed the issue in the asynchronous materialized view build where the
store_row_column attribute specified was not being recognized by the core.
+
+- Fixed the problem in the asynchronous materialized view build where
specifying the storage_medium was not taking effect.
+
+- Resolved the error occurring in the asynchronous materialized view show
partitions after the base table is deleted.
- Fixed the issue where asynchronous materialized views caused backup and
restore exceptions. [#35703](https://github.com/apache/doris/pull/35703)
- Fixed the issue where partition rewrite could lead to incorrect results.
[#35236](https://github.com/apache/doris/pull/35236)
-### 04 Semi-structured
+### Semi-structured
- Fixed the core dump problem when a VARIANT with an empty key is used.
[#35671](https://github.com/apache/doris/pull/35671)
- Bitmap and BloomFilter index should not perform light index changes.
[#35225](https://github.com/apache/doris/pull/35225)
-### 05 Primary Key
+### Primary key
- Fixed the issue where an exception BE restart occurred in the case of
partial column updates during import, which could result in duplicate keys.
[#35678](https://github.com/apache/doris/pull/35678)
- Fixed the issue where BE might core dump during clone operations when memory
is tight. [#34702](https://github.com/apache/doris/pull/34702)
-### 06 Lakehouse
+### Lakehouse
- Fixed the issue where a Hive table could not be created with a fully
qualified name such as `ctl.db.tbl`
[#34984](https://github.com/apache/doris/pull/34984)
@@ -230,7 +238,7 @@ Apache Doris version 2.1.4 was officially released on June
26, 2024. In this upd
- Support retain and pass the additional user-defined properties fo Table
Valued Functions to the S3 SDK.
[#35515](https://github.com/apache/doris/pull/35515)
-### 07 Data Import
+### Data import
- Fixed the issue where `CANCEL LOAD` did not work
[#35352](https://github.com/apache/doris/pull/35352)
@@ -238,7 +246,7 @@ Apache Doris version 2.1.4 was officially released on June
26, 2024. In this upd
- Fixed the issue with bRPC serializing large data files when sent via HTTP
[#36169](https://github.com/apache/doris/pull/36169)
-### 08 Data Management
+### Data management
- Fixed the isseu that the resource tag in ConnectionContext was not set after
forwarding DDL or DML to master FE.
[#35618](https://github.com/apache/doris/pull/35618)
@@ -252,13 +260,13 @@ Apache Doris version 2.1.4 was officially released on
June 26, 2024. In this upd
- Fixed the issue where the partition column of a table changed when querying
or inserting into an automatic partition table using the old optimizer
[#36514](https://github.com/apache/doris/pull/36514)
-### 09 Memory Management
+### Memory management
- Fixed the issue of frequent errors in the logs due to failure in obtaining
Cgroup meminfo. [#35425](https://github.com/apache/doris/pull/35425)
- Fixed the issue where the Segment cache size was uncontrolled when using
BloomFilter, leading to abnormal process memory growth.
[#34871](https://github.com/apache/doris/pull/34871)
-### 10 Permissions
+### Permissions
- Fixed the issue where permission settings were ineffective after enabling
case-insensitive table names.
[#36557](https://github.com/apache/doris/pull/36557)
@@ -266,7 +274,7 @@ Apache Doris version 2.1.4 was officially released on June
26, 2024. In this upd
- Fixed the issue where authorization could not be checked for the `SELECT
COUNT(*)` statement. [#35465](https://github.com/apache/doris/pull/35465)
-### 11 Others
+### Others
- Fixed the issue where the client JDBC program could not close the connection
if the MySQL connection was broken.
[#36616](https://github.com/apache/doris/pull/36616)
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/releasenotes/release-2.1.4.md
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/releasenotes/release-2.1.4.md
index 2a901aac749..205a17c8212 100644
---
a/i18n/zh-CN/docusaurus-plugin-content-docs/current/releasenotes/release-2.1.4.md
+++
b/i18n/zh-CN/docusaurus-plugin-content-docs/current/releasenotes/release-2.1.4.md
@@ -24,7 +24,7 @@ specific language governing permissions and limitations
under the License.
-->
-**Apache Doris 2.1.4 版本已于 2024 年 6 月 26 日正式发布。**在 2.1.4
版本中，我们对数据湖分析场景进行了多项功能体验优化，重点修复了旧版本中异常内存占用的问题，同时提交了若干改进项以及问题修复，进一步提升了系统的性能、稳定性及易用性，欢迎大家下载使用。
+**Apache Doris 2.1.4 版本已于 2024 年 6 月 26 日正式发布。** 在 2.1.4
版本中，我们对数据湖分析场景进行了多项功能体验优化，重点修复了旧版本中异常内存占用的问题，同时提交了若干改进项以及问题修复，进一步提升了系统的性能、稳定性及易用性，欢迎大家下载使用。
**官网下载：** https://doris.apache.org/download/
@@ -40,21 +40,21 @@ under the License.
关于更多信息，请参考文档：
- - [BE
日志管理](https://doris.apache.org/zh-CN/docs/admin-manual/log-management/be-log)
+ - [BE 日志管理](../admin-manual/log-management/be-log.md)
- - - [FE
日志管理](https://doris.apache.org/zh-CN/docs/admin-manual/log-management/fe-log)
+ - [FE 日志管理](../admin-manual/log-management/fe-log.md)
- 如果建表时没有填写表注释，默认注释为空，不再使用表类型作为默认表注释。
[#36025](https://github.com/apache/doris/pull/36025)
- DECIMALV3 的默认精度从 (9, 0) 调整为 (38,9) ，以和最初发布此功能的版本保持兼容。
[#36316](https://github.com/apache/doris/pull/36316)
-## 新功能
+## 新增功能
-### 01 查询优化器
+### 查询优化器
- **支持 FE 火焰图工具**：在 FE 部署目录 `${DORIS_FE_HOME}/bin` 中会增加`profile_fe.sh` 脚本，可以利用
async-profiler 工具生成 FE 的火焰图，用以发现性能瓶颈点。
- 关于更多信息，请参考文档：[使用 FE Profiler
生成火焰图](https://doris.apache.org/zh-CN/community/developer-guide/fe-profiler)
+ 关于更多信息，请参考文档：[使用 FE Profiler
生成火焰图](/community/developer-guide/fe-profiler.md)
- **支持 SELECT DISTINCT 与聚合函数同时使用**：支持 `SELECT DISTINCT`
与聚合函数同时使用，在一个查询中同时去重和进行聚合操作，如 SUM、MIN/MAX 等。
@@ -62,47 +62,45 @@ under the License.
- **查询优化器全面支持高并发点查询功能**：在 2.1.4 版本之后，查询优化器全面支持高并发点查询功能，所有符合点查询条件的 SQL
语句会自动走短路径查询，无需用户在客户端额外设置 `set experimental_enable_nereids_planner = false`。
[#36205](https://github.com/apache/doris/pull/36205).
-### 02 湖仓一体
+### 湖仓一体
- **支持 Paimon 的原生读取器来处理 Deletion Vector：** Deletion Vector
主要用于标记或追踪哪些数据已被删除或标记为删除，通常应用在需要保留历史数据的场景，基于本优化可以提升大量数据更新或删除时的处理效率。
[#35241](https://github.com/apache/doris/pull/35241)
- 关于更多信息，请参考文档：[数据湖分析 -
Paimon](https://doris.apache.org/zh-CN/docs/lakehouse/datalake-analytics/paimon)
+ 关于更多信息，请参考文档：[数据湖分析 - Paimon](../lakehouse/datalake-analytics/paimon.md)
- **支持在表值函数（TVF）中使用 Resource**：TVF 功能为 Apache Doris 提供了直接将对象存储或 HDFS 上的文件作为
Table 进行查询分析的能力。通过在 TVF 中引用 Resource，可以避免重复填写连接信息，提升使用体验。
[#35139](https://github.com/apache/doris/pull/35139)
- 关于更多信息，请参考文档：[表函数 -
HDFS](https://doris.apache.org/zh-CN/docs/sql-manual/sql-functions/table-functions/hdfs/)
+ 关于更多信息，请参考文档：[表函数 -
HDFS](../sql-manual/sql-functions/table-functions/hdfs.md)
- **支持通过 Ranger 插件实现数据脱敏**：开启 Ranger 鉴权功能后，支持使用 Ranger 中的 Data Mask 功能进行数据脱敏。
- 关于更多信息，请参考文档：[基于 Apache Ranger
的鉴权管理](https://doris.apache.org/zh-CN/docs/admin-manual/auth/ranger/#data-mask-%E7%A4%BA%E4%BE%8B)
+ 关于更多信息，请参考文档：[基于 Apache Ranger
的鉴权管理](https://doris.apache.org/zh-CN/docs/admin-manual/auth/ranger/#%E5%AE%89%E8%A3%85-doris-ranger-%E6%8F%92%E4%BB%B6)
-### 03 异步物化视图
+### 异步物化视图
-- 构建支持分区上卷。 [#31812](https://github.com/apache/doris/pull/31812)
+- 构建支持内表触发式更新，如果物化视图使用的是内表，如果内表数据发生变化，可以触发物化视图刷新，需要在创建物化视图时指定 REFRESH ON
COMMIT。
-- 构建支持触发式更新。 [#34548](https://github.com/apache/doris/pull/34548)
+- 支持单表透明改写。
-- 构建支持指定 `store_row_column`。
[#35860](https://github.com/apache/doris/pull/35860)
+
关于更多信息，请参考文档：[查询异步物化视图](../query/view-materialized-view/query-async-materialized-view.md)
-- 构建支持指定 Storage Medium。
+- 透明改写支持 agg_state, agg_union 类型的聚合上卷，物化视图可以定义为 agg_state 或者
agg_union，查询使用具体的聚合函数，或者使用 agg_merge
-- 透明改写支持单表异步物化视图。[#34646](https://github.com/apache/doris/pull/34646)
+ 关于更多信息，请参考文档：[AGG_STATE](../sql-manual/sql-types/Data-Types/AGG_STATE.md)
-- 透明改写支持 AGG_STATE 类型的聚合上卷。
[#35026](https://github.com/apache/doris/pull/35026)
-
-### 04 其他
+### 其他
- **新增 `replace_empty` 函数**：将字符串中的子字符串进行替换，当旧字符串为空时，会将新字符串插入到原有字符串的每个字符前以及最后。
- 关于更多信息，请参考文档：[字符串函数 -
REPLACE_EMPTY](https://doris.apache.org/zh-CN/docs/sql-manual/sql-functions/string-functions/replace_empty)
+ 关于更多信息，请参考文档：[字符串函数 -
REPLACE_EMPTY](../sql-manual/sql-functions/string-functions/replace_empty.md)
- 支持 `show storage policy using` 语句：支持查看所有或指定存储策略关联的表和分区。
- 关于更多信息，请参考文档：[SQL 语句 -
SHOW](https://doris.apache.org/zh-CN/docs/sql-manual/sql-statements/Show-Statements/SHOW-STORAGE-POLICY-USING)
+ 关于更多信息，请参考文档：[SQL 语句 -
SHOW](../sql-manual/sql-statements/Show-Statements/SHOW-STORAGE-POLICY-USING.md)
- **支持 BE 侧的 JVM 指标：** 通过在 `be.conf` 配置文件中设置`enable_jvm_monitor=true`，可以启用对 BE
节点 JVM 的监控和指标收集，有助于了解 BE JVM 的资源使用情况，以便进行故障排除和性能优化。
-## 功能优化
+## 改进优化
- 支持为中文列名创建倒排索引。[#36321](https://github.com/apache/doris/pull/36321)
@@ -124,9 +122,17 @@ under the License.
- 支持分批获取 Hudi 和 Hive 文件列表，当存在大量数据文件时可以提升数据扫描性能。
[#35107](https://github.com/apache/doris/pull/35107)
-## 问题修复
+ - 120 万文件场景中，获取文件列表的时间由 390 秒缩减到 46 秒。
+
+- 创建异步物化试图时，禁止使用动态分区。
+
+- 支持检测 Hive 外表分区数据是否和异步物化试图同步。
+
+- 允许异步物化试图创建索引。
-### 01 查询优化器
+## 缺陷修复
+
+### 查询优化器
- 修复 SQL Cache 在 `truncate paritition`
后依然返回旧结果的问题。[#34698](https://github.com/apache/doris/pull/34698)
@@ -162,31 +168,37 @@ under the License.
- 修复偶现的 `<=>` 被错误转换为 `=`
的问题。[#36521](https://github.com/apache/doris/pull/36521)
-### 02 查询执行
+### 查询执行
- 修复 Pipeline 引擎上达到限定的行数且内存没有释放时查询被挂起的问题。
[#35746](https://github.com/apache/doris/pull/35746)
- 修复当设置 `enable_decimal256 =true` 且查询优化器回退到旧版本时 BE 发生 Core
的问题。[#35731](https://github.com/apache/doris/pull/35731)
-### 03 物化视图
+### 物化视图
+
+- 修复构建异步物化试图指定 store_row_column 属性，be core 的问题。
+
+- 修复构建异步物化视图指定 storage_medium 不生效的问题。
+- 修复基表删除后，异步物化视图 show partitions 报错的问题。
+
- 修复异步物化视图引起备份恢复异常的问题。
- 修复分区改写可能导致错误结果的问题。
-### 04 半结构化数据分析
+### 半结构化数据分析
- 修复带有空 Key 的 VARIANT 类型发生 Core
的问题。[#35671](https://github.com/apache/doris/pull/35671)
- Bitmap 索引和 BloomFilter
索引不应支持轻量级索引变更。[#35225](https://github.com/apache/doris/pull/35225)
-### 05 主键模型
+### 主键模型
- 修复在有部分列更新导入的情况下发生异常重启，可能会产生重复 Key
的问题。[#35678](https://github.com/apache/doris/pull/35678)
- 修复在内存紧张时发生 Clone 时 BE 可能会发生 Core
的问题。[#34702](https://github.com/apache/doris/pull/34702)
-### 06 湖仓一体
+### 湖仓一体
- 修复创建 Hive 表时无法使用完全限定名（如 `ctl.db.tbl`）的问题。
[#34984](https://github.com/apache/doris/pull/34984)
@@ -228,7 +240,7 @@ under the License.
- 允许用户定义的属性通过表函数传递给 S3 SDK。[#35515](https://github.com/apache/doris/pull/35515)
-### 07 数据导入
+### 数据导入
- 修复 `CANCEL LOAD`
命令不生效的问题。[#35352](https://github.com/apache/doris/pull/35352)
@@ -236,7 +248,7 @@ under the License.
- 修复 bRPC 通过 HTTP 发送大数据文件序列化的问题。
[#36169](https://github.com/apache/doris/pull/36169)
-### 08 数据管控
+### 数据管控
- 修复了在将 DDL 或 DML 转发到主 FE 后，ConnectionContext 中的资源标签未设置的问题。
[#35618](https://github.com/apache/doris/pull/35618)
@@ -250,13 +262,13 @@ under the License.
- 修复了使用旧优化器查询或插入自动分区表时，表的分区列发生变化的问题。
[#36514](https://github.com/apache/doris/pull/36514)
-### 09 内存管理
+### 内存管理
- 修复日志中频繁报错 Cgroup meminfo 获取失败的问题。
[#35425](https://github.com/apache/doris/pull/35425)
- 修复使用 BloomFilter 时 Segment
缓存大小不受控制导致进程内存异常增长的问题。[#34871](https://github.com/apache/doris/pull/34871)
-### 10 权限
+### 权限
- 修复开启表名大小写不敏感后，权限设置无效的问题。[#36557](https://github.com/apache/doris/pull/36557)
@@ -264,7 +276,7 @@ under the License.
- 修复了无法检查 `SELECT COUNT(*)`
语句授权的问题。[#35465](https://github.com/apache/doris/pull/35465)
-### 11 其他
+### 其他
- 修复 MySQL 连接损坏情况下，客户端 JDBC 程序无法关闭连接的问题。
[#36616](https://github.com/apache/doris/pull/36616)
diff --git
a/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/releasenotes/release-2.1.4.md
b/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/releasenotes/release-2.1.4.md
index 2a901aac749..205a17c8212 100644
---
a/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/releasenotes/release-2.1.4.md
+++
b/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/releasenotes/release-2.1.4.md
@@ -24,7 +24,7 @@ specific language governing permissions and limitations
under the License.
-->
-**Apache Doris 2.1.4 版本已于 2024 年 6 月 26 日正式发布。**在 2.1.4
版本中，我们对数据湖分析场景进行了多项功能体验优化，重点修复了旧版本中异常内存占用的问题，同时提交了若干改进项以及问题修复，进一步提升了系统的性能、稳定性及易用性，欢迎大家下载使用。
+**Apache Doris 2.1.4 版本已于 2024 年 6 月 26 日正式发布。** 在 2.1.4
版本中，我们对数据湖分析场景进行了多项功能体验优化，重点修复了旧版本中异常内存占用的问题，同时提交了若干改进项以及问题修复，进一步提升了系统的性能、稳定性及易用性，欢迎大家下载使用。
**官网下载：** https://doris.apache.org/download/
@@ -40,21 +40,21 @@ under the License.
关于更多信息，请参考文档：
- - [BE
日志管理](https://doris.apache.org/zh-CN/docs/admin-manual/log-management/be-log)
+ - [BE 日志管理](../admin-manual/log-management/be-log.md)
- - - [FE
日志管理](https://doris.apache.org/zh-CN/docs/admin-manual/log-management/fe-log)
+ - [FE 日志管理](../admin-manual/log-management/fe-log.md)
- 如果建表时没有填写表注释，默认注释为空，不再使用表类型作为默认表注释。
[#36025](https://github.com/apache/doris/pull/36025)
- DECIMALV3 的默认精度从 (9, 0) 调整为 (38,9) ，以和最初发布此功能的版本保持兼容。
[#36316](https://github.com/apache/doris/pull/36316)
-## 新功能
+## 新增功能
-### 01 查询优化器
+### 查询优化器
- **支持 FE 火焰图工具**：在 FE 部署目录 `${DORIS_FE_HOME}/bin` 中会增加`profile_fe.sh` 脚本，可以利用
async-profiler 工具生成 FE 的火焰图，用以发现性能瓶颈点。
- 关于更多信息，请参考文档：[使用 FE Profiler
生成火焰图](https://doris.apache.org/zh-CN/community/developer-guide/fe-profiler)
+ 关于更多信息，请参考文档：[使用 FE Profiler
生成火焰图](/community/developer-guide/fe-profiler.md)
- **支持 SELECT DISTINCT 与聚合函数同时使用**：支持 `SELECT DISTINCT`
与聚合函数同时使用，在一个查询中同时去重和进行聚合操作，如 SUM、MIN/MAX 等。
@@ -62,47 +62,45 @@ under the License.
- **查询优化器全面支持高并发点查询功能**：在 2.1.4 版本之后，查询优化器全面支持高并发点查询功能，所有符合点查询条件的 SQL
语句会自动走短路径查询，无需用户在客户端额外设置 `set experimental_enable_nereids_planner = false`。
[#36205](https://github.com/apache/doris/pull/36205).
-### 02 湖仓一体
+### 湖仓一体
- **支持 Paimon 的原生读取器来处理 Deletion Vector：** Deletion Vector
主要用于标记或追踪哪些数据已被删除或标记为删除，通常应用在需要保留历史数据的场景，基于本优化可以提升大量数据更新或删除时的处理效率。
[#35241](https://github.com/apache/doris/pull/35241)
- 关于更多信息，请参考文档：[数据湖分析 -
Paimon](https://doris.apache.org/zh-CN/docs/lakehouse/datalake-analytics/paimon)
+ 关于更多信息，请参考文档：[数据湖分析 - Paimon](../lakehouse/datalake-analytics/paimon.md)
- **支持在表值函数（TVF）中使用 Resource**：TVF 功能为 Apache Doris 提供了直接将对象存储或 HDFS 上的文件作为
Table 进行查询分析的能力。通过在 TVF 中引用 Resource，可以避免重复填写连接信息，提升使用体验。
[#35139](https://github.com/apache/doris/pull/35139)
- 关于更多信息，请参考文档：[表函数 -
HDFS](https://doris.apache.org/zh-CN/docs/sql-manual/sql-functions/table-functions/hdfs/)
+ 关于更多信息，请参考文档：[表函数 -
HDFS](../sql-manual/sql-functions/table-functions/hdfs.md)
- **支持通过 Ranger 插件实现数据脱敏**：开启 Ranger 鉴权功能后，支持使用 Ranger 中的 Data Mask 功能进行数据脱敏。
- 关于更多信息，请参考文档：[基于 Apache Ranger
的鉴权管理](https://doris.apache.org/zh-CN/docs/admin-manual/auth/ranger/#data-mask-%E7%A4%BA%E4%BE%8B)
+ 关于更多信息，请参考文档：[基于 Apache Ranger
的鉴权管理](https://doris.apache.org/zh-CN/docs/admin-manual/auth/ranger/#%E5%AE%89%E8%A3%85-doris-ranger-%E6%8F%92%E4%BB%B6)
-### 03 异步物化视图
+### 异步物化视图
-- 构建支持分区上卷。 [#31812](https://github.com/apache/doris/pull/31812)
+- 构建支持内表触发式更新，如果物化视图使用的是内表，如果内表数据发生变化，可以触发物化视图刷新，需要在创建物化视图时指定 REFRESH ON
COMMIT。
-- 构建支持触发式更新。 [#34548](https://github.com/apache/doris/pull/34548)
+- 支持单表透明改写。
-- 构建支持指定 `store_row_column`。
[#35860](https://github.com/apache/doris/pull/35860)
+
关于更多信息，请参考文档：[查询异步物化视图](../query/view-materialized-view/query-async-materialized-view.md)
-- 构建支持指定 Storage Medium。
+- 透明改写支持 agg_state, agg_union 类型的聚合上卷，物化视图可以定义为 agg_state 或者
agg_union，查询使用具体的聚合函数，或者使用 agg_merge
-- 透明改写支持单表异步物化视图。[#34646](https://github.com/apache/doris/pull/34646)
+ 关于更多信息，请参考文档：[AGG_STATE](../sql-manual/sql-types/Data-Types/AGG_STATE.md)
-- 透明改写支持 AGG_STATE 类型的聚合上卷。
[#35026](https://github.com/apache/doris/pull/35026)
-
-### 04 其他
+### 其他
- **新增 `replace_empty` 函数**：将字符串中的子字符串进行替换，当旧字符串为空时，会将新字符串插入到原有字符串的每个字符前以及最后。
- 关于更多信息，请参考文档：[字符串函数 -
REPLACE_EMPTY](https://doris.apache.org/zh-CN/docs/sql-manual/sql-functions/string-functions/replace_empty)
+ 关于更多信息，请参考文档：[字符串函数 -
REPLACE_EMPTY](../sql-manual/sql-functions/string-functions/replace_empty.md)
- 支持 `show storage policy using` 语句：支持查看所有或指定存储策略关联的表和分区。
- 关于更多信息，请参考文档：[SQL 语句 -
SHOW](https://doris.apache.org/zh-CN/docs/sql-manual/sql-statements/Show-Statements/SHOW-STORAGE-POLICY-USING)
+ 关于更多信息，请参考文档：[SQL 语句 -
SHOW](../sql-manual/sql-statements/Show-Statements/SHOW-STORAGE-POLICY-USING.md)
- **支持 BE 侧的 JVM 指标：** 通过在 `be.conf` 配置文件中设置`enable_jvm_monitor=true`，可以启用对 BE
节点 JVM 的监控和指标收集，有助于了解 BE JVM 的资源使用情况，以便进行故障排除和性能优化。
-## 功能优化
+## 改进优化
- 支持为中文列名创建倒排索引。[#36321](https://github.com/apache/doris/pull/36321)
@@ -124,9 +122,17 @@ under the License.
- 支持分批获取 Hudi 和 Hive 文件列表，当存在大量数据文件时可以提升数据扫描性能。
[#35107](https://github.com/apache/doris/pull/35107)
-## 问题修复
+ - 120 万文件场景中，获取文件列表的时间由 390 秒缩减到 46 秒。
+
+- 创建异步物化试图时，禁止使用动态分区。
+
+- 支持检测 Hive 外表分区数据是否和异步物化试图同步。
+
+- 允许异步物化试图创建索引。
-### 01 查询优化器
+## 缺陷修复
+
+### 查询优化器
- 修复 SQL Cache 在 `truncate paritition`
后依然返回旧结果的问题。[#34698](https://github.com/apache/doris/pull/34698)
@@ -162,31 +168,37 @@ under the License.
- 修复偶现的 `<=>` 被错误转换为 `=`
的问题。[#36521](https://github.com/apache/doris/pull/36521)
-### 02 查询执行
+### 查询执行
- 修复 Pipeline 引擎上达到限定的行数且内存没有释放时查询被挂起的问题。
[#35746](https://github.com/apache/doris/pull/35746)
- 修复当设置 `enable_decimal256 =true` 且查询优化器回退到旧版本时 BE 发生 Core
的问题。[#35731](https://github.com/apache/doris/pull/35731)
-### 03 物化视图
+### 物化视图
+
+- 修复构建异步物化试图指定 store_row_column 属性，be core 的问题。
+
+- 修复构建异步物化视图指定 storage_medium 不生效的问题。
+- 修复基表删除后，异步物化视图 show partitions 报错的问题。
+
- 修复异步物化视图引起备份恢复异常的问题。
- 修复分区改写可能导致错误结果的问题。
-### 04 半结构化数据分析
+### 半结构化数据分析
- 修复带有空 Key 的 VARIANT 类型发生 Core
的问题。[#35671](https://github.com/apache/doris/pull/35671)
- Bitmap 索引和 BloomFilter
索引不应支持轻量级索引变更。[#35225](https://github.com/apache/doris/pull/35225)
-### 05 主键模型
+### 主键模型
- 修复在有部分列更新导入的情况下发生异常重启，可能会产生重复 Key
的问题。[#35678](https://github.com/apache/doris/pull/35678)
- 修复在内存紧张时发生 Clone 时 BE 可能会发生 Core
的问题。[#34702](https://github.com/apache/doris/pull/34702)
-### 06 湖仓一体
+### 湖仓一体
- 修复创建 Hive 表时无法使用完全限定名（如 `ctl.db.tbl`）的问题。
[#34984](https://github.com/apache/doris/pull/34984)
@@ -228,7 +240,7 @@ under the License.
- 允许用户定义的属性通过表函数传递给 S3 SDK。[#35515](https://github.com/apache/doris/pull/35515)
-### 07 数据导入
+### 数据导入
- 修复 `CANCEL LOAD`
命令不生效的问题。[#35352](https://github.com/apache/doris/pull/35352)
@@ -236,7 +248,7 @@ under the License.
- 修复 bRPC 通过 HTTP 发送大数据文件序列化的问题。
[#36169](https://github.com/apache/doris/pull/36169)
-### 08 数据管控
+### 数据管控
- 修复了在将 DDL 或 DML 转发到主 FE 后，ConnectionContext 中的资源标签未设置的问题。
[#35618](https://github.com/apache/doris/pull/35618)
@@ -250,13 +262,13 @@ under the License.
- 修复了使用旧优化器查询或插入自动分区表时，表的分区列发生变化的问题。
[#36514](https://github.com/apache/doris/pull/36514)
-### 09 内存管理
+### 内存管理
- 修复日志中频繁报错 Cgroup meminfo 获取失败的问题。
[#35425](https://github.com/apache/doris/pull/35425)
- 修复使用 BloomFilter 时 Segment
缓存大小不受控制导致进程内存异常增长的问题。[#34871](https://github.com/apache/doris/pull/34871)
-### 10 权限
+### 权限
- 修复开启表名大小写不敏感后，权限设置无效的问题。[#36557](https://github.com/apache/doris/pull/36557)
@@ -264,7 +276,7 @@ under the License.
- 修复了无法检查 `SELECT COUNT(*)`
语句授权的问题。[#35465](https://github.com/apache/doris/pull/35465)
-### 11 其他
+### 其他
- 修复 MySQL 连接损坏情况下，客户端 JDBC 程序无法关闭连接的问题。
[#36616](https://github.com/apache/doris/pull/36616)
diff --git a/versioned_docs/version-2.1/releasenotes/release-2.1.4.md
b/versioned_docs/version-2.1/releasenotes/release-2.1.4.md
index bff23d629b5..d04a8be0cf6 100644
--- a/versioned_docs/version-2.1/releasenotes/release-2.1.4.md
+++ b/versioned_docs/version-2.1/releasenotes/release-2.1.4.md
@@ -24,7 +24,7 @@ specific language governing permissions and limitations
under the License.
-->
-Apache Doris version 2.1.4 was officially released on June 26, 2024. In this
update, we have optimized various functional experiences for data lakehouse
scenarios, with a focus on resolving the abnormal memory usage issue in the
previous version. Additionally, we have implemented several improvemnents and
bug fixes to enhance the stability. Welcome to download and use it.
+**Apache Doris version 2.1.4 was officially released on June 26, 2024.** In
this update, we have optimized various functional experiences for data
lakehouse scenarios, with a focus on resolving the abnormal memory usage issue
in the previous version. Additionally, we have implemented several
improvemnents and bug fixes to enhance the stability. Welcome to download and
use it.
**Quick Download:** https://doris.apache.org/download/
@@ -32,7 +32,7 @@ Apache Doris version 2.1.4 was officially released on June
26, 2024. In this upd
**GitHub Release:** https://github.com/apache/doris/releases
-## Behavior Changed
+## Behavior changes
- Non-existent files will be ignored when querying external tables such as
Hive. [#35319](https://github.com/apache/doris/pull/35319)
@@ -48,21 +48,21 @@ Apache Doris version 2.1.4 was officially released on June
26, 2024. In this upd
For more infomation, please see the documentations:
- - [Log Management - FE
Log](https://doris.apache.org/docs/admin-manual/log-management/fe-log)
+ - [Log Management - FE Log](../admin-manual/log-management/fe-log.md)
- - [Log Management - FE
Log](https://doris.apache.org/docs/admin-manual/log-management/be-log)
+ - [Log Management - FE Log](../admin-manual/log-management/be-log.md)
- If no table comment is provided when creating a table, the default comment
will be empty instead of using the table type as the default comment.
[#36025](https://github.com/apache/doris/pull/36025)
- The default precision of DECIMALV3 has been adjusted from (9, 0) to (38, 9)
to maintain compatibility with the version in which this feature was initially
released. [#36316](https://github.com/apache/doris/pull/36316)
-## New Features
+## New features
-### 01 Query Optimizer
+### Query optimizer
- Support FE flame graph tool
- For more information, see the
[documentation](https://doris.apache.org/community/developer-guide/fe-profiler)
+ For more information, see the
[documentation](/community/developer-guide/fe-profiler.md)
- Support `SELECT DISTINCT` to be used with aggregation.
@@ -70,7 +70,7 @@ Apache Doris version 2.1.4 was officially released on June
26, 2024. In this upd
- The new optimizer fully supports point query functionality
[#36205](https://github.com/apache/doris/pull/36205).
-### 02 Lakehouse
+### Lakehouse
- Support native reader of Apache Paimon deletion vector
[#35241](https://github.com/apache/doris/pull/35241)
@@ -78,33 +78,29 @@ Apache Doris version 2.1.4 was officially released on June
26, 2024. In this upd
- Access controller with Hive Ranger plugin supports Data Mask
-### 03 Asynchronous Materialized Views
+### Asynchronous materialized views
-- Support partition roll-up during construction.
[#31812](https://github.com/apache/doris/pull/31812)
+- Build support for internal table triggered updates, where if a materialized
view uses an internal table and the data in the internal table changes, it can
trigger a refresh of the materialized view, specifying REFRESH ON COMMIT when
creating the materialized view.
-- Support triggered updates during construction.
[#34548](https://github.com/apache/doris/pull/34548)
+- Support transparent rewriting for single tables. For more information, see
[Querying Async Materialized
View](../query/view-materialized-view/query-async-materialized-view.md).
-- Support specifying the `store_row_column` and `storage_medium` attribute
during construction. [#35860](https://github.com/apache/doris/pull/35860)
+- Transparent rewriting supports aggregation roll-up for agg_state, agg_union
types; materialized views can be defined as agg_state or agg_union, queries can
use specific aggregation functions, or use agg_merge. For more information, see
[AGG_STATE](../sql-manual/sql-types/Data-Types/AGG_STATE.md).
-- Transparent rewrite supports single table asynchronous materialized views.
[#34646](https://github.com/apache/doris/pull/34646)
-
-- Transparent rewrite supports `AGG_STATE` type aggregation roll-up.
[#35026](https://github.com/apache/doris/pull/35026)
-
-### 04 Others
+### Others
- Added function `replace_empty`.
- For more information, see
[documentation](https://doris.apache.org/docs/sql-manual/sql-functions/string-functions/replace_empty).
+ For more information, see
[documentation]../sql-manual/sql-functions/string-functions/replace_empty).
- Support `show storage policy using` statement.
- For more information, see
[documentation](https://doris.apache.org/docs/sql-manual/sql-statements/Show-Statements/SHOW-STORAGE-POLICY-USING/).
+ For more information, see
[documentation](../sql-manual/sql-statements/Show-Statements/SHOW-STORAGE-POLICY-USING.md).
- Support JVM metrics on the BE side.
By setting `enable_jvm_monitor=true` in `be.conf` to enable this feature.
-## Optimization
+## Improvements
- Supported creating inverted indexes for columns with Chinese names.
[#36321](https://github.com/apache/doris/pull/36321)
@@ -124,11 +120,17 @@ Apache Doris version 2.1.4 was officially released on
June 26, 2024. In this upd
- Optimize the `INSERT` operation that when the source is empty, the BE will
not execute. [#34418](https://github.com/apache/doris/pull/34418)
-- Support fetching file lists of Hive/Hudi tables in batches.
[#35107](https://github.com/apache/doris/pull/35107)
+- Support fetching file lists of Hive/Hudi tables in batches. In a senario
with 1.2 million files, the time taken to obtain the list of files has been
reduced from 390 seconds to 46 seconds.
[#35107](https://github.com/apache/doris/pull/35107)
+
+- Forbid dynamic partitioning when creating asynchronous materialized views.
+
+- Support detecting whether the partition data of external data of external
tables in Hive is synchronized with asynchronous materialized views.
-## Bugfix
+- Allow to create index for asynchronous materialized views.
-### 01 Query Optimizer
+## Bug fixes
+
+### Query optimizer
- Fixed the issue where SQL cache returns old results after truncating a
partition. [#34698](https://github.com/apache/doris/pull/34698)
@@ -162,30 +164,36 @@ Apache Doris version 2.1.4 was officially released on
June 26, 2024. In this upd
- Fixed the occasional issue where `<=>` was incorrectly converted to `=`.
[#36521](https://github.com/apache/doris/pull/36521)
-### 02 Query Execution
+### Query execution
- Fixed the issue where the query hangs if the limited rows are reached on the
pipeline engine and memory is not released.
[#35746](https://github.com/apache/doris/pull/35746)
- Fixed the BE coredump when `enable_decimal256` is true but falls back to the
old planner. [#35731](https://github.com/apache/doris/pull/35731)
-### 03 Asynchronous Materialized Views
+### Asynchronous materialized views
+
+- Fixed the issue in the asynchronous materialized view build where the
store_row_column attribute specified was not being recognized by the core.
+
+- Fixed the problem in the asynchronous materialized view build where
specifying the storage_medium was not taking effect.
+
+- Resolved the error occurring in the asynchronous materialized view show
partitions after the base table is deleted.
- Fixed the issue where asynchronous materialized views caused backup and
restore exceptions. [#35703](https://github.com/apache/doris/pull/35703)
- Fixed the issue where partition rewrite could lead to incorrect results.
[#35236](https://github.com/apache/doris/pull/35236)
-### 04 Semi-structured
+### Semi-structured
- Fixed the core dump problem when a VARIANT with an empty key is used.
[#35671](https://github.com/apache/doris/pull/35671)
- Bitmap and BloomFilter index should not perform light index changes.
[#35225](https://github.com/apache/doris/pull/35225)
-### 05 Primary Key
+### Primary key
- Fixed the issue where an exception BE restart occurred in the case of
partial column updates during import, which could result in duplicate keys.
[#35678](https://github.com/apache/doris/pull/35678)
- Fixed the issue where BE might core dump during clone operations when memory
is tight. [#34702](https://github.com/apache/doris/pull/34702)
-### 06 Lakehouse
+### Lakehouse
- Fixed the issue where a Hive table could not be created with a fully
qualified name such as `ctl.db.tbl`
[#34984](https://github.com/apache/doris/pull/34984)
@@ -230,7 +238,7 @@ Apache Doris version 2.1.4 was officially released on June
26, 2024. In this upd
- Support retain and pass the additional user-defined properties fo Table
Valued Functions to the S3 SDK.
[#35515](https://github.com/apache/doris/pull/35515)
-### 07 Data Import
+### Data import
- Fixed the issue where `CANCEL LOAD` did not work
[#35352](https://github.com/apache/doris/pull/35352)
@@ -238,7 +246,7 @@ Apache Doris version 2.1.4 was officially released on June
26, 2024. In this upd
- Fixed the issue with bRPC serializing large data files when sent via HTTP
[#36169](https://github.com/apache/doris/pull/36169)
-### 08 Data Management
+### Data management
- Fixed the isseu that the resource tag in ConnectionContext was not set after
forwarding DDL or DML to master FE.
[#35618](https://github.com/apache/doris/pull/35618)
@@ -252,13 +260,13 @@ Apache Doris version 2.1.4 was officially released on
June 26, 2024. In this upd
- Fixed the issue where the partition column of a table changed when querying
or inserting into an automatic partition table using the old optimizer
[#36514](https://github.com/apache/doris/pull/36514)
-### 09 Memory Management
+### Memory management
- Fixed the issue of frequent errors in the logs due to failure in obtaining
Cgroup meminfo. [#35425](https://github.com/apache/doris/pull/35425)
- Fixed the issue where the Segment cache size was uncontrolled when using
BloomFilter, leading to abnormal process memory growth.
[#34871](https://github.com/apache/doris/pull/34871)
-### 10 Permissions
+### Permissions
- Fixed the issue where permission settings were ineffective after enabling
case-insensitive table names.
[#36557](https://github.com/apache/doris/pull/36557)
@@ -266,7 +274,7 @@ Apache Doris version 2.1.4 was officially released on June
26, 2024. In this upd
- Fixed the issue where authorization could not be checked for the `SELECT
COUNT(*)` statement. [#35465](https://github.com/apache/doris/pull/35465)
-### 11 Others
+### Others
- Fixed the issue where the client JDBC program could not close the connection
if the MySQL connection was broken.
[#36616](https://github.com/apache/doris/pull/36616)
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
