This is an automated email from the ASF dual-hosted git repository. wangbo pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris-website.git
The following commit(s) were added to refs/heads/master by this push: new 9e6eb168170 add some note for workload group/resource group (#793) 9e6eb168170 is described below commit 9e6eb16817029b178971356e6bf7f6d333e80683 Author: wangbo <wan...@apache.org> AuthorDate: Thu Jun 27 09:58:16 2024 +0800 add some note for workload group/resource group (#793) --- docs/admin-manual/resource-admin/multi-tenant.md | 2 ++ .../admin-manual/resource-admin/multi-tenant.md | 3 ++- .../admin-manual/resource-admin/workload-group.md | 21 ++++++++++++++------ .../admin-manual/resource-admin/multi-tenant.md | 3 ++- .../admin-manual/resource-admin/workload-group.md | 23 +++++++++++++++------- .../admin-manual/resource-admin/multi-tenant.md | 2 ++ 6 files changed, 39 insertions(+), 15 deletions(-) diff --git a/docs/admin-manual/resource-admin/multi-tenant.md b/docs/admin-manual/resource-admin/multi-tenant.md index ef7cf47613f..1a6c1d505ba 100644 --- a/docs/admin-manual/resource-admin/multi-tenant.md +++ b/docs/admin-manual/resource-admin/multi-tenant.md @@ -181,6 +181,8 @@ At present, Doris's resource restrictions on single queries are mainly divided i 2. CPU limitations + > Note: Since Doris 2.1, cpu_resource_limit will gradually be replaced by workload group, so it is not recommended to use it. + Users can limit the CPU resources of the query in the following ways: ```sql diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/current/admin-manual/resource-admin/multi-tenant.md b/i18n/zh-CN/docusaurus-plugin-content-docs/current/admin-manual/resource-admin/multi-tenant.md index f4daa8d8ddc..9c5b0f86610 100644 --- a/i18n/zh-CN/docusaurus-plugin-content-docs/current/admin-manual/resource-admin/multi-tenant.md +++ b/i18n/zh-CN/docusaurus-plugin-content-docs/current/admin-manual/resource-admin/multi-tenant.md @@ -180,6 +180,8 @@ FE 不参与用户数据的处理计算等工作,因此是一个资源消耗 2. CPU 限制 + >注:从Doris 2.1 之后,cpu_resource_limit 将逐渐被 workload group 替代,因此不建议使用该参数。 + 用户可以通过以下方式限制查询的 CPU 资源: ```sql @@ -192,7 +194,6 @@ FE 不参与用户数据的处理计算等工作,因此是一个资源消耗 `cpu_resource_limit` 的取值是一个相对值,取值越大则能够使用的 CPU 资源越多。但一个查询能使用的 CPU 上限也取决于表的分区分桶数。原则上,一个查询的最大 CPU 使用量和查询涉及到的 tablet 数量正相关。极端情况下,假设一个查询仅涉及到一个 tablet,则即使 `cpu_resource_limit` 设置一个较大值,也仅能使用 1 个 CPU 资源。 通过内存和 CPU 的资源限制。我们可以在一个资源组内,将用户的查询进行更细粒度的资源划分。比如我们可以让部分时效性要求不高,但是计算量很大的离线任务使用更少的 CPU 资源和更多的内存资源。而部分延迟敏感的在线任务,使用更多的 CPU 资源以及合理的内存资源。 -注:在 Doris 2.1 之后,cpu_resource_limit 将逐渐被 workload group 替代。 ## 最佳实践和向前兼容 diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.0/admin-manual/resource-admin/workload-group.md b/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.0/admin-manual/resource-admin/workload-group.md index a1212e1b9f4..aa20fb79415 100644 --- a/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.0/admin-manual/resource-admin/workload-group.md +++ b/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.0/admin-manual/resource-admin/workload-group.md @@ -40,14 +40,23 @@ Workload Group 可限制组内任务在单个 BE 节点上的计算资源和内 ## Workload Group 使用 -1. 开启 experimental_enable_workload_group 配置项,在 fe.conf 中设置: +1. 手动创建名为`normal`的Workload Group,该Group不可删除。也可以在打开Workload Group开关后重启FE,会自动创建这个Group。 +``` +create workload group if not exists normal +properties ( + "cpu_share"="10", + "memory_limit"="30%", + "enable_memory_overcommit"="true" +); +``` + +2. 开启 experimental_enable_workload_group 配置项,在 fe.conf 中设置: ```bash experimental_enable_workload_group=true ``` -在开启该配置后系统会自动创建名为`normal`的默认 Workload Group。 -2. 创建 Workload Group: +3. 创建 Workload Group: ``` create workload group if not exists g1 @@ -60,13 +69,13 @@ properties ( 创建 workload group 详细可参考:[CREATE-WORKLOAD-GROUP](../../sql-manual/sql-reference/Data-Definition-Statements/Create/CREATE-WORKLOAD-GROUP),另删除 Workload Group 可参考[DROP-WORKLOAD-GROUP](../../sql-manual/sql-reference/Data-Definition-Statements/Drop/DROP-WORKLOAD-GROUP);修改 Workload Group 可参考:[ALTER-WORKLOAD-GROUP](../../sql-manual/sql-reference/Data-Definition-Statements/Alter/ALTER-WORKLOAD-GROUP);查看 Workload Group 可参考:[WORKLOAD_GROUPS()](../../sql-manual/sql-functions/table-functions/workload [...] -3. 开启 Pipeline 执行引擎,Workload Group CPU 隔离基于 Pipeline 执行引擎实现,因此需开启 Session 变量: +4. 开启 Pipeline 执行引擎,Workload Group CPU 隔离基于 Pipeline 执行引擎实现,因此需开启 Session 变量: ```bash set experimental_enable_pipeline_engine = true; ``` -4. 绑定 Workload Group。 +5. 绑定 Workload Group。 * 通过设置 user property 将 user 默认绑定到 workload group,默认为`normal`: ``` @@ -84,7 +93,7 @@ session 变量`workload_group`优先于 user property `default_workload_group`, 如果是非 Admin 用户,需要先执行[SHOW-WORKLOAD-GROUPS](../../sql-manual/sql-reference/Show-Statements/SHOW-WORKLOAD-GROUPS) 确认下当前用户能否看到该 workload group,不能看到的 workload group 可能不存在或者当前用户没有权限,执行查询时会报错。给 worklaod group 授权参考:[grant 语句](../../sql-manual/sql-reference/Account-Management-Statements/GRANT)。 -5. 执行查询,查询将关联到指定的 Workload Group。 +6. 执行查询,查询将关联到指定的 Workload Group。 ### 查询排队功能 ``` diff --git a/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/admin-manual/resource-admin/multi-tenant.md b/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/admin-manual/resource-admin/multi-tenant.md index f4daa8d8ddc..7d01895ab02 100644 --- a/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/admin-manual/resource-admin/multi-tenant.md +++ b/i18n/zh-CN/docusaurus-plugin-content-docs/version-2.1/admin-manual/resource-admin/multi-tenant.md @@ -180,6 +180,8 @@ FE 不参与用户数据的处理计算等工作,因此是一个资源消耗 2. CPU 限制 + >注:从Doris 2.1 之后,cpu_resource_limit 将逐渐被 workload group 替代,因此不建议使用该参数。 + 用户可以通过以下方式限制查询的 CPU 资源: ```sql @@ -192,7 +194,6 @@ FE 不参与用户数据的处理计算等工作,因此是一个资源消耗 `cpu_resource_limit` 的取值是一个相对值,取值越大则能够使用的 CPU 资源越多。但一个查询能使用的 CPU 上限也取决于表的分区分桶数。原则上,一个查询的最大 CPU 使用量和查询涉及到的 tablet 数量正相关。极端情况下,假设一个查询仅涉及到一个 tablet,则即使 `cpu_resource_limit` 设置一个较大值,也仅能使用 1 个 CPU 资源。 通过内存和 CPU 的资源限制。我们可以在一个资源组内,将用户的查询进行更细粒度的资源划分。比如我们可以让部分时效性要求不高,但是计算量很大的离线任务使用更少的 CPU 资源和更多的内存资源。而部分延迟敏感的在线任务,使用更多的 CPU 资源以及合理的内存资源。 -注:在 Doris 2.1 之后,cpu_resource_limit 将逐渐被 workload group 替代。 ## 最佳实践和向前兼容 diff --git a/versioned_docs/version-2.0/admin-manual/resource-admin/workload-group.md b/versioned_docs/version-2.0/admin-manual/resource-admin/workload-group.md index 3a148d2c9c6..c6bdd18d02d 100644 --- a/versioned_docs/version-2.0/admin-manual/resource-admin/workload-group.md +++ b/versioned_docs/version-2.0/admin-manual/resource-admin/workload-group.md @@ -38,13 +38,22 @@ The workload group can limit the use of compute and memory resources on a single ## Workload group usage -1. Enable the experimental_enable_workload_group configuration, set in fe.conf to +1. Manually create a Workload Group named 'normal', which cannot be deleted. You can also restart FE after set FE config ```experimental_enable_workload_group=true``` to automatically create this group. ``` -experimental_enable_workload_group=true +create workload group if not exists normal +properties ( + "cpu_share"="10", + "memory_limit"="30%", + "enable_memory_overcommit"="true" +); +``` + +2. Enable the experimental_enable_workload_group configuration, set in fe.conf to ``` -The system will automatically create a default workload group named ``normal`` after this configuration is enabled. +experimental_enable_workload_group=true +``` -2. To create a workload group: +3. To create a workload group: ``` create workload group if not exists g1 properties ( @@ -56,12 +65,12 @@ properties ( For details on creating a workload group, see [CREATE-WORKLOAD-GROUP](../../sql-manual/sql-reference/Data-Definition-Statements/Create/CREATE-WORKLOAD-GROUP.md), and to delete a workload group, refer to [DROP-WORKLOAD-GROUP](../../sql-manual/sql-reference/Data-Definition-Statements/Drop/DROP-WORKLOAD-GROUP.md); to modify a workload group, refer to [ALTER-WORKLOAD-GROUP](../../sql-manual/sql-reference/Data-Definition-Statements/Alter/ALTER-WORKLOAD-GROUP.md); to view the workload group, r [...] -3. turn on the pipeline execution engine, the workload group cpu isolation is based on the implementation of the pipeline execution engine, so you need to turn on the session variable: +4. turn on the pipeline execution engine, the workload group cpu isolation is based on the implementation of the pipeline execution engine, so you need to turn on the session variable: ``` set experimental_enable_pipeline_engine = true. ``` -4. Bind the workload group. +5. Bind the workload group. * Bind the user to the workload group by default by setting the user property to ``normal``. ``` set property 'default_workload_group' = 'g1'. @@ -75,7 +84,7 @@ session variable `workload_group` takes precedence over user property `default_w If you are a non-admin user, you need to execute [SHOW-WORKLOAD-GROUPS](../../sql-manual/sql-reference/Show-Statements/SHOW-WORKLOAD-GROUPS.md) to check if the current user can see the workload group, if not, the workload group may not exist or the current user does not have permission to execute the query. If you cannot see the workload group, the workload group may not exist or the current user does not have privileges. To authorize the workload group, refer to: [grant statement](../.. [...] -5. Execute the query, which will be associated with the g1 workload group. +6. Execute the query, which will be associated with the g1 workload group. ### Query queue Function ``` diff --git a/versioned_docs/version-2.1/admin-manual/resource-admin/multi-tenant.md b/versioned_docs/version-2.1/admin-manual/resource-admin/multi-tenant.md index ef7cf47613f..bfdc4e219ec 100644 --- a/versioned_docs/version-2.1/admin-manual/resource-admin/multi-tenant.md +++ b/versioned_docs/version-2.1/admin-manual/resource-admin/multi-tenant.md @@ -181,6 +181,8 @@ At present, Doris's resource restrictions on single queries are mainly divided i 2. CPU limitations + >Note: Since Doris 2.1, cpu_resource_limit will gradually be replaced by workload group, so it is not recommended to use it. + Users can limit the CPU resources of the query in the following ways: ```sql --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org