[GitHub] [incubator-doris] yiguolei commented on a diff in pull request #8855: [Refactor] Change ALL OLAPStatus to Status
yiguolei commented on code in PR #8855:
URL: https://github.com/apache/incubator-doris/pull/8855#discussion_r847008717
##
be/src/agent/agent_server.cpp:
##
@@ -208,17 +208,14 @@ void AgentServer::submit_tasks(TAgentResult& agent_result,
void AgentServer::make_snapshot(TAgentResult& t_agent_result,
const TSnapshotRequest& snapshot_request) {
-Status ret_st;
string snapshot_path;
bool allow_incremental_clone = false;
-OLAPStatus err_code =
+Status err_code =
SnapshotManager::instance()->make_snapshot(snapshot_request,
&snapshot_path, &allow_incremental_clone);
-if (err_code != OLAP_SUCCESS) {
+if (!err_code) {
Review Comment:
I think we should not modify variable name in this PR. Because there maybe
multiple status variable in same function. Modify the variable name may cause
logic bug. And also, there are too many variables, it may take lot of time.
We may refactor the status variable name in other PRs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #8949: [feature-wip](array-type) permute impl for column array
github-actions[bot] commented on PR #8949: URL: https://github.com/apache/incubator-doris/pull/8949#issuecomment-1094633112 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #8949: [feature-wip](array-type) permute impl for column array
github-actions[bot] commented on PR #8949: URL: https://github.com/apache/incubator-doris/pull/8949#issuecomment-1094633075 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #8925: [fix][mem tracker] Fix MemTracker null pointer in vectorized
github-actions[bot] commented on PR #8925: URL: https://github.com/apache/incubator-doris/pull/8925#issuecomment-1094637302 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #8925: [fix][mem tracker] Fix MemTracker null pointer in vectorized
github-actions[bot] commented on PR #8925: URL: https://github.com/apache/incubator-doris/pull/8925#issuecomment-1094637346 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] hf200012 opened a new pull request, #8953: [Refactor][Doc]DML help doc（INSERT，UPDATE，DELETE，OUTFILE）
hf200012 opened a new pull request, #8953: URL: https://github.com/apache/incubator-doris/pull/8953 DML help doc # Proposed changes Issue Number: close #xxx ## Problem Summary: Describe the overview of changes. ## Checklist(Required) 1. Does it affect the original behavior: (Yes/No/I Don't know) 2. Has unit tests been added: (Yes/No/No Need) 3. Has document been added or modified: (Yes/No/No Need) 4. Does it need to update dependencies: (Yes/No) 5. Are there any changes that cannot be rolled back: (Yes/No) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] EmmyMiao87 commented on a diff in pull request #8906: [feat](sql) support star except modifier
EmmyMiao87 commented on code in PR #8906: URL: https://github.com/apache/incubator-doris/pull/8906#discussion_r846036375 ## fe/fe-core/src/main/cup/sql_parser.cup: ## @@ -544,6 +544,7 @@ precedence left KW_LIKE, KW_REGEXP; precedence left EQUAL, LESSTHAN, GREATERTHAN; precedence left ADD, SUBTRACT; precedence left AT, STAR, DIVIDE, MOD, KW_DIV; +precedence left KW_EXCEPT; Review Comment: What is the purpose of adding this line? ## fe/fe-core/src/main/cup/sql_parser.cup: ## @@ -4007,6 +4015,12 @@ select_sublist ::= list.addItem(SelectListItem.createStarItem(null)); RESULT = list; :} +| select_sublist:list COMMA STAR except_list:exceptList Review Comment: What's this mean ```select a, * except (a)``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] littleeleventhwolf commented on a diff in pull request #8906: [feat](sql) support star except modifier
littleeleventhwolf commented on code in PR #8906:
URL: https://github.com/apache/incubator-doris/pull/8906#discussion_r847029355
##
fe/fe-core/src/main/cup/sql_parser.cup:
##
@@ -544,6 +544,7 @@ precedence left KW_LIKE, KW_REGEXP;
precedence left EQUAL, LESSTHAN, GREATERTHAN;
precedence left ADD, SUBTRACT;
precedence left AT, STAR, DIVIDE, MOD, KW_DIV;
+precedence left KW_EXCEPT;
Review Comment:
Because `select * except (field)` always match syntax below.
```
select_list ::=
... ...
| STAR
{:
SelectList list = new SelectList();
list.addItem(SelectListItem.createStarItem(null));
RESULT = list;
:}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #8952: [Refactor][Doc]Utility-Statements SQL Help
github-actions[bot] commented on PR #8952: URL: https://github.com/apache/incubator-doris/pull/8952#issuecomment-1094658793 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #8952: [Refactor][Doc]Utility-Statements SQL Help
github-actions[bot] commented on PR #8952: URL: https://github.com/apache/incubator-doris/pull/8952#issuecomment-1094658836 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] littleeleventhwolf commented on a diff in pull request #8906: [feat](sql) support star except modifier
littleeleventhwolf commented on code in PR #8906: URL: https://github.com/apache/incubator-doris/pull/8906#discussion_r847033485 ## fe/fe-core/src/main/cup/sql_parser.cup: ## @@ -4007,6 +4015,12 @@ select_sublist ::= list.addItem(SelectListItem.createStarItem(null)); RESULT = list; :} +| select_sublist:list COMMA STAR except_list:exceptList Review Comment: It's up to the user to decide their own select_list. If user give sql `select a, * except (a)`, field `a` before comma also be projected, `* except (a)` after comma will remove `a` from all columns and then be projected. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris-manager] hf200012 merged pull request #40: [Fix] Fix the disk occupancy display error and initialize error
hf200012 merged PR #40: URL: https://github.com/apache/incubator-doris-manager/pull/40 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[incubator-doris-manager] branch master updated: [Fix] Fix the disk occupancy display error and initialize error (#40)
This is an automated email from the ASF dual-hosted git repository.
jiafengzheng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-doris-manager.git
The following commit(s) were added to refs/heads/master by this push:
new 533b267 [Fix] Fix the disk occupancy display error and initialize
error (#40)
533b267 is described below
commit 533b267c3d9009824c7871a29ca79fafc1b5b7c4
Author: zhengbowen <101616192+zhengbowe...@users.noreply.github.com>
AuthorDate: Mon Apr 11 15:51:50 2022 +0800
[Fix] Fix the disk occupancy display error and initialize error (#40)
---
frontend/src/common/common.data.ts| 2 +-
frontend/src/hooks/use-auth.ts| 2 +-
.../src/routes/cluster/components/liquid-fill-chart/index.tsx | 8
frontend/src/routes/cluster/overview/index.tsx| 2 +-
frontend/src/routes/initialize/auths/auth.tsx | 4 ++--
.../routes/initialize/auths/components/admin-user/admin-user.tsx | 2 +-
frontend/src/routes/initialize/initialize-set-type.tsx| 6 +++---
frontend/src/routes/initialize/initialize.tsx | 4 ++--
frontend/src/routes/passport/login.tsx| 4
9 files changed, 19 insertions(+), 15 deletions(-)
diff --git a/frontend/src/common/common.data.ts
b/frontend/src/common/common.data.ts
index d3572e1..cda7ec7 100644
--- a/frontend/src/common/common.data.ts
+++ b/frontend/src/common/common.data.ts
@@ -92,5 +92,5 @@ export const FIRST_COLUMN_FIELD_TYPES = FIELD_TYPES.filter(
export enum AuthTypeEnum {
LDAP = 'ldap',
-LOCAL = 'local',
+STUDIO = 'studio',
}
diff --git a/frontend/src/hooks/use-auth.ts b/frontend/src/hooks/use-auth.ts
index aef9988..7c79a1c 100644
--- a/frontend/src/hooks/use-auth.ts
+++ b/frontend/src/hooks/use-auth.ts
@@ -35,7 +35,7 @@ export function useAuth() {
const res = await InitializeAPI.getInitProperties();
if (isSuccess(res)) {
setInitStep(res.data.initStep);
-setAuthType(res.data.auth_type === 'studio' ? AuthTypeEnum.LOCAL :
AuthTypeEnum.LDAP);
+setAuthType(res.data.auth_type === 'studio' ? AuthTypeEnum.STUDIO
: AuthTypeEnum.LDAP);
if (res.data.completed) {
localStorage.setItem('initialized', 'true');
setInitialized(true);
diff --git a/frontend/src/routes/cluster/components/liquid-fill-chart/index.tsx
b/frontend/src/routes/cluster/components/liquid-fill-chart/index.tsx
index b37b028..8962de0 100644
--- a/frontend/src/routes/cluster/components/liquid-fill-chart/index.tsx
+++ b/frontend/src/routes/cluster/components/liquid-fill-chart/index.tsx
@@ -19,7 +19,7 @@ import React from 'react';
import ReactEChartsCore from 'echarts-for-react/lib/core';
import * as echarts from 'echarts/core';
import 'echarts-liquidfill';
-import styles from './index.module.less'
+import styles from './index.module.less';
interface LiquidFillChartProps {
label?: string;
@@ -39,7 +39,7 @@ export default function LiquidFillChart(props:
LiquidFillChartProps) {
series: [
{
type: 'liquidFill',
-data: [value],
+data: [Number(value) / 100],
radius: '150vw',
backgroundStyle: {
color: '#fff',
@@ -62,8 +62,8 @@ export default function LiquidFillChart(props:
LiquidFillChartProps) {
},
waveAnimation: false,
label: {
-formatter: (param: any) => {
-return (param.value * 100).toFixed(2) +
'%';
+formatter: () => {
+return Number(value).toFixed(2) + '%';
},
fontSize: 25,
},
diff --git a/frontend/src/routes/cluster/overview/index.tsx
b/frontend/src/routes/cluster/overview/index.tsx
index aeb6215..49646b4 100644
--- a/frontend/src/routes/cluster/overview/index.tsx
+++ b/frontend/src/routes/cluster/overview/index.tsx
@@ -52,7 +52,7 @@ export default function ClusterOverview() {
},
});
const getClusterInfo = useCallback(
-(setStartLoading: boolean = false) => {
+(setStartLoading = false) => {
return runGetClusterInfo(
Promise.all([
SpaceAPI.spaceGet(userInfo.space_id + '').then(res => {
diff --git a/frontend/src/routes/initialize/auths/auth.tsx
b/frontend/src/routes/initialize/auths/auth.tsx
index e351d71..93b65d2 100644
--- a/frontend/src/routes/initialize/auths/auth.tsx
+++ b/frontend/src/routes/initialize/auths/auth.tsx
@@ -22,8 +22,8 @@ export fu
[GitHub] [incubator-doris] hf200012 commented on a diff in pull request #8951: [Refactor][doc] add doc for auth management and conf
hf200012 commented on code in PR #8951: URL: https://github.com/apache/incubator-doris/pull/8951#discussion_r847061446 ## new-docs/zh-CN/advanced/broker.md: ## @@ -24,4 +24,205 @@ specific language governing permissions and limitations under the License. --> -# Broker \ No newline at end of file +# Broker + +Broker 是 Doris 集群中一种可选进程，主要用于支持 Doris 读写远端存储上的文件和目录，如 HDFS、BOS 和 AFS 等。 + +Broker 通过提供一个 RPC 服务端口来提供服务，是一个无状态的 Java 进程，负责为远端存储的读写操作封装一些类 POSIX 的文件操作，如 open，pread，pwrite 等等。除此之外，Broker 不记录任何其他信息，所以包括远端存储的连接信息、文件信息、权限信息等等，都需要通过参数在 RPC 调用中传递给 Broker 进程，才能使得 Broker 能够正确读写文件。 + +Broker 仅作为一个数据通路，并不参与任何计算，因此仅需占用较少的内存。通常一个 Doris 系统中会部署一个或多个 Broker 进程。并且相同类型的 Broker 会组成一个组，并设定一个 **名称（Broker name）**。 + +Broker 在 Doris 系统架构中的位置如下： + +```text +++ ++ +| FE | | BE | ++-^--+ +--^-+ + | | + | | ++-v-v-+ +| Broker| ++--^--+ + | + | ++--v--+ +|HDFS/BOS/AFS | ++-+ +``` + +本文档主要介绍 Broker 在访问不同远端存储时需要的参数，如连接信息、权限认证信息等等。 + +## 支持的存储系统 + +不同的 Broker 类型支持不同的存储系统。 + +1. 社区版 HDFS + - 支持简单认证访问 + - 支持通过 kerberos 认证访问 + - 支持 HDFS HA 模式访问 +2. 百度 HDFS/AFS（开源版本不支持） Review Comment: Delete this information in Baidu community and directly introduce the version supported ## new-docs/zh-CN/admin-manual/user-privilege.md: ## @@ -24,4 +24,192 @@ specific language governing permissions and limitations under the License. --> -# 权限管理 \ No newline at end of file +# 权限管理 + +Doris 新的权限管理系统参照了 Mysql 的权限管理机制，做到了表级别细粒度的权限控制，基于角色的权限访问控制，并且支持白名单机制。 + +## 名词解释 + +1. 用户标识 user_identity + + 在权限系统中，一个用户被识别为一个 User Identity（用户标识）。用户标识由两部分组成：username 和 userhost。其中 username 为用户名，由英文大小写组成。userhost 表示该用户链接来自的 IP。user_identity 以 username@'userhost' 的方式呈现，表示来自 userhost 的 username。 + + user_identity 的另一种表现方式为 username@['domain']，其中 domain 为域名，可以通过 DNS 或 BNS（百度名字服务）解析为一组 ip。最终表现为一组 username@'userhost'，所以后面我们统一使用 username@'userhost' 来表示。 + +2. 权限 Privilege + + 权限作用的对象是节点、数据库或表。不同的权限代表不同的操作许可。 + +3. 角色 Role + + Doris可以创建自定义命名的角色。角色可以被看做是一组权限的集合。新创建的用户可以被赋予某一角色，则自动被赋予该角色所拥有的权限。后续对角色的权限变更，也会体现在所有属于该角色的用户权限上。 + +4. 用户属性 user_property + + 用户属性直接附属于某一用户，而不是用户标识。即 cmy@'192.%' 和 cmy@['domain'] 都拥有同一组用户属性，该属性属于用户 cmy，而不是 cmy@'192.%' 或 cmy@['domain']。 + + 用户属性包括但不限于： 用户最大连接数、导入集群配置等等。 + +## 支持的操作 + +1. 创建用户：CREATE USER +2. 删除用户：DROP USER +3. 授权：GRANT +4. 撤权：REVOKE +5. 创建角色：CREATE ROLE +6. 删除角色：DROP ROLE +7. 查看当前用户权限：SHOW GRANTS +8. 查看所有用户权限：SHOW ALL GRANTS +9. 查看已创建的角色：SHOW ROLES +10. 查看用户属性：SHOW PROPERTY + +关于以上命令的详细帮助，可以通过 mysql 客户端连接 Doris 后，使用 help + command 获取帮助。如 `HELP CREATE USER`。 + +## 权限类型 + +Doris 目前支持以下几种权限 + +1. Node_priv + + 节点变更权限。包括 FE、BE、BROKER 节点的添加、删除、下线等操作。目前该权限只能授予 Root 用户。 + +2. Grant_priv + + 权限变更权限。允许执行包括授权、撤权、添加/删除/变更 用户/角色 等操作。 + +3. Select_priv + + 对数据库、表的只读权限。 + +4. Load_priv + + 对数据库、表的写权限。包括 Load、Insert、Delete 等。 + +5. Alter_priv + + 对数据库、表的更改权限。包括重命名 库/表、添加/删除/变更 列、添加/删除 分区等操作。 + +6. Create_priv + + 创建数据库、表、视图的权限。 + +7. Drop_priv + + 删除数据库、表、视图的权限。 + +8. Usage_priv + + 资源的使用权限。 + +## 权限层级 + +同时，根据权限适用范围的不同，我们将库表的权限分为以下三个层级： + +1. GLOBAL LEVEL：全局权限。即通过 GRANT 语句授予的 `*.*` 上的权限。被授予的权限适用于任意数据库中的任意表。 +2. DATABASE LEVEL：数据库级权限。即通过 GRANT 语句授予的 `db.*` 上的权限。被授予的权限适用于指定数据库中的任意表。 +3. TABLE LEVEL：表级权限。即通过 GRANT 语句授予的 `db.tbl` 上的权限。被授予的权限适用于指定数据库中的指定表。 + +将资源的权限分为以下两个层级： + +1. GLOBAL LEVEL：全局权限。即通过 GRANT 语句授予的 `*` 上的权限。被授予的权限适用于资源。 +2. RESOURCE LEVEL: 资源级权限。即通过 GRANT 语句授予的 `resource_name` 上的权限。被授予的权限适用于指定资源。 + +## ADMIN/GRANT 权限说明 + +ADMIN_PRIV 和 GRANT_PRIV 权限同时拥有**授予权限**的权限，较为特殊。这里对和这两个权限相关的操作逐一说明。 + +1. CREATE USER + - 拥有 ADMIN 权限，或任意层级的 GRANT 权限的用户可以创建新用户。 +2. DROP USER + - 只有 ADMIN 权限可以删除用户。 +3. CREATE/DROP ROLE + - 只有 ADMIN 权限可以创建角色。 +4. GRANT/REVOKE + - 拥有 ADMIN 权限，或者 GLOBAL 层级 GRANT 权限的用户，可以授予或撤销任意用户的权限。 + - 拥有 DATABASE 层级 GRANT 权限的用户，可以授予或撤销任意用户对指定数据库的权限。 + - 拥有 TABLE 层级 GRANT 权限的用户，可以授予或撤销任意用户对指定数据库中指定表的权限。 +5. SET PASSWORD + - 拥有 ADMIN 权限，或者 GLOBAL 层级 GRANT 权限的用户，可以设置任意用户的密码。 + - 普通用户可以设置自己对应的 UserIdentity 的密码。自己对应的 UserIdentity 可以通过 `SELECT CURRENT_USER();` 命令查看。 + - 拥有非 GLOBAL 层级 GRANT 权限的用户，不可以设置已存在用户的密码，仅能在创建用户时指定密码。 + +## 一些说明 + +1. Doris 初始化时，会自动创建如下用户和角色： + 1. operator 角色：该角色拥有 Node_priv 和 Admin_priv，即对Doris的所有权限。后续某个升级版本中，我们可能会将该角色的权限限制为 Node_priv，即仅授予节点变更权限。以满足某些云上部署需求。 + 2. admin 角色：该角色拥有 Admin_priv，即除节点变更以外的所有权限。 + 3. root@'%'：root 用户，允许从任意节点登陆，角色为 operator。 + 4. admin@'%'：admin 用户，允许从任意节点登陆，角色为 admin。 +2. 不支持删除或更改默认创建的角色或用户的权限。 + +3. operator 角色的用户有且只有一个。admin 角色的用户可以创建多个。 + +4. 一些可能产生冲突的操作说明 + + 1. 域名与ip冲突： + + 假设创建了如下用户： + + CREATE USER cmy@['domain']; + + 并且授权： + + GRANT SELECT_PRIV ON *.* TO cmy@['domain'] + + 该 domain 被解析为两个 ip：ip1 和 ip2 + + 假设之后，我们对 cmy@'ip1' 进行一次单独授权： + + GRANT ALTER_PRIV ON *.* TO cmy@'ip1'; + + 则 cmy@'ip1' 的权限会被修改为 SELECT_PRIV, ALTER_PRIV。并且当我们再次变更 cmy@['domain'] 的权限时，cmy@'ip1' 也不会跟随改变。 + + 2. 重复ip冲突： + + 假设创建了如下用户： + + CREATE USER cmy@'%' IDENTIFIE
[GitHub] [incubator-doris] Henry2SS commented on a diff in pull request #8919: [Feature] (sql digest) support sql digest
Henry2SS commented on code in PR #8919:
URL: https://github.com/apache/incubator-doris/pull/8919#discussion_r847072683
##
fe/fe-core/src/main/java/org/apache/doris/analysis/Expr.java:
##
@@ -871,12 +871,20 @@ public String toSql() {
return (printSqlInParens) ? "(" + toSqlImpl() + ")" : toSqlImpl();
}
+public String toDigest() {
+return (printSqlInParens) ? "(" + toDigestImpl() + ")" :
toDigestImpl();
+}
+
/**
* Returns a SQL string representing this expr. Subclasses should override
this method
* instead of toSql() to ensure that parenthesis are properly added around
the toSql().
*/
protected abstract String toSqlImpl();
+protected String toDigestImpl() {
+return toSqlImpl();
Review Comment:
Thx for your reply.
The abstract method `toSqlImpl()` will be overrided in each subclass. So
there is no need to throw out exceptions.
The only situation is that a newly added subclass, which its sql digest
needs to be different from sql impl, should override this method.
Added comments over this method to remind developers.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] morrySnow opened a new pull request, #8954: [enhancement] take all previous broadcast join consumption into account when choose join algorithm
morrySnow opened a new pull request, #8954:
URL: https://github.com/apache/incubator-doris/pull/8954
# Proposed changes
According to discussion in the issue #8695
https://github.com/apache/incubator-doris/pull/8695#discussion_r838442611
We need to take all broadcast cost in one fragment when compute current node
broadcast join consumption.
This patch is a simple implement to do this.
The algorithm is very simple:
```
long spaceUsedInFragment = spaceUsedByPreviousJoinNodes +
spaceUsedByCurrentNode;
if (spaceUsedInFragment <= threshold) {
doBroadcast = true;
} else {
doBroadcast = false;
}
```
we could update this algorithm to choose best split position in future.
## Checklist(Required)
1. Does it affect the original behavior: (Yes)
2. Has unit tests been added: (Yes)
3. Has document been added or modified: (No)
4. Does it need to update dependencies: (No)
5. Are there any changes that cannot be rolled back: (No)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] hf200012 opened a new pull request, #8955: [improvement]Disable mini load
hf200012 opened a new pull request, #8955: URL: https://github.com/apache/incubator-doris/pull/8955 # Proposed changes Issue Number: close #xxx ## Problem Summary: Describe the overview of changes. ## Checklist(Required) 1. Does it affect the original behavior: (Yes/No/I Don't know) 2. Has unit tests been added: (Yes/No/No Need) 3. Has document been added or modified: (Yes/No/No Need) 4. Does it need to update dependencies: (Yes/No) 5. Are there any changes that cannot be rolled back: (Yes/No) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #8955: [improvement]Disable mini load
github-actions[bot] commented on PR #8955: URL: https://github.com/apache/incubator-doris/pull/8955#issuecomment-1094738116 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #8955: [improvement]Disable mini load
github-actions[bot] commented on PR #8955: URL: https://github.com/apache/incubator-doris/pull/8955#issuecomment-1094738190 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] jaingmengmeng opened a new pull request, #8956: [docs]correct spelling of the word 'LABEL' #3265
jaingmengmeng opened a new pull request, #8956: URL: https://github.com/apache/incubator-doris/pull/8956 # Proposed changes Issue Number: close #3265 ## Problem Summary: Doris document description error for version 0.11 English and Chinese ## Checklist(Required) 1. Does it affect the original behavior: No 2. Has unit tests been added: No Need 3. Has document been added or modified: Yes 4. Does it need to update dependencies: No 5. Are there any changes that cannot be rolled back: No -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] JNSimba commented on a diff in pull request #8951: [Refactor][doc] add doc for auth management and conf
JNSimba commented on code in PR #8951: URL: https://github.com/apache/incubator-doris/pull/8951#discussion_r847130139 ## new-docs/zh-CN/admin-manual/user-privilege.md: ## @@ -24,4 +24,192 @@ specific language governing permissions and limitations under the License. --> -# 权限管理 \ No newline at end of file +# 权限管理 + +Doris 新的权限管理系统参照了 Mysql 的权限管理机制，做到了表级别细粒度的权限控制，基于角色的权限访问控制，并且支持白名单机制。 + +## 名词解释 + +1. 用户标识 user_identity + + 在权限系统中，一个用户被识别为一个 User Identity（用户标识）。用户标识由两部分组成：username 和 userhost。其中 username 为用户名，由英文大小写组成。userhost 表示该用户链接来自的 IP。user_identity 以 username@'userhost' 的方式呈现，表示来自 userhost 的 username。 + + user_identity 的另一种表现方式为 username@['domain']，其中 domain 为域名，可以通过 DNS 或 BNS（百度名字服务）解析为一组 ip。最终表现为一组 username@'userhost'，所以后面我们统一使用 username@'userhost' 来表示。 + +2. 权限 Privilege + + 权限作用的对象是节点、数据库或表。不同的权限代表不同的操作许可。 + +3. 角色 Role + + Doris可以创建自定义命名的角色。角色可以被看做是一组权限的集合。新创建的用户可以被赋予某一角色，则自动被赋予该角色所拥有的权限。后续对角色的权限变更，也会体现在所有属于该角色的用户权限上。 + +4. 用户属性 user_property + + 用户属性直接附属于某一用户，而不是用户标识。即 cmy@'192.%' 和 cmy@['domain'] 都拥有同一组用户属性，该属性属于用户 cmy，而不是 cmy@'192.%' 或 cmy@['domain']。 + + 用户属性包括但不限于： 用户最大连接数、导入集群配置等等。 + +## 支持的操作 + +1. 创建用户：CREATE USER +2. 删除用户：DROP USER +3. 授权：GRANT +4. 撤权：REVOKE +5. 创建角色：CREATE ROLE +6. 删除角色：DROP ROLE +7. 查看当前用户权限：SHOW GRANTS +8. 查看所有用户权限：SHOW ALL GRANTS +9. 查看已创建的角色：SHOW ROLES +10. 查看用户属性：SHOW PROPERTY + +关于以上命令的详细帮助，可以通过 mysql 客户端连接 Doris 后，使用 help + command 获取帮助。如 `HELP CREATE USER`。 + +## 权限类型 + +Doris 目前支持以下几种权限 + +1. Node_priv + + 节点变更权限。包括 FE、BE、BROKER 节点的添加、删除、下线等操作。目前该权限只能授予 Root 用户。 + +2. Grant_priv + + 权限变更权限。允许执行包括授权、撤权、添加/删除/变更 用户/角色 等操作。 + +3. Select_priv + + 对数据库、表的只读权限。 + +4. Load_priv + + 对数据库、表的写权限。包括 Load、Insert、Delete 等。 + +5. Alter_priv + + 对数据库、表的更改权限。包括重命名 库/表、添加/删除/变更 列、添加/删除 分区等操作。 + +6. Create_priv + + 创建数据库、表、视图的权限。 + +7. Drop_priv + + 删除数据库、表、视图的权限。 + +8. Usage_priv + + 资源的使用权限。 + +## 权限层级 + +同时，根据权限适用范围的不同，我们将库表的权限分为以下三个层级： + +1. GLOBAL LEVEL：全局权限。即通过 GRANT 语句授予的 `*.*` 上的权限。被授予的权限适用于任意数据库中的任意表。 +2. DATABASE LEVEL：数据库级权限。即通过 GRANT 语句授予的 `db.*` 上的权限。被授予的权限适用于指定数据库中的任意表。 +3. TABLE LEVEL：表级权限。即通过 GRANT 语句授予的 `db.tbl` 上的权限。被授予的权限适用于指定数据库中的指定表。 + +将资源的权限分为以下两个层级： + +1. GLOBAL LEVEL：全局权限。即通过 GRANT 语句授予的 `*` 上的权限。被授予的权限适用于资源。 +2. RESOURCE LEVEL: 资源级权限。即通过 GRANT 语句授予的 `resource_name` 上的权限。被授予的权限适用于指定资源。 + +## ADMIN/GRANT 权限说明 + +ADMIN_PRIV 和 GRANT_PRIV 权限同时拥有**授予权限**的权限，较为特殊。这里对和这两个权限相关的操作逐一说明。 + +1. CREATE USER + - 拥有 ADMIN 权限，或任意层级的 GRANT 权限的用户可以创建新用户。 +2. DROP USER + - 只有 ADMIN 权限可以删除用户。 +3. CREATE/DROP ROLE + - 只有 ADMIN 权限可以创建角色。 +4. GRANT/REVOKE + - 拥有 ADMIN 权限，或者 GLOBAL 层级 GRANT 权限的用户，可以授予或撤销任意用户的权限。 + - 拥有 DATABASE 层级 GRANT 权限的用户，可以授予或撤销任意用户对指定数据库的权限。 + - 拥有 TABLE 层级 GRANT 权限的用户，可以授予或撤销任意用户对指定数据库中指定表的权限。 +5. SET PASSWORD + - 拥有 ADMIN 权限，或者 GLOBAL 层级 GRANT 权限的用户，可以设置任意用户的密码。 + - 普通用户可以设置自己对应的 UserIdentity 的密码。自己对应的 UserIdentity 可以通过 `SELECT CURRENT_USER();` 命令查看。 + - 拥有非 GLOBAL 层级 GRANT 权限的用户，不可以设置已存在用户的密码，仅能在创建用户时指定密码。 + +## 一些说明 + +1. Doris 初始化时，会自动创建如下用户和角色： + 1. operator 角色：该角色拥有 Node_priv 和 Admin_priv，即对Doris的所有权限。后续某个升级版本中，我们可能会将该角色的权限限制为 Node_priv，即仅授予节点变更权限。以满足某些云上部署需求。 + 2. admin 角色：该角色拥有 Admin_priv，即除节点变更以外的所有权限。 + 3. root@'%'：root 用户，允许从任意节点登陆，角色为 operator。 + 4. admin@'%'：admin 用户，允许从任意节点登陆，角色为 admin。 +2. 不支持删除或更改默认创建的角色或用户的权限。 + +3. operator 角色的用户有且只有一个。admin 角色的用户可以创建多个。 + +4. 一些可能产生冲突的操作说明 + + 1. 域名与ip冲突： + + 假设创建了如下用户： + + CREATE USER cmy@['domain']; + + 并且授权： + + GRANT SELECT_PRIV ON *.* TO cmy@['domain'] + + 该 domain 被解析为两个 ip：ip1 和 ip2 + + 假设之后，我们对 cmy@'ip1' 进行一次单独授权： + + GRANT ALTER_PRIV ON *.* TO cmy@'ip1'; + + 则 cmy@'ip1' 的权限会被修改为 SELECT_PRIV, ALTER_PRIV。并且当我们再次变更 cmy@['domain'] 的权限时，cmy@'ip1' 也不会跟随改变。 + + 2. 重复ip冲突： + + 假设创建了如下用户： + + CREATE USER cmy@'%' IDENTIFIED BY "12345"; + + CREATE USER cmy@'192.%' IDENTIFIED BY "abcde"; + + 在优先级上，'192.%' 优先于 '%'，因此，当用户 cmy 从 192.168.1.1 这台机器尝试使用密码 '12345' 登陆 Doris 会被拒绝。 + +5. 忘记密码 + + 如果忘记了密码无法登陆 Doris，可以在 Doris FE 节点所在机器，使用如下命令无密码登陆 Doris： + + `mysql-client -h 127.0.0.1 -P query_port -uroot` + + 登陆后，可以通过 SET PASSWORD 命令重置密码。 + +6. 任何用户都不能重置 root 用户的密码，除了 root 用户自己。 + +7. ADMIN_PRIV 权限只能在 GLOBAL 层级授予或撤销。 + +8. 拥有 GLOBAL 层级 GRANT_PRIV 其实等同于拥有 ADMIN_PRIV，因为该层级的 GRANT_PRIV 有授予任意权限的权限，请谨慎使用。 + +9. `current_user()` 和 `user()` + + 用户可以通过 `SELECT current_user();` 和 `SELECT user();` 分别查看 `current_user` 和 `user`。其中 `current_user` 表示当前用户是以哪种身份通过认证系统的，而 `user` 则是用户当前实际的 `user_identity`。举例说明： + + 假设创建了 `user1@'192.%'` 这个用户，然后以为来自 192.168.10.1 的用户 user1 登陆了系统，则此时的 `current_user` 为 `user1@'192.%'`，而 `user` 为 `user1@'192.168.10.1'`。 + + 所有的权限都是赋予某一个 `current_user` 的，真实用户拥有对应的 `current_user` 的所有权限。 + +## 最佳实践 + +这里举例一些 Doris 权限系统的使用场景。 + +1. 场景一 + + Doris 集群的使用者分为管理员（Admin）、开发工程师（RD）和用户（Client）。其中管理员拥有整个集群的所有权限，主要负责集群的搭建、节点管理等。开发工程师负责业务建模，包括建库建表、数据的导入和修改等。用户访问不同的数据库和表来获取数据。 + + 在这种场景下，可以为管理员赋予 ADMIN 权限或 GRANT 权限。对 RD 赋予对任意或指定数据库表的 CREATE、DROP、ALTER
[GitHub] [incubator-doris] JNSimba commented on a diff in pull request #8951: [Refactor][doc] add doc for auth management and conf
JNSimba commented on code in PR #8951: URL: https://github.com/apache/incubator-doris/pull/8951#discussion_r847130537 ## new-docs/zh-CN/advanced/broker.md: ## @@ -24,4 +24,205 @@ specific language governing permissions and limitations under the License. --> -# Broker \ No newline at end of file +# Broker + +Broker 是 Doris 集群中一种可选进程，主要用于支持 Doris 读写远端存储上的文件和目录，如 HDFS、BOS 和 AFS 等。 + +Broker 通过提供一个 RPC 服务端口来提供服务，是一个无状态的 Java 进程，负责为远端存储的读写操作封装一些类 POSIX 的文件操作，如 open，pread，pwrite 等等。除此之外，Broker 不记录任何其他信息，所以包括远端存储的连接信息、文件信息、权限信息等等，都需要通过参数在 RPC 调用中传递给 Broker 进程，才能使得 Broker 能够正确读写文件。 + +Broker 仅作为一个数据通路，并不参与任何计算，因此仅需占用较少的内存。通常一个 Doris 系统中会部署一个或多个 Broker 进程。并且相同类型的 Broker 会组成一个组，并设定一个 **名称（Broker name）**。 + +Broker 在 Doris 系统架构中的位置如下： + +```text +++ ++ +| FE | | BE | ++-^--+ +--^-+ + | | + | | ++-v-v-+ +| Broker| ++--^--+ + | + | ++--v--+ +|HDFS/BOS/AFS | ++-+ +``` + +本文档主要介绍 Broker 在访问不同远端存储时需要的参数，如连接信息、权限认证信息等等。 + +## 支持的存储系统 + +不同的 Broker 类型支持不同的存储系统。 + +1. 社区版 HDFS + - 支持简单认证访问 + - 支持通过 kerberos 认证访问 + - 支持 HDFS HA 模式访问 +2. 百度 HDFS/AFS（开源版本不支持） Review Comment: thanks ,already modified -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #8953: [Refactor][Doc]DML help doc（INSERT，UPDATE，DELETE，OUTFILE）
github-actions[bot] commented on PR #8953: URL: https://github.com/apache/incubator-doris/pull/8953#issuecomment-1094817884 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #8953: [Refactor][Doc]DML help doc（INSERT，UPDATE，DELETE，OUTFILE）
github-actions[bot] commented on PR #8953: URL: https://github.com/apache/incubator-doris/pull/8953#issuecomment-1094817937 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] smallhibiscus opened a new pull request, #8957: [Refactor][doc] Modify the documentation of alter, backup and restore in DDL
smallhibiscus opened a new pull request, #8957: URL: https://github.com/apache/incubator-doris/pull/8957 # Proposed changes Issue Number: close #xxx ## Problem Summary: Modify the en and zh-CN documentation of alter, backup and restore in DDL. ## Checklist(Required) 1. Does it affect the original behavior: (Yes/No/I Don't know) 2. Has unit tests been added: (Yes/No/No Need) 3. Has document been added or modified: (Yes/No/No Need) 4. Does it need to update dependencies: (Yes/No) 5. Are there any changes that cannot be rolled back: (Yes/No) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #8956: [docs]correct spelling of the word 'LABEL' #3265
github-actions[bot] commented on PR #8956: URL: https://github.com/apache/incubator-doris/pull/8956#issuecomment-1094822285 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #8956: [docs]correct spelling of the word 'LABEL' #3265
github-actions[bot] commented on PR #8956: URL: https://github.com/apache/incubator-doris/pull/8956#issuecomment-1094822327 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] 924060929 commented on a diff in pull request #8930: [Enhancement][Regression-Test] enhancement for regression-test framework
924060929 commented on code in PR #8930:
URL: https://github.com/apache/incubator-doris/pull/8930#discussion_r847164901
##
regression-test/framework/src/main/groovy/org/apache/doris/regression/util/JdbcUtils.groovy:
##
@@ -18,37 +18,40 @@
package org.apache.doris.regression.util
import com.google.common.collect.ImmutableList
+import org.apache.doris.regression.util.Metaholder
import java.sql.Connection
import java.sql.ResultSet
+import java.sql.ResultSetMetaData
class JdbcUtils {
-static List> executeToList(Connection conn, String sql) {
+static List> executeToList(Connection conn, String sql,
Metaholder holder) {
conn.prepareStatement(sql).withCloseable { stmt ->
Review Comment:
suggest `new Metaholder` in `JdbcUtils` and return, and you can extract the
interest metadata out of the `Connection` life cycle safely.
e.g.
```groovy
class Metadata {
public final List columnNames;
public final List columnTypes;
}
static Tuple2>, Metadata> executeToList(Connection conn,
String sql) {
def metadata = new Metadata()
resultList = ...
metadata.columnNames = ...
metadata.columnTypes = ...
return [resultList, metadata]
}
```
then, you can take result by this code
```groovy
def (result, metadata) = JdbcUtils.executeToList(connnection, sql)
```
##
regression-test/framework/src/main/groovy/org/apache/doris/regression/util/OutputUtils.groovy:
##
@@ -44,9 +53,36 @@ class OutputUtils {
return writer.toString()
}
+static String checkCell(String info, int line, String expect_cell, String
real_cell, String data_type) {
+if(data_type == "FLOAT" || data_type == "DOUBLE") {
Review Comment:
suggest change to camel case
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] yangzhg opened a new pull request, #8958: [refactor] Unify all unit tests into one binary file
yangzhg opened a new pull request, #8958: URL: https://github.com/apache/incubator-doris/pull/8958 # Proposed changes 1. solved the previous delayed unit test file size is too large (1.7G+) and the unit test link time is too long problem problems 2. Unify all unit tests into one file to significantly reduce unit test execution time to less than 3 mins 3. temporarily disable `stream_load_test.cpp`, `metrics_action_test.cpp`, `load_channel_mgr_test.cpp` because it will re-implement part of the code and affect other tests ## Checklist(Required) 1. Does it affect the original behavior: (No) 2. Has unit tests been added: (No Need) 3. Has document been added or modified: No Need) 4. Does it need to update dependencies: (No) 5. Are there any changes that cannot be rolled back: (No) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] xinyiZzz closed issue #8928: [Bug] No matching function with signature: hex(hll)
xinyiZzz closed issue #8928: [Bug] No matching function with signature: hex(hll) URL: https://github.com/apache/incubator-doris/issues/8928 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] xinyiZzz commented on issue #8928: [Bug] No matching function with signature: hex(hll)
xinyiZzz commented on issue #8928: URL: https://github.com/apache/incubator-doris/issues/8928#issuecomment-1094899784 thanks for answer @jackwener @cambyzju @HappenLee -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] qidaye commented on a diff in pull request #8808: [feature](cold-hot) support s3 resource
qidaye commented on code in PR #8808:
URL: https://github.com/apache/incubator-doris/pull/8808#discussion_r846551943
##
fe/fe-core/src/main/java/org/apache/doris/catalog/OdbcCatalogResource.java:
##
@@ -93,6 +93,29 @@ private void checkProperties(String propertiesKey) throws
DdlException {
}
+@Override
+public void modifyProperties(Map properties) throws
DdlException {
+// check properties
+String host = properties.remove(HOST);
+String port = properties.remove(PORT);
+String user = properties.remove(USER);
+String password = properties.remove(PASSWORD);
+String type = properties.remove(TYPE);
+String driver = properties.remove(DRIVER);
+
+if (!properties.isEmpty()) {
+throw new DdlException("Unknown ODBC catalog resource: " +
properties);
Review Comment:
Because we have multiple resources, we cannot check all types of resources
in the analysis phase, so I put the check in specific resouce.
I have moved the check to analysis phase.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] fwshrimp opened a new issue, #8959: [Bug] Hive UDF bitmap function and/or/xor return wrong result
fwshrimp opened a new issue, #8959: URL: https://github.com/apache/incubator-doris/issues/8959 ### Search before asking - [X] I had searched in the [issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and found no similar issues. ### Version Master branch ### What's Wrong? fe/hive-udf/src/main/java/org/apache/doris/udf/BitmapAndUDF.java fe/hive-udf/src/main/java/org/apache/doris/udf/BitmapOrUDF.java fe/hive-udf/src/main/java/org/apache/doris/udf/BitmapXorUDF.java return wrong result ### What You Expected? for BitmapAndUDF.java line 64 return BitmapValueUtil.serializeToBytes(bitmapValue1); need change to BitmapValueUtil.serializeToBytes(bitmapValue0); the same to BitmapOrUDF.java and BitmapXorUDF.java ### How to Reproduce? _No response_ ### Anything Else? _No response_ ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] morningman commented on a diff in pull request #8958: [refactor] Unify all unit tests into one binary file
morningman commented on code in PR #8958:
URL: https://github.com/apache/incubator-doris/pull/8958#discussion_r847387912
##
be/src/geo/geo_functions.cpp:
##
@@ -236,7 +236,8 @@ void
GeoFunctions::st_contains_prepare(doris_udf::FunctionContext* ctx,
if (str->is_null) {
contains_ctx->is_null = true;
} else {
-contains_ctx->shapes[i] = GeoShape::from_encoded(str->ptr,
str->len);
+contains_ctx->shapes[i] =
+
std::shared_ptr(GeoShape::from_encoded(str->ptr, str->len));
Review Comment:
Why using shared_ptr instead of unique_ptr?
##
run-be-ut.sh:
##
@@ -242,15 +229,11 @@ if [ ${VECTORIZED_ONLY} -eq 1 ]; then
echo "Run Vectorized ut only"
export DORIS_TEST_BINARY_DIR=${DORIS_TEST_BINARY_DIR}/vec
fi
-
-test_files=`find ${DORIS_TEST_BINARY_DIR} -type f -perm -111 -name "*test"`
-
-for test in ${test_files[@]}
-do
-file_name=${test##*/}
-if [ -z $RUN_FILE ] || [ $file_name == $RUN_FILE ]; then
-echo "=== Run $file_name ==="
-$test --gtest_output=xml:${GTEST_OUTPUT_DIR}/${file_name}.xml
-fi
-done
-echo "=== Finished. Gtest output: ${GTEST_OUTPUT_DIR}"
+test=${DORIS_TEST_BINARY_DIR}doris_be_test
+echo "${FILTER}"
+if [ -f "$test" ]; then
+$test --gtest_output=xml:${GTEST_OUTPUT_DIR}/${file_name}.xml
--gtest_print_time=true "${FILTER}"
Review Comment:
`${file_name}` is not defined
##
be/test/CMakeLists.txt:
##
@@ -0,0 +1,389 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# where to put generated libraries
+set(LIBRARY_OUTPUT_PATH "${BUILD_DIR}/test")
+
+# where to put generated libraries
+set(EXECUTABLE_OUTPUT_PATH "${BUILD_DIR}/test")
+
+set(AGENT_TEST_FILES
+agent/utils_test.cpp
+# agent/agent_server_test.cpp
+# agent/cgroups_mgr_test.cpp
+# agent/heartbeat_server_test.cpp
+)
+set(COMMON_TEST_FILES
+common/resource_tls_test.cpp
+common/status_test.cpp
+common/config_test.cpp
+)
+set(ENV_TEST_FILES
+env/env_posix_test.cpp
+)
+
+set(EXEC_TEST_FILES
+exec/hash_table_test.cpp
+exec/olap_common_test.cpp
+exec/json_scanner_test.cpp
+exec/json_scanner_with_jsonpath_test.cpp
+exec/parquet_scanner_test.cpp
+exec/orc_scanner_test.cpp
+exec/plain_text_line_reader_uncompressed_test.cpp
+exec/plain_text_line_reader_gzip_test.cpp
+exec/plain_text_line_reader_bzip_test.cpp
+exec/plain_text_line_reader_lz4frame_test.cpp
+exec/broker_scanner_test.cpp
+exec/broker_scan_node_test.cpp
+exec/tablet_info_test.cpp
+exec/tablet_sink_test.cpp
+exec/buffered_reader_test.cpp
+exec/es_http_scan_node_test.cpp
+exec/es_predicate_test.cpp
+exec/es_query_builder_test.cpp
+exec/es_scan_reader_test.cpp
+exec/s3_reader_test.cpp
+exec/multi_bytes_separator_test.cpp
+# exec/hdfs_file_reader_test.cpp
Review Comment:
How to address these tests?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #8808: [feature](cold-hot) support s3 resource
github-actions[bot] commented on PR #8808: URL: https://github.com/apache/incubator-doris/pull/8808#issuecomment-1095148819 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] zbtzbtzbt commented on pull request #8958: [refactor] Unify all unit tests into one binary file
zbtzbtzbt commented on PR #8958: URL: https://github.com/apache/incubator-doris/pull/8958#issuecomment-1095149930 cool！ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] zbtzbtzbt commented on a diff in pull request #8947: [feature](statistics) Statistics derivation.Step 1:ScanNode implement…
zbtzbtzbt commented on code in PR #8947:
URL: https://github.com/apache/incubator-doris/pull/8947#discussion_r847418638
##
fe/fe-core/src/main/java/org/apache/doris/planner/OlapScanNode.java:
##
@@ -131,6 +131,7 @@ public class OlapScanNode extends ScanNode {
private int selectedPartitionNum = 0;
private Collection selectedPartitionIds = Lists.newArrayList();
private long totalBytes = 0;
+private long tmpRowCount = 0;
Review Comment:
what does this `tmpRowCount` mean ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] yiguolei merged pull request #8952: [Refactor][Doc]Utility-Statements SQL Help
yiguolei merged PR #8952: URL: https://github.com/apache/incubator-doris/pull/8952 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[incubator-doris] branch master updated: Utility-Statements SQL Help (#8952)
This is an automated email from the ASF dual-hosted git repository. yiguolei pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-doris.git The following commit(s) were added to refs/heads/master by this push: new 667eac9b7d Utility-Statements SQL Help (#8952) 667eac9b7d is described below commit 667eac9b7d89b2390e7e0e0ad2ce6c50435fdf1c Author: jiafeng.zhang AuthorDate: Tue Apr 12 08:44:16 2022 +0800 Utility-Statements SQL Help (#8952) Utility-Statements SQL Help --- .../Utility-Statements/DESCRIBE.md | 28 .../sql-reference-v2/Utility-Statements/HELP.md| 76 ++ .../sql-reference-v2/Utility-Statements/USE.md | 19 ++ .../Utility-Statements/DESCRIBE.md | 30 - .../sql-reference-v2/Utility-Statements/HELP.md| 75 + .../sql-reference-v2/Utility-Statements/USE.md | 19 ++ 6 files changed, 246 insertions(+), 1 deletion(-) diff --git a/new-docs/en/sql-manual/sql-reference-v2/Utility-Statements/DESCRIBE.md b/new-docs/en/sql-manual/sql-reference-v2/Utility-Statements/DESCRIBE.md index e854bdd6df..ae0f168073 100644 --- a/new-docs/en/sql-manual/sql-reference-v2/Utility-Statements/DESCRIBE.md +++ b/new-docs/en/sql-manual/sql-reference-v2/Utility-Statements/DESCRIBE.md @@ -26,10 +26,38 @@ under the License. ## DESCRIBE +### Name + +DESCRIBE + ### Description +This statement is used to display the schema information of the specified table + +grammar: + +```sql +DESC[RIBE] [db_name.]table_name [ALL]; + + +illustrate: + +1. If ALL is specified, the schemas of all indexes (rollup) of the table will be displayed + ### Example +1. Display the Base table schema + +```sql +DESC table_name; + + +2. Display the schema of all indexes of the table + +```sql +DESC db1.table_name ALL; + + ### Keywords DESCRIBE diff --git a/new-docs/en/sql-manual/sql-reference-v2/Utility-Statements/HELP.md b/new-docs/en/sql-manual/sql-reference-v2/Utility-Statements/HELP.md index 29a43b975a..33ba93c402 100644 --- a/new-docs/en/sql-manual/sql-reference-v2/Utility-Statements/HELP.md +++ b/new-docs/en/sql-manual/sql-reference-v2/Utility-Statements/HELP.md @@ -26,10 +26,86 @@ under the License. ## HELP +### Name + +HELP + ### Description +The directory of help can be queried by changing the command + +grammar: + + sql +HELP + + +All Doris commands can be listed with `help` + +```sql +List of all MySQL commands: +Note that all text commands must be first on line and end with ';' +? (\?) Synonym for `help'. +clear (\c) Clear the current input statement. +connect (\r) Reconnect to the server. Optional arguments are db and host. +delimiter (\d) Set statement delimiter. +edit (\e) Edit command with $EDITOR. +ego (\G) Send command to mysql server, display result vertically. +exit (\q) Exit mysql. Same as quit. +go (\g) Send command to mysql server. +help (\h) Display this help. +nopager (\n) Disable pager, print to stdout. +note (\t) Don't write into outfile. +pager (\P) Set PAGER [to_pager]. Print the query results via PAGER. +print (\p) Print current command. +prompt (\R) Change your mysql prompt. +quit (\q) Quit mysql. +rehash (\#) Rebuild completion hash. +source (\.) Execute an SQL script file. Takes a file name as an argument. +status (\s) Get status information from the server. +system (\!) Execute a system shell command. +tee (\T) Set outfile [to_outfile]. Append everything into given outfile. +use (\u) Use another database. Takes database name as argument. +charset (\C) Switch to another charset. Might be needed for processing binlog with multi-byte charsets. +warnings (\W) Show warnings after every statement. +nowarning (\w) Don't show warnings after every statement. +resetconnection(\x) Clean session context. + +For server side help, type 'help contents' + + +Get the Doris SQL help contents via `help contents` + +```sql +Many help items for your request exist. +To make a more specific request, please type 'help ', +where is one of the following +categories: + sql-functions + sql-statements + + ### Example +1. List all SQL help contents of Doris + + ```sql + help contents + + +2. The command to list all function directories of the Doris cluster + + ```sql + help sql-functions + + +3. List all functions under the date function + + ```sql + help date-time-functions + + + ### Keywords HELP diff --git a/new-docs/en/sql-manual/sql-reference-v2/Utility-Statements/USE.md b/new-docs/en/sql-manual/sql-reference-v2/Utility-Statements/USE.md index 00ce4dc298..79cab5508e 100644 --- a/new-docs/en/sql-manual/sql-reference-v2/Utility-Statements/USE.md +++ b/new-docs/en/sql-manual/sql-reference-v2/Utility-Statements/USE.md @@ -26,10 +26,29 @@ under the License. ## USE +### Name + +USE + ### Description +The USE command allows us to use the database
[GitHub] [incubator-doris] yiguolei closed issue #3265: Doris docment description error for version 0.11 English and Chinese
yiguolei closed issue #3265: Doris docment description error for version 0.11 English and Chinese URL: https://github.com/apache/incubator-doris/issues/3265 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] yiguolei merged pull request #8956: [docs]correct spelling of the word 'LABEL' #3265
yiguolei merged PR #8956: URL: https://github.com/apache/incubator-doris/pull/8956 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[incubator-doris] branch master updated: [docs]correct spelling of the word 'LABEL' #3265 (#8956)
This is an automated email from the ASF dual-hosted git repository. yiguolei pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-doris.git The following commit(s) were added to refs/heads/master by this push: new 81495b0ff1 [docs]correct spelling of the word 'LABEL' #3265 (#8956) 81495b0ff1 is described below commit 81495b0ff17f6e8adc93ba9612aed85eba0c96be Author: jiangmengmeng <1804226...@qq.com> AuthorDate: Tue Apr 12 08:44:33 2022 +0800 [docs]correct spelling of the word 'LABEL' #3265 (#8956) --- docs/en/getting-started/basic-usage.md | 2 +- new-docs/en/data-table/basic-usage.md | 2 +- samples/mini_load/python/mini_load_utils.py | 2 +- 3 files changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/en/getting-started/basic-usage.md b/docs/en/getting-started/basic-usage.md index 4819c76da1..f3b4cc44a6 100644 --- a/docs/en/getting-started/basic-usage.md +++ b/docs/en/getting-started/basic-usage.md @@ -313,7 +313,7 @@ PROPERTIES Broker imports are asynchronous commands. Successful execution of the above commands only indicates successful submission of tasks. Successful imports need to be checked through `SHOW LOAD;' Such as: -`SHOW LOAD WHERE LABLE = "table1_20170708";` +`SHOW LOAD WHERE LABEL = "table1_20170708";` In the return result, FINISHED in the `State` field indicates that the import was successful. diff --git a/new-docs/en/data-table/basic-usage.md b/new-docs/en/data-table/basic-usage.md index 56c2581d82..4f45044dfe 100644 --- a/new-docs/en/data-table/basic-usage.md +++ b/new-docs/en/data-table/basic-usage.md @@ -321,7 +321,7 @@ PROPERTIES Broker imports are asynchronous commands. Successful execution of the above commands only indicates successful submission of tasks. Successful imports need to be checked through `SHOW LOAD;' Such as: ```sql -SHOW LOAD WHERE LABLE = "table1_20170708"; +SHOW LOAD WHERE LABEL = "table1_20170708"; ``` In the return result, `FINISHED` in the `State` field indicates that the import was successful. diff --git a/samples/mini_load/python/mini_load_utils.py b/samples/mini_load/python/mini_load_utils.py index e8bb1a569c..88753680e0 100644 --- a/samples/mini_load/python/mini_load_utils.py +++ b/samples/mini_load/python/mini_load_utils.py @@ -51,7 +51,7 @@ class DorisMiniLoadClient(object): def get_label(self): """ 获取label前缀 -:return: lable +:return: label """ return '_'.join([self.table, os.path.basename(self.file_name)]) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] yiguolei merged pull request #8953: [Refactor][Doc]DML help doc（INSERT，UPDATE，DELETE，OUTFILE）
yiguolei merged PR #8953: URL: https://github.com/apache/incubator-doris/pull/8953 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[incubator-doris] branch master updated: DML help doc (#8953)
This is an automated email from the ASF dual-hosted git repository.
yiguolei pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-doris.git
The following commit(s) were added to refs/heads/master by this push:
new 7494a4cbdf DML help doc (#8953)
7494a4cbdf is described below
commit 7494a4cbdfafbd04af5cee8decd25c0c1da628ca
Author: jiafeng.zhang
AuthorDate: Tue Apr 12 08:45:16 2022 +0800
DML help doc (#8953)
DML help doc
---
.../Manipulation/DELETE.md | 51 +
.../Manipulation/INSERT.md | 82 +++
.../Manipulation/UPDATE.md | 46
.../Data-Manipulation-Statements/OUTFILE.md| 246 -
.../Manipulation/DELETE.md | 49
.../Manipulation/INSERT.md | 82 +++
.../Manipulation/UPDATE.md | 46
.../Data-Manipulation-Statements/OUTFILE.md| 127 ---
8 files changed, 697 insertions(+), 32 deletions(-)
diff --git
a/new-docs/en/sql-manual/sql-reference-v2/Data-Manipulation-Statements/Manipulation/DELETE.md
b/new-docs/en/sql-manual/sql-reference-v2/Data-Manipulation-Statements/Manipulation/DELETE.md
index e233d84024..447a64cc78 100644
---
a/new-docs/en/sql-manual/sql-reference-v2/Data-Manipulation-Statements/Manipulation/DELETE.md
+++
b/new-docs/en/sql-manual/sql-reference-v2/Data-Manipulation-Statements/Manipulation/DELETE.md
@@ -26,10 +26,61 @@ under the License.
## DELETE
+### Name
+
+DELETE
+
### Description
+This statement is used to conditionally delete data in the specified table
(base index) partition.
+
+This operation will also delete the data of the rollup index related to this
base index.
+
+grammar:
+
+SQL
+DELETE FROM table_name [PARTITION partition_name | PARTITIONS (p1, p2)]
+WHERE
+column_name1 op { value | value_list } [ AND column_name2 op { value |
value_list } ...];
+
+
+illustrate:
+
+1. The optional types of op include: =, >, <, >=, <=, !=, in, not in
+2. Only conditions on the key column can be specified.
+3. When the selected key column does not exist in a rollup, delete cannot be
performed.
+4. Conditions can only have an "and" relationship. If you want to achieve an
"or" relationship, you need to write the conditions in two DELETE statements.
+5. If it is a partitioned table, you can specify a partition. If not
specified, and the session variable delete_without_partition is true, it will
be applied to all partitions. If it is a single-partition table, it can be left
unspecified.
+
+Notice:
+
+1. This statement may reduce query efficiency for a period of time after
execution.
+2. The degree of impact depends on the number of delete conditions specified
in the statement.
+3. The more conditions you specify, the greater the impact.
+
### Example
+1. Delete the data row whose k1 column value is 3 in my_table partition p1
+
+ ```sql
+ DELETE FROM my_table PARTITION p1
+ WHERE k1 = 3;
+
+
+2. Delete the data rows where the value of column k1 is greater than or equal
to 3 and the value of column k2 is "abc" in my_table partition p1
+
+ ```sql
+ DELETE FROM my_table PARTITION p1
+ WHERE k1 >= 3 AND k2 = "abc";
+
+
+3. Delete the data rows where the value of column k1 is greater than or equal
to 3 and the value of column k2 is "abc" in my_table partition p1, p2
+
+ ```sql
+ DELETE FROM my_table PARTITIONS (p1, p2)
+ WHERE k1 >= 3 AND k2 = "abc";
+
+
### Keywords
DELETE
diff --git
a/new-docs/en/sql-manual/sql-reference-v2/Data-Manipulation-Statements/Manipulation/INSERT.md
b/new-docs/en/sql-manual/sql-reference-v2/Data-Manipulation-Statements/Manipulation/INSERT.md
index 9931392c45..70bf4fa2b1 100644
---
a/new-docs/en/sql-manual/sql-reference-v2/Data-Manipulation-Statements/Manipulation/INSERT.md
+++
b/new-docs/en/sql-manual/sql-reference-v2/Data-Manipulation-Statements/Manipulation/INSERT.md
@@ -26,10 +26,92 @@ under the License.
## INSERT
+### Name
+
+INSERT
+
### Description
+The change statement is to complete the data insertion operation.
+
+```sql
+INSERT INTO table_name
+[ PARTITION (p1, ...) ]
+[ WITH LABEL label]
+[ (column [, ...]) ]
+[ [ hint [, ...] ] ]
+{ VALUES ( { expression | DEFAULT } [, ...] ) [, ...] | query }
+
+
+ Parameters
+
+> tablet_name: The destination table for importing data. Can be of the form
`db_name.table_name`
+>
+> partitions: Specify the partitions to be imported, which must be partitions
that exist in `table_name`. Multiple partition names are separated by commas
+>
+> label: specify a label for the Insert task
+>
+> column_name: The specified destination column, must be a column that exists
in `table_name`
+>
+> expression: the corresponding expression that needs to be assigned to a
column
+>
+> DEFAULT: let the corresponding column use the default value
+>
+> query:
[GitHub] [incubator-doris] yiguolei merged pull request #8915: [docs] update alter table doc
yiguolei merged PR #8915: URL: https://github.com/apache/incubator-doris/pull/8915 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[incubator-doris] branch master updated: [docs] update alter table doc (#8915)
This is an automated email from the ASF dual-hosted git repository.
yiguolei pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-doris.git
The following commit(s) were added to refs/heads/master by this push:
new f992247779 [docs] update alter table doc (#8915)
f992247779 is described below
commit f992247779d2b191e866a35801fdce1a7fbf0540
Author: Hao Wang <1727386...@qq.com>
AuthorDate: Tue Apr 12 09:26:40 2022 +0800
[docs] update alter table doc (#8915)
---
.../sql-statements/Data Definition/ALTER TABLE.md | 14 ++
.../sql-statements/Data Definition/ALTER TABLE.md | 17 -
2 files changed, 30 insertions(+), 1 deletion(-)
diff --git a/docs/en/sql-reference/sql-statements/Data Definition/ALTER
TABLE.md b/docs/en/sql-reference/sql-statements/Data Definition/ALTER TABLE.md
index 984220a9c3..3cb5f5304a 100644
--- a/docs/en/sql-reference/sql-statements/Data Definition/ALTER TABLE.md
+++ b/docs/en/sql-reference/sql-statements/Data Definition/ALTER TABLE.md
@@ -228,6 +228,14 @@ under the License.
3. Modify the partition name
grammar:
RENAME PARTITION old_partition_name new_partition_name;
+
+Replace supports swap data between two tables:
+1. swap data between two tables
+geammar:
+REPLACE WITH TABLE new_table [PROPERTIES('swap' = 'true')];
+note:
+1. if swap is true, swap data between two tables.
+2. if swap is fasle, replace the old_table with the new_table, and
delete the new_table.
Bitmap index supports the following modifications:
1. create bitmap index
@@ -418,6 +426,12 @@ under the License.
3. Modify the partition named p1 in the table example_table to p2
ALTER TABLE example_table RENAME PARTITION p1 p2;
+
+[replace]
+1. swap data between two tables
+ALTER TABLE table1 REPLACE WITH TABLE table2;
+2. replace the table1 with the table2, and delete the table2.
+ALTER TABLE table1 REPLACE WITH TABLE table2 PROPERTIES('swap' =
'false');
[index]
1. create index on table1 column siteid using bitmap
diff --git a/docs/zh-CN/sql-reference/sql-statements/Data Definition/ALTER
TABLE.md b/docs/zh-CN/sql-reference/sql-statements/Data Definition/ALTER
TABLE.md
index c5a4a3796b..9a11ef2a73 100644
--- a/docs/zh-CN/sql-reference/sql-statements/Data Definition/ALTER TABLE.md
+++ b/docs/zh-CN/sql-reference/sql-statements/Data Definition/ALTER TABLE.md
@@ -226,6 +226,14 @@ under the License.
3. 修改 partition 名称
语法：
RENAME PARTITION old_partition_name new_partition_name;
+
+replace 支持对两个表进行原子的替换操作:
+1. 替换表
+语法：
+REPLACE WITH TABLE new_table [PROPERTIES('swap' = 'true')];
+注意：
+1. 如果 swap 参数为 true，两表交换数据。
+2. 如果 swap 参数为 false，用 new_table 表中的数据替换 old_table 表中的数据，并删除
new_table 表。
bitmap index 支持如下几种修改方式
1. 创建bitmap 索引
@@ -233,7 +241,7 @@ under the License.
ADD INDEX [IF NOT EXISTS] index_name (column [, ...],) [USING
BITMAP] [COMMENT 'balabala'];
注意：
1. 目前仅支持bitmap 索引
-1. BITMAP 索引仅在单列上创建
+2. BITMAP 索引仅在单列上创建
2. 删除索引
语法：
@@ -414,6 +422,13 @@ under the License.
3. 将表 example_table 中名为 p1 的 partition 修改为 p2
ALTER TABLE example_table RENAME PARTITION p1 p2;
+
+[replace]
+1. 将 tabel1 和 table2 的数据互换
+ALTER TABLE table1 REPLACE WITH TABLE table2;
+2. 用 table2 表中的数据替换 tabel1 表中的数据, 并删除 table2
+ALTER TABLE table1 REPLACE WITH TABLE table2 PROPERTIES('swap' =
'false');
+
[index]
1. 在table1 上为siteid 创建bitmap 索引
ALTER TABLE table1 ADD INDEX [IF NOT EXISTS] index_name (siteid)
[USING BITMAP] COMMENT 'balabala';
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] yangzhg commented on a diff in pull request #8958: [refactor] Unify all unit tests into one binary file
yangzhg commented on code in PR #8958:
URL: https://github.com/apache/incubator-doris/pull/8958#discussion_r847863370
##
be/test/CMakeLists.txt:
##
@@ -0,0 +1,389 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# where to put generated libraries
+set(LIBRARY_OUTPUT_PATH "${BUILD_DIR}/test")
+
+# where to put generated libraries
+set(EXECUTABLE_OUTPUT_PATH "${BUILD_DIR}/test")
+
+set(AGENT_TEST_FILES
+agent/utils_test.cpp
+# agent/agent_server_test.cpp
+# agent/cgroups_mgr_test.cpp
+# agent/heartbeat_server_test.cpp
+)
+set(COMMON_TEST_FILES
+common/resource_tls_test.cpp
+common/status_test.cpp
+common/config_test.cpp
+)
+set(ENV_TEST_FILES
+env/env_posix_test.cpp
+)
+
+set(EXEC_TEST_FILES
+exec/hash_table_test.cpp
+exec/olap_common_test.cpp
+exec/json_scanner_test.cpp
+exec/json_scanner_with_jsonpath_test.cpp
+exec/parquet_scanner_test.cpp
+exec/orc_scanner_test.cpp
+exec/plain_text_line_reader_uncompressed_test.cpp
+exec/plain_text_line_reader_gzip_test.cpp
+exec/plain_text_line_reader_bzip_test.cpp
+exec/plain_text_line_reader_lz4frame_test.cpp
+exec/broker_scanner_test.cpp
+exec/broker_scan_node_test.cpp
+exec/tablet_info_test.cpp
+exec/tablet_sink_test.cpp
+exec/buffered_reader_test.cpp
+exec/es_http_scan_node_test.cpp
+exec/es_predicate_test.cpp
+exec/es_query_builder_test.cpp
+exec/es_scan_reader_test.cpp
+exec/s3_reader_test.cpp
+exec/multi_bytes_separator_test.cpp
+# exec/hdfs_file_reader_test.cpp
Review Comment:
use testName
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] yangzhg commented on a diff in pull request #8958: [refactor] Unify all unit tests into one binary file
yangzhg commented on code in PR #8958:
URL: https://github.com/apache/incubator-doris/pull/8958#discussion_r847863370
##
be/test/CMakeLists.txt:
##
@@ -0,0 +1,389 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+# where to put generated libraries
+set(LIBRARY_OUTPUT_PATH "${BUILD_DIR}/test")
+
+# where to put generated libraries
+set(EXECUTABLE_OUTPUT_PATH "${BUILD_DIR}/test")
+
+set(AGENT_TEST_FILES
+agent/utils_test.cpp
+# agent/agent_server_test.cpp
+# agent/cgroups_mgr_test.cpp
+# agent/heartbeat_server_test.cpp
+)
+set(COMMON_TEST_FILES
+common/resource_tls_test.cpp
+common/status_test.cpp
+common/config_test.cpp
+)
+set(ENV_TEST_FILES
+env/env_posix_test.cpp
+)
+
+set(EXEC_TEST_FILES
+exec/hash_table_test.cpp
+exec/olap_common_test.cpp
+exec/json_scanner_test.cpp
+exec/json_scanner_with_jsonpath_test.cpp
+exec/parquet_scanner_test.cpp
+exec/orc_scanner_test.cpp
+exec/plain_text_line_reader_uncompressed_test.cpp
+exec/plain_text_line_reader_gzip_test.cpp
+exec/plain_text_line_reader_bzip_test.cpp
+exec/plain_text_line_reader_lz4frame_test.cpp
+exec/broker_scanner_test.cpp
+exec/broker_scan_node_test.cpp
+exec/tablet_info_test.cpp
+exec/tablet_sink_test.cpp
+exec/buffered_reader_test.cpp
+exec/es_http_scan_node_test.cpp
+exec/es_predicate_test.cpp
+exec/es_query_builder_test.cpp
+exec/es_scan_reader_test.cpp
+exec/s3_reader_test.cpp
+exec/multi_bytes_separator_test.cpp
+# exec/hdfs_file_reader_test.cpp
Review Comment:
use testName like TestEngineStorageMigrationTask
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #8951: [Refactor][doc] add doc for auth management and conf
github-actions[bot] commented on PR #8951: URL: https://github.com/apache/incubator-doris/pull/8951#issuecomment-1095808412 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #8951: [Refactor][doc] add doc for auth management and conf
github-actions[bot] commented on PR #8951: URL: https://github.com/apache/incubator-doris/pull/8951#issuecomment-1095808470 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] yangzhg commented on a diff in pull request #8958: [refactor] Unify all unit tests into one binary file
yangzhg commented on code in PR #8958:
URL: https://github.com/apache/incubator-doris/pull/8958#discussion_r847874533
##
be/src/geo/geo_functions.cpp:
##
@@ -236,7 +236,8 @@ void
GeoFunctions::st_contains_prepare(doris_udf::FunctionContext* ctx,
if (str->is_null) {
contains_ctx->is_null = true;
} else {
-contains_ctx->shapes[i] = GeoShape::from_encoded(str->ptr,
str->len);
+contains_ctx->shapes[i] =
+
std::shared_ptr(GeoShape::from_encoded(str->ptr, str->len));
Review Comment:
shapes may share at line 275
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] yangzhg commented on a diff in pull request #8924: [Refactor][Doc]add cluster admin help
yangzhg commented on code in PR #8924: URL: https://github.com/apache/incubator-doris/pull/8924#discussion_r847877235 ## new-docs/en/sql-manual/sql-reference-v2/Cluster-Management-Statements/ALTER-SYSTEM-ADD-BACKEND.md: ## @@ -26,10 +26,45 @@ under the License. ## ALTER-SYSTEM-ADD-BACKEND Review Comment: `-` in header is useless -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #8916: [Refactor][doc] add data model and index doc
github-actions[bot] commented on PR #8916: URL: https://github.com/apache/incubator-doris/pull/8916#issuecomment-1095828263 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #8916: [Refactor][doc] add data model and index doc
github-actions[bot] commented on PR #8916: URL: https://github.com/apache/incubator-doris/pull/8916#issuecomment-1095828305 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] yangzhg commented on a diff in pull request #8923: [Feature] Support compression prop
yangzhg commented on code in PR #8923:
URL: https://github.com/apache/incubator-doris/pull/8923#discussion_r847878329
##
be/src/olap/tablet_meta.cpp:
##
@@ -89,7 +90,21 @@ TabletMeta::TabletMeta(int64_t table_id, int64_t
partition_id, int64_t tablet_id
LOG(WARNING) << "unknown tablet keys type";
break;
}
-schema->set_compress_kind(COMPRESS_LZ4);
+
+switch (compression_type)
+{
+case TCompressionType::LZ4:
+schema->set_compress_kind(COMPRESS_LZ4);
+break;
+#ifdef DORIS_WITH_LZO
+case TCompressionType::LZ0:
+schema->set_compress_kind(COMPRESS_LZ0);
Review Comment:
```suggestion
schema->set_compress_kind(COMPRESS_LZO);
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] yangzhg commented on a diff in pull request #8923: [Feature] Support compression prop
yangzhg commented on code in PR #8923:
URL: https://github.com/apache/incubator-doris/pull/8923#discussion_r84788
##
gensrc/thrift/AgentService.thrift:
##
@@ -84,6 +84,12 @@ struct TStorageParam {
3: optional TS3StorageParam s3_storage_param
}
+enum TCompressionType {
Review Comment:
use `DEFAULT` is not a good idea, it is better to specify an explicit type
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] yangzhg commented on a diff in pull request #8923: [Feature] Support compression prop
yangzhg commented on code in PR #8923:
URL: https://github.com/apache/incubator-doris/pull/8923#discussion_r847880569
##
gensrc/thrift/AgentService.thrift:
##
@@ -84,6 +84,12 @@ struct TStorageParam {
3: optional TS3StorageParam s3_storage_param
}
+enum TCompressionType {
+DEFAULT = 0,
+LZ4 = 1,
+LZ0 = 2
Review Comment:
maybe `LZO`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] EmmyMiao87 closed issue #8920: [Bug] `ThreadMemTrackerMgr::update_tracker` null pointer in vectorized
EmmyMiao87 closed issue #8920: [Bug] `ThreadMemTrackerMgr::update_tracker` null pointer in vectorized URL: https://github.com/apache/incubator-doris/issues/8920 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] EmmyMiao87 merged pull request #8925: [fix][mem tracker] Fix MemTracker null pointer in vectorized
EmmyMiao87 merged PR #8925: URL: https://github.com/apache/incubator-doris/pull/8925 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[incubator-doris] branch master updated: [fix][mem tracker] Fix MemTracker null pointer in vectorized (#8925)
This is an automated email from the ASF dual-hosted git repository.
lingmiao pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-doris.git
The following commit(s) were added to refs/heads/master by this push:
new 66d2f4e1fd [fix][mem tracker] Fix MemTracker null pointer in
vectorized (#8925)
66d2f4e1fd is described below
commit 66d2f4e1fdc490536d383e0878ed2e3ad011bd6c
Author: Xinyi Zou
AuthorDate: Tue Apr 12 10:17:10 2022 +0800
[fix][mem tracker] Fix MemTracker null pointer in vectorized (#8925)
Fix ThreadMemTrackerMgr::update_tracker null pointer and some details.
Issue Number: close #8920
---
be/src/exec/exchange_node.cpp | 1 +
be/src/runtime/data_stream_recvr.h| 1 +
be/src/runtime/fold_constant_executor.cpp | 4 ++--
be/src/runtime/thread_context.h | 2 +-
be/src/runtime/thread_mem_tracker_mgr.h | 13 -
be/src/vec/exec/vexchange_node.cpp| 5 +
be/src/vec/runtime/vdata_stream_recvr.h | 1 +
be/src/vec/sink/vtablet_sink.cpp | 3 ++-
8 files changed, 21 insertions(+), 9 deletions(-)
diff --git a/be/src/exec/exchange_node.cpp b/be/src/exec/exchange_node.cpp
index 083e40518e..d79170f912 100644
--- a/be/src/exec/exchange_node.cpp
+++ b/be/src/exec/exchange_node.cpp
@@ -212,6 +212,7 @@ Status ExchangeNode::get_next_merging(RuntimeState* state,
RowBatch* output_batc
RETURN_IF_CANCELLED(state);
RETURN_IF_ERROR(state->check_query_state("Exchange, while merging next."));
+ADD_THREAD_LOCAL_MEM_TRACKER(_stream_recvr->mem_tracker());
RETURN_IF_ERROR(_stream_recvr->get_next(output_batch, eos));
while ((_num_rows_skipped < _offset)) {
_num_rows_skipped += output_batch->num_rows();
diff --git a/be/src/runtime/data_stream_recvr.h
b/be/src/runtime/data_stream_recvr.h
index 3b1edac081..28a4a9dadf 100644
--- a/be/src/runtime/data_stream_recvr.h
+++ b/be/src/runtime/data_stream_recvr.h
@@ -100,6 +100,7 @@ public:
const TUniqueId& fragment_instance_id() const { return
_fragment_instance_id; }
PlanNodeId dest_node_id() const { return _dest_node_id; }
const RowDescriptor& row_desc() const { return _row_desc; }
+const std::shared_ptr& mem_tracker() const { return
_mem_tracker; }
void add_sub_plan_statistics(const PQueryStatistics& statistics, int
sender_id) {
_sub_plan_query_statistics_recvr->insert(statistics, sender_id);
diff --git a/be/src/runtime/fold_constant_executor.cpp
b/be/src/runtime/fold_constant_executor.cpp
index 9b5300f16a..274750b8b3 100644
--- a/be/src/runtime/fold_constant_executor.cpp
+++ b/be/src/runtime/fold_constant_executor.cpp
@@ -44,7 +44,6 @@ TUniqueId FoldConstantExecutor::_dummy_id;
Status FoldConstantExecutor::fold_constant_expr(
const TFoldConstantParams& params, PConstantExprResult* response) {
-SCOPED_SWITCH_THREAD_LOCAL_MEM_TRACKER(_mem_tracker);
const auto& expr_map = params.expr_map;
auto expr_result_map = response->mutable_expr_result_map();
@@ -54,6 +53,7 @@ Status FoldConstantExecutor::fold_constant_expr(
if (UNLIKELY(!status.ok())) {
return status;
}
+SCOPED_SWITCH_THREAD_LOCAL_MEM_TRACKER(_mem_tracker);
for (const auto& m : expr_map) {
PExprResultMap pexpr_result_map;
@@ -108,9 +108,9 @@ Status FoldConstantExecutor::fold_constant_vexpr(
// init
Status status = _init(query_globals);
if (UNLIKELY(!status.ok())) {
-LOG(WARNING) << "Failed to init mem trackers, msg: " <<
status.get_error_msg();
return status;
}
+SCOPED_SWITCH_THREAD_LOCAL_MEM_TRACKER(_mem_tracker);
for (const auto& m : expr_map) {
PExprResultMap pexpr_result_map;
diff --git a/be/src/runtime/thread_context.h b/be/src/runtime/thread_context.h
index a7c8261930..4d9d60078c 100644
--- a/be/src/runtime/thread_context.h
+++ b/be/src/runtime/thread_context.h
@@ -156,7 +156,7 @@ private:
// The func provided by pthread and std::thread doesn't help either.
//
// So, kudu Class-scoped static thread local implementation was introduced.
Solve the above problem by
-// Thread-scopedthread local + Class-scoped thread local.
+// Thread-scoped thread local + Class-scoped thread local.
//
// This may look very trick, but it's the best way I can find.
//
diff --git a/be/src/runtime/thread_mem_tracker_mgr.h
b/be/src/runtime/thread_mem_tracker_mgr.h
index a709a536bf..79c152c243 100644
--- a/be/src/runtime/thread_mem_tracker_mgr.h
+++ b/be/src/runtime/thread_mem_tracker_mgr.h
@@ -65,6 +65,7 @@ public:
_mem_trackers[0] = MemTracker::get_process_tracker();
_untracked_mems[0] = 0;
_tracker_id = 0;
+_mem_tracker_labels[0] = MemTracker::get_process_tracker()->label();
start_thread_mem_tracker = true;
}
~ThreadMemTrackerMgr() {
@@ -75,7 +76,8 @@ public:
void clear_untracked_mems() {
for (const auto& untracked_mem : _untracked_mems) {
[GitHub] [incubator-doris] yangzhg commented on pull request #8923: [Feature] Support compression prop
yangzhg commented on PR #8923: URL: https://github.com/apache/incubator-doris/pull/8923#issuecomment-1095851195 `compression type` you mean used to compress page? page compress use LZ4F not LZ4， and compress type is defined in `CompressionTypePB` in `gensrc/proto/segment_v2.proto` no need to create new define in thrift, or thrift should keep consistent with proto, the actual compress algorithm is hard code in `be/src/olap/rowset/segment_v2/segment_writer.cpp` so I think we need more discuss for this pr -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] smallhibiscus opened a new pull request, #8960: [improvement](config) Modify the maximum and minimum number of threads in jetty
smallhibiscus opened a new pull request, #8960: URL: https://github.com/apache/incubator-doris/pull/8960 # Proposed changes Issue Number: close #xxx ## Problem Summary: Modify the maximum and minimum number of threads in jetty. ## Checklist(Required) 1. Does it affect the original behavior: (Yes/No/I Don't know) 2. Has unit tests been added: (Yes/No/No Need) 3. Has document been added or modified: (Yes/No/No Need) 4. Does it need to update dependencies: (Yes/No) 5. Are there any changes that cannot be rolled back: (Yes/No) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] BiteTheDDDDt opened a new issue, #8961: [Enhancement] unnecessary date/datetime cast to double
BiteThet opened a new issue, #8961: URL: https://github.com/apache/incubator-doris/issues/8961 ### Search before asking - [X] I had searched in the [issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and found no similar issues. ### Description ```sql SELECT sum(LO_ORDERDATE DIV 1) FROM lineorder_flat; 5.327 sec SELECT sum(cast(LO_ORDERDATE as bigint) DIV 1) FROM lineorder_flat; 4.845 sec ``` ``` | 3:AGGREGATE (merge finalize) | | | output: sum( sum(CAST(CAST(`LO_ORDERDATE` AS DOUBLE) AS BIGINT) DIV 1)) | | | group by: | | | cardinality=-1 | | | tuple ids: 2 | | | | | 2:VEXCHANGE | | tuple ids: 1 ``` ### Solution _No response_ ### Are you willing to submit PR? - [X] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] yangzhg commented on a diff in pull request #8921: [Bug] fix backup timeout on a table without partition
yangzhg commented on code in PR #8921:
URL: https://github.com/apache/incubator-doris/pull/8921#discussion_r847892735
##
fe/fe-core/src/main/java/org/apache/doris/backup/BackupJob.java:
##
@@ -358,6 +358,18 @@ private void prepareAndSendSnapshotTask() {
taskProgress.clear();
taskErrMsg.clear();
AgentBatchTask batchTask = new AgentBatchTask();
+
+// remove table which has no partition
+tableRefs = tableRefs.stream().filter(tableRef -> {
+String tblName = tableRef.getName().getTbl();
+Table tbl = db.getTableNullable(tblName);
+if (tbl != null && tbl.getType() == TableType.OLAP) {
+OlapTable olapTable = (OlapTable) tbl;
+return !olapTable.getAllPartitions().isEmpty();
Review Comment:
What kind of table does not have a partition？
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] zhangstar333 opened a new pull request, #8962: [Feature]support stddev/variance agg functions to window function
zhangstar333 opened a new pull request, #8962: URL: https://github.com/apache/incubator-doris/pull/8962 # Proposed changes Issue Number: close #7749 ## Problem Summary: Describe the overview of changes. ## Checklist(Required) 1. Does it affect the original behavior: (Yes/No/I Don't know) 2. Has unit tests been added: (Yes/No/No Need) 3. Has document been added or modified: (Yes/No/No Need) 4. Does it need to update dependencies: (Yes/No) 5. Are there any changes that cannot be rolled back: (Yes/No) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #8908: [fix](load) start transaction before we need it (#8819)
github-actions[bot] commented on PR #8908: URL: https://github.com/apache/incubator-doris/pull/8908#issuecomment-1095921048 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #8908: [fix](load) start transaction before we need it (#8819)
github-actions[bot] commented on PR #8908: URL: https://github.com/apache/incubator-doris/pull/8908#issuecomment-1095921088 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] zhangstar333 commented on a diff in pull request #8877: [feature](function) Support nth value window function on vectorized e…
zhangstar333 commented on code in PR #8877:
URL: https://github.com/apache/incubator-doris/pull/8877#discussion_r847915588
##
fe/fe-core/src/main/java/org/apache/doris/catalog/FunctionSet.java:
##
@@ -2336,6 +2336,10 @@ private void initAggregateBuiltins() {
prefix + OFFSET_FN_UPDATE_SYMBOL.get(t),
null, null, null, true));
+addBuiltin(AggregateFunction.createAnalyticBuiltin(
+ "nth_value", Lists.newArrayList(t, Type.BIGINT, t), t, t,
Review Comment:
It seems that MySQL has no default value and directly returns null
##
fe/fe-core/src/main/java/org/apache/doris/catalog/FunctionSet.java:
##
@@ -2336,6 +2336,10 @@ private void initAggregateBuiltins() {
prefix + OFFSET_FN_UPDATE_SYMBOL.get(t),
null, null, null, true));
+addBuiltin(AggregateFunction.createAnalyticBuiltin(
+ "nth_value", Lists.newArrayList(t, Type.BIGINT, t), t, t,
Review Comment:
It seems that MySQL has no default value and directly returns null
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] wucheng opened a new pull request, #8963: add support for tencent chdfs
wucheng opened a new pull request, #8963: URL: https://github.com/apache/incubator-doris/pull/8963 add support for tencent cloud [chdfs](https://cloud.tencent.com/document/product/1105). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] wucheng opened a new pull request, #8964: add support for tencent chdfs
wucheng opened a new pull request, #8964: URL: https://github.com/apache/incubator-doris/pull/8964 add support for tencent cloud [chdfs](https://cloud.tencent.com/document/product/1105). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] yiguolei commented on a diff in pull request #8958: [refactor] Unify all unit tests into one binary file
yiguolei commented on code in PR #8958:
URL: https://github.com/apache/incubator-doris/pull/8958#discussion_r847956940
##
be/src/geo/geo_functions.cpp:
##
@@ -267,22 +268,21 @@ doris_udf::BooleanVal
GeoFunctions::st_contains(doris_udf::FunctionContext* ctx,
if (state != nullptr && state->is_null) {
return BooleanVal::null();
}
-GeoShape* shapes[2] = {nullptr, nullptr};
+std::vector> shapes = {nullptr, nullptr};
const StringVal* strs[2] = {&lhs, &rhs};
-// use this to delete new
-StContainsState local_state;
for (int i = 0; i < 2; ++i) {
if (state != nullptr && state->shapes[i] != nullptr) {
shapes[i] = state->shapes[i];
} else {
-shapes[i] = local_state.shapes[i] =
GeoShape::from_encoded(strs[i]->ptr, strs[i]->len);
+shapes[i] =
+
std::shared_ptr(GeoShape::from_encoded(strs[i]->ptr, strs[i]->len));
if (shapes[i] == nullptr) {
return BooleanVal::null();
}
}
}
-return shapes[0]->contains(shapes[1]);
+return shapes[0]->contains(shapes[1].get());
Review Comment:
Has to add get() here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #8960: [improvement](config) Modify the maximum and minimum number of threads in jetty
github-actions[bot] commented on PR #8960: URL: https://github.com/apache/incubator-doris/pull/8960#issuecomment-1096000581 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #8960: [improvement](config) Modify the maximum and minimum number of threads in jetty
github-actions[bot] commented on PR #8960: URL: https://github.com/apache/incubator-doris/pull/8960#issuecomment-1096000543 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] zhangstar333 commented on a diff in pull request #8877: [feature](function) Support nth value window function on vectorized e…
zhangstar333 commented on code in PR #8877:
URL: https://github.com/apache/incubator-doris/pull/8877#discussion_r847915589
##
fe/fe-core/src/main/java/org/apache/doris/catalog/FunctionSet.java:
##
@@ -2336,6 +2336,10 @@ private void initAggregateBuiltins() {
prefix + OFFSET_FN_UPDATE_SYMBOL.get(t),
null, null, null, true));
+addBuiltin(AggregateFunction.createAnalyticBuiltin(
+ "nth_value", Lists.newArrayList(t, Type.BIGINT, t), t, t,
Review Comment:
It seems that MySQL has no default value and directly returns null
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] lide-reed commented on pull request #8964: add support for tencent chdfs
lide-reed commented on PR #8964: URL: https://github.com/apache/incubator-doris/pull/8964#issuecomment-1096107979 Thanks wucheng, LGTM -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #8964: add support for tencent chdfs
github-actions[bot] commented on PR #8964: URL: https://github.com/apache/incubator-doris/pull/8964#issuecomment-1096122821 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #8964: add support for tencent chdfs
github-actions[bot] commented on PR #8964: URL: https://github.com/apache/incubator-doris/pull/8964#issuecomment-1096122857 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] BiteTheDDDDt opened a new issue, #8965: [Enhancement][Vectorized] optimize and clean int_div apply
BiteThet opened a new issue, #8965: URL: https://github.com/apache/incubator-doris/issues/8965 ### Search before asking - [X] I had searched in the [issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and found no similar issues. ### Description 1. remove unsigned type check because we don't support unsigned type. 2. use `a / (b + null_map[index])` to replace`a / (b + (b==0))`, this change on query`SELECT sum(cast(LO_ORDERDATE as double) DIV 1) FROM lineorder_flat;` have little faster(5.322 sec -> 5.138 sec). ### Solution _No response_ ### Are you willing to submit PR? - [X] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] BiteTheDDDDt opened a new pull request, #8966: [Enhancement][Vectorized] optimize and clean int_div apply
BiteThet opened a new pull request, #8966: URL: https://github.com/apache/incubator-doris/pull/8966 # Proposed changes Issue Number: close #8965 ## Problem Summary: Describe the overview of changes. ## Checklist(Required) 1. Does it affect the original behavior: (Yes/No/I Don't know) 2. Has unit tests been added: (Yes/No/No Need) 3. Has document been added or modified: (Yes/No/No Need) 4. Does it need to update dependencies: (Yes/No) 5. Are there any changes that cannot be rolled back: (Yes/No) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #8963: add support for tencent chdfs
github-actions[bot] commented on PR #8963: URL: https://github.com/apache/incubator-doris/pull/8963#issuecomment-1096126577 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #8963: add support for tencent chdfs
github-actions[bot] commented on PR #8963: URL: https://github.com/apache/incubator-doris/pull/8963#issuecomment-1096126622 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] yangzhg commented on a diff in pull request #8958: [refactor] Unify all unit tests into one binary file
yangzhg commented on code in PR #8958:
URL: https://github.com/apache/incubator-doris/pull/8958#discussion_r848006064
##
be/src/geo/geo_functions.cpp:
##
@@ -267,22 +268,21 @@ doris_udf::BooleanVal
GeoFunctions::st_contains(doris_udf::FunctionContext* ctx,
if (state != nullptr && state->is_null) {
return BooleanVal::null();
}
-GeoShape* shapes[2] = {nullptr, nullptr};
+std::vector> shapes = {nullptr, nullptr};
const StringVal* strs[2] = {&lhs, &rhs};
-// use this to delete new
-StContainsState local_state;
for (int i = 0; i < 2; ++i) {
if (state != nullptr && state->shapes[i] != nullptr) {
shapes[i] = state->shapes[i];
} else {
-shapes[i] = local_state.shapes[i] =
GeoShape::from_encoded(strs[i]->ptr, strs[i]->len);
+shapes[i] =
+
std::shared_ptr(GeoShape::from_encoded(strs[i]->ptr, strs[i]->len));
if (shapes[i] == nullptr) {
return BooleanVal::null();
}
}
}
-return shapes[0]->contains(shapes[1]);
+return shapes[0]->contains(shapes[1].get());
Review Comment:
`contains` only accept raw pointer
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #8966: [Enhancement][Vectorized] optimize and clean int_div apply
github-actions[bot] commented on PR #8966: URL: https://github.com/apache/incubator-doris/pull/8966#issuecomment-1096130247 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #8966: [Enhancement][Vectorized] optimize and clean int_div apply
github-actions[bot] commented on PR #8966: URL: https://github.com/apache/incubator-doris/pull/8966#issuecomment-1096130194 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] wucheng closed pull request #8964: add support for tencent chdfs
wucheng closed pull request #8964: add support for tencent chdfs URL: https://github.com/apache/incubator-doris/pull/8964 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] jackwener commented on pull request #8910: [WIP][feature](optimizer) : convert out join to inner join
jackwener commented on PR #8910: URL: https://github.com/apache/incubator-doris/pull/8910#issuecomment-1096159553 TODO： Now, this rule apply just predicate all is `AND` like `expr1 And expr2 And expr3`. The future ticklet is normalize `Expr`, we should conjunct all `OR`, like ```sql (A and B) or C -> (A or C) and (B or C) (A and B or C) or (D or E)-> (A and B and E) or (C and D) -> (A or C) and (A or D) and (B or C) and (B or D) and (E or C) and (E or D) ``` For each (`left` `OR` `right`), `left` and `right` must have same `column`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] wangbo opened a new pull request, #8967: [optimize](storage)optimize date in storage layer
wangbo opened a new pull request, #8967: URL: https://github.com/apache/incubator-doris/pull/8967 # Proposed changes Issue Number: close #8796 ## Problem Summary: Describe the overview of changes. ## Checklist(Required) 1. Does it affect the original behavior: (Yes/No/I Don't know) 2. Has unit tests been added: (Yes/No/No Need) 3. Has document been added or modified: (Yes/No/No Need) 4. Does it need to update dependencies: (Yes/No) 5. Are there any changes that cannot be rolled back: (Yes/No) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] wangbo commented on pull request #8967: [optimize](storage)optimize date in storage layer
wangbo commented on PR #8967: URL: https://github.com/apache/incubator-doris/pull/8967#issuecomment-1096193037 # Performance Test ## SQL1 ``` SELECT sum(LO_EXTENDEDPRICE * LO_DISCOUNT) AS revenue FROM lineorder_flat WHERE LO_ORDERDATE >= 19930101 and LO_ORDERDATE <= 19931231 AND LO_DISCOUNT BETWEEN 1 AND 3 AND LO_QUANTITY < 25; before: - BlockLoadTime: 1s226ms - PredColumnReadTime: 734.907ms - VectorPredEvalTime: 296.752ms - OutputColumnTime: 179.064ms after: - BlockLoadTime: 903.229ms - PredColumnReadTime: 738.852ms - VectorPredEvalTime: 83.223ms - OutputColumnTime: 65.080ms ``` ## SQL2: pred only contains date ``` SELECT sum(LO_DISCOUNT) AS revenue FROM lineorder_flat WHERE LO_ORDERDATE >= 19930101 and LO_ORDERDATE <= 19931231 before: - BlockLoadTime: 1s982ms - PredColumnReadTime: 478.911ms - VectorPredEvalTime: 307.892ms - OutputColumnTime: 1s180ms after: - BlockLoadTime: 879.068ms - PredColumnReadTime: 489.480ms - VectorPredEvalTime: 100.503ms - OutputColumnTime: 274.997ms ``` ## SQL3: no predicate ``` select count(distinct LO_ORDERDATE) from lineorder_flat before: - BlockLoadTime: 8s395ms - PredColumnReadTime: 8s002ms after: - BlockLoadTime: 6s641ms - PredColumnReadTime: 6s252ms ``` We can see ```VectorPredEvalTime``` has significant performance improvement. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
