[GitHub] [incubator-doris] github-actions[bot] commented on pull request #7886: [docs]correct SeaTunnel website hyperlink
github-actions[bot] commented on pull request #7886: URL: https://github.com/apache/incubator-doris/pull/7886#issuecomment-1020913272 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #7886: [docs]correct SeaTunnel website hyperlink
github-actions[bot] commented on pull request #7886: URL: https://github.com/apache/incubator-doris/pull/7886#issuecomment-1020916041 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] qidaye opened a new issue #7887: [Bug] Alias function `digital_masking` result is wrong
qidaye opened a new issue #7887: URL: https://github.com/apache/incubator-doris/issues/7887 ### Search before asking - [X] I had searched in the [issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and found no similar issues. ### Version ee0037e1af504e6945c3a5d0578f4a945f7e3f8f ### What's Wrong? ```sql mysql> select digital_masking(13012345678); +-+ | digital_masking(13012345678) | +-+ | 1273790 | +-+ ``` ### What You Expected? ```sql mysql> select digital_masking(13012345678); +--+ | digital_masking(13012345678) | +--+ | 1305678 | +--+ ``` ### How to Reproduce? See above. ### Anything Else? The default function arg in `digital_masking` is `INT`, which range is [-2147483648, 2147483647]. The reason of this wrong result is that the number `13012345678` is out of the range. ### Are you willing to submit PR? - [X] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] qidaye opened a new pull request #7888: [fix](function) Change digital_masking function arg type to BIGINT
qidaye opened a new pull request #7888: URL: https://github.com/apache/incubator-doris/pull/7888 # Proposed changes Issue Number: close #7887 Change digital_masking function arg type to BIGINT to fix the wrong result. ## Problem Summary: Describe the overview of changes. ## Checklist(Required) 1. Does it affect the original behavior: (No) 2. Has unit tests been added: (No Need) 3. Has document been added or modified: (Yes) 4. Does it need to update dependencies: (No) 5. Are there any changes that cannot be rolled back: (Yes) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #7888: [fix](function) Change digital_masking function arg type to BIGINT
github-actions[bot] commented on pull request #7888: URL: https://github.com/apache/incubator-doris/pull/7888#issuecomment-1020997318 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #7851: [improvement](executor) Displays cast information with implicit conversions in verbose
github-actions[bot] commented on pull request #7851: URL: https://github.com/apache/incubator-doris/pull/7851#issuecomment-1020999899 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] Henry2SS opened a new issue #7889: [Bug] there are some errors while compiling with option -O3
Henry2SS opened a new issue #7889: URL: https://github.com/apache/incubator-doris/issues/7889 ### Search before asking - [X] I had searched in the [issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and found no similar issues. ### Description compile failed when compile with option -O3 ``` error: writing 1 byte into a region of size 0 [-Werror=stringop-overflow=] at be/src/vec/common/cow/h:150 ``` ``` t' may be used uninitialized [-Werror=maybe-uninitialized] at be/src/vec/functions/function_case.h:66 ``` ### Solution _No response_ ### Are you willing to submit PR? - [X] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] Henry2SS opened a new pull request #7890: fix error while compiling with -O3
Henry2SS opened a new pull request #7890: URL: https://github.com/apache/incubator-doris/pull/7890 # Proposed changes Issue Number: close #7889 ## Problem Summary: Describe the overview of changes. ## Checklist(Required) 1. Does it affect the original behavior: (No) 2. Has unit tests been added: (No Need) 3. Has document been added or modified: (No Need) 4. Does it need to update dependencies: (No) 5. Are there any changes that cannot be rolled back: (No) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] HappenLee closed issue #7825: [Vectorized][Bug] coalesce function get wrong data when result type is int/decimal type
HappenLee closed issue #7825: URL: https://github.com/apache/incubator-doris/issues/7825 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] HappenLee merged pull request #7827: [Vectorized][Bug] fix bug of coalesce function
HappenLee merged pull request #7827: URL: https://github.com/apache/incubator-doris/pull/7827 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[incubator-doris] branch master updated (ee0037e -> a683153)
This is an automated email from the ASF dual-hosted git repository. lihaopeng pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-doris.git. from ee0037e [fix](ldap) fix ldap password logic (#7862) add a683153 [Vectorized][Bug] fix bug of coalesce function (#7827) No new revisions were added by this update. Summary of changes: be/src/vec/columns/column_complex.h| 5 + be/src/vec/columns/column_decimal.h| 6 ++ be/src/vec/columns/column_nullable.h | 5 + be/src/vec/columns/column_string.h | 14 ++ be/src/vec/columns/column_vector.h | 6 ++ be/src/vec/functions/function_coalesce.cpp | 2 +- 6 files changed, 37 insertions(+), 1 deletion(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] morningman commented on a change in pull request #7883: [Bug] fix clang compile error
morningman commented on a change in pull request #7883: URL: https://github.com/apache/incubator-doris/pull/7883#discussion_r791683698 ## File path: be/CMakeLists.txt ## @@ -350,6 +350,11 @@ if (COMPILER_GCC) endif () if (COMPILER_CLANG) + +if(MAKE_TEST STREQUAL "OFF") +add_compile_options(-Qunused-arguments) Review comment: Why adding this? Add comment to explain -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] morningman commented on a change in pull request #7883: [Bug] fix clang compile error
morningman commented on a change in pull request #7883: URL: https://github.com/apache/incubator-doris/pull/7883#discussion_r791683698 ## File path: be/CMakeLists.txt ## @@ -350,6 +350,11 @@ if (COMPILER_GCC) endif () if (COMPILER_CLANG) + +if(MAKE_TEST STREQUAL "OFF") +add_compile_options(-Qunused-arguments) Review comment: Why adding this? Add comment to explain -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #7883: [Bug] fix clang compile error
github-actions[bot] commented on pull request #7883: URL: https://github.com/apache/incubator-doris/pull/7883#issuecomment-1021199650 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #7890: fix error while compiling with -O3
github-actions[bot] commented on pull request #7890: URL: https://github.com/apache/incubator-doris/pull/7890#issuecomment-1021200632 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] morningman closed issue #7887: [Bug] Alias function `digital_masking` result is wrong
morningman closed issue #7887: URL: https://github.com/apache/incubator-doris/issues/7887 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] morningman merged pull request #7888: [fix](function) Change digital_masking function arg type to BIGINT
morningman merged pull request #7888: URL: https://github.com/apache/incubator-doris/pull/7888 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[incubator-doris] branch master updated: [fix](function) Change digital_masking function arg type to BIGINT (#7888)
This is an automated email from the ASF dual-hosted git repository.
morningman pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-doris.git
The following commit(s) were added to refs/heads/master by this push:
new 461b352 [fix](function) Change digital_masking function arg type to
BIGINT (#7888)
461b352 is described below
commit 461b352d3ee741ff8781d474c3907dabc26fe670
Author: qiye
AuthorDate: Tue Jan 25 22:28:05 2022 +0800
[fix](function) Change digital_masking function arg type to BIGINT (#7888)
Change digital_masking function arg type to BIGINT to fix the wrong result.
---
docs/en/sql-reference/sql-functions/digital-masking.md | 2 +-
docs/en/sql-reference/sql-statements/Data Definition/create-function.md | 2 +-
docs/en/sql-reference/sql-statements/Data Definition/show-functions.md | 2 +-
docs/zh-CN/sql-reference/sql-functions/digital-masking.md | 2 +-
.../sql-reference/sql-statements/Data Definition/create-function.md | 2 +-
.../sql-reference/sql-statements/Data Definition/show-functions.md | 2 +-
fe/fe-core/src/main/java/org/apache/doris/catalog/AliasFunction.java| 2 +-
.../src/test/java/org/apache/doris/catalog/CreateFunctionTest.java | 2 +-
8 files changed, 8 insertions(+), 8 deletions(-)
diff --git a/docs/en/sql-reference/sql-functions/digital-masking.md
b/docs/en/sql-reference/sql-functions/digital-masking.md
index bd2b22b..f8ce744 100644
--- a/docs/en/sql-reference/sql-functions/digital-masking.md
+++ b/docs/en/sql-reference/sql-functions/digital-masking.md
@@ -36,7 +36,7 @@ digital_masking(digital_number)
Alias function, the original function is
`concat(left(id,3),'',right(id,4))`.
-Desensitizes the input `digital_number` and returns the result after masking
desensitization.
+Desensitizes the input `digital_number` and returns the result after masking
desensitization. `digital_number` is `BIGINT` data type.
## example
diff --git a/docs/en/sql-reference/sql-statements/Data
Definition/create-function.md b/docs/en/sql-reference/sql-statements/Data
Definition/create-function.md
index fdd08ed..ca581dd 100644
--- a/docs/en/sql-reference/sql-statements/Data Definition/create-function.md
+++ b/docs/en/sql-reference/sql-statements/Data Definition/create-function.md
@@ -131,7 +131,7 @@ If the `function_name` contains the database name, the
custom function will be c
```
-- create a custom functional alias function
-CREATE ALIAS FUNCTION id_masking(INT) WITH PARAMETER(id)
+CREATE ALIAS FUNCTION id_masking(BIGINT) WITH PARAMETER(id)
AS CONCAT(LEFT(id, 3), '', RIGHT(id, 4));
-- create a custom cast alias function
diff --git a/docs/en/sql-reference/sql-statements/Data
Definition/show-functions.md b/docs/en/sql-reference/sql-statements/Data
Definition/show-functions.md
index b942046..59b5bcb 100644
--- a/docs/en/sql-reference/sql-statements/Data Definition/show-functions.md
+++ b/docs/en/sql-reference/sql-statements/Data Definition/show-functions.md
@@ -60,7 +60,7 @@ Intermediate Type: NULL
Intermediate Type: NULL
Properties:
{"object_file":"http://host:port/libudasample.so","finalize_fn":"_ZN9doris_udf13CountFinalizeEPNS_15FunctionContextERKNS_9BigIntValE","init_fn":"_ZN9doris_udf9CountInitEPNS_15FunctionContextEPNS_9BigIntValE","merge_fn":"_ZN9doris_udf10CountMergeEPNS_15FunctionContextERKNS_9BigIntValEPS2_","md5":"37d185f80f95569e2676da3d5b5b9d2f","update_fn":"_ZN9doris_udf11CountUpdateEPNS_15FunctionContextERKNS_6IntValEPNS_9BigIntValE"}
*** 3. row ***
-Signature: id_masking(INT)
+Signature: id_masking(BIGINT)
Return Type: VARCHAR
Function Type: Alias
Intermediate Type: NULL
diff --git a/docs/zh-CN/sql-reference/sql-functions/digital-masking.md
b/docs/zh-CN/sql-reference/sql-functions/digital-masking.md
index 32ea5a3..f049a30 100644
--- a/docs/zh-CN/sql-reference/sql-functions/digital-masking.md
+++ b/docs/zh-CN/sql-reference/sql-functions/digital-masking.md
@@ -36,7 +36,7 @@ digital_masking(digital_number)
别名函数，原始函数为 `concat(left(id,3),'',right(id,4))`。
-将输入的 `digital_number` 进行脱敏处理，返回遮盖脱敏后的结果。
+将输入的 `digital_number` 进行脱敏处理，返回遮盖脱敏后的结果。`digital_number` 为 `BIGINT` 数据类型。
## example
diff --git a/docs/zh-CN/sql-reference/sql-statements/Data
Definition/create-function.md b/docs/zh-CN/sql-reference/sql-statements/Data
Definition/create-function.md
index 98d9ce0..f6b2c7b 100644
--- a/docs/zh-CN/sql-reference/sql-statements/Data
Definition/create-function.md
+++ b/docs/zh-CN/sql-reference/sql-statements/Data
Definition/create-function.md
@@ -132,7 +132,7 @@ CREATE [AGGREGATE] [ALIAS] FUNCTION function_name
```
-- 创建自定义功能别名函数
-CREATE ALIAS FUNCTION id_masking(INT) WITH PARAMETER(id)
+CREATE ALIAS FUNCTION id_masking(BIGINT) WITH PARAMETER(id)
AS CONCAT(LEFT(id, 3
[GitHub] [incubator-doris] aubdiy opened a new issue #7891: [Bug]
aubdiy opened a new issue #7891: URL: https://github.com/apache/incubator-doris/issues/7891 ### Search before asking - [X] I had searched in the [issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and found no similar issues. ### Version 0.15.0-incubating ### What's Wrong? compile error: sh build.sh ``` lz4 libraries installed make[1]: Leaving directory '/data/sae-dp-files/tmp/apache-doris-0.15.0-incubating-src/thirdparty/src/lz4-1.9.3/lib' make[1]: Entering directory '/data/sae-dp-files/tmp/apache-doris-0.15.0-incubating-src/thirdparty/src/lz4-1.9.3/programs' Installing binaries ln: failed to create symbolic link '/data/sae-dp-files/tmp/apache-doris-0.15.0-incubating-src/thirdparty/installed/bin/lz4c': File exists make[1]: *** [Makefile:166: install] Error 1 make[1]: Leaving directory '/data/sae-dp-files/tmp/apache-doris-0.15.0-incubating-src/thirdparty/src/lz4-1.9.3/programs' make: *** [Makefile:95: install] Error 2 ``` ### What You Expected? that's ok ### How to Reproduce? _No response_ ### Anything Else? _No response_ ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] adonis0147 edited a comment on pull request #7874: [Improvement] (fe-unit-test) Pass conf map to create Doris cluster.
adonis0147 edited a comment on pull request #7874: URL: https://github.com/apache/incubator-doris/pull/7874#issuecomment-1020760870 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] morningman commented on a change in pull request #7884: [Vectorized][Bug] This pr main fix the bug:
morningman commented on a change in pull request #7884:
URL: https://github.com/apache/incubator-doris/pull/7884#discussion_r791375122
##
File path: be/src/vec/columns/column_string.cpp
##
@@ -292,6 +292,7 @@ ColumnPtr ColumnString::replicate(const Offsets&
replicate_offsets) const {
void ColumnString::reserve(size_t n) {
offsets.reserve(n);
+chars.reserve(n * 5);
Review comment:
why multi 5? add comment to explain, or define it as a variable.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] EmmyMiao87 commented on a change in pull request #7880: [New feature](statistics) Step1: Statistics collection framework
EmmyMiao87 commented on a change in pull request #7880: URL: https://github.com/apache/incubator-doris/pull/7880#discussion_r791341879 ## File path: fe/fe-core/src/main/java/org/apache/doris/common/Config.java ## @@ -1582,4 +1582,23 @@ */ @ConfField public static boolean allow_replica_on_same_host = false; + +// statistics +/* + * the max unfinished statistics job number + */ +@ConfField(mutable = true, masterOnly = true) Review comment: Sure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] morningman merged pull request #7888: [fix](function) Change digital_masking function arg type to BIGINT
morningman merged pull request #7888: URL: https://github.com/apache/incubator-doris/pull/7888 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] EmmyMiao87 commented on pull request #7874: [Improvement] (fe-unit-test) Pass conf map to create Doris cluster.
EmmyMiao87 commented on pull request #7874: URL: https://github.com/apache/incubator-doris/pull/7874#issuecomment-1020758021 At present, Doris's single test framework will set the configuration directly through Config.xxx=xxx, such as QueryPlanTest.java:1806 What's the difference between the two settings? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] morningman commented on a change in pull request #7880: [New feature](statistics) Step1: Statistics collection framework
morningman commented on a change in pull request #7880: URL: https://github.com/apache/incubator-doris/pull/7880#discussion_r791338035 ## File path: fe/fe-core/src/main/java/org/apache/doris/common/Config.java ## @@ -1582,4 +1582,23 @@ */ @ConfField public static boolean allow_replica_on_same_host = false; + +// statistics +/* + * the max unfinished statistics job number + */ +@ConfField(mutable = true, masterOnly = true) Review comment: maybe add a `cbo_` prefix for all these config? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] EmmyMiao87 commented on pull request #7880: [New feature](statistics) Step1: Statistics collection framework
EmmyMiao87 commented on pull request #7880: URL: https://github.com/apache/incubator-doris/pull/7880#issuecomment-1020784586 Linked #6370 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] morningman edited a comment on issue #7502: Doris Roadmap 2022
morningman edited a comment on issue #7502: URL: https://github.com/apache/incubator-doris/issues/7502#issuecomment-1001839293 The following is the Roadmap for the Doris community in 2022. The plan includes all aspects of code features, documentation, community building, etc. that are to be developed, have already been developed, and have been completed but require ongoing optimization. > The plan is currently under discussion, so if you have comments or suggestions on any aspect of the plan or beyond, please feel free to leave a comment or send an email to d...@doris.apache.org. > We will gradually create issues or jira for each direction of the plan to describe and track the progress in detail. Developers who wish to contribute are also welcome to create issues directly and associate with them (just leave a comment) > The directions marked (**Good First Issue**) in the plan are more independent modules, which are more suitable for newbie tasks or developers who are new to Doris. If you are interested in the relevant direction, please contact us at d...@doris.apache.org or under this issue, and we will provide detailed guidance, help and discussion. > The directions marked with (**Q1**) are the current work to be completed in the first quarter of 2022. We will update the schedule and progress of other directions gradually. > The marked (**Done & Optimizing**) directions are the directions that are currently completed but need continuous optimization. Such as ease of use, feature additions, and documentation additions. > We encourage developers to discuss anything in the dev mailing list, to subscribe to the mailing list please refer to [How to subscribe](http://doris.incubator.apache.org/master/en/community/subscribe-mail-list.html). ## Features - [ ] #7571 - [ ] Extensible new query optimizer framework - [ ] #6370 - [ ] Standard test set support and performance enhancements + TPC-DS feature pass rate 100% + TPC-H performance enhancements - [ ] #7572 - [ ] Pipeline execution engine - [ ] Algorithm Concurrency Control and Resource Control - [ ] #7573 - [ ] #7570 - [ ] Map - [ ] Struct - [ ] #7574 Provides Schemaless semantics for fast analysis of semi-structured data. - [ ] Json - [ ] #7575 (Q1) Supports cold data storage to object storage at partition granularity with remote access capabilities and local Cache acceleration. - [ ] #7503 Doris' current "materialized view" is more of a "materialized index" concept. Doris will later implement a true Materialized View to support full and incremental construction of single and multi-table views. - [ ] #7576 Provide Kudu-like data update support. - [ ] #7577 - [ ] WindowFunnel - [ ] #7578 Support for the new UDF framework has solved the problems of high writing difficulty, poor isolation, and poor compatibility with existing C++ frameworks. - [ ] UDF - [ ] UDAF - [ ] UDTF - [ ] #7579 (**Good First Issue**) - [ ] #7552 - [ ] #7650 - [ ] Add more resource limits - [ ] #7129 - [ ] More builtin function support - [ ] #7678 ## Performance Optimization - [ ] #7580 (Q1) - [ ] Query layer vectorization - [ ] Storage level vectorization - [ ] Vectorization function supplementation - [ ] Query layer storage layer arithmetic unification - [ ] Import Vectorization - [ ] Json Parsing Optimization (**Good First Issue**) - [ ] #7551 - [ ] #7743 Optimize the performance of compaction task. And try to refactor the compaction logic. For example, only one replica do the compaction and sync to other replicas. ## Stability and Observability - [ ] #7553 (Q1) Solve the problems of inaccurate memory prediction and OOM, and improve memory observability by global + thread + task level memory management. - [ ] #7581 Provides fine-grained IO speed limit, priority scheduling, etc. through global IO management. - [ ] #7582 Introduces OpenTelemetry to enhance system internal state observability and unify monitoring data format. ## Testing - [ ] #7583 - [ ] FE Refine the FE single test framework to support multi-node simulation testing of features. - [ ] BE Provide testing framework to simplify the difficulty of writing complex unit tests (e.g. data builds) for BE. - [ ] #7584 Provide Case collection or submission framework for refining and accumulating regression test sets. - [ ] #7585
[GitHub] [incubator-doris] morningman commented on a change in pull request #7883: [Bug] fix clang compile error
morningman commented on a change in pull request #7883: URL: https://github.com/apache/incubator-doris/pull/7883#discussion_r791683698 ## File path: be/CMakeLists.txt ## @@ -350,6 +350,11 @@ if (COMPILER_GCC) endif () if (COMPILER_CLANG) + +if(MAKE_TEST STREQUAL "OFF") +add_compile_options(-Qunused-arguments) Review comment: Why adding this? Add comment to explain ## File path: be/CMakeLists.txt ## @@ -350,6 +350,11 @@ if (COMPILER_GCC) endif () if (COMPILER_CLANG) + +if(MAKE_TEST STREQUAL "OFF") +add_compile_options(-Qunused-arguments) Review comment: Why adding this? Add comment to explain -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #7867: [build][scripts] add set -e to download_thirdparty.sh
github-actions[bot] commented on pull request #7867: URL: https://github.com/apache/incubator-doris/pull/7867#issuecomment-1020727642 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] adonis0147 commented on pull request #7874: [Improvement] (fe-unit-test) Pass conf map to create Doris cluster.
adonis0147 commented on pull request #7874: URL: https://github.com/apache/incubator-doris/pull/7874#issuecomment-1020760870 This alternative initializes the configuration before the cluster starts by serializing the configuration to `fe.conf`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #7868: [typo](sync-job) Corrigendum document backup-restore.md
github-actions[bot] commented on pull request #7868: URL: https://github.com/apache/incubator-doris/pull/7868#issuecomment-1020726510 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] morningman commented on pull request #7866: [fix](toolchain) Allow building when system contains libunwind
morningman commented on pull request #7866: URL: https://github.com/apache/incubator-doris/pull/7866#issuecomment-1020730993 > Btw, this is a strong indication to hosting all thirdparty libraries as cmake modules. We should gradually adopt the `contrib` folder from ClickHouse. This can be done after we graduated from Apache Incubator. So that we can use a new repo to save all contribs -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #7851: [improvement](executor) Displays cast information with implicit conversions in verbose
github-actions[bot] commented on pull request #7851: URL: https://github.com/apache/incubator-doris/pull/7851#issuecomment-1020999899 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] EmmyMiao87 commented on a change in pull request #7865: [fix](lateral-view) fix bugs of lateral view with CTE or where clause
EmmyMiao87 commented on a change in pull request #7865:
URL: https://github.com/apache/incubator-doris/pull/7865#discussion_r791407625
##
File path:
fe/fe-core/src/test/java/org/apache/doris/planner/TableFunctionPlanTest.java
##
@@ -470,4 +478,55 @@ public void aggColumnInOuterQuery() throws Exception {
Assert.assertTrue(explainString.contains("output slot id: 2"));
Assert.assertTrue(explainString.contains("tuple ids: 1 3"));
}
+
+@Test
+public void testLaterViewWithView() throws Exception {
+// test 1
+String createViewStr = "create view db1.v1 (k1,e1) as select k1,e1
from db1.table_for_view lateral view explode_split(k3,',') tmp as e1;";
+CreateViewStmt createViewStmt = (CreateViewStmt)
UtFrameUtils.parseAndAnalyzeStmt(createViewStr, ctx);
+Catalog.getCurrentCatalog().createView(createViewStmt);
+
+String sql = "select * from db1.v1;";
+String explainString = UtFrameUtils.getSQLPlanOrErrorMsg(ctx, sql,
true);
+Assert.assertTrue(explainString.contains("output slot id: 1 2"));
+// query again to see if it has error
+explainString = UtFrameUtils.getSQLPlanOrErrorMsg(ctx, sql, true);
+Assert.assertTrue(explainString.contains("output slot id: 1 2"));
+}
+
+@Test
+public void testLaterViewWithWhere() throws Exception {
+String sql = "select k1,e1 from db1.table_for_view lateral view
explode_split(k3,',') tmp as e1 where k1 in (select k2 from
db1.table_for_view);";
+String explainString = UtFrameUtils.getSQLPlanOrErrorMsg(ctx, sql,
true);
+Assert.assertTrue(explainString.contains("join op: LEFT SEMI JOIN
(BROADCAST)"));
+Assert.assertTrue(explainString.contains("equal join conjunct: `k1` =
`k2`"));
+Assert.assertTrue(!explainString.contains("equal join conjunct: `k2` =
`k2`"));
+}
+
+@Test
+public void testLaterViewWithCTE() throws Exception {
+String sql = "with tmp as (select k1,e1 from db1.table_for_view
lateral view explode_split(k3,',') tmp2 as e1) select * from tmp;";
+String explainString = UtFrameUtils.getSQLPlanOrErrorMsg(ctx, sql,
true);
+Assert.assertTrue(explainString.contains("table function:
explode_split(`default_cluster:db1`.`table_for_view`.`k3`, ',') "));
+}
+
+@Ignore
+// errCode = 2, detailMessage = Unknown column 'e1' in 'table list'
+public void testLaterViewWithCTEBug() throws Exception {
+String sql = "with tmp as (select * from db1.table_for_view where
k2=1) select k1,e1 from tmp lateral view explode_split(k3,',') tmp2 as e1;";
+String explainString = UtFrameUtils.getSQLPlanOrErrorMsg(ctx, sql,
true);
+Assert.assertTrue(!explainString.contains("Unknown column 'e1' in
'table list'"));
+}
+
+@Ignore
+// errCode = 2, detailMessage = Unknown column 'e1' in 'table list'
+public void testLaterViewUnknowColumnBug() throws Exception {
Review comment:
Why ignore this single test
##
File path: fe/fe-core/src/main/java/org/apache/doris/analysis/TableRef.java
##
@@ -611,7 +622,16 @@ public String tableRefToSql() {
// if (resolvedPath_ != null) path =
resolvedPath_.getFullyQualifiedRawPath();
// return ToSqlUtils.getPathSql(path) + ((aliasSql != null) ? " " +
aliasSql : "");
-return name.toSql() + ((aliasSql != null) ? " " + aliasSql : "");
+// tbl1
+// tbl1 alias_tbl1
+// tbl1 alias_tbl1 lateral view explode_split(k1, ",") tmp1 as e1
+String tblName = name.toSql() + ((aliasSql != null) ? " " + aliasSql :
"");
Review comment:
Which printing will be mainly affected after the change here?
##
File path:
fe/fe-core/src/test/java/org/apache/doris/planner/TableFunctionPlanTest.java
##
@@ -470,4 +478,55 @@ public void aggColumnInOuterQuery() throws Exception {
Assert.assertTrue(explainString.contains("output slot id: 2"));
Assert.assertTrue(explainString.contains("tuple ids: 1 3"));
}
+
+@Test
+public void testLaterViewWithView() throws Exception {
+// test 1
+String createViewStr = "create view db1.v1 (k1,e1) as select k1,e1
from db1.table_for_view lateral view explode_split(k3,',') tmp as e1;";
+CreateViewStmt createViewStmt = (CreateViewStmt)
UtFrameUtils.parseAndAnalyzeStmt(createViewStr, ctx);
+Catalog.getCurrentCatalog().createView(createViewStmt);
+
+String sql = "select * from db1.v1;";
+String explainString = UtFrameUtils.getSQLPlanOrErrorMsg(ctx, sql,
true);
+Assert.assertTrue(explainString.contains("output slot id: 1 2"));
+// query again to see if it has error
+explainString = UtFrameUtils.getSQLPlanOrErrorMsg(ctx, sql, true);
+Assert.assertTrue(explainString.contains("output slot id: 1 2"));
+}
+
+@Test
+public void testLaterViewWithWhere() throws Exception {
+Str
[GitHub] [incubator-doris] wangbo commented on a change in pull request #7852: (refactor)(vectorized)refactor SegmentIterator::BitmapRangeIterator and SegmentIterator
wangbo commented on a change in pull request #7852:
URL: https://github.com/apache/incubator-doris/pull/7852#discussion_r791328561
##
File path: be/src/olap/rowset/segment_v2/segment_iterator.cpp
##
@@ -872,13 +867,13 @@ Status SegmentIterator::next_batch(vectorized::Block*
block) {
RETURN_IF_ERROR(_init(true));
_inited = true;
if (!_vec_pred_column_ids.empty() ||
!_short_cir_pred_column_ids.empty()) {
-_block_rowids.reserve(_opts.block_row_max);
+_block_rowids.resize(_opts.block_row_max);
Review comment:
When first to init SegmentIterator, _block_rowids needs to reserve;
Because _block_rowids is accessed by operator[] directly
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] morningman closed issue #7887: [Bug] Alias function `digital_masking` result is wrong
morningman closed issue #7887: URL: https://github.com/apache/incubator-doris/issues/7887 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] caiconghui commented on pull request #7876: [improvement](rewrite) Make RewriteDateLiteralRule to be compatible with mysql
caiconghui commented on pull request #7876: URL: https://github.com/apache/incubator-doris/pull/7876#issuecomment-1020763094 cc @morningman @xinghuayu007 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] HappenLee closed issue #7825: [Vectorized][Bug] coalesce function get wrong data when result type is int/decimal type
HappenLee closed issue #7825: URL: https://github.com/apache/incubator-doris/issues/7825 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #7886: [docs]correct SeaTunnel website hyperlink
github-actions[bot] commented on pull request #7886: URL: https://github.com/apache/incubator-doris/pull/7886#issuecomment-1020902421 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #7883: [Bug] fix clang compile error
github-actions[bot] commented on pull request #7883: URL: https://github.com/apache/incubator-doris/pull/7883#issuecomment-1021199650 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #7836: (optimization)(vectorized)(olap) optimize pageDecoder by batch processing
github-actions[bot] commented on pull request #7836: URL: https://github.com/apache/incubator-doris/pull/7836#issuecomment-1020864917 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #7888: [fix](function) Change digital_masking function arg type to BIGINT
github-actions[bot] commented on pull request #7888: URL: https://github.com/apache/incubator-doris/pull/7888#issuecomment-1020997318 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] HappenLee merged pull request #7827: [Vectorized][Bug] fix bug of coalesce function
HappenLee merged pull request #7827: URL: https://github.com/apache/incubator-doris/pull/7827 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] EmmyMiao87 commented on a change in pull request #7851: (executor) Displays cast information with implicit conversions in verbose
EmmyMiao87 commented on a change in pull request #7851: URL: https://github.com/apache/incubator-doris/pull/7851#discussion_r791312807 ## File path: fe/fe-core/src/main/java/org/apache/doris/qe/StmtExecutor.java ## @@ -676,6 +676,21 @@ private void analyzeAndGenerateQueryPlan(TQueryOptions tQueryOptions) throws Use } if (explainOptions != null) parsedStmt.setIsExplain(explainOptions); } + Review comment: Why do you need to set explain option repeatedly here? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] blackstar-baba commented on pull request #7879: [improvement](FeMetaVersion)Some if conditions for comparing CatalogJournalVersion do not use the FeMetaVersion constant
blackstar-baba commented on pull request #7879: URL: https://github.com/apache/incubator-doris/pull/7879#issuecomment-1020782685 PTAL -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] morningman merged pull request #7862: [fix](ldap) fix ldap password logic
morningman merged pull request #7862: URL: https://github.com/apache/incubator-doris/pull/7862 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] luzhijing closed pull request #7849: [docs]correct SeaTunnel website hyperlink
luzhijing closed pull request #7849: URL: https://github.com/apache/incubator-doris/pull/7849 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] wangbo commented on a change in pull request #7855: (optimization)(vectorized) improving Column inserter
wangbo commented on a change in pull request #7855:
URL: https://github.com/apache/incubator-doris/pull/7855#discussion_r791325529
##
File path: be/src/vec/columns/column_vector.h
##
@@ -120,10 +120,13 @@ class ColumnVector final : public
COWHelper>
ColumnVector(std::initializer_list il) : data {il} {}
void insert_res_column(const uint16_t* sel, size_t sel_size,
vectorized::ColumnVector* res_ptr) {
+auto& res_data = res_ptr->data;
+res_data.reserve(res_ptr->size() + sel_size);
Review comment:
```suggestion
res_data.reserve(sel_size);
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] zuochunwei commented on a change in pull request #7852: (refactor)(vectorized)refactor SegmentIterator::BitmapRangeIterator and SegmentIterator
zuochunwei commented on a change in pull request #7852:
URL: https://github.com/apache/incubator-doris/pull/7852#discussion_r791334747
##
File path: be/src/olap/rowset/segment_v2/segment_iterator.cpp
##
@@ -872,13 +867,13 @@ Status SegmentIterator::next_batch(vectorized::Block*
block) {
RETURN_IF_ERROR(_init(true));
_inited = true;
if (!_vec_pred_column_ids.empty() ||
!_short_cir_pred_column_ids.empty()) {
-_block_rowids.reserve(_opts.block_row_max);
+_block_rowids.resize(_opts.block_row_max);
Review comment:
reverve just alloc memory for optimization reason.
there is one basic rule: any reserve calls can be removed, this removing
operation will not influence the correctness of result.
according this rule, if we remove this _block_rowids.reserve(), the will
crash, so we should use resize instead.
##
File path: be/src/olap/rowset/segment_v2/segment_iterator.cpp
##
@@ -872,13 +867,13 @@ Status SegmentIterator::next_batch(vectorized::Block*
block) {
RETURN_IF_ERROR(_init(true));
_inited = true;
if (!_vec_pred_column_ids.empty() ||
!_short_cir_pred_column_ids.empty()) {
-_block_rowids.reserve(_opts.block_row_max);
+_block_rowids.resize(_opts.block_row_max);
Review comment:
reserve只是预分配内存，是而且只是性能的考虑，代码中任何调用reserve()的地方应该可以被删除，而不会影响正确性。
比如vector，调用reserve后不会改变vector的size，随后的at(index)会崩，operator[index]的行为也依赖于实现，所以，这个地方用resize更合适
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] zhengshiJ commented on a change in pull request #7851: (executor) Displays cast information with implicit conversions in verbose
zhengshiJ commented on a change in pull request #7851: URL: https://github.com/apache/incubator-doris/pull/7851#discussion_r791336129 ## File path: fe/fe-core/src/main/java/org/apache/doris/qe/StmtExecutor.java ## @@ -676,6 +676,21 @@ private void analyzeAndGenerateQueryPlan(TQueryOptions tQueryOptions) throws Use } if (explainOptions != null) parsedStmt.setIsExplain(explainOptions); } + Review comment: deleted -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #7864: remove useless if statement in segment_writer.cpp
github-actions[bot] commented on pull request #7864: URL: https://github.com/apache/incubator-doris/pull/7864#issuecomment-1020732262 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #7890: fix error while compiling with -O3
github-actions[bot] commented on pull request #7890: URL: https://github.com/apache/incubator-doris/pull/7890#issuecomment-1021200632 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] morningman merged pull request #7784: [Bug] Consider backend status when more than one backends exists in same host
morningman merged pull request #7784: URL: https://github.com/apache/incubator-doris/pull/7784 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] morningman closed issue #7783: [Bug] Consider backend status when more than one backends exists in same host
morningman closed issue #7783: URL: https://github.com/apache/incubator-doris/issues/7783 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[incubator-doris] branch master updated (461b352 -> b435a54)
This is an automated email from the ASF dual-hosted git repository. morningman pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-doris.git. from 461b352 [fix](function) Change digital_masking function arg type to BIGINT (#7888) add b435a54 [fix] Consider backend status when more than one backends exists in same host (#7784) No new revisions were added by this update. Summary of changes: fe/fe-core/src/main/java/org/apache/doris/system/SystemInfoService.java | 1 + 1 file changed, 1 insertion(+) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[incubator-doris] branch master updated: [chore][fix][doc](fe-plugin)(mysqldump) fix build auditlog plugin error (#7804)
This is an automated email from the ASF dual-hosted git repository.
morningman pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-doris.git
The following commit(s) were added to refs/heads/master by this push:
new 4bdeef3 [chore][fix][doc](fe-plugin)(mysqldump) fix build auditlog
plugin error (#7804)
4bdeef3 is described below
commit 4bdeef3b64413847f4a0b0c011d3eacad1e15b9e
Author: Zhengguo Yang
AuthorDate: Wed Jan 26 09:11:23 2022 +0800
[chore][fix][doc](fe-plugin)(mysqldump) fix build auditlog plugin error
(#7804)
1. fix problems when build fe_plugins
2. format
3. add docs about dump data using mysql dump
---
build_plugin.sh| 6 -
docs/.vuepress/sidebar/en.js | 1 +
docs/.vuepress/sidebar/zh-CN.js| 1 +
.../administrator-guide/export_with_mysql_dump.md | 41 +
docs/en/community/how-to-deploy-to-maven.md| 16 +-
.../administrator-guide/export_with_mysql_dump.md | 41 +
docs/zh-CN/community/how-to-deploy-to-maven.md | 16 +-
extension/DataX/doriswriter/pom.xml| 9 +-
.../doriswriter/src/main/assembly/package.xml | 12 +-
extension/flink-doris-connector/pom.xml| 15 +-
extension/spark-doris-connector/pom.xml| 18 +--
fe/checkstyle.xml | 5 -
fe/fe-common/pom.xml | 19 ---
fe/fe-core/pom.xml | 121 +-
fe/fe-core/src/test/resources/log4j2.xml | 18 +--
fe/pom.xml | 135 +---
fe/spark-dpp/pom.xml | 58 ++-
fe_plugins/auditdemo/pom.xml | 15 +-
fe_plugins/auditdemo/src/main/assembly/zip.xml | 3 +-
fe_plugins/auditloader/pom.xml | 15 +-
fe_plugins/auditloader/src/main/assembly/zip.xml | 3 +-
fe_plugins/pom.xml | 14 --
fs_brokers/apache_hdfs_broker/conf/hdfs-site.xml | 3 -
fs_brokers/apache_hdfs_broker/pom.xml | 64 +---
samples/connect/java/client/pom.xml| 15 +-
samples/doris-demo/flink-demo/pom.xml | 12 +-
samples/doris-demo/pom.xml | 85 +-
samples/doris-demo/spark-demo/pom.xml | 13 +-
samples/doris-demo/spring-jdbc-demo/pom.xml| 179 +
.../src/main/resources/logback.xml | 62 ---
.../src/main/resources/mybatis/DemoMapper.xml | 7 +-
.../src/main/resources/mybatis/mybatis-config.xml | 20 +--
samples/doris-demo/stream-load-demo/pom.xml| 6 +-
33 files changed, 339 insertions(+), 709 deletions(-)
diff --git a/build_plugin.sh b/build_plugin.sh
index b6c1375..c8b35ed 100755
--- a/build_plugin.sh
+++ b/build_plugin.sh
@@ -82,12 +82,6 @@ echo "Get params:
CLEAN -- $CLEAN
"
-# check if palo-fe.jar exist
-if [ ! -f "$DORIS_HOME/fe/fe-core/target/palo-fe.jar" ]; then
-echo "ERROR: palo-fe.jar does not exist. Please build FE first"
-exit -1
-fi
-
cd ${DORIS_HOME}
PLUGIN_MODULE=
if [ ${ALL_PLUGIN} -eq 1 ] ; then
diff --git a/docs/.vuepress/sidebar/en.js b/docs/.vuepress/sidebar/en.js
index 9081d69..8fee736 100644
--- a/docs/.vuepress/sidebar/en.js
+++ b/docs/.vuepress/sidebar/en.js
@@ -202,6 +202,7 @@ module.exports = [
"bucket-shuffle-join",
"dynamic-partition",
"export-manual",
+ "export_with_mysql_dump",
"outfile",
"privilege",
"ldap",
diff --git a/docs/.vuepress/sidebar/zh-CN.js b/docs/.vuepress/sidebar/zh-CN.js
index e94119d..0d2e382 100644
--- a/docs/.vuepress/sidebar/zh-CN.js
+++ b/docs/.vuepress/sidebar/zh-CN.js
@@ -201,6 +201,7 @@ module.exports = [
"bucket-shuffle-join",
"dynamic-partition",
"export-manual",
+ "export_with_mysql_dump",
"outfile",
"partition_cache",
"privilege",
diff --git a/docs/en/administrator-guide/export_with_mysql_dump.md
b/docs/en/administrator-guide/export_with_mysql_dump.md
new file mode 100644
index 000..ee2ce76
--- /dev/null
+++ b/docs/en/administrator-guide/export_with_mysql_dump.md
@@ -0,0 +1,41 @@
+---
+{
+"title": "Use mysqldump data to export table structure or data",
+"language": "en"
+}
+---
+
+
+
+# Use mysqldump data to export table structure or data
+Doris has supported exporting data or table structures through the `mysqldump`
tool after version 0.15
+
+## Example
+### Export
+ 1. Export the table1 table in the test database: `mysqldump -h127.0.0.1
-P9030 -uroot --no-tablespaces --databases test --tables table1`
+ 2. Export the table1 table structure in the test database: `mysqldump
-h127.0.0.1 -P9030 -uroot --no-tablespaces --databases test --tables table1
--no-data`
+ 3. Export all tables in the test1, test2 database: `mysqldump -h127.
[GitHub] [incubator-doris] morningman merged pull request #7804: [build] fix build auditlog plugin error
morningman merged pull request #7804: URL: https://github.com/apache/incubator-doris/pull/7804 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] morningman merged pull request #7830: [Doc] OS Installation Requirements
morningman merged pull request #7830: URL: https://github.com/apache/incubator-doris/pull/7830 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[incubator-doris] branch master updated: [docs] OS Installation Requirements (#7830)
This is an automated email from the ASF dual-hosted git repository. morningman pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-doris.git The following commit(s) were added to refs/heads/master by this push: new e3bc232 [docs] OS Installation Requirements (#7830) e3bc232 is described below commit e3bc23257869115fdd5a06ac6a7d76f6fbc6b1c0 Author: jiafeng.zhang AuthorDate: Wed Jan 26 09:11:41 2022 +0800 [docs] OS Installation Requirements (#7830) --- docs/en/installing/install-deploy.md| 22 ++ docs/zh-CN/installing/install-deploy.md | 22 ++ 2 files changed, 44 insertions(+) diff --git a/docs/en/installing/install-deploy.md b/docs/en/installing/install-deploy.md index ff0a5b5..4bf683c 100644 --- a/docs/en/installing/install-deploy.md +++ b/docs/en/installing/install-deploy.md @@ -50,6 +50,28 @@ Doris, as an open source MPP architecture OLAP database, can run on most mainstr | Java | 1.8 and above | | GCC | 4.8.2 and above | + OS Installation Requirements + +# Set the maximum number of open file handles in the system + + +vi /etc/security/limits.conf +*soft nofile 65536 +*hard nofile 65536 + + +# Clock synchronization + +The metadata of Doris requires the time precision to be less than 5000ms, so all machines in the cluster need to synchronize the clocks to avoid service exceptions caused by inconsistencies in metadata caused by clock problems. + +# Close the swap partition (swap) + +The Linux swap partition will cause serious performance problems for Doris, you need to disable the swap partition before installation + +# Linux file system + +Here we recommend using the ext4 file system. When installing the operating system, please select the ext4 file system. + Development Test Environment | Module | CPU | Memory | Disk | Network | Instance Number| diff --git a/docs/zh-CN/installing/install-deploy.md b/docs/zh-CN/installing/install-deploy.md index 78ac756..07a9c13 100644 --- a/docs/zh-CN/installing/install-deploy.md +++ b/docs/zh-CN/installing/install-deploy.md @@ -49,6 +49,28 @@ Doris 作为一款开源的 MPP 架构 OLAP 数据库，能够运行在绝大多 | Java | 1.8 及以上 | | GCC | 4.8.2 及以上 | + 操作系统安装要求 + +# 设置系统最大打开文件句柄数 + +``` +vi /etc/security/limits.conf +* soft nofile 65536 +* hard nofile 65536 +``` + +# 时钟同步 + +Doris 的元数据要求时间精度要小于5000ms，所以所有集群所有机器要进行时钟同步，避免因为时钟问题引发的元数据不一致导致服务出现异常。 + +# 关闭交换分区（swap） + +Linux交换分区会给Doris带来很严重的性能问题，需要在安装之前禁用交换分区 + +# Liunx文件系统 + +这里我们推荐使用ext4文件系统，在安装操作系统的时候，请选择ext4文件系统。 + 开发测试环境 | 模块 | CPU | 内存 | 磁盘 | 网络 | 实例数量 | - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] blackstar-baba commented on pull request #7879: [improvement](FeMetaVersion)Some if conditions for comparing CatalogJournalVersion do not use the FeMetaVersion constant
blackstar-baba commented on pull request #7879: URL: https://github.com/apache/incubator-doris/pull/7879#issuecomment-1021816346 @morningman PTAL -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] zenoyang opened a new pull request #7892: [Refactor](Vectorized) create columns by tuple slots
zenoyang opened a new pull request #7892: URL: https://github.com/apache/incubator-doris/pull/7892 # Proposed changes Issue Number: close #7881 - create columns by tuple slots - insert columns from tuple slots ## Problem Summary: Describe the overview of changes. ## Checklist(Required) 1. Does it affect the original behavior: (Yes/No/I Don't know) 2. Has unit tests been added: (Yes/No/No Need) 3. Has document been added or modified: (Yes/No/No Need) 4. Does it need to update dependencies: (Yes/No) 5. Are there any changes that cannot be rolled back: (Yes/No) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] wangbo commented on a change in pull request #7892: [Refactor](Vectorized) create columns by tuple slots
wangbo commented on a change in pull request #7892:
URL: https://github.com/apache/incubator-doris/pull/7892#discussion_r792307629
##
File path: be/src/olap/rowset/segment_v2/binary_dict_page.cpp
##
@@ -266,8 +267,9 @@ Status BinaryDictPageDecoder::next_batch(size_t* n,
vectorized::MutableColumnPtr
// fill null bitmap here, not null;
// todo(wb) using SIMD speed up here
+auto& null_map_data = nullable_column->get_null_map_data();
for (int i = 0; i < max_fetch; i++) {
-nullable_column->get_null_map_data().push_back(0);
+null_map_data.push_back(0);
Review comment:
the TODO batch insert has been solved by #7836
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] wangbo commented on a change in pull request #7855: (optimization)(vectorized) improving Column inserter
wangbo commented on a change in pull request #7855:
URL: https://github.com/apache/incubator-doris/pull/7855#discussion_r792308864
##
File path: be/src/vec/columns/column_vector.h
##
@@ -120,10 +120,13 @@ class ColumnVector final : public
COWHelper>
ColumnVector(std::initializer_list il) : data {il} {}
void insert_res_column(const uint16_t* sel, size_t sel_size,
vectorized::ColumnVector* res_ptr) {
+auto& res_data = res_ptr->data;
+res_data.reserve(res_ptr->size() + sel_size);
Review comment:
```suggestion
res_data.reserve(sel_size);
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] morningman closed issue #7882: [Bug] Clang13 compile error
morningman closed issue #7882: URL: https://github.com/apache/incubator-doris/issues/7882 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] morningman merged pull request #7883: [Bug] fix clang compile error
morningman merged pull request #7883: URL: https://github.com/apache/incubator-doris/pull/7883 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[incubator-doris] branch master updated: [chore] fix clang compile error (#7883)
This is an automated email from the ASF dual-hosted git repository.
morningman pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-doris.git
The following commit(s) were added to refs/heads/master by this push:
new cd73a6b [chore] fix clang compile error (#7883)
cd73a6b is described below
commit cd73a6b84bf147816e9f02a8db50be5883fe7ae3
Author: Pxl <952130...@qq.com>
AuthorDate: Wed Jan 26 12:53:35 2022 +0800
[chore] fix clang compile error (#7883)
---
be/CMakeLists.txt | 5 +
be/src/olap/storage_engine.cpp| 10 +-
be/test/exprs/bitmap_function_test.cpp| 4 ++--
be/test/olap/delta_writer_test.cpp| 7 +--
be/test/olap/rowset/beta_rowset_test.cpp | 2 +-
be/test/olap/rowset/rowset_converter_test.cpp | 10 +-
be/test/olap/rowset/segment_v2/bitmap_index_test.cpp | 5 +++--
.../segment_v2/bloom_filter_index_reader_writer_test.cpp | 2 +-
.../olap/rowset/segment_v2/column_reader_writer_test.cpp | 6 +++---
.../olap/rowset/segment_v2/ordinal_page_index_test.cpp| 5 +++--
be/test/olap/rowset/segment_v2/segment_test.cpp | 2 +-
be/test/olap/rowset/segment_v2/zone_map_index_test.cpp| 15 ++-
be/test/tools/benchmark_tool.cpp | 5 +++--
run-be-ut.sh | 10 +-
14 files changed, 56 insertions(+), 32 deletions(-)
diff --git a/be/CMakeLists.txt b/be/CMakeLists.txt
index 4e919c5..b7cd148 100644
--- a/be/CMakeLists.txt
+++ b/be/CMakeLists.txt
@@ -350,6 +350,11 @@ if (COMPILER_GCC)
endif ()
if (COMPILER_CLANG)
+
+if(MAKE_TEST STREQUAL "OFF")
+add_compile_options(-Qunused-arguments)
+endif()
+
# TODO: Fix all these warnings
add_compile_options(
-Wno-address-of-packed-member
diff --git a/be/src/olap/storage_engine.cpp b/be/src/olap/storage_engine.cpp
index 9de74cd..aa1af9c 100644
--- a/be/src/olap/storage_engine.cpp
+++ b/be/src/olap/storage_engine.cpp
@@ -570,11 +570,11 @@ void StorageEngine::stop() {
THREAD_JOIN(_tablet_checkpoint_tasks_producer_thread);
#undef THREAD_JOIN
-#define THREADS_JOIN(threads) \
-for (const auto thread : threads) { \
-if (thread) { \
-thread->join(); \
-} \
+#define THREADS_JOIN(threads)\
+for (const auto& thread : threads) { \
+if (thread) {\
+thread->join(); \
+}\
}
THREADS_JOIN(_path_gc_threads);
diff --git a/be/test/exprs/bitmap_function_test.cpp
b/be/test/exprs/bitmap_function_test.cpp
index 346a841..d789164 100644
--- a/be/test/exprs/bitmap_function_test.cpp
+++ b/be/test/exprs/bitmap_function_test.cpp
@@ -854,7 +854,7 @@ TEST_F(BitmapFunctionsTest, bitmap_not) {
expected = BigIntVal(0);
ASSERT_EQ(expected, result);
-bitmap1 = BitmapValue({1});
+bitmap1 = BitmapValue(1);
bitmap2 = BitmapValue({2, 1});
bitmap_src = convert_bitmap_to_string(ctx, bitmap1);
@@ -1151,7 +1151,7 @@ TEST_F(BitmapFunctionsTest, sub_bitmap) {
ASSERT_EQ(res, convert_bitmap_to_string(ctx, bitmap5));
res = BitmapFunctions::sub_bitmap(ctx, bitmap_src, BigIntVal(-1),
BigIntVal(2));
-BitmapValue bitmap6({500});
+BitmapValue bitmap6(500);
ASSERT_EQ(res, convert_bitmap_to_string(ctx, bitmap6));
res = BitmapFunctions::sub_bitmap(ctx, bitmap_src, BigIntVal(-7),
BigIntVal(6));
diff --git a/be/test/olap/delta_writer_test.cpp
b/be/test/olap/delta_writer_test.cpp
index c4eb7f9..159e9e7 100644
--- a/be/test/olap/delta_writer_test.cpp
+++ b/be/test/olap/delta_writer_test.cpp
@@ -438,7 +438,8 @@ TEST_F(TestDeltaWriter, write) {
memcpy(var_ptr->ptr, "abcde", 5);
var_ptr->len = 5;
-DecimalV2Value decimal_value(1.1);
+DecimalV2Value decimal_value;
+decimal_value.assign_from_double(1.1);
*(DecimalV2Value*)(tuple->get_slot(slots[9]->tuple_offset())) =
decimal_value;
*(int8_t*)(tuple->get_slot(slots[10]->tuple_offset())) = -127;
@@ -463,7 +464,9 @@ TEST_F(TestDeltaWriter, write) {
memcpy(var_ptr->ptr, "abcde", 5);
var_ptr->len = 5;
-DecimalV2Value val_decimal(1.1);
+DecimalV2Value val_decimal;
+val_decimal.assign_from_double(1.1);
+
*(DecimalV2Value*)(tuple->get_slot(slots[19]->tuple_offset())) =
val_decimal;
res = delta_writer->write(tuple);
diff --git a/be/test/olap/rowset/beta_rowset_test.cpp
b/be/test/olap/rowset/beta_rowset_test.cpp
index 7f8190f..c3b11ef 100644
--- a/be/test/olap/rowset/beta_rowset_test.cpp
+++ b/be/test/olap/rowset/beta_rowset_test.cpp
@@ -353,7 +353,7 @@ TEST_F(BetaRowsetTest, BasicFunctionTest) {
} // namespace dori
[GitHub] [incubator-doris] morningman closed issue #7889: [Bug] there are some errors while compiling with option -O3
morningman closed issue #7889: URL: https://github.com/apache/incubator-doris/issues/7889 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] morningman merged pull request #7890: fix error while compiling with -O3
morningman merged pull request #7890: URL: https://github.com/apache/incubator-doris/pull/7890 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[incubator-doris] branch master updated: [chore] fix error while compiling with -O3 (#7890)
This is an automated email from the ASF dual-hosted git repository.
morningman pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-doris.git
The following commit(s) were added to refs/heads/master by this push:
new f227472 [chore] fix error while compiling with -O3 (#7890)
f227472 is described below
commit f227472db2d8ec207efac2fc144b08a969acb98f
Author: Henry2SS <45096548+henry...@users.noreply.github.com>
AuthorDate: Wed Jan 26 12:53:56 2022 +0800
[chore] fix error while compiling with -O3 (#7890)
---
be/CMakeLists.txt | 1 +
be/src/vec/common/cow.h | 3 +++
2 files changed, 4 insertions(+)
diff --git a/be/CMakeLists.txt b/be/CMakeLists.txt
index b7cd148..cfe4a46 100644
--- a/be/CMakeLists.txt
+++ b/be/CMakeLists.txt
@@ -347,6 +347,7 @@ if (COMPILER_GCC)
#
https://stackoverflow.com/questions/67584073/gcc-11-false-array-subscript-is-partly-outside-array-bounds-warning
#
https://stackoverflow.com/questions/69426070/gcc-11-order-of-arguments-triggers-false-positive-wstringop-overflow-is-this-bu
set(CXX_COMMON_FLAGS "${CXX_COMMON_FLAGS} -Wno-array-bounds
-Wno-stringop-overread")
+add_compile_options(-Wno-stringop-overflow)
endif ()
if (COMPILER_CLANG)
diff --git a/be/src/vec/common/cow.h b/be/src/vec/common/cow.h
index 58ae14d..a235c64 100644
--- a/be/src/vec/common/cow.h
+++ b/be/src/vec/common/cow.h
@@ -147,9 +147,12 @@ protected:
return *this;
}
+#pragma GCC diagnostic push
+#pragma GCC diagnostic ignored "-Wmaybe-uninitialized"
intrusive_ptr(intrusive_ptr&& rhs) : t(rhs.t) {
rhs.t = nullptr;
}
+#pragma GCC diagnostic pop
intrusive_ptr& operator=(intrusive_ptr&& rhs) {
intrusive_ptr(static_cast(rhs)).swap(*this);
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] morningman opened a new pull request #7893: [chore] Remove BUILD_TYPE bcc
morningman opened a new pull request #7893: URL: https://github.com/apache/incubator-doris/pull/7893 # Proposed changes Issue Number: close #xxx ## Problem Summary: no need anymore ## Checklist(Required) 1. Does it affect the original behavior: (Yes/No/I Don't know) 2. Has unit tests been added: (Yes/No/No Need) 3. Has document been added or modified: (Yes/No/No Need) 4. Does it need to update dependencies: (Yes/No) 5. Are there any changes that cannot be rolled back: (Yes/No) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] zenoyang commented on a change in pull request #7892: [refactor](vectorized) Create and insert columns by tuple slots
zenoyang commented on a change in pull request #7892:
URL: https://github.com/apache/incubator-doris/pull/7892#discussion_r792328979
##
File path: be/src/olap/rowset/segment_v2/binary_dict_page.cpp
##
@@ -266,8 +267,9 @@ Status BinaryDictPageDecoder::next_batch(size_t* n,
vectorized::MutableColumnPtr
// fill null bitmap here, not null;
// todo(wb) using SIMD speed up here
+auto& null_map_data = nullable_column->get_null_map_data();
for (int i = 0; i < max_fetch; i++) {
-nullable_column->get_null_map_data().push_back(0);
+null_map_data.push_back(0);
Review comment:
ok, I removed this change.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] morningman commented on a change in pull request #7391: [feature-wip](iceberg) Step1: Support create Iceberg external table
morningman commented on a change in pull request #7391:
URL: https://github.com/apache/incubator-doris/pull/7391#discussion_r792333809
##
File path: fe/fe-core/src/main/java/org/apache/doris/catalog/Catalog.java
##
@@ -2631,7 +2636,15 @@ public Frontend getFeByName(String name) {
public void createDb(CreateDbStmt stmt) throws DdlException {
final String clusterName = stmt.getClusterName();
String fullDbName = stmt.getFullDbName();
-long id = 0L;
+Map properties = stmt.getProperties();
+
+long id = getNextId();
+Database db = new Database(id, fullDbName);
+db.setClusterName(clusterName);
+// check and analyze database properties before create database
+db.getDbProperties().putAll(properties);
Review comment:
merge these 2 methods:
```
db.getDbProperties().putAll(properties);
db.getDbProperties().checkAndBuildProperties();
```
to
```
db.addAndBuildProperties(properties);
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] FireFreedomK commented on issue #5796: flinksql doris insert exception:stream load error: too many filtered rows
FireFreedomK commented on issue #5796: URL: https://github.com/apache/incubator-doris/issues/5796#issuecomment-1021931259 你好，我用的0.15.0版本的doris，用spark dataframe将数据导入doris，也出现了一样的问题，请问你是怎么解决的 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] hf200012 commented on issue #5796: flinksql doris insert exception:stream load error: too many filtered rows
hf200012 commented on issue #5796: URL: https://github.com/apache/incubator-doris/issues/5796#issuecomment-1021932959 > 你好，我用的0.15.0版本的doris，用spark dataframe将数据导入doris，也出现了一样的问题，请问你是怎么解决的 This is your data problem, there may be several cases, the field separator (the number of columns that cause the field split to be inconsistent with the number of columns in the table, generally more), the row separator, the actual number of imported fields and the table are inconsistent, Imported data causes some fields to be overly long, etc. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] hf200012 merged pull request #7886: [docs]correct SeaTunnel website hyperlink
hf200012 merged pull request #7886: URL: https://github.com/apache/incubator-doris/pull/7886 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[incubator-doris] branch master updated (f227472 -> 38660d5)
This is an automated email from the ASF dual-hosted git repository. jiafengzheng pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-doris.git. from f227472 [chore] fix error while compiling with -O3 (#7890) add 38660d5 [docs]correct SeaTunnel website hyperlink and PPMC into the Member List (#7886) No new revisions were added by this update. Summary of changes: docs/en/community/members.md | 18 +- docs/en/extending-doris/seatunnel/flink-sink.md| 2 +- docs/en/extending-doris/seatunnel/spark-sink.md| 2 +- docs/zh-CN/community/members.md| 8 docs/zh-CN/extending-doris/seatunnel/flink-sink.md | 2 +- docs/zh-CN/extending-doris/seatunnel/spark-sink.md | 2 +- 6 files changed, 17 insertions(+), 17 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] obobj opened a new pull request #7895: [docs] add function substring document
obobj opened a new pull request #7895: URL: https://github.com/apache/incubator-doris/pull/7895 # Proposed changes 1. add function substring document ## Checklist(Required) 1. Does it affect the original behavior: (Yes/No/I Don't know) 2. Has unit tests been added: (Yes/No/No Need) 3. Has document been added or modified: (Yes/No/No Need) 4. Does it need to update dependencies: (Yes/No) 5. Are there any changes that cannot be rolled back: (Yes/No) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #7391: [feature-wip](iceberg) Step1: Support create Iceberg external table
github-actions[bot] commented on pull request #7391: URL: https://github.com/apache/incubator-doris/pull/7391#issuecomment-1021945487 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] morningman commented on a change in pull request #7821: [feature](meta) Support show compactionTooSlowTablets and largeDataSizeTablets
morningman commented on a change in pull request #7821:
URL: https://github.com/apache/incubator-doris/pull/7821#discussion_r792379415
##
File path:
fe/fe-core/src/main/java/org/apache/doris/common/proc/StatisticProcDir.java
##
@@ -56,6 +58,7 @@
.add("DbId").add("DbName").add("TableNum").add("PartitionNum")
.add("IndexNum").add("TabletNum").add("ReplicaNum").add("UnhealthyTabletNum")
.add("InconsistentTabletNum").add("CloningTabletNum").add("BadTabletNum")
+.add("CompactionTooSlowTabletNum").add("LargeDataSizeTabletNum")
Review comment:
```suggestion
.add("CompactionTooSlowTabletNum").add("OversizeTablets")
```
##
File path: docs/zh-CN/administrator-guide/config/fe_config.md
##
@@ -2144,3 +2144,33 @@ load 标签清理器将每隔 `label_clean_interval_second` 运行一次以清
是否为 Master FE 节点独有的配置项：false
是否允许同一个 tablet 的多个副本分布在同一个 host 上。这个参数主要用于本地测试是，方便搭建多个 BE 已测试某些多副本情况。不要用于非测试环境。
+
+### min_version_count_indicate_replica_compaction_too_slow
+
+默认值：300
+
+是否可以动态配置：true
+
+是否为 Master FE 节点独有的配置项：true
+
+版本计数阈值，用来判断副本做 compaction 的速度是否太慢
+
+### valid_version_count_delta_ratio_between_replicas
+
+默认值：0.5
+
+是否可以动态配置：true
+
+是否为 Master FE 节点独有的配置项：true
+
+最慢副本的版本计数与最快副本的差异有效比率阈值，如果设置 repair_slow_replica 为 true，则用于判断是否修复最慢的副本
Review comment:
```suggestion
最慢副本的版本计数与最快副本的差异有效比率阈值，如果设置 `repair_slow_replica` 为 true，则用于判断是否修复最慢的副本
```
##
File path: docs/en/administrator-guide/config/fe_config.md
##
@@ -2121,3 +2121,32 @@ Only for Master FE: false
Whether to allow multiple replicas of the same tablet to be distributed on the
same host. This parameter is mainly used for local testing, to facilitate
building multiple BEs to test certain multi-replica situations. Do not use it
for non-test environments.
+### min_version_count_indicate_replica_compaction_too_slow
+
+Default: 300
+
+Dynamically configured: true
+
+Only for Master FE: true
+
+The version count threshold used to judge whether replica compaction is too
slow
+
+### valid_version_count_delta_ratio_between_replicas
+
+Default: 0.5
+
+Dynamically configured: true
+
+Only for Master FE: true
+
+The valid ratio threshold of the difference between the version count of the
slowest replica and the fastest replica. If repair_slow_replica is set to true,
it is used to determine whether to repair the slowest replica
Review comment:
```suggestion
The valid ratio threshold of the difference between the version count of the
slowest replica and the fastest replica. If `repair_slow_replica` is set to
true, it is used to determine whether to repair the slowest replica
```
##
File path:
fe/fe-core/src/main/java/org/apache/doris/common/proc/StatisticProcDir.java
##
@@ -113,10 +116,15 @@ public ProcNodeInterface lookup(String dbIdStr) throws
AnalysisException {
int inconsistentTabletNum;
int cloningTabletNum;
int badTabletNum;
+int compactionTooSlowTabletNum;
+int largeDataSizeTabletNum;
Review comment:
```suggestion
int oversizeTabletNum;
```
##
File path:
fe/fe-core/src/main/java/org/apache/doris/common/proc/IncompleteTabletsProcNode.java
##
@@ -29,6 +29,7 @@
public class IncompleteTabletsProcNode implements ProcNodeInterface {
public static final ImmutableList TITLE_NAMES = new
ImmutableList.Builder()
.add("UnhealthyTablets").add("InconsistentTablets").add("CloningTablets").add("BadTablets")
+.add("CompactionTooSlowTablets").add("LargeDataSizeTablets")
Review comment:
```suggestion
.add("CompactionTooSlowTablets").add("OversizeTablets")
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
