[GitHub] [incubator-doris] EmmyMiao87 commented on a change in pull request #8084: [community] modify the doc of verifying apache release
EmmyMiao87 commented on a change in pull request #8084: URL: https://github.com/apache/incubator-doris/pull/8084#discussion_r807643798 ## File path: docs/zh-CN/extending-doris/flink-doris-connector.md ## @@ -65,18 +65,13 @@ Flink Doris Connector 可以支持通过 Flink 操作（读取、插入、修改 ## 编译与安装 -在 `extension/flink-doris-connector/` 源码目录下执行： Review comment: Please change the English doc together ~ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] BiteTheDDDDt opened a new issue #8085: [Feature][Vectorized] support function `group_concat()`
BiteThet opened a new issue #8085: URL: https://github.com/apache/incubator-doris/issues/8085 ### Search before asking - [X] I had searched in the [issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and found no similar issues. ### Description support function `group_concat()` ### Use case select group_concat(substr(lo_orderpriority,1,1)) from lineorder group by lo_custkey order by lo_custkey; ### Related issues _No response_ ### Are you willing to submit PR? - [X] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] BiteTheDDDDt opened a new pull request #8086: [Feature][Vectorized] support function `group_concat()`
BiteThet opened a new pull request #8086: URL: https://github.com/apache/incubator-doris/pull/8086 # Proposed changes Issue Number: close #8085 ## Problem Summary: Describe the overview of changes. ## Checklist(Required) 1. Does it affect the original behavior: (Yes/No/I Don't know) 2. Has unit tests been added: (Yes/No/No Need) 3. Has document been added or modified: (Yes/No/No Need) 4. Does it need to update dependencies: (Yes/No) 5. Are there any changes that cannot be rolled back: (Yes/No) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #8084: [community] modify the doc of verifying apache release
github-actions[bot] commented on pull request #8084: URL: https://github.com/apache/incubator-doris/pull/8084#issuecomment-1041222076 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] adonis0147 closed issue #8002: [Feature] [array-type] Create table with nested array type.
adonis0147 closed issue #8002: URL: https://github.com/apache/incubator-doris/issues/8002 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #8042: [Improvement] check simd instructions before start
github-actions[bot] commented on pull request #8042: URL: https://github.com/apache/incubator-doris/pull/8042#issuecomment-1041231694 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] adonis0147 opened a new pull request #8089: [Feature][array-type] Support ArrayLiteral in SQL.
adonis0147 opened a new pull request #8089: URL: https://github.com/apache/incubator-doris/pull/8089 # Proposed changes Issue Number: close #8074 . ## Problem Summary: Please refer to #8074 . ## Checklist(Required) 1. Does it affect the original behavior: No. 2. Has unit tests been added: Yes. 3. Has document been added or modified: No. 4. Does it need to update dependencies: No. 5. Are there any changes that cannot be rolled back: No. ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] zuochunwei opened a new pull request #8090: (Vectorized)(Optimization) refactor RowRef/RowRefList to decrease hash table memory size
zuochunwei opened a new pull request #8090: URL: https://github.com/apache/incubator-doris/pull/8090 # Proposed changes 1. use union + bitfield C++ language features to reuse the memory, reduce the size of RowRefList from 32 to 16 bytes 2. ssb test indicate（Q4.1）: benefit from RowRefList size reduce, hash table's memory consuming is greatly decreased, almost 40% 3. refactor RowRefList iterate logic, i think it becomes more simple after modifing 4. run performance, change is little, i need more time to test for it Issue Number: close #xxx ## Problem Summary: Describe the overview of changes. ## Checklist(Required) 1. Does it affect the original behavior: (Yes/No/I Don't know) 2. Has unit tests been added: (Yes/No/No Need) 3. Has document been added or modified: (Yes/No/No Need) 4. Does it need to update dependencies: (Yes/No) 5. Are there any changes that cannot be rolled back: (Yes/No) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] adonis0147 closed pull request #8075: [Feature] [array-type] Support ArrayLiteral in SQL.
adonis0147 closed pull request #8075: URL: https://github.com/apache/incubator-doris/pull/8075 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] adonis0147 commented on pull request #8075: [Feature] [array-type] Support ArrayLiteral in SQL.
adonis0147 commented on pull request #8075: URL: https://github.com/apache/incubator-doris/pull/8075#issuecomment-1041249689 Hi @morningman , I rebased the master in order to merge the branch into master for convenience in future. I'm sorry to mess the commit history up. I tried to resolve it by force push my commits to my own branch, but it didn't help. Therefore, I opened a new pull request #8089 . Please take a look when you are free. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] wangbo opened a new pull request #8091: [feature][storage]support delete condition
wangbo opened a new pull request #8091: URL: https://github.com/apache/incubator-doris/pull/8091 # Proposed changes [Vectorization]Support delete condition in Storage layer ## Problem Summary: Describe the overview of changes. ## Checklist(Required) 1. Does it affect the original behavior: (Yes/No/I Don't know) 2. Has unit tests been added: (Yes/No/No Need) 3. Has document been added or modified: (Yes/No/No Need) 4. Does it need to update dependencies: (Yes/No) 5. Are there any changes that cannot be rolled back: (Yes/No) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] wangbo commented on pull request #8091: [feature][storage]support delete condition
wangbo commented on pull request #8091: URL: https://github.com/apache/incubator-doris/pull/8091#issuecomment-1041259592 I'll add some UT tests later -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris-flink-connector] JNSimba commented on pull request #4: When doing checkpoint, write cache data to doris to prevent loss
JNSimba commented on pull request #4: URL: https://github.com/apache/incubator-doris-flink-connector/pull/4#issuecomment-1041288537 Great, thank you for your pr. And `DorisDynamicTableSink.getSinkRuntimeProvider` maybe also need to be modified ? To ensure the correct task submitted by FlinkSQL -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] yiguolei opened a new pull request #8092: [Refactor] Remove unused code in data dir
yiguolei opened a new pull request #8092: URL: https://github.com/apache/incubator-doris/pull/8092 # Proposed changes Issue Number: close #xxx ## Problem Summary: Describe the overview of changes. ## Checklist(Required) 1. Does it affect the original behavior: (Yes/No/I Don't know) 2. Has unit tests been added: (Yes/No/No Need) 3. Has document been added or modified: (Yes/No/No Need) 4. Does it need to update dependencies: (Yes/No) 5. Are there any changes that cannot be rolled back: (Yes/No) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] 254288935 commented on issue #8073: [Bug] BE crashed after altering int field to string
254288935 commented on issue #8073: URL: https://github.com/apache/incubator-doris/issues/8073#issuecomment-1041352870 alter table时，目前不支持BIGINT到CHAR的转换， 是否需要支持BIGINT到VARCHAR的转换呢？ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] BiteTheDDDDt commented on a change in pull request #8090: (Vectorized)(Optimization) refactor RowRef/RowRefList to decrease hash table memory size
BiteThet commented on a change in pull request #8090:
URL: https://github.com/apache/incubator-doris/pull/8090#discussion_r807805187
##
File path: be/src/vec/exec/join/join_op.h
##
@@ -22,19 +22,42 @@
#include "vec/core/block.h"
namespace doris::vectorized {
+
/// Reference to the row in block.
struct RowRef {
using SizeT = uint32_t; /// Do not use size_t cause of memory economy
-const Block* block = nullptr;
-SizeT row_num = 0;
-// Use in right join to mark row is visited
-// TODO: opt the varaible to use it only need
-bool visited = false;
+// using union to represent two cases
+// 1. when RowRefList containing only one RowRef, visited + blockptr are
valid
+// 2. when RowRefList contaning multi RowRef, it's used through next
pointer
+union {
+struct {
+uint64_t visited : 1;
Review comment:
Why is it defined as uint64 here, not use automatic memory alignment?
##
File path: be/src/vec/exec/join/vhash_join_node.cpp
##
@@ -286,9 +286,12 @@ struct ProcessHashTableProbe {
// use in right join to change visited state after
// exec the vother join conjunt
-std::vector visited_map;
+std::vector visited_map;
visited_map.reserve(1.2 * _batch_size);
+#define visited_map_or(index, value) do { if (value)
((RowRef*)visited_map[index])->visited = 1; } while (0)
Review comment:
The macros in our code style need uppercase
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] caiconghui opened a new issue #8093: [Bug] Execute showVariablesStmt with where expression return empty resultset
caiconghui opened a new issue #8093: URL: https://github.com/apache/incubator-doris/issues/8093 ### Search before asking - [X] I had searched in the [issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and found no similar issues. ### Version master ### What's Wrong? MySQL [(none)]> show variables where Variable_name = "wait_timeout"; Empty set (0.01 sec) ### What You Expected? MySQL [(none)]> show variables where Variable_name = "wait_timeout"; +---+---+ | Variable_name | Value | +---+---+ | wait_timeout | 28800 | +---+---+ 1 row in set (0.02 sec) ### How to Reproduce? execute showVariablesStmt with where expression ### Anything Else? _No response_ ### Are you willing to submit PR? - [X] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] caiconghui opened a new pull request #8094: [Bug](Variables) Fix bug that execute showVariablesStmt with where expression return empty resultset
caiconghui opened a new pull request #8094: URL: https://github.com/apache/incubator-doris/pull/8094 # Proposed changes Issue Number: close #8093 ## Problem Summary: This Bug is introduced by PR #7936 , which change key type of connectionMap from Long to Integer, which cause connectionMap could not find connectContext by connectionId ## Checklist(Required) 1. Does it affect the original behavior: (No) 2. Has unit tests been added: (No) 3. Has document been added or modified: (No) 4. Does it need to update dependencies: (No) 5. Are there any changes that cannot be rolled back: (No) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] yangzhg opened a new issue #8095: [Enhancement] Data scan may very slow and create too much scanner for small tablet in some case
yangzhg opened a new issue #8095: URL: https://github.com/apache/incubator-doris/issues/8095 ### Search before asking - [X] I had searched in the [issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and found no similar issues. ### Description Since OlapScanNode calculates the number of scanners per tablet based on the scan range, https://github.com/apache/incubator-doris/blob/143c4085ee58007954f3eef8910556f5b8ce6b39/be/src/exec/olap_scan_node.cpp#L771 when the query conditions are few, the number of scanners for the tablet may be too large, and when calculating the amount of data scanned by each scanner in the next step, it will be use `scanners_per_tablet` as the denominator, https://github.com/apache/incubator-doris/blob/143c4085ee58007954f3eef8910556f5b8ce6b39/be/src/exec/olap_scan_node.cpp#L786 so each scanner scans only a small amount of data. The total number of scanners is roughly the number of scan ranges multiplied by the number of conds in each range, which will result in a large number of scanners scanning a small amount of data at the same time. ### Solution Consider the tablet size when calculating scanners_per_tablet, no more scanners are generated for small tablets ### Are you willing to submit PR? - [X] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] yangzhg opened a new pull request #8096: [optimize] optimze tablet read, avoid to create too much scanner for small tablet
yangzhg opened a new pull request #8096: URL: https://github.com/apache/incubator-doris/pull/8096 # Proposed changes Issue Number: close #8095 ## Problem Summary: optimze tablet read, avoid to create too much scanner for small tablet ## Checklist(Required) 1. Does it affect the original behavior: (No) 2. Has unit tests been added: (No Need) 3. Has document been added or modified: (Yes) 4. Does it need to update dependencies: (No) 5. Are there any changes that cannot be rolled back: (No) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] morningman commented on issue #8073: [Bug] BE crashed after altering int field to string
morningman commented on issue #8073: URL: https://github.com/apache/incubator-doris/issues/8073#issuecomment-1041434275 > alter table时，目前不支持BIGINT到CHAR的转换， 是否需要支持BIGINT到VARCHAR的转换呢？ I think we do support convert BIGINT to CHAR. Why you say it not support? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #8072: [fix](load)(compaction) Fix NodeChannel coredump bug and modify some compaction logic
github-actions[bot] commented on pull request #8072: URL: https://github.com/apache/incubator-doris/pull/8072#issuecomment-1041461196 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] jackwener commented on pull request #8071: Correct the wrong words in the prompt information
jackwener commented on pull request #8071: URL: https://github.com/apache/incubator-doris/pull/8071#issuecomment-1041469968 `be/src/olap/olap_common.h` `UNKNOW OLAP_FIELD_TYPE_STRING` can be replaced with `UNKNOWN ... ` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] jackwener commented on pull request #8092: [Refactor] Remove unused code in data dir
jackwener commented on pull request #8092: URL: https://github.com/apache/incubator-doris/pull/8092#issuecomment-1041475756 LGTM -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #8092: [Refactor] Remove unused code in data dir
github-actions[bot] commented on pull request #8092: URL: https://github.com/apache/incubator-doris/pull/8092#issuecomment-1041476788 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] yiguolei commented on pull request #8094: [Bug](Variables) Fix bug that execute showVariablesStmt with where expression return empty resultset
yiguolei commented on pull request #8094: URL: https://github.com/apache/incubator-doris/pull/8094#issuecomment-1042471030 Why change connection id from long to int? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] caiconghui commented on pull request #8094: [Bug](Variables) Fix bug that execute showVariablesStmt with where expression return empty resultset
caiconghui commented on pull request #8094: URL: https://github.com/apache/incubator-doris/pull/8094#issuecomment-1042500205 > Why change connection id from long to int? the original type of connection id in connect context in is int, -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] yangzhg commented on a change in pull request #8089: [Feature][array-type] Support ArrayLiteral in SQL.
yangzhg commented on a change in pull request #8089: URL: https://github.com/apache/incubator-doris/pull/8089#discussion_r808611931 ## File path: fe/fe-core/src/main/java/org/apache/doris/catalog/ArrayType.java ## @@ -17,16 +17,15 @@ package org.apache.doris.catalog; +import com.google.common.base.Preconditions; +import com.google.common.base.Strings; +import com.google.gson.annotations.SerializedName; Review comment: the import order is ``` org.apache.doris com org java javax ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] yangzhg commented on a change in pull request #8089: [Feature][array-type] Support ArrayLiteral in SQL.
yangzhg commented on a change in pull request #8089:
URL: https://github.com/apache/incubator-doris/pull/8089#discussion_r808612432
##
File path: fe/fe-core/src/main/java/org/apache/doris/catalog/Column.java
##
@@ -159,6 +159,7 @@ public Column(Column column) {
public void createChildrenColumn(Type type, Column column) {
if (type.isArrayType()) {
Column c = new Column(COLUMN_ARRAY_CHILDREN, ((ArrayType)
type).getItemType());
+c.setIsAllowNull(true);
Review comment:
why always allow null
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] caiconghui merged pull request #8035: [Refactor][heartbeat] Make get fe heart response by thrift
caiconghui merged pull request #8035: URL: https://github.com/apache/incubator-doris/pull/8035 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] caiconghui closed issue #8037: [Enhancement] Make get fe heartbeat response more stable by use thrift as default method
caiconghui closed issue #8037: URL: https://github.com/apache/incubator-doris/issues/8037 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] caiconghui commented on pull request #8094: [Bug](Variables) Fix bug that execute showVariablesStmt with where expression return empty resultset
caiconghui commented on pull request #8094: URL: https://github.com/apache/incubator-doris/pull/8094#issuecomment-1042505989 > Why change connection id from long to int? The value range of connection id is from int_min to int_max, I think it is enough, so I don't change the connection id type in connection context -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #8057: [Doc]Add version upgrade instructions
github-actions[bot] commented on pull request #8057: URL: https://github.com/apache/incubator-doris/pull/8057#issuecomment-1042506141 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[incubator-doris] branch master updated (143c408 -> e6fedff)
This is an automated email from the ASF dual-hosted git repository. caiconghui pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-doris.git. from 143c408 [Feature][Vectorized] support aggregate function ndv()/approx_count_distinct() (#8044) add e6fedff [Refactor][heartbeat] Make get fe heart response by thrift (#8035) No new revisions were added by this update. Summary of changes: docs/en/administrator-guide/config/fe_config.md| 10 docs/zh-CN/administrator-guide/config/fe_config.md | 10 .../main/java/org/apache/doris/common/Config.java | 9 .../java/org/apache/doris/system/HeartbeatMgr.java | 61 +- .../org/apache/doris/system/HeartbeatMgrTest.java | 53 +-- 5 files changed, 3 insertions(+), 140 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] yangzhg commented on a change in pull request #8053: [docs] fix document date-time-functions typo
yangzhg commented on a change in pull request #8053: URL: https://github.com/apache/incubator-doris/pull/8053#discussion_r808617190 ## File path: docs/zh-CN/sql-reference/sql-functions/date-time-functions/convert_tz.md ## @@ -25,14 +25,12 @@ under the License. --> # convert_tz - -## Syntax +## description +### Syntax Review comment: why？ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] yangzhg commented on a change in pull request #8053: [docs] fix document date-time-functions typo
yangzhg commented on a change in pull request #8053: URL: https://github.com/apache/incubator-doris/pull/8053#discussion_r808617190 ## File path: docs/zh-CN/sql-reference/sql-functions/date-time-functions/convert_tz.md ## @@ -25,14 +25,12 @@ under the License. --> # convert_tz - -## Syntax +## description +### Syntax Review comment: Why doesn't `docs/en/sql-reference/sql-functions/date-time-functions/curtime.md` follow this structure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] yiguolei opened a new pull request #8099: Remove version hash fe
yiguolei opened a new pull request #8099: URL: https://github.com/apache/incubator-doris/pull/8099 # Proposed changes Issue Number: close #xxx ## Problem Summary: Describe the overview of changes. ## Checklist(Required) 1. Does it affect the original behavior: (Yes/No/I Don't know) 2. Has unit tests been added: (Yes/No/No Need) 3. Has document been added or modified: (Yes/No/No Need) 4. Does it need to update dependencies: (Yes/No) 5. Are there any changes that cannot be rolled back: (Yes/No) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #8046: [Bug] fix incorrect serialized_size of TDigest object
github-actions[bot] commented on pull request #8046: URL: https://github.com/apache/incubator-doris/pull/8046#issuecomment-1042512373 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] morningman merged pull request #7828: [Vectorized][Feature] add ColumnHLL to support hll type
morningman merged pull request #7828: URL: https://github.com/apache/incubator-doris/pull/7828 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] morningman closed issue #7826: [Vectorized][Feature]add ColumnHLL to support HLL type
morningman closed issue #7826: URL: https://github.com/apache/incubator-doris/issues/7826 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[incubator-doris] branch master updated (e6fedff -> 0003822)
This is an automated email from the ASF dual-hosted git repository.
morningman pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-doris.git.
from e6fedff [Refactor][heartbeat] Make get fe heart response by thrift
(#8035)
add 0003822 [feature](vec) add ColumnHLL to support hll type (#7828)
No new revisions were added by this update.
Summary of changes:
be/src/exec/text_converter.hpp | 5 +-
be/src/olap/hll.h | 123 -
be/src/olap/row_block2.cpp | 22 +++-
be/src/olap/schema.cpp | 4 +-
be/src/runtime/types.h | 3 +-
be/src/vec/CMakeLists.txt | 1 +
.../aggregate_function_hll_union_agg.h | 23 ++--
be/src/vec/columns/column_complex.h| 17 +++
be/src/vec/core/block.cpp | 13 ++-
be/src/vec/core/types.h| 8 ++
be/src/vec/data_types/data_type.cpp| 11 +-
be/src/vec/data_types/data_type_bitmap.h | 10 +-
be/src/vec/data_types/data_type_hll.cpp| 116 +++
.../{data_type_bitmap.h => data_type_hll.h}| 34 ++
be/src/vec/exec/volap_scanner.cpp | 20 ++--
be/src/vec/exec/vschema_scan_node.cpp | 10 +-
be/src/vec/functions/hll_cardinality.cpp | 67 +--
be/src/vec/functions/hll_empty.cpp | 13 ++-
be/src/vec/functions/hll_hash.cpp | 57 +++---
be/src/vec/sink/vtablet_sink.cpp | 20
20 files changed, 431 insertions(+), 146 deletions(-)
create mode 100644 be/src/vec/data_types/data_type_hll.cpp
copy be/src/vec/data_types/{data_type_bitmap.h => data_type_hll.h} (77%)
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] morningman merged pull request #8036: [feature](spark-load) add Hive Bitmap UDFs
morningman merged pull request #8036: URL: https://github.com/apache/incubator-doris/pull/8036 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[incubator-doris] branch master updated (0003822 -> 264f384)
This is an automated email from the ASF dual-hosted git repository.
morningman pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-doris.git.
from 0003822 [feature](vec) add ColumnHLL to support hll type (#7828)
add 264f384 [feature](spark-load) add Hive Bitmap UDFs (#8036)
No new revisions were added by this update.
Summary of changes:
docs/en/extending-doris/hive-bitmap-udf.md | 97 ++
docs/zh-CN/extending-doris/hive-bitmap-udf.md | 104 +++
fe/hive-udf/pom.xml| 124 ++
.../org/apache/doris/common/BitmapValueUtil.java} | 46 +++
.../java/org/apache/doris/udf/BitmapAndUDF.java| 75 +++
.../java/org/apache/doris/udf/BitmapCountUDF.java | 70 ++
.../java/org/apache/doris/udf/BitmapOrUDF.java | 76 +++
.../java/org/apache/doris/udf/BitmapUnionUDAF.java | 132 +++
.../java/org/apache/doris/udf/BitmapXorUDF.java| 76 +++
.../java/org/apache/doris/udf/ToBitmapUDAF.java| 143 +
fe/pom.xml | 1 +
11 files changed, 916 insertions(+), 28 deletions(-)
create mode 100644 docs/en/extending-doris/hive-bitmap-udf.md
create mode 100644 docs/zh-CN/extending-doris/hive-bitmap-udf.md
create mode 100644 fe/hive-udf/pom.xml
copy fe/{fe-core/src/main/java/org/apache/doris/common/MetaIndex.java =>
hive-udf/src/main/java/org/apache/doris/common/BitmapValueUtil.java} (51%)
create mode 100644
fe/hive-udf/src/main/java/org/apache/doris/udf/BitmapAndUDF.java
create mode 100644
fe/hive-udf/src/main/java/org/apache/doris/udf/BitmapCountUDF.java
create mode 100644
fe/hive-udf/src/main/java/org/apache/doris/udf/BitmapOrUDF.java
create mode 100644
fe/hive-udf/src/main/java/org/apache/doris/udf/BitmapUnionUDAF.java
create mode 100644
fe/hive-udf/src/main/java/org/apache/doris/udf/BitmapXorUDF.java
create mode 100644
fe/hive-udf/src/main/java/org/apache/doris/udf/ToBitmapUDAF.java
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] ztgoto opened a new issue #8100: stream load fe 307 redirect Access denied for default_cluster:default_cluster@host
ztgoto opened a new issue #8100:
URL: https://github.com/apache/incubator-doris/issues/8100
### Search before asking
- [X] I had searched in the
[issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and
found no similar issues.
### Version
0.15.0
### What's Wrong?
我在三台主机上部署fe be各三个节点 启用enable_http_server_v2。
通过postman调用stream load api来导入数据，采用basic认证，发现会出现随机成功或报Access denied for
default_cluster的问题
```
{
"TxnId": -1,
"Label": "6eadd95f-cc7d-46e8-9749-c7cef0bb846b",
"Status": "Fail",
"Message": "errCode = 2, detailMessage = Access denied for
default_cluster:default_cluster@x",
"NumberTotalRows": 0,
"NumberLoadedRows": 0,
"NumberFilteredRows": 0,
"NumberUnselectedRows": 0,
"LoadBytes": 0,
"LoadTimeMs": 0,
"BeginTxnTimeMs": 0,
"StreamLoadPutTimeMs": 0,
"ReadDataTimeMs": 0,
"WriteDataTimeMs": 0,
"CommitAndPublishTimeMs": 0
}
```
后通过网络抓包发现请求fe通过307跳转到be 认证信息会被改写。
奇怪的是如果跳转到的be所在ip如果与请求的fe所在ip为同一个Authorization就是正常的（这个暂不清楚是不是postman争对同一host
307缓存所致）。
后来查看fe源码 org.apache.doris.httpv2.rest.RestBaseController#redirectTo
在重定向到be时会在重定向请求上加入authInfo.fullUserName + ":" + authInfo.password形式
而authInfo.fullUserName 来自
org.apache.doris.httpv2.controller.BaseController#parseAuthInfo
```
int index = authString.indexOf(":");
authInfo.fullUserName = authString.substring(0, index);
final String[] elements = authInfo.fullUserName.split("@");
if (elements != null && elements.length < 2) {
authInfo.fullUserName =
ClusterNamespace.getFullName(SystemInfoService.DEFAULT_CLUSTER,
authInfo.fullUserName);
authInfo.cluster = SystemInfoService.DEFAULT_CLUSTER;
} else if (elements != null && elements.length == 2) {
authInfo.fullUserName = ClusterNamespace.getFullName(elements[1],
elements[0]);
authInfo.cluster = elements[1];
}
```
说实话这块我是真心没看懂，认证信息采用的是 basic username:password的形式，代码取出用户名，然后在用户名前面又用冒号拼接成
default_cluster:username
最后导致307跳转到be的地址
Location:
http://default_cluster:username:password@ip:8040/api/xxx/xxx/_stream_load?
变成了这样 导致认证失败
但是我没理解的是如果是这个问题应该每次都失败不可能有成功的才对。后来用Wireshark抓包发现307重定向到be Authorization 不一样。
如果fe与重定向的be ip相同 Authorization相同 为basic username:password形式
如果fe与重定向的be ip不同 Authorization发生了变化 为basic
default_cluster:username:password形式
这就是我前面所说的不清楚是不是postman缓存的问题导致的成功。
暂不清楚是不是我配置问题导致
### What You Expected?
请核实
### How to Reproduce?
_No response_
### Anything Else?
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of
Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] morningman merged pull request #8042: [Improvement] check simd instructions before start
morningman merged pull request #8042: URL: https://github.com/apache/incubator-doris/pull/8042 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[incubator-doris] branch master updated: [improvement] check simd instructions before start (#8042)
This is an automated email from the ASF dual-hosted git repository.
morningman pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-doris.git
The following commit(s) were added to refs/heads/master by this push:
new d1cb291 [improvement] check simd instructions before start (#8042)
d1cb291 is described below
commit d1cb2913c1da372dd9711569e9e3609a0cb22113
Author: yiguolei
AuthorDate: Thu Feb 17 10:46:03 2022 +0800
[improvement] check simd instructions before start (#8042)
Sometimes BE is build on a machine with SIMD instruction such as AVX2.
But the BE binary will be copied to a machine without AVX2. It will crashed
without any error message.
This PR will check the required SIMD instructions and print error messages
during startup.
---
be/src/common/utils.h | 2 +
be/src/service/doris_main.cpp | 199 ++
2 files changed, 201 insertions(+)
diff --git a/be/src/common/utils.h b/be/src/common/utils.h
index 61fed1d..c0ca459 100644
--- a/be/src/common/utils.h
+++ b/be/src/common/utils.h
@@ -21,6 +21,8 @@
namespace doris {
+#define ARRAY_SIZE(a) (sizeof(a)/sizeof((a)[0]))
+
struct AuthInfo {
std::string user;
std::string passwd;
diff --git a/be/src/service/doris_main.cpp b/be/src/service/doris_main.cpp
index fba3669..6c59b72 100644
--- a/be/src/service/doris_main.cpp
+++ b/be/src/service/doris_main.cpp
@@ -20,8 +20,12 @@
#include
#include
+#include
+#include
#include
+#include
#include
+#include
#include
#if defined(LEAK_SANITIZER)
@@ -40,6 +44,7 @@
#include "common/logging.h"
#include "common/resource_tls.h"
#include "common/status.h"
+#include "common/utils.h"
#include "env/env.h"
#include "olap/options.h"
#include "olap/storage_engine.h"
@@ -74,6 +79,200 @@ static void thrift_output(const char* x) {
} // namespace doris
+// These code is referenced from clickhouse
+// It is used to check the SIMD instructions
+enum class InstructionFail {
+NONE = 0,
+SSE3 = 1,
+SSSE3 = 2,
+SSE4_1 = 3,
+SSE4_2 = 4,
+POPCNT = 5,
+AVX = 6,
+AVX2 = 7,
+AVX512 = 8,
+ARM_NEON = 9
+};
+
+auto instruction_fail_to_string(InstructionFail fail)
+{
+switch (fail)
+{
+#define ret(x) return std::make_tuple(STDERR_FILENO, x, ARRAY_SIZE(x) - 1)
+case InstructionFail::NONE:
+ret("NONE");
+case InstructionFail::SSE3:
+ret("SSE3");
+case InstructionFail::SSSE3:
+ret("SSSE3");
+case InstructionFail::SSE4_1:
+ret("SSE4.1");
+case InstructionFail::SSE4_2:
+ret("SSE4.2");
+case InstructionFail::POPCNT:
+ret("POPCNT");
+case InstructionFail::AVX:
+ret("AVX");
+case InstructionFail::AVX2:
+ret("AVX2");
+case InstructionFail::AVX512:
+ret("AVX512");
+case InstructionFail::ARM_NEON:
+ret("ARM_NEON");
+}
+__builtin_unreachable();
+}
+
+
+sigjmp_buf jmpbuf;
+
+void sig_ill_check_handler(int, siginfo_t *, void *)
+{
+siglongjmp(jmpbuf, 1);
+}
+
+/// Check if necessary SSE extensions are available by trying to execute some
sse instructions.
+/// If instruction is unavailable, SIGILL will be sent by kernel.
+void check_required_instructions_impl(volatile InstructionFail & fail)
+{
+#if defined(__SSE3__)
+fail = InstructionFail::SSE3;
+__asm__ volatile ("addsubpd %%xmm0, %%xmm0" : : : "xmm0");
+#endif
+
+#if defined(__SSSE3__)
+fail = InstructionFail::SSSE3;
+__asm__ volatile ("pabsw %%xmm0, %%xmm0" : : : "xmm0");
+
+#endif
+
+#if defined(__SSE4_1__)
+fail = InstructionFail::SSE4_1;
+__asm__ volatile ("pmaxud %%xmm0, %%xmm0" : : : "xmm0");
+#endif
+
+#if defined(__SSE4_2__)
+fail = InstructionFail::SSE4_2;
+__asm__ volatile ("pcmpgtq %%xmm0, %%xmm0" : : : "xmm0");
+#endif
+
+/// Defined by -msse4.2
+#if defined(__POPCNT__)
+fail = InstructionFail::POPCNT;
+{
+uint64_t a = 0;
+uint64_t b = 0;
+__asm__ volatile ("popcnt %1, %0" : "=r"(a) :"r"(b) :);
+}
+#endif
+
+#if defined(__AVX__)
+fail = InstructionFail::AVX;
+__asm__ volatile ("vaddpd %%ymm0, %%ymm0, %%ymm0" : : : "ymm0");
+#endif
+
+#if defined(__AVX2__)
+fail = InstructionFail::AVX2;
+__asm__ volatile ("vpabsw %%ymm0, %%ymm0" : : : "ymm0");
+#endif
+
+#if defined(__AVX512__)
+fail = InstructionFail::AVX512;
+__asm__ volatile ("vpabsw %%zmm0, %%zmm0" : : : "zmm0");
+#endif
+
+#if defined(__ARM_NEON__)
+fail = InstructionFail::ARM_NEON;
+__asm__ volatile ("vadd.i32 q8, q8, q8" : : : "q8");
+#endif
+
+fail = InstructionFail::NONE;
+}
+
+bool write_retry(int fd, const char * data, size_t size)
+{
+if (!size)
+size = strlen(data);
+
+while (size != 0)
+{
+ssize_t res = ::write(fd, data, size);
+
+if ((-1 == res || 0 == res) && errno != E
[GitHub] [incubator-doris] morningman merged pull request #8046: [Bug] fix incorrect serialized_size of TDigest object
morningman merged pull request #8046: URL: https://github.com/apache/incubator-doris/pull/8046 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[incubator-doris] branch master updated (d1cb291 -> 53f22bb)
This is an automated email from the ASF dual-hosted git repository. morningman pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-doris.git. from d1cb291 [improvement] check simd instructions before start (#8042) add 53f22bb [fix] fix incorrect serialized_size of TDigest object (#8046) No new revisions were added by this update. Summary of changes: be/src/util/tdigest.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] morningman merged pull request #8048: [Doc]Added be -238 error code description
morningman merged pull request #8048: URL: https://github.com/apache/incubator-doris/pull/8048 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[incubator-doris] branch master updated: [doc] Added be -238 error code description (#8048)
This is an automated email from the ASF dual-hosted git repository. morningman pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-doris.git The following commit(s) were added to refs/heads/master by this push: new 79fd81f [doc] Added be -238 error code description (#8048) 79fd81f is described below commit 79fd81f035217682c9610ed35724c62d71ecbebc Author: jiafeng.zhang AuthorDate: Thu Feb 17 10:47:52 2022 +0800 [doc] Added be -238 error code description (#8048) Added be -238 error code description --- docs/en/administrator-guide/operation/be-olap-error-code.md| 3 ++- docs/zh-CN/administrator-guide/operation/be-olap-error-code.md | 1 + 2 files changed, 3 insertions(+), 1 deletion(-) diff --git a/docs/en/administrator-guide/operation/be-olap-error-code.md b/docs/en/administrator-guide/operation/be-olap-error-code.md index 7b6736a..5b4514c 100644 --- a/docs/en/administrator-guide/operation/be-olap-error-code.md +++ b/docs/en/administrator-guide/operation/be-olap-error-code.md @@ -70,7 +70,7 @@ under the License. | OLAP_ERR_CHECKSUM_ERROR | -208 | Check value error | | OLAP_ERR_SIGNATURE_ERROR | -209 | Signature error | | OLAP_ERR_CATCH_EXCEPTION | -210 | Exception caught | -| OLAP_ERR_PARSE_PROTOBUF_ERROR | -211 | Error parsing Protobuf +| OLAP_ERR_PARSE_PROTOBUF_ERROR | -211 | Error parsing Protobuf| | OLAP_ERR_INVALID_ROOT_PATH | -222 | Invalid root directory | | OLAP_ERR_NO_AVAILABLE_ROOT_PATH | -223 | No valid root directory | | OLAP_ERR_CHECK_LINES_ERROR | -224 | Check the number of lines error | @@ -87,6 +87,7 @@ under the License. | OLAP_ERR_TOO_MANY_VERSION | -235 | The tablet data version exceeds the maximum limit (default 500) | | OLAP_ERR_NOT_INITIALIZED | -236 | Cannot initialize | | OLAP_ERR_ALREADY_CANCELLED | -237 | Has been cancelled | +| OLAP_ERR_TOO_MANY_SEGMENTS | -238 | usually occurs when the amount of imported data in the same batch is too large, resulting in too many segment files for a tablet | | Command execution exception code | | | | OLAP_ERR_CE_CMD_PARAMS_ERROR | -300 | Command parameter error | | OLAP_ERR_CE_BUFFER_TOO_SMALL | -301 | Too many small files in the buffer | diff --git a/docs/zh-CN/administrator-guide/operation/be-olap-error-code.md b/docs/zh-CN/administrator-guide/operation/be-olap-error-code.md index 18ea2dc..7acfc77 100644 --- a/docs/zh-CN/administrator-guide/operation/be-olap-error-code.md +++ b/docs/zh-CN/administrator-guide/operation/be-olap-error-code.md @@ -96,6 +96,7 @@ under the License. | OLAP_ERR_TOO_MANY_VERSION| -235 | tablet的数据版本超过了最大限制（默认500）| | OLAP_ERR_NOT_INITIALIZED | -236 | 不能初始化 | | OLAP_ERR_ALREADY_CANCELLED | -237 | 已经被取消 | +| OLAP_ERR_TOO_MANY_SEGMENTS | -238 | 通常出现在同一批导入数据量过大的情况，从而导致某一个 tablet 的 Segment 文件过多 | | 命令执行异常代码 || | | OLAP_ERR_CE_CMD_PARAMS_ERROR | -300 | 命令参数错误 | | OLAP_ERR_CE_BUFFER_TOO_SMALL | -301 | 缓冲区太多小文件 | - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] yangzhg commented on pull request #8014: [Enhancement] Open clang compile warning option and resolve compile error Part1
yangzhg commented on pull request #8014: URL: https://github.com/apache/incubator-doris/pull/8014#issuecomment-1042514814 I had a pr #8069 have fix all warnings -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] morningman merged pull request #8050: [Function][Vectorized] Fix multi args function call the DATETIME type not effective in DATE type and add the alias function
morningman merged pull request #8050: URL: https://github.com/apache/incubator-doris/pull/8050 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] morningman closed issue #8047: [Vectorized][Bug] Some multi arg funtion use in DATETIME not effective in DATE type
morningman closed issue #8047: URL: https://github.com/apache/incubator-doris/issues/8047 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[incubator-doris] branch master updated: [feature][fix](vec)(function) Fix multi args function call the DATETIME type not effective in DATE type and add the alias function (#8050)
This is an automated email from the ASF dual-hosted git repository.
morningman pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-doris.git
The following commit(s) were added to refs/heads/master by this push:
new bef1b55 [feature][fix](vec)(function) Fix multi args function call
the DATETIME type not effective in DATE type and add the alias function (#8050)
bef1b55 is described below
commit bef1b55c1f86ec4e3efe9eee57497127d6d9648c
Author: HappenLee
AuthorDate: Thu Feb 17 10:49:25 2022 +0800
[feature][fix](vec)(function) Fix multi args function call the DATETIME
type not effective in DATE type and add the alias function (#8050)
1. Support some function alias of mod/fmod, adddate/add_data
2. Support some function of multi args: week, yearweek
3. Fix bug of multi args function call the DATETIME type not effective in
DATE type
---
be/src/olap/block_column_predicate.h | 1 -
be/src/vec/data_types/data_type_date.h | 3 +-
be/src/vec/data_types/data_type_date_time.cpp | 4 ---
be/src/vec/data_types/data_type_date_time.h| 2 +-
be/src/vec/functions/date_time_transforms.h| 22 ++--
.../function_date_or_datetime_computation.cpp | 11 ++
.../function_date_or_datetime_computation.h| 32 -
.../function_date_or_datetime_to_something.h | 7
be/src/vec/functions/modulo.cpp| 1 +
be/src/vec/functions/simple_function_factory.h | 10 +++---
be/src/vec/functions/time_of_function.cpp | 3 +-
be/src/vec/functions/to_time_function.cpp | 2 +-
be/test/vec/function/function_time_test.cpp| 42 ++
gensrc/script/doris_builtins_functions.py | 20 +--
14 files changed, 133 insertions(+), 27 deletions(-)
diff --git a/be/src/olap/block_column_predicate.h
b/be/src/olap/block_column_predicate.h
index ef2a4ca..6f679db 100644
--- a/be/src/olap/block_column_predicate.h
+++ b/be/src/olap/block_column_predicate.h
@@ -29,7 +29,6 @@ namespace doris {
// in the future
// TODO: support do predicate on Bitmap and ZoneMap, So we can use index of
column to do predicate on
// page and segment
-
class BlockColumnPredicate {
public:
BlockColumnPredicate() = default;
diff --git a/be/src/vec/data_types/data_type_date.h
b/be/src/vec/data_types/data_type_date.h
index b5d148b..685363c 100644
--- a/be/src/vec/data_types/data_type_date.h
+++ b/be/src/vec/data_types/data_type_date.h
@@ -27,7 +27,8 @@ namespace doris::vectorized {
class DataTypeDate final : public DataTypeNumberBase {
public:
TypeIndex get_type_id() const override { return TypeIndex::Date; }
-const char* get_family_name() const override { return "Date"; }
+const char* get_family_name() const override { return "DateTime"; }
+std::string do_get_name() const override {return "Date"; }
bool can_be_used_as_version() const override { return true; }
bool can_be_inside_nullable() const override { return true; }
diff --git a/be/src/vec/data_types/data_type_date_time.cpp
b/be/src/vec/data_types/data_type_date_time.cpp
index ae6c94d..4b26194 100644
--- a/be/src/vec/data_types/data_type_date_time.cpp
+++ b/be/src/vec/data_types/data_type_date_time.cpp
@@ -28,10 +28,6 @@ namespace doris::vectorized {
DataTypeDateTime::DataTypeDateTime() {}
-std::string DataTypeDateTime::do_get_name() const {
-return "DateTime";
-}
-
bool DataTypeDateTime::equals(const IDataType& rhs) const {
return typeid(rhs) == typeid(*this);
}
diff --git a/be/src/vec/data_types/data_type_date_time.h
b/be/src/vec/data_types/data_type_date_time.h
index 6b6af04..a0b21f1 100644
--- a/be/src/vec/data_types/data_type_date_time.h
+++ b/be/src/vec/data_types/data_type_date_time.h
@@ -52,7 +52,7 @@ public:
DataTypeDateTime();
const char* get_family_name() const override { return "DateTime"; }
-std::string do_get_name() const override;
+std::string do_get_name() const override {return "DateTime"; }
TypeIndex get_type_id() const override { return TypeIndex::DateTime; }
bool can_be_used_as_version() const override { return true; }
diff --git a/be/src/vec/functions/date_time_transforms.h
b/be/src/vec/functions/date_time_transforms.h
index eaab918..7640a27 100644
--- a/be/src/vec/functions/date_time_transforms.h
+++ b/be/src/vec/functions/date_time_transforms.h
@@ -27,8 +27,10 @@
#include "vec/columns/column_vector.h"
#include "vec/common/exception.h"
#include "vec/core/types.h"
+#include "vec/data_types/data_type_date_time.h"
#include "vec/functions/function_helpers.h"
#include "vec/runtime/vdatetime_value.h"
+
namespace doris::vectorized {
#define TIME_FUNCTION_IMPL(CLASS, UNIT, FUNCTION) \
@@ -46,7 +48,6 @@ namespace doris::vectorized {
TO_TIME_FUNCTION(ToYearImpl, year);
TO_TIME_FUNCTION(ToQuarterImpl, quarter);
TO_TIME_FUNCTION(ToMonthImpl, month);
-TO_T
[GitHub] [incubator-doris] adonis0147 commented on a change in pull request #8089: [Feature][array-type] Support ArrayLiteral in SQL.
adonis0147 commented on a change in pull request #8089: URL: https://github.com/apache/incubator-doris/pull/8089#discussion_r808624158 ## File path: fe/fe-core/src/main/java/org/apache/doris/catalog/ArrayType.java ## @@ -17,16 +17,15 @@ package org.apache.doris.catalog; +import com.google.common.base.Preconditions; +import com.google.common.base.Strings; +import com.google.gson.annotations.SerializedName; Review comment: Ok, I will fix it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] BiteTheDDDDt closed pull request #8014: [Enhancement] Open clang compile warning option and resolve compile error Part1
BiteThet closed pull request #8014: URL: https://github.com/apache/incubator-doris/pull/8014 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[incubator-doris] branch master updated: [feature](vec)(function) support function `convert_tz()` (#8060)
This is an automated email from the ASF dual-hosted git repository.
morningman pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-doris.git
The following commit(s) were added to refs/heads/master by this push:
new f06c13a [feature](vec)(function) support function `convert_tz()`
(#8060)
f06c13a is described below
commit f06c13a82870bf115866e0afeea2b327f9f336b8
Author: Pxl <952130...@qq.com>
AuthorDate: Thu Feb 17 10:51:32 2022 +0800
[feature](vec)(function) support function `convert_tz()` (#8060)
---
be/src/vec/CMakeLists.txt | 1 +
be/src/vec/columns/columns_number.h| 3 +
.../function_convert_tz.cpp} | 27 +
be/src/vec/functions/function_convert_tz.h | 111 +
be/src/vec/functions/simple_function_factory.h | 2 +
be/test/vec/function/function_test_util.h | 29 +++---
be/test/vec/function/function_time_test.cpp| 16 +++
7 files changed, 153 insertions(+), 36 deletions(-)
diff --git a/be/src/vec/CMakeLists.txt b/be/src/vec/CMakeLists.txt
index d9d9c43..0ad0ab1 100644
--- a/be/src/vec/CMakeLists.txt
+++ b/be/src/vec/CMakeLists.txt
@@ -145,6 +145,7 @@ set(VEC_FILES
functions/function_datetime_string_to_string.cpp
functions/function_grouping.cpp
functions/function_rpc.cpp
+ functions/function_convert_tz.cpp
olap/vgeneric_iterators.cpp
olap/vcollect_iterator.cpp
olap/block_reader.cpp
diff --git a/be/src/vec/columns/columns_number.h
b/be/src/vec/columns/columns_number.h
index da3c011..500bab4 100644
--- a/be/src/vec/columns/columns_number.h
+++ b/be/src/vec/columns/columns_number.h
@@ -39,6 +39,9 @@ using ColumnInt32 = ColumnVector;
using ColumnInt64 = ColumnVector;
using ColumnInt128 = ColumnVector;
+using ColumnDate = ColumnVector;
+using ColumnDateTime = ColumnVector;
+
using ColumnFloat32 = ColumnVector;
using ColumnFloat64 = ColumnVector;
diff --git a/be/src/vec/columns/columns_number.h
b/be/src/vec/functions/function_convert_tz.cpp
similarity index 53%
copy from be/src/vec/columns/columns_number.h
copy to be/src/vec/functions/function_convert_tz.cpp
index da3c011..56461db 100644
--- a/be/src/vec/columns/columns_number.h
+++ b/be/src/vec/functions/function_convert_tz.cpp
@@ -14,32 +14,13 @@
// KIND, either express or implied. See the License for the
// specific language governing permissions and limitations
// under the License.
-// This file is copied from
-//
https://github.com/ClickHouse/ClickHouse/blob/master/src/Columns/ColumnsNumber.h
-// and modified by Doris
-#pragma once
-
-#include "vec/columns/column_vector.h"
-#include "vec/core/types.h"
+#include "vec/functions/function_convert_tz.h"
namespace doris::vectorized {
-/** Columns with numbers. */
-
-using ColumnUInt8 = ColumnVector;
-using ColumnUInt16 = ColumnVector;
-using ColumnUInt32 = ColumnVector;
-using ColumnUInt64 = ColumnVector;
-using ColumnUInt128 = ColumnVector;
-
-using ColumnInt8 = ColumnVector;
-using ColumnInt16 = ColumnVector;
-using ColumnInt32 = ColumnVector;
-using ColumnInt64 = ColumnVector;
-using ColumnInt128 = ColumnVector;
-
-using ColumnFloat32 = ColumnVector;
-using ColumnFloat64 = ColumnVector;
+void register_function_convert_tz(SimpleFunctionFactory& factory) {
+factory.register_function();
+}
} // namespace doris::vectorized
diff --git a/be/src/vec/functions/function_convert_tz.h
b/be/src/vec/functions/function_convert_tz.h
new file mode 100644
index 000..739279d
--- /dev/null
+++ b/be/src/vec/functions/function_convert_tz.h
@@ -0,0 +1,111 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+// This file is copied from
+//
https://github.com/ClickHouse/ClickHouse/blob/master/src/Functions/FunctionStringOrArrayToT.h
+// and modified by Doris
+
+#include "vec/columns/columns_number.h"
+#include "vec/common/string_ref.h"
+#include "vec/core/types.h"
+#include "vec/functions/simple_function_factory.h"
+#include "vec/utils/util.hpp"
+
+namespace doris::vectorized {
+
+class FunctionConvertTZ : public IFunction {
+public:
+static constexpr auto name = "convert_tz";
+
+static FunctionPtr create() { return
std::make_shared(); }
+
[GitHub] [incubator-doris] morningman merged pull request #8060: [Feature][Vectorized] support function `convert_tz()`
morningman merged pull request #8060: URL: https://github.com/apache/incubator-doris/pull/8060 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] morningman closed issue #8059: [Feature][Vectorized] support function `convert_tz()`
morningman closed issue #8059: URL: https://github.com/apache/incubator-doris/issues/8059 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] morningman merged pull request #8072: [fix](load)(compaction) Fix NodeChannel coredump bug and modify some compaction logic
morningman merged pull request #8072: URL: https://github.com/apache/incubator-doris/pull/8072 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] morningman closed issue #8058: [Bug] BE crashed when calling IndexCahnnel::mark_as_failed
morningman closed issue #8058: URL: https://github.com/apache/incubator-doris/issues/8058 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[incubator-doris] branch master updated: [fix](load)(compaction) Fix NodeChannel coredump bug and modify some compaction logic (#8072)
This is an automated email from the ASF dual-hosted git repository.
morningman pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-doris.git
The following commit(s) were added to refs/heads/master by this push:
new 26289c2 [fix](load)(compaction) Fix NodeChannel coredump bug and
modify some compaction logic (#8072)
26289c2 is described below
commit 26289c28b0adf06e96f3789bb5560e5759005ea6
Author: Mingyu Chen
AuthorDate: Thu Feb 17 10:52:08 2022 +0800
[fix](load)(compaction) Fix NodeChannel coredump bug and modify some
compaction logic (#8072)
1. Fix the problem of BE crash caused by destruct sequence. (close #8058)
2. Add a new BE config `compaction_task_num_per_fast_disk`
This config specify the max concurrent compaction task num on fast
disk(typically .SSD).
So that for high speed disk, we can execute more compaction task at
same time,
to compact the data as soon as possible
3. Avoid frequent selection of unqualified tablet to perform compaction.
4. Modify some log level to reduce the log size of BE.
5. Modify some clone logic to handle error correctly.
---
be/src/agent/task_worker_pool.cpp | 8 ++--
be/src/common/config.h | 3 ++
be/src/exec/tablet_sink.cpp| 14 +++
be/src/exec/tablet_sink.h | 17 +++-
be/src/http/action/compaction_action.cpp | 2 +-
be/src/http/http_client.h | 13 +--
be/src/olap/base_compaction.cpp| 32 +++
be/src/olap/compaction_permit_limiter.h| 2 +
be/src/olap/cumulative_compaction.cpp | 8 ++--
be/src/olap/data_dir.cpp | 10 +++--
be/src/olap/olap_define.h | 2 +-
be/src/olap/olap_server.cpp| 28 +-
be/src/olap/rowset/beta_rowset.cpp | 10 ++---
be/src/olap/rowset/beta_rowset_writer.cpp | 3 +-
be/src/olap/tablet.cpp | 45 --
be/src/olap/tablet.h | 5 ++-
be/src/olap/tablet_meta_manager.cpp| 7 ++--
be/src/olap/task/engine_clone_task.cpp | 2 +-
be/src/olap/task/engine_publish_version_task.cpp | 8 ++--
be/src/olap/txn_manager.cpp| 20 +-
be/src/olap/utils.cpp | 2 +-
be/src/runtime/load_channel.cpp| 2 +-
be/src/runtime/load_channel_mgr.cpp| 2 +-
be/src/runtime/plan_fragment_executor.cpp | 6 +--
docs/en/administrator-guide/config/be_config.md| 20 --
docs/zh-CN/administrator-guide/config/be_config.md | 20 --
.../org/apache/doris/clone/TabletSchedCtx.java | 8 ++--
27 files changed, 164 insertions(+), 135 deletions(-)
diff --git a/be/src/agent/task_worker_pool.cpp
b/be/src/agent/task_worker_pool.cpp
index a0eaf57..457f8da 100644
--- a/be/src/agent/task_worker_pool.cpp
+++ b/be/src/agent/task_worker_pool.cpp
@@ -252,7 +252,7 @@ void TaskWorkerPool::submit_task(const TAgentTaskRequest&
task) {
_worker_thread_condition_variable.notify_one();
}
LOG(INFO) << "success to submit task. type=" << type_str << ",
signature=" << signature
- << ", task_count_in_queue=" << task_count_in_queue;
+ << ", queue size=" << task_count_in_queue;
} else {
LOG(INFO) << "fail to register task. type=" << type_str << ",
signature=" << signature;
}
@@ -280,8 +280,8 @@ void TaskWorkerPool::_remove_task_info(const
TTaskType::type task_type, int64_t
std::string type_str;
EnumToString(TTaskType, task_type, type_str);
-LOG(INFO) << "remove task info. type=" << type_str << ", signature=" <<
signature
- << ", queue_size=" << queue_size;
+VLOG_NOTICE << "remove task info. type=" << type_str << ", signature=" <<
signature
+<< ", queue_size=" << queue_size;
TRACE("remove task info");
}
@@ -713,7 +713,7 @@ void
TaskWorkerPool::_publish_version_worker_thread_callback() {
}
DorisMetrics::instance()->publish_task_request_total->increment(1);
-LOG(INFO) << "get publish version task, signature:" <<
agent_task_req.signature;
+VLOG_NOTICE << "get publish version task, signature:" <<
agent_task_req.signature;
Status st;
std::vector error_tablet_ids;
diff --git a/be/src/common/config.h b/be/src/common/config.h
index c5eed39..0c6cf36 100644
--- a/be/src/common/config.h
+++ b/be/src/common/config.h
@@ -289,7 +289,10 @@ CONF_mInt32(generate_compaction_tasks_min_interval_ms,
"10");
// Compaction task number per disk.
// Must be greater than 2, because Base compaction and Cumulative compaction
have at least one thread each.
CONF_mInt32(compaction_t
[GitHub] [incubator-doris] morningman merged pull request #8076: [Docs] Update alter table docs
morningman merged pull request #8076: URL: https://github.com/apache/incubator-doris/pull/8076 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[incubator-doris] branch master updated: [doc] update alter table docs (#8076)
This is an automated email from the ASF dual-hosted git repository.
morningman pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-doris.git
The following commit(s) were added to refs/heads/master by this push:
new d9535c2 [doc] update alter table docs (#8076)
d9535c2 is described below
commit d9535c29f6fa93280b1dde9ec071a590c49eb051
Author: GRX <2578518...@qq.com>
AuthorDate: Thu Feb 17 10:52:59 2022 +0800
[doc] update alter table docs (#8076)
---
docs/en/sql-reference/sql-statements/Data Definition/ALTER TABLE.md | 5 -
.../sql-reference/sql-statements/Data Definition/ALTER TABLE.md | 3 +++
2 files changed, 7 insertions(+), 1 deletion(-)
diff --git a/docs/en/sql-reference/sql-statements/Data Definition/ALTER
TABLE.md b/docs/en/sql-reference/sql-statements/Data Definition/ALTER TABLE.md
index 6e7c901..2df6a15 100644
--- a/docs/en/sql-reference/sql-statements/Data Definition/ALTER TABLE.md
+++ b/docs/en/sql-reference/sql-statements/Data Definition/ALTER TABLE.md
@@ -54,6 +54,7 @@ under the License.
2) If the bucket mode is not specified, the bucket method used by
the built-in table is automatically used.
3) If the bucket mode is specified, only the bucket number can be
modified, and the bucket mode or bucket column cannot be modified.
4) ["key"="value"] section can set some properties of the
partition, see CREATE TABLE for details.
+5) Adding partitions to non-partitioned table is not supported.
2. Delete the partition
grammar:
@@ -376,9 +377,11 @@ under the License.
ALTER TABLE example_db.my_table set ("dynamic_partition.enable" =
"false");
-If you need to add dynamic partition attributes to a table without
dynamic partition attributes, you need to specify all dynamic partition
attributes
+If you need to add dynamic partition attributes to a table without
dynamic partition attributes, you need to specify all dynamic partition
attributes.
+(Note:Adding dynamic partition attributes to non-partitioned table is
not supported)
ALTER TABLE example_db.my_table set ("dynamic_partition.enable"=
"true", "dynamic_partition.time_unit" = "DAY", "dynamic_partition.end "= "3",
"dynamic_partition.prefix" = "p", "dynamic_partition.buckets" = "32");
+
15. Modify the in_memory property of the table
diff --git a/docs/zh-CN/sql-reference/sql-statements/Data Definition/ALTER
TABLE.md b/docs/zh-CN/sql-reference/sql-statements/Data Definition/ALTER
TABLE.md
index 1746d2c..1738b1a 100644
--- a/docs/zh-CN/sql-reference/sql-statements/Data Definition/ALTER TABLE.md
+++ b/docs/zh-CN/sql-reference/sql-statements/Data Definition/ALTER TABLE.md
@@ -54,6 +54,7 @@ under the License.
2) 如果没有指定分桶方式，则自动使用建表使用的分桶方式
3) 如指定分桶方式，只能修改分桶数，不可修改分桶方式或分桶列
4) ["key"="value"] 部分可以设置分区的一些属性，具体说明见 CREATE TABLE
+5) 如果建表时用户未显式创建Partition,则不支持通过ALTER的方式增加分区
2. 删除分区
语法：
@@ -376,7 +377,9 @@ under the License.
ALTER TABLE example_db.my_table set ("dynamic_partition.enable" =
"false");
如果需要在未添加动态分区属性的表中添加动态分区属性，则需要指定所有的动态分区属性
+(注:非分区表不支持添加动态分区属性)
ALTER TABLE example_db.my_table set ("dynamic_partition.enable" =
"true", "dynamic_partition.time_unit" = "DAY", "dynamic_partition.end" = "3",
"dynamic_partition.prefix" = "p", "dynamic_partition.buckets" = "32");
+
15. 修改表的 in_memory 属性
ALTER TABLE example_db.my_table set ("in_memory" = "true");
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] morningman merged pull request #8084: [community] modify the doc of verifying apache release
morningman merged pull request #8084: URL: https://github.com/apache/incubator-doris/pull/8084 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[incubator-doris] branch master updated: [community] modify the doc of verifying apache release (#8084)
This is an automated email from the ASF dual-hosted git repository. morningman pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/incubator-doris.git The following commit(s) were added to refs/heads/master by this push: new 486a058 [community] modify the doc of verifying apache release (#8084) 486a058 is described below commit 486a0586ac545643bd40807727ba18d65cd861be Author: Mingyu Chen AuthorDate: Thu Feb 17 10:53:31 2022 +0800 [community] modify the doc of verifying apache release (#8084) --- docs/en/community/verify-apache-release.md | 30 ++ docs/en/extending-doris/flink-doris-connector.md | 21 ++- docs/en/extending-doris/spark-doris-connector.md | 13 +++--- docs/zh-CN/community/verify-apache-release.md | 30 ++ .../zh-CN/extending-doris/flink-doris-connector.md | 13 +++--- .../zh-CN/extending-doris/spark-doris-connector.md | 12 +++-- 6 files changed, 57 insertions(+), 62 deletions(-) diff --git a/docs/en/community/verify-apache-release.md b/docs/en/community/verify-apache-release.md index 34bc4eb..e83fb08 100644 --- a/docs/en/community/verify-apache-release.md +++ b/docs/en/community/verify-apache-release.md @@ -30,7 +30,7 @@ To verify the release, following checklist can used to reference: 1. [ ] Download links are valid. 2. [ ] Checksums and PGP signatures are valid. -3. [ ] DISCLAIMER-WIP is included. +3. [ ] DISCLAIMER or DISCLAIMER-WIP is included. 4. [ ] Source code artifacts have correct names matching the current release. 5. [ ] LICENSE and NOTICE files are correct for the repository. 6. [ ] All files have license headers if necessary. @@ -55,6 +55,8 @@ wget https://www.apache.org/dist/incubator/doris/KEYS GnuPG is recommended, which can install by yum install gnupg or apt-get install gnupg. +Here we use Doris main code release as an example. Other releases are similar. + ``` shell gpg --import KEYS gpg --verify apache-doris-a.b.c-incubating-src.tar.gz.asc apache-doris-a.b.c-incubating-src.tar.gz @@ -63,20 +65,28 @@ sha512sum --check apache-doris-a.b.c-incubating-src.tar.gz.sha512 ## 3. Verify license header -Apache RAT is recommended to verify license header, which can download as following command. +Here we use [apache/skywalking-eyes](https://github.com/apache/skywalking-eyes) for source license header validation. -``` shell -wget http://mirrors.tuna.tsinghua.edu.cn/apache/creadur/apache-rat-0.13/apache-rat-0.13-bin.tar.gz -tar zxvf apache-rat-0.13-bin.tar.gz +Go to the source directory and execute the following command (requires a Docker environment). + +``` +docker run -it --rm -v $(pwd):/github/workspace apache/skywalking-eyes header check ``` -Given your source dir is apache-doris-a.b.c-incubating-src, you can check with following command. -It will output a file list which don't include ASF license header, and these files used other licenses. +The output is similar to the following: -``` shell -/usr/java/jdk/bin/java -jar apache-rat-0.13/apache-rat-0.13.jar -a -d apache-doris-a.b.c-incubating-src -E apache-doris-a.b.c-incubating-src/.rat-excludes ``` +INFO GITHUB_TOKEN is not set, license-eye won't comment on the pull request +INFO Loading configuration from file: .licenserc.yaml +INFO Totally checked 5611 files, valid: 3926, invalid: 0, ignored: 1685, fixed: 0 +``` + +where an invalid of 0 means the check passed. + +> Some non-Apache License header files are documented in `.licenserc.yaml`. ## 4. Verify building -To compile the Doris, please read [Compilation](../installing/compilation.html) +* For Doris main code compilation, see [compilation documentation](../installing/compilation.html) +* Flink Doris Connector compilation, see [compilation documentation](../extending-doris/flink-doris-connector.md) +* Spark Doris Connector compilation, see [compilation documentation](../extending-doris/spark-doris-connector.md) diff --git a/docs/en/extending-doris/flink-doris-connector.md b/docs/en/extending-doris/flink-doris-connector.md index 3538f52..9b2ec5e 100644 --- a/docs/en/extending-doris/flink-doris-connector.md +++ b/docs/en/extending-doris/flink-doris-connector.md @@ -28,14 +28,14 @@ under the License. - The Flink Doris Connector can support operations (read, insert, modify, delete) data stored in Doris through Flink. -Github: https://github.com/apache/incubator-doris-connectors +Github: https://github.com/apache/incubator-doris-flink-connector - * `Doris` table can be mapped to `DataStream` or `Table`. +* `Doris` table can be mapped to `DataStream` or `Table`. - >**Note:** - > - >1. Modification and deletion are only supported on the Unique Key model - >2. The current deletion is to support Flink CDC to access data to achieve automatic deletion. If it is to delete other data access methods, you need to implement it yourself. For the data deletion usage of Flink CDC, please refer to
[GitHub] [incubator-doris] morningman commented on a change in pull request #7994: [Feature](ThreadPool) Support thread pool per disk for scanners
morningman commented on a change in pull request #7994:
URL: https://github.com/apache/incubator-doris/pull/7994#discussion_r808626294
##
File path: be/src/util/priority_thread_pool.hpp
##
@@ -39,6 +39,7 @@ class PriorityThreadPool {
public:
int priority;
WorkFunction work_function;
+int queue_id;
Review comment:
Oh, i read it as `query_id`...
lgtm
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] zuochunwei commented on a change in pull request #8090: (Vectorized)(Optimization) refactor RowRef/RowRefList to decrease hash table memory size
zuochunwei commented on a change in pull request #8090:
URL: https://github.com/apache/incubator-doris/pull/8090#discussion_r808627987
##
File path: be/src/vec/exec/join/join_op.h
##
@@ -22,19 +22,42 @@
#include "vec/core/block.h"
namespace doris::vectorized {
+
/// Reference to the row in block.
struct RowRef {
using SizeT = uint32_t; /// Do not use size_t cause of memory economy
-const Block* block = nullptr;
-SizeT row_num = 0;
-// Use in right join to mark row is visited
-// TODO: opt the varaible to use it only need
-bool visited = false;
+// using union to represent two cases
+// 1. when RowRefList containing only one RowRef, visited + blockptr are
valid
+// 2. when RowRefList contaning multi RowRef, it's used through next
pointer
+union {
+struct {
+uint64_t visited : 1;
Review comment:
i need 64bit wide to union with void*
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] zuochunwei commented on a change in pull request #8090: (Vectorized)(Optimization) refactor RowRef/RowRefList to decrease hash table memory size
zuochunwei commented on a change in pull request #8090:
URL: https://github.com/apache/incubator-doris/pull/8090#discussion_r808628352
##
File path: be/src/vec/exec/join/vhash_join_node.cpp
##
@@ -286,9 +286,12 @@ struct ProcessHashTableProbe {
// use in right join to change visited state after
// exec the vother join conjunt
-std::vector visited_map;
+std::vector visited_map;
visited_map.reserve(1.2 * _batch_size);
+#define visited_map_or(index, value) do { if (value)
((RowRef*)visited_map[index])->visited = 1; } while (0)
Review comment:
got， i will change as you say
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] EmmyMiao87 opened a new pull request #8101: [docs] Modify the SSB document
EmmyMiao87 opened a new pull request #8101: URL: https://github.com/apache/incubator-doris/pull/8101 Add one step of preparation before load ## Checklist(Required) 1. Does it affect the original behavior: (Yes/No/I Don't know) 2. Has unit tests been added: (Yes/No/No Need) 3. Has document been added or modified: (Yes/No/No Need) 4. Does it need to update dependencies: (Yes/No) 5. Are there any changes that cannot be rolled back: (Yes/No) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] adonis0147 commented on a change in pull request #8089: [Feature][array-type] Support ArrayLiteral in SQL.
adonis0147 commented on a change in pull request #8089:
URL: https://github.com/apache/incubator-doris/pull/8089#discussion_r808631826
##
File path: fe/fe-core/src/main/java/org/apache/doris/catalog/Column.java
##
@@ -159,6 +159,7 @@ public Column(Column column) {
public void createChildrenColumn(Type type, Column column) {
if (type.isArrayType()) {
Column c = new Column(COLUMN_ARRAY_CHILDREN, ((ArrayType)
type).getItemType());
+c.setIsAllowNull(true);
Review comment:
I'm not sure whether there is a way to declare the item type in array
type nullable.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] adonis0147 commented on a change in pull request #8089: [Feature][array-type] Support ArrayLiteral in SQL.
adonis0147 commented on a change in pull request #8089:
URL: https://github.com/apache/incubator-doris/pull/8089#discussion_r808631826
##
File path: fe/fe-core/src/main/java/org/apache/doris/catalog/Column.java
##
@@ -159,6 +159,7 @@ public Column(Column column) {
public void createChildrenColumn(Type type, Column column) {
if (type.isArrayType()) {
Column c = new Column(COLUMN_ARRAY_CHILDREN, ((ArrayType)
type).getItemType());
+c.setIsAllowNull(true);
Review comment:
I'm not sure whether there is a way to declare the item type in array
type nullable.
For scalar type, if we declare it without `NOT NULL`, the type is nullable
by default. If we don't set the item type `isAllowNull` and insert value
`[NULL]`, we will get a wrong result, because the item_writer won't append null
in be.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #8086: [Feature][Vectorized] support function `group_concat()`
github-actions[bot] commented on pull request #8086: URL: https://github.com/apache/incubator-doris/pull/8086#issuecomment-1042524673 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] 254288935 commented on issue #8073: [Bug] BE crashed after altering int field to string
254288935 commented on issue #8073:
URL:
https://github.com/apache/incubator-doris/issues/8073#issuecomment-1042525539
> > alter table时，目前不支持BIGINT到CHAR的转换， 是否需要支持BIGINT到VARCHAR的转换呢？
>
> I think we do support convert BIGINT to CHAR. Why you say it not support?
mysql> create table t(a int, b bigint) duplicate key(a) distributed by
hash(a) buckets 5 PROPERTIES ("replication_num" = "1");
Query OK, 0 rows affected (0.01 sec)
mysql> insert into t values(1,1);
Query OK, 1 row affected (0.04 sec)
{'label':'insert_37ddf33352aa45e2-be78d5486b802511', 'status':'VISIBLE',
'txnId':'23'}
mysql> select * from t;
+--+---+
| a| b |
+--+---+
|1 | 1 |
+--+---+
1 row in set (0.01 sec)
mysql> alter table t modify column b char(10);
ERROR 1105 (HY000): errCode = 2, detailMessage = Can not change BIGINT to
CHAR
master分支上，我是这样操作的
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #7994: [Feature](ThreadPool) Support thread pool per disk for scanners
github-actions[bot] commented on pull request #7994: URL: https://github.com/apache/incubator-doris/pull/7994#issuecomment-1042527884 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #8092: [Refactor] Remove unused code in data dir
github-actions[bot] commented on pull request #8092: URL: https://github.com/apache/incubator-doris/pull/8092#issuecomment-1042528809 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] morningman merged pull request #8034: enable check_java_version
morningman merged pull request #8034: URL: https://github.com/apache/incubator-doris/pull/8034 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[incubator-doris] branch master updated: [improvement] enable check_java_version (#8034)
This is an automated email from the ASF dual-hosted git repository.
morningman pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-doris.git
The following commit(s) were added to refs/heads/master by this push:
new 289aacb [improvement] enable check_java_version (#8034)
289aacb is described below
commit 289aacb78c98e86dbf3be96ccbb396d109307946
Author: jiafeng.zhang
AuthorDate: Thu Feb 17 11:16:45 2022 +0800
[improvement] enable check_java_version (#8034)
Enable to check the Java version when Doris starts, to prevent the user
experience caused by the inconsistency
between the compiled version and the running version.
If the Java version is compiled and the Java version is run, it will not
start, and a prompt message will be given.
---
docs/en/administrator-guide/config/fe_config.md | 6 +++---
docs/zh-CN/administrator-guide/config/fe_config.md | 4 ++--
fe/fe-core/src/main/java/org/apache/doris/common/Config.java | 2 +-
3 files changed, 6 insertions(+), 6 deletions(-)
diff --git a/docs/en/administrator-guide/config/fe_config.md
b/docs/en/administrator-guide/config/fe_config.md
index f2cf45f..1f8ab3f 100644
--- a/docs/en/administrator-guide/config/fe_config.md
+++ b/docs/en/administrator-guide/config/fe_config.md
@@ -394,7 +394,7 @@ IsMutable：true
MasterOnly：true
If disable_storage_medium_check is true, ReportHandler would not check
tablet's storage medium and disable storage cool down function, the default
value is false. You can set the value true when you don't care what the storage
medium of the tablet is.
-
+
### drop_backend_after_decommission
Default：false
@@ -448,9 +448,9 @@ This variable is a dynamic configuration, and users can
modify the configuration
### check_java_version
-Default：false
+Default：true
-If set to true, Doris will check whether the compiled and running Java
versions are compatible
+Doris will check whether the compiled and run Java versions are compatible, if
not, it will throw a Java version mismatch exception message and terminate the
startup
### max_running_rollup_job_num_per_table
diff --git a/docs/zh-CN/administrator-guide/config/fe_config.md
b/docs/zh-CN/administrator-guide/config/fe_config.md
index 8978452..b373719 100644
--- a/docs/zh-CN/administrator-guide/config/fe_config.md
+++ b/docs/zh-CN/administrator-guide/config/fe_config.md
@@ -434,9 +434,9 @@ show data （其他用法：HELP SHOW DATA）
### check_java_version
-默认值：false
+默认值：true
-如果设置为 true，Doris 将检查已编译和运行的 Java 版本是否兼容
+Doris 将检查已编译和运行的 Java 版本是否兼容，如果不兼容将抛出Java版本不匹配的异常信息，并终止启动
### max_running_rollup_job_num_per_table
diff --git a/fe/fe-core/src/main/java/org/apache/doris/common/Config.java
b/fe/fe-core/src/main/java/org/apache/doris/common/Config.java
index 12dd3ad..4f9089e 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/common/Config.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/common/Config.java
@@ -1277,7 +1277,7 @@ public class Config extends ConfigBase {
* If set to true, Doris will check if the compiled and running versions
of Java are compatible
*/
@ConfField
-public static boolean check_java_version = false;
+public static boolean check_java_version = true;
/**
* control materialized view
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] morningman closed issue #8033: [Bug] Startup exception due to inconsistent java versions
morningman closed issue #8033: URL: https://github.com/apache/incubator-doris/issues/8033 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] HappenLee opened a new issue #8102: [Vectorized][Feature] Support function `least/greast`
HappenLee opened a new issue #8102: URL: https://github.com/apache/incubator-doris/issues/8102 ### Search before asking - [X] I had searched in the [issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and found no similar issues. ### Description Support function `least/greast` ### Use case _No response_ ### Related issues _No response_ ### Are you willing to submit PR? - [X] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] yangzhg closed issue #8080: [refactor] (mysql_table_writer)split into two parts of vectorized and row mode&&clang compile error
yangzhg closed issue #8080: URL: https://github.com/apache/incubator-doris/issues/8080 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] yangzhg merged pull request #8081: [refactor](mysql_table_writer)split into two parts of vectorized and row mode
yangzhg merged pull request #8081: URL: https://github.com/apache/incubator-doris/pull/8081 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[incubator-doris] branch master updated (289aacb -> f8411f3)
This is an automated email from the ASF dual-hosted git repository.
yangzhg pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-doris.git.
from 289aacb [improvement] enable check_java_version (#8034)
add f8411f3 [refactor](mysql_table_writer)split into two parts of
vectorized and row mode (#8081)
No new revisions were added by this update.
Summary of changes:
be/src/exec/data_sink.cpp | 1 +
be/src/runtime/mysql_table_writer.cpp | 152 +-
be/src/runtime/mysql_table_writer.h| 18 +-
be/src/vec/CMakeLists.txt | 1 +
...ysql_table_writer.cpp => vmysql_table_sink.cpp} | 12 +-
.../{vmysql_table_writer.h => vmysql_table_sink.h} | 8 +-
be/src/vec/sink/vmysql_table_writer.cpp| 218 -
be/src/vec/sink/vmysql_table_writer.h | 58 ++
8 files changed, 205 insertions(+), 263 deletions(-)
copy be/src/vec/sink/{vmysql_table_writer.cpp => vmysql_table_sink.cpp} (89%)
copy be/src/vec/sink/{vmysql_table_writer.h => vmysql_table_sink.h} (93%)
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] HappenLee opened a new issue #8103: [Bug] [Vectorized] Nullable column compute the hash value error cause error result of left/right join
HappenLee opened a new issue #8103: URL: https://github.com/apache/incubator-doris/issues/8103 ### Search before asking - [X] I had searched in the [issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and found no similar issues. ### Version vectorized ### What's Wrong? wrong hash compute of nullable column ### What You Expected? nullable column and normal column have same value get the same hash value ### How to Reproduce? _No response_ ### Anything Else? _No response_ ### Are you willing to submit PR? - [X] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #8101: [docs] Modify the SSB document
github-actions[bot] commented on pull request #8101: URL: https://github.com/apache/incubator-doris/pull/8101#issuecomment-1042545563 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] adonis0147 commented on a change in pull request #8089: [Feature][array-type] Support ArrayLiteral in SQL.
adonis0147 commented on a change in pull request #8089:
URL: https://github.com/apache/incubator-doris/pull/8089#discussion_r808631826
##
File path: fe/fe-core/src/main/java/org/apache/doris/catalog/Column.java
##
@@ -159,6 +159,7 @@ public Column(Column column) {
public void createChildrenColumn(Type type, Column column) {
if (type.isArrayType()) {
Column c = new Column(COLUMN_ARRAY_CHILDREN, ((ArrayType)
type).getItemType());
+c.setIsAllowNull(true);
Review comment:
I'm not sure whether there is a way to declare the item type in array
type nullable.
For scalar type, if we declare it without `NOT NULL`, the type is nullable
by default. If we don't set the item type `isAllowNull` here and insert value
`[NULL]`, we will get a wrong result, because the item_writer won't append null
in be.
There are three ways to resolve it:
1. `Array` indicates the item type is not null by default and add some
checks to forbid values like `[NULL]` from being inserted.
2. `Array` indicates the item type is null by default.
3. Provide some sql syntax to declare whether the item type is nullable or
not.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] yangzhg commented on a change in pull request #8089: [Feature][array-type] Support ArrayLiteral in SQL.
yangzhg commented on a change in pull request #8089:
URL: https://github.com/apache/incubator-doris/pull/8089#discussion_r808654484
##
File path: fe/fe-core/src/main/java/org/apache/doris/catalog/Column.java
##
@@ -159,6 +159,7 @@ public Column(Column column) {
public void createChildrenColumn(Type type, Column column) {
if (type.isArrayType()) {
Column c = new Column(COLUMN_ARRAY_CHILDREN, ((ArrayType)
type).getItemType());
+c.setIsAllowNull(true);
Review comment:
add a todo comment to his
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] yangzhg commented on a change in pull request #8089: [Feature][array-type] Support ArrayLiteral in SQL.
yangzhg commented on a change in pull request #8089:
URL: https://github.com/apache/incubator-doris/pull/8089#discussion_r808654484
##
File path: fe/fe-core/src/main/java/org/apache/doris/catalog/Column.java
##
@@ -159,6 +159,7 @@ public Column(Column column) {
public void createChildrenColumn(Type type, Column column) {
if (type.isArrayType()) {
Column c = new Column(COLUMN_ARRAY_CHILDREN, ((ArrayType)
type).getItemType());
+c.setIsAllowNull(true);
Review comment:
please add a todo comment to this
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #8056: (unitTest) resolve unit test failed problem for VGenericIteratorsTest
github-actions[bot] commented on pull request #8056: URL: https://github.com/apache/incubator-doris/pull/8056#issuecomment-1042548772 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] spaces-X commented on a change in pull request #8090: (Vectorized)(Optimization) refactor RowRef/RowRefList to decrease hash table memory size
spaces-X commented on a change in pull request #8090:
URL: https://github.com/apache/incubator-doris/pull/8090#discussion_r808661055
##
File path: be/src/vec/exec/join/vhash_join_node.cpp
##
@@ -317,10 +319,10 @@ struct ProcessHashTableProbe {
for (auto it = mapped.begin(); it.ok(); ++it) {
++current_offset;
for (size_t j = 0; j < right_col_len; ++j) {
-auto& column = *it->block->get_by_position(j).column;
+auto& column = *it->block()->get_by_position(j).column;
mcol[j + right_col_idx]->insert_from(column,
it->row_num);
}
-visited_map.emplace_back(&it->visited);
+visited_map.emplace_back(&(*it));
Review comment:
why not just `emplace_back(it)`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
commits@doris.apache.org
HappenLee commented on a change in pull request #8066:
URL: https://github.com/apache/incubator-doris/pull/8066#discussion_r808663527
##
File path: be/src/vec/aggregate_functions/aggregate_function_percentile_approx.h
##
@@ -0,0 +1,266 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#pragma once
+
+#include "common/status.h"
+#include "util/counts.h"
+#include "util/tdigest.h"
+#include "vec/aggregate_functions/aggregate_function.h"
+#include "vec/columns/columns_number.h"
+#include "vec/data_types/data_type_decimal.h"
+#include "vec/data_types/data_type_number.h"
+#include "vec/io/io_helper.h"
+namespace doris::vectorized {
+
+struct PercentileApproxState {
+static constexpr double INIT_QUANTILE = -1.0;
+PercentileApproxState() {}
+~PercentileApproxState() {}
+
+void init(double compression = 1) {
+if (!init_flag) {
+digest.reset(new TDigest(compression));
+init_flag = true;
+}
+}
+
+void write(BufferWritable& buf) const {
+write_binary(init_flag, buf);
+write_binary(targetQuantile, buf);
+
+uint32_t serialize_size = digest->serialized_size();
+std::string result(serialize_size, '0');
+DCHECK(digest.get() != nullptr);
+digest->serialize((uint8_t*)result.c_str());
+
+write_binary(result, buf);
+}
+
+void read(BufferReadable& buf) {
+read_binary(init_flag, buf);
+read_binary(targetQuantile, buf);
+
+std::string str;
+read_binary(str, buf);
+digest.reset(new TDigest());
+digest->unserialize((uint8_t*)str.c_str());
+}
+
+double get() const { return digest->quantile(targetQuantile); }
+
+void merge(const PercentileApproxState& rhs) {
+if (init_flag) {
+DCHECK(digest.get() != nullptr);
+digest->merge(rhs.digest.get());
+} else {
+digest.reset(new TDigest());
+digest->merge(rhs.digest.get());
+init_flag = true;
+}
+if (targetQuantile == PercentileApproxState::INIT_QUANTILE) {
+targetQuantile = rhs.targetQuantile;
+}
+}
+
+void add(double source, double quantile) {
+digest->add(source);
+targetQuantile = quantile;
+}
+
+void reset() {
+targetQuantile = INIT_QUANTILE;
+init_flag = false;
+digest.reset();
+}
+
+bool init_flag = false;
+std::shared_ptr digest;
+double targetQuantile = INIT_QUANTILE;
Review comment:
`target_quantile`
##
File path: be/src/vec/aggregate_functions/aggregate_function_percentile_approx.h
##
@@ -0,0 +1,266 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#pragma once
+
+#include "common/status.h"
+#include "util/counts.h"
+#include "util/tdigest.h"
+#include "vec/aggregate_functions/aggregate_function.h"
+#include "vec/columns/columns_number.h"
+#include "vec/data_types/data_type_decimal.h"
+#include "vec/data_types/data_type_number.h"
+#include "vec/io/io_helper.h"
+namespace doris::vectorized {
+
+struct PercentileApproxState {
+static constexpr double INIT_QUANTILE = -1.0;
+PercentileApproxState() {}
+~PercentileApproxState() {}
+
+void init(double compression = 1) {
+if (!init_flag) {
+digest.reset(new TDigest(compression));
+init_flag = true;
+}
+}
+
+void write(BufferWritable& buf) co
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #8068: [Vectorized][Function] Support year/month/week/hour/mintue/day/second floor/ceil function
github-actions[bot] commented on pull request #8068: URL: https://github.com/apache/incubator-doris/pull/8068#issuecomment-1042564769 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] HappenLee opened a new issue #8104: [Bug] [Vectorized] Core dump with query table have a `delete condition`
HappenLee opened a new issue #8104: URL: https://github.com/apache/incubator-doris/issues/8104 ### Search before asking - [X] I had searched in the [issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and found no similar issues. ### Version vectorized ### What's Wrong? vec block size is different with RowBlock v2 size， the mismatch cause the problem ### What You Expected? query normal ### How to Reproduce? _No response_ ### Anything Else? _No response_ ### Are you willing to submit PR? - [X] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] HappenLee commented on a change in pull request #8051: eliminate branch prediction
HappenLee commented on a change in pull request #8051:
URL: https://github.com/apache/incubator-doris/pull/8051#discussion_r808668296
##
File path: be/src/vec/exec/join/vhash_join_node.cpp
##
@@ -545,7 +532,7 @@ struct ProcessHashTableProbe {
if (_join_node->_is_outer_join) {
Review comment:
here also use if constexpr
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #8051: eliminate branch prediction
github-actions[bot] commented on pull request #8051: URL: https://github.com/apache/incubator-doris/pull/8051#issuecomment-1042577252 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] zuochunwei commented on a change in pull request #8090: (Vectorized)(Optimization) refactor RowRef/RowRefList to decrease hash table memory size
zuochunwei commented on a change in pull request #8090:
URL: https://github.com/apache/incubator-doris/pull/8090#discussion_r808677199
##
File path: be/src/vec/exec/join/vhash_join_node.cpp
##
@@ -317,10 +319,10 @@ struct ProcessHashTableProbe {
for (auto it = mapped.begin(); it.ok(); ++it) {
++current_offset;
for (size_t j = 0; j < right_col_len; ++j) {
-auto& column = *it->block->get_by_position(j).column;
+auto& column = *it->block()->get_by_position(j).column;
mcol[j + right_col_idx]->insert_from(column,
it->row_num);
}
-visited_map.emplace_back(&it->visited);
+visited_map.emplace_back(&(*it));
Review comment:
&(*it) is not same to it, this is because ForwardIterator has overloaded
the operator*
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] HappenLee opened a new pull request #8105: [Vectorized][Bug] Fix nullable column compute the hash value error
HappenLee opened a new pull request #8105: URL: https://github.com/apache/incubator-doris/pull/8105 # Proposed changes Issue Number: close #8103 ## Problem Summary: Fix nullable column compute the hash value error 6 and not null 6 compute the same hash value ## Checklist(Required) 1. Does it affect the original behavior: (Yes) 2. Has unit tests been added: (Yes) 3. Has document been added or modified: (No) 4. Does it need to update dependencies: (Yes/No) 5. Are there any changes that cannot be rolled back: (Yes) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] liuzhuang2017 opened a new pull request #8106: Update sql-block.md
liuzhuang2017 opened a new pull request #8106: URL: https://github.com/apache/incubator-doris/pull/8106 CREATE SQL_BLOCK_RULE test_rule PROPERTIES( "sql"="select * from order_analysis", "global"="false", "enable"="true", "sqlHash"="" ) in "global":"false" should be "=" # Proposed changes Issue Number: close #xxx ## Problem Summary: Describe the overview of changes. ## Checklist(Required) 1. Does it affect the original behavior: (Yes/No/I Don't know) 2. Has unit tests been added: (Yes/No/No Need) 3. Has document been added or modified: (Yes/No/No Need) 4. Does it need to update dependencies: (Yes/No) 5. Are there any changes that cannot be rolled back: (Yes/No) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] liuzhuang2017 closed pull request #8106: Update sql-block.md
liuzhuang2017 closed pull request #8106: URL: https://github.com/apache/incubator-doris/pull/8106 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] HappenLee opened a new pull request #8107: [Function][Vectorized] Support least/greast function
HappenLee opened a new pull request #8107: URL: https://github.com/apache/incubator-doris/pull/8107 # Proposed changes Issue Number: close #8102 ## Problem Summary: Describe the overview of changes. ## Checklist(Required) 1. Does it affect the original behavior: (No) 2. Has unit tests been added: (Yes) 3. Has document been added or modified: (No) 4. Does it need to update dependencies: (No) 5. Are there any changes that cannot be rolled back: (Yes) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
