[GitHub] [doris] englefly opened a new pull request, #18331: [improve](nereids)compute statsRange.length() according to the column datatype
englefly opened a new pull request, #18331: URL: https://github.com/apache/doris/pull/18331 # Proposed changes we map date/datetime/V2 to double. this map reserves date order, but it does not reserve range length. For example, from 1990-01-01 to 2000-01-01, there are 10 years. for filter `A < 1991-01-01`, the selectivity should be 1/10. if we compute this filter by their corresponding double value, the error is about 10 times. This pr aims to fix this error. Issue Number: close #xxx ## Problem summary Describe your changes. ## Checklist(Required) * [ ] Does it affect the original behavior * [ ] Has unit tests been added * [ ] Has document been added or modified * [ ] Does it need to update dependencies * [ ] Is this PR support rollback (If NO, please explain WHY) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] zy-kkk commented on pull request #18329: [typo](docs) add regression test doc & fix api doc
zy-kkk commented on PR #18329: URL: https://github.com/apache/doris/pull/18329#issuecomment-1493800455 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] HappenLee commented on a diff in pull request #18278: [Improvement](predicate) optimize short-circuit predicates
HappenLee commented on code in PR #18278:
URL: https://github.com/apache/doris/pull/18278#discussion_r1155563145
##
be/src/olap/null_predicate.cpp:
##
@@ -54,10 +54,17 @@ uint16_t NullPredicate::evaluate(const vectorized::IColumn&
column, uint16_t* se
return _is_null ? 0 : size;
}
auto& null_map = nullable->get_null_map_data();
-for (uint16_t i = 0; i < size; ++i) {
-uint16_t idx = sel[i];
-sel[new_size] = idx;
-new_size += (null_map[idx] == _is_null);
+if (null_map.size() == size) {
+for (uint16_t i = 0; i < size; ++i) {
+sel[new_size] = i;
+new_size += (null_map[i] == _is_null);
+}
+} else {
+for (uint16_t i = 0; i < size; ++i) {
+uint16_t idx = sel[i];
+sel[new_size] = idx;
+new_size += (null_map[idx] == _is_null);
+}
Review Comment:
better need a function to do:
```
uint16_t idx = is_parse_column ? i : sel[i];
```
to simple the function logic
`is_parse_column` better be const bool
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] morrySnow commented on pull request #18280: [enhancement](Nereids) cast do not relay on legacy planner anymore
morrySnow commented on PR #18280: URL: https://github.com/apache/doris/pull/18280#issuecomment-1493800934 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] yuanyuan8983 commented on issue #18322: Process one quert failed because org.apache.doris.common.AnalysisException:errCode=2,detailMessage=Syntax error in line1
yuanyuan8983 commented on issue #18322: URL: https://github.com/apache/doris/issues/18322#issuecomment-1493803239 Hello, is your Doris 1.2.3 environment new or upgraded to 1.2.3? Could you please provide me with more detailed information? Can you add me to WeChat and let's take a look together? My WeChat address is: cylly810222 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] Mryange commented on pull request #18166: [refactor](config) Delete the environment variable enable_vectorized_engine
Mryange commented on PR #18166: URL: https://github.com/apache/doris/pull/18166#issuecomment-1493814131 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #18329: [typo](docs) add regression test doc & fix api doc
github-actions[bot] commented on PR #18329: URL: https://github.com/apache/doris/pull/18329#issuecomment-1493818433 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] englefly commented on pull request #18331: [improve](nereids)compute statsRange.length() according to the column datatype
englefly commented on PR #18331: URL: https://github.com/apache/doris/pull/18331#issuecomment-1493818447 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #18329: [typo](docs) add regression test doc & fix api doc
github-actions[bot] commented on PR #18329: URL: https://github.com/apache/doris/pull/18329#issuecomment-1493818493 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] HappenLee merged pull request #18282: [Bug](decimal) Fix string to decimal
HappenLee merged PR #18282: URL: https://github.com/apache/doris/pull/18282 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated: [Bug](decimal) Fix string to decimal (#18282)
This is an automated email from the ASF dual-hosted git repository.
lihaopeng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 368a2f7ace [Bug](decimal) Fix string to decimal (#18282)
368a2f7ace is described below
commit 368a2f7ace19e3ec854facfc5f0f278a3103ee71
Author: Gabriel
AuthorDate: Mon Apr 3 15:30:48 2023 +0800
[Bug](decimal) Fix string to decimal (#18282)
---
be/src/util/string_parser.hpp | 8 ++--
1 file changed, 2 insertions(+), 6 deletions(-)
diff --git a/be/src/util/string_parser.hpp b/be/src/util/string_parser.hpp
index ff473bf401..5c776029da 100644
--- a/be/src/util/string_parser.hpp
+++ b/be/src/util/string_parser.hpp
@@ -647,7 +647,6 @@ T StringParser::string_to_decimal(const char* s, int len,
int type_precision, in
}
// Find the number of truncated digits before adjusting the precision for
an exponent.
-int truncated_digit_count = precision - type_precision;
if (exponent > scale) {
// Ex: 0.1e3 (which at this point would have precision == 1 and scale
== 1), the
// scale must be set to 0 and the value set to 100 which means a
precision of 3.
@@ -679,9 +678,6 @@ T StringParser::string_to_decimal(const char* s, int len,
int type_precision, in
} else if (UNLIKELY(scale > type_scale)) {
*result = StringParser::PARSE_UNDERFLOW;
int shift = scale - type_scale;
-if (UNLIKELY(truncated_digit_count > 0)) {
-shift -= truncated_digit_count;
-}
if (shift > 0) {
T divisor;
if constexpr (std::is_same_v) {
@@ -689,14 +685,14 @@ T StringParser::string_to_decimal(const char* s, int len,
int type_precision, in
} else {
divisor = get_scale_multiplier(shift);
}
-if (LIKELY(divisor >= 0)) {
+if (LIKELY(divisor > 0)) {
T remainder = value % divisor;
value /= divisor;
if ((remainder > 0 ? T(remainder) : T(-remainder)) >= (divisor
>> 1)) {
value += 1;
}
} else {
-DCHECK(divisor == -1); // //DCHECK_EQ doesn't work with
__int128.
+DCHECK(divisor == -1 || divisor == 0); // //DCHECK_EQ doesn't
work with __int128.
value = 0;
}
}
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] mrhhsg opened a new pull request, #18332: [fix](scan) Avoid using incorrect cache code in ComparisonPredicate
mrhhsg opened a new pull request, #18332: URL: https://github.com/apache/doris/pull/18332 # Proposed changes The cache code for ComparisonPredicate can only be used by predicates that have been newly cloned. ## Problem summary Describe your changes. ## Checklist(Required) * [ ] Does it affect the original behavior * [ ] Has unit tests been added * [ ] Has document been added or modified * [ ] Does it need to update dependencies * [ ] Is this PR support rollback (If NO, please explain WHY) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] HappenLee merged pull request #18177: [Improvement](pipeline) Use bloom runtime filter by default for pipeline engine
HappenLee merged PR #18177: URL: https://github.com/apache/doris/pull/18177 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated (368a2f7ace -> 96a64dc9e8)
This is an automated email from the ASF dual-hosted git repository. lihaopeng pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/doris.git from 368a2f7ace [Bug](decimal) Fix string to decimal (#18282) add 96a64dc9e8 [Improvement](pipeline) Use bloom runtime filter by default for pipeline engine (#18177) No new revisions were added by this update. Summary of changes: .../doris/nereids/glue/translator/RuntimeFilterTranslator.java | 7 +++ .../main/java/org/apache/doris/planner/RuntimeFilterGenerator.java | 6 ++ 2 files changed, 13 insertions(+) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] mrhhsg commented on pull request #18332: [fix](scan) Avoid using incorrect cache code in ComparisonPredicate
mrhhsg commented on PR #18332: URL: https://github.com/apache/doris/pull/18332#issuecomment-1493822354 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #18332: [fix](scan) Avoid using incorrect cache code in ComparisonPredicate
github-actions[bot] commented on PR #18332: URL: https://github.com/apache/doris/pull/18332#issuecomment-1493825014 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] mrhhsg opened a new pull request, #18333: [feature](string_functions) The 'split_part' function supports non-constant parameters
mrhhsg opened a new pull request, #18333: URL: https://github.com/apache/doris/pull/18333 ## Proposed changes pick from #18029 ## Problem summary Describe your changes. ## Checklist(Required) * [ ] Does it affect the original behavior * [ ] Has unit tests been added * [ ] Has document been added or modified * [ ] Does it need to update dependencies * [ ] Is this PR support rollback (If NO, please explain WHY) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] zhangstar333 opened a new pull request, #18334: [vectorized](function) suppoort date_trunc function truncate week mode
zhangstar333 opened a new pull request, #18334:
URL: https://github.com/apache/doris/pull/18334
# Proposed changes
support date_trunc could truncate week eg:
`select date_trunc('2023-4-3 19:28:30', 'week');`
Issue Number: close #xxx
## Problem summary
Describe your changes.
## Checklist(Required)
* [ ] Does it affect the original behavior
* [ ] Has unit tests been added
* [ ] Has document been added or modified
* [ ] Does it need to update dependencies
* [ ] Is this PR support rollback (If NO, please explain WHY)
## Further comments
If this is a relatively large or complex change, kick off the discussion at
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you
chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] zhannngchen commented on pull request #18291: [fix](profile) fix problem on stream load profile log
zhannngchen commented on PR #18291: URL: https://github.com/apache/doris/pull/18291#issuecomment-1493827950 > FYI @gitccl #18015 This is a fix for #18015 , which would cause stream load failed. @hust-hhb it's a good idea to add a new header to control this behavior, you can try to implement in that way. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] morrySnow commented on pull request #18297: [fix](planner) cast be removed after substitute
morrySnow commented on PR #18297: URL: https://github.com/apache/doris/pull/18297#issuecomment-1493830008 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] yixiutt opened a new pull request, #18335: [test](compaction) add delete test
yixiutt opened a new pull request, #18335: URL: https://github.com/apache/doris/pull/18335 # Proposed changes Issue Number: close #xxx ## Problem summary Describe your changes. ## Checklist(Required) * [ ] Does it affect the original behavior * [ ] Has unit tests been added * [ ] Has document been added or modified * [ ] Does it need to update dependencies * [ ] Is this PR support rollback (If NO, please explain WHY) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] nextdreamblue commented on pull request #18163: [feature](multi-catalog) support select current_catalog();
nextdreamblue commented on PR #18163: URL: https://github.com/apache/doris/pull/18163#issuecomment-1493830200 @morrySnow @morningman pls review -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #18334: [vectorized](function) suppoort date_trunc function truncate week mode
github-actions[bot] commented on PR #18334: URL: https://github.com/apache/doris/pull/18334#issuecomment-1493830797 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #18311: [opt](array) optimize array_sortby using qsort instead of bubble sort
github-actions[bot] commented on PR #18311: URL: https://github.com/apache/doris/pull/18311#issuecomment-1493833165 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] Gabriel39 opened a new pull request, #18336: [Bug](DECIMALV3) report failure if a decimal value is overflow
Gabriel39 opened a new pull request, #18336: URL: https://github.com/apache/doris/pull/18336 # Proposed changes Issue Number: close #xxx ## Problem summary Describe your changes. ## Checklist(Required) * [ ] Does it affect the original behavior * [ ] Has unit tests been added * [ ] Has document been added or modified * [ ] Does it need to update dependencies * [ ] Is this PR support rollback (If NO, please explain WHY) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] dataroaring opened a new pull request, #18337: [fix](segment_iter) do not init segment_iterator twice
dataroaring opened a new pull request, #18337: URL: https://github.com/apache/doris/pull/18337 Initing segment_iterator twice would duplicate predicates and we should not use like this. # Proposed changes Issue Number: close #xxx ## Problem summary Describe your changes. ## Checklist(Required) * [ ] Does it affect the original behavior * [ ] Has unit tests been added * [ ] Has document been added or modified * [ ] Does it need to update dependencies * [ ] Is this PR support rollback (If NO, please explain WHY) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #18337: [fix](segment_iter) do not init segment_iterator twice
github-actions[bot] commented on PR #18337: URL: https://github.com/apache/doris/pull/18337#issuecomment-1493840275 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #18336: [Bug](DECIMALV3) report failure if a decimal value is overflow
github-actions[bot] commented on PR #18336: URL: https://github.com/apache/doris/pull/18336#issuecomment-1493841027 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #18337: [fix](segment_iter) do not init segment_iterator twice
github-actions[bot] commented on PR #18337: URL: https://github.com/apache/doris/pull/18337#issuecomment-1493842165 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #18336: [Bug](DECIMALV3) report failure if a decimal value is overflow
github-actions[bot] commented on PR #18336: URL: https://github.com/apache/doris/pull/18336#issuecomment-1493844289 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] GronwD opened a new issue, #18338: [Bug]
GronwD opened a new issue, #18338: URL: https://github.com/apache/doris/issues/18338 ### Search before asking - [X] I had searched in the [issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no similar issues. ### Version 1.2.0 ### What's Wrong? One of our BE was not able to be started, as a result we removed the BE; After that other BEs fails with below error frequently. Please kindly help. *** Query id: 0-0 *** *** Aborted at 1680507855 (unix time) try "date -d @1680507855" if you are using GNU date *** *** Current BE git commitID: Unknown *** *** SIGSEGV address not mapped to object (@0x35) received by PID 2863096 (TID 0x7f9f169ff700) from PID 53; stack trace: *** 0# doris::signal::(anonymous namespace)::FailureSignalHandler(int, siginfo_t*, void*) at /root/doris/be/src/common/signal_handler.h:420 1# 0x7F9F13DFF2B7 in /usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server/libjvm.so 2# JVM_handle_linux_signal in /usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server/libjvm.so 3# 0x7F9F13DF80AC in /usr/lib/jvm/java-8-openjdk-amd64/jre/lib/amd64/server/libjvm.so 4# 0x7F9F173ED090 in /lib/x86_64-linux-gnu/libc.so.6 5# bvar::Reducer, bvar::detail::MinusFrom >::SeriesSampler::take_sample() in /root/apache-doris-be-1.2.0-bin-x86_64/lib/doris_be 6# bvar::detail::SamplerCollector::run() in /root/apache-doris-be-1.2.0-bin-x86_64/lib/doris_be 7# bvar::detail::SamplerCollector::sampling_thread(void*) in /root/apache-doris-be-1.2.0-bin-x86_64/lib/doris_be 8# start_thread in /lib/x86_64-linux-gnu/libpthread.so.0 9# __clone in /lib/x86_64-linux-gnu/libc.so.6 ### What You Expected? BE should not exit ### How to Reproduce? It's like when we run some query on big tables such exception shows in be.out ### Anything Else? _No response_ ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] TomPkuer opened a new pull request, #18339: Update run-docker-cluster.md----The hyphen writing is not standardized
TomPkuer opened a new pull request, #18339: URL: https://github.com/apache/doris/pull/18339 The hyphen writing is not standardized # Proposed changes Issue Number: close #xxx ## Problem summary Describe your changes. ## Checklist(Required) * [ ] Does it affect the original behavior * [ ] Has unit tests been added * [ ] Has document been added or modified * [ ] Does it need to update dependencies * [ ] Is this PR support rollback (If NO, please explain WHY) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] AshinGau opened a new pull request, #18340: [fix](file_cache) turn on file cache by FE session variable
AshinGau opened a new pull request, #18340: URL: https://github.com/apache/doris/pull/18340 # Proposed changes Issue Number: close #xxx ## Problem summary Describe your changes. ## Checklist(Required) * [ ] Does it affect the original behavior * [ ] Has unit tests been added * [ ] Has document been added or modified * [ ] Does it need to update dependencies * [ ] Is this PR support rollback (If NO, please explain WHY) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] yixiutt commented on pull request #18335: [test](compaction) add delete test
yixiutt commented on PR #18335: URL: https://github.com/apache/doris/pull/18335#issuecomment-1493849483 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] wsjz commented on pull request #18005: [feature-wip](multi-catalog)add properties converter
wsjz commented on PR #18005: URL: https://github.com/apache/doris/pull/18005#issuecomment-1493852105 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] mrhhsg opened a new pull request, #18341: [fix](scan) Avoid using incorrect cache code in ComparisonPredicate
mrhhsg opened a new pull request, #18341: URL: https://github.com/apache/doris/pull/18341 # Proposed changes pick from #18332 ## Problem summary Describe your changes. ## Checklist(Required) * [ ] Does it affect the original behavior * [ ] Has unit tests been added * [ ] Has document been added or modified * [ ] Does it need to update dependencies * [ ] Is this PR support rollback (If NO, please explain WHY) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on a diff in pull request #18340: [fix](file_cache) turn on file cache by FE session variable
github-actions[bot] commented on code in PR #18340:
URL: https://github.com/apache/doris/pull/18340#discussion_r1155610410
##
be/src/vec/exec/format/parquet/parquet_thrift_util.h:
##
@@ -35,12 +35,13 @@ namespace doris::vectorized {
constexpr uint8_t PARQUET_VERSION_NUMBER[4] = {'P', 'A', 'R', '1'};
constexpr uint32_t PARQUET_FOOTER_SIZE = 8;
-static Status parse_thrift_footer(io::FileReaderSPtr file, FileMetaData**
file_metadata) {
+static Status parse_thrift_footer(io::FileReaderSPtr file, FileMetaData**
file_metadata,
Review Comment:
warning: unused function 'parse_thrift_footer'
[clang-diagnostic-unused-function]
```cpp
static Status parse_thrift_footer(io::FileReaderSPtr file, FileMetaData**
file_metadata,
^
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] HYBG-1126 commented on pull request #18339: Update run-docker-cluster.md----The hyphen writing is not standardized
HYBG-1126 commented on PR #18339: URL: https://github.com/apache/doris/pull/18339#issuecomment-1493860339 真细心，牛！ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] AshinGau commented on pull request #18340: [fix](file_cache) turn on file cache by FE session variable
AshinGau commented on PR #18340: URL: https://github.com/apache/doris/pull/18340#issuecomment-1493861188 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #18332: [fix](scan) Avoid using incorrect cache code in ComparisonPredicate
github-actions[bot] commented on PR #18332: URL: https://github.com/apache/doris/pull/18332#issuecomment-1493860780 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] Jibing-Li opened a new pull request, #18342: [Refactor](multi catalog)Split ExternalFileScanNode into FileQueryScanNode and FileLoadScanNode.
Jibing-Li opened a new pull request, #18342: URL: https://github.com/apache/doris/pull/18342 Split ExternalFileScanNode into FileQueryScanNode and FileLoadScanNode. Remove some useless code in FileLoadScanNode. * [ ] Does it affect the original behavior * [ ] Has unit tests been added * [ ] Has document been added or modified * [ ] Does it need to update dependencies * [ ] Is this PR support rollback (If NO, please explain WHY) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] 03/03: [fix](load) handle null map right in OlapDataConvertor (#18236)
This is an automated email from the ASF dual-hosted git repository.
zhangchen pushed a commit to branch branch-1.2-lts
in repository https://gitbox.apache.org/repos/asf/doris.git
commit f4456fc0e91ba8dcb757db5f23238d87f7ee909b
Author: Xin Liao
AuthorDate: Mon Apr 3 09:14:05 2023 +0800
[fix](load) handle null map right in OlapDataConvertor (#18236)
The offset of _nullmap and _value are inconsistent in OlapDataConvertor, so
the obtained null flag is incorrect when calling get_ data_ at function. When
the key column or sequence column has null values, the encoding of the short
key index or primary key index may be wrong.
This was introduced by #10883 #10925.
---
be/src/vec/olap/olap_data_convertor.cpp | 17 +
be/src/vec/olap/olap_data_convertor.h | 16
2 files changed, 17 insertions(+), 16 deletions(-)
diff --git a/be/src/vec/olap/olap_data_convertor.cpp
b/be/src/vec/olap/olap_data_convertor.cpp
index e9b4c8fd24..476f20c9a1 100644
--- a/be/src/vec/olap/olap_data_convertor.cpp
+++ b/be/src/vec/olap/olap_data_convertor.cpp
@@ -172,8 +172,9 @@ void
OlapBlockDataConvertor::OlapColumnDataConvertorBase::clear_source_column()
_nullmap = nullptr;
}
-// This should be called only in SegmentWriter. If you want to access nullmap
in Convertor,
-// use `_nullmap` directly.
+// Obtain the converted nullmap with an offset of _row_pos.
+// This should be called only in SegmentWriter and `get_data_at` in Convertor.
+// If you want to access origin nullmap without offset, use `_nullmap`
directly.
const UInt8*
OlapBlockDataConvertor::OlapColumnDataConvertorBase::get_nullmap() const {
assert(_typed_column.column);
return _nullmap ? _nullmap + _row_pos : nullptr;
@@ -195,8 +196,8 @@ const void*
OlapBlockDataConvertor::OlapColumnDataConvertorObject::get_data() co
const void* OlapBlockDataConvertor::OlapColumnDataConvertorObject::get_data_at(
size_t offset) const {
UInt8 null_flag = 0;
-if (_nullmap) {
-null_flag = _nullmap[offset];
+if (get_nullmap()) {
+null_flag = get_nullmap()[offset];
}
return null_flag ? nullptr : _slice.data() + offset;
}
@@ -372,8 +373,8 @@ const void*
OlapBlockDataConvertor::OlapColumnDataConvertorChar::get_data() cons
const void*
OlapBlockDataConvertor::OlapColumnDataConvertorChar::get_data_at(size_t offset)
const {
UInt8 null_flag = 0;
-if (_nullmap) {
-null_flag = _nullmap[offset];
+if (get_nullmap()) {
+null_flag = get_nullmap()[offset];
}
return null_flag ? nullptr : _slice.data() + offset;
}
@@ -428,8 +429,8 @@ const void*
OlapBlockDataConvertor::OlapColumnDataConvertorVarChar::get_data_at(
size_t offset) const {
assert(offset < _slice.size());
UInt8 null_flag = 0;
-if (_nullmap) {
-null_flag = _nullmap[offset];
+if (get_nullmap()) {
+null_flag = get_nullmap()[offset];
}
return null_flag ? nullptr : _slice.data() + offset;
}
diff --git a/be/src/vec/olap/olap_data_convertor.h
b/be/src/vec/olap/olap_data_convertor.h
index 6898b44a9d..037a1f4a70 100644
--- a/be/src/vec/olap/olap_data_convertor.h
+++ b/be/src/vec/olap/olap_data_convertor.h
@@ -179,8 +179,8 @@ private:
const void* get_data() const override { return _values.data(); }
const void* get_data_at(size_t offset) const override {
UInt8 null_flag = 0;
-if (_nullmap) {
-null_flag = _nullmap[offset];
+if (get_nullmap()) {
+null_flag = get_nullmap()[offset];
}
return null_flag ? nullptr : _values.data() + offset;
}
@@ -221,8 +221,8 @@ private:
const void* get_data_at(size_t offset) const override {
assert(offset < _num_rows);
UInt8 null_flag = 0;
-if (_nullmap) {
-null_flag = _nullmap[offset];
+if (get_nullmap()) {
+null_flag = get_nullmap()[offset];
}
return null_flag ? nullptr : _values + offset;
}
@@ -263,8 +263,8 @@ private:
const void* get_data_at(size_t offset) const override {
assert(offset < _num_rows);
UInt8 null_flag = 0;
-if (_nullmap) {
-null_flag = _nullmap[offset];
+if (get_nullmap()) {
+null_flag = get_nullmap()[offset];
}
return null_flag ? nullptr : values_ + offset;
}
@@ -305,8 +305,8 @@ private:
const void* get_data_at(size_t offset) const override {
assert(offset < _num_rows);
UInt8 null_flag = 0;
-if (_nullmap) {
-null_flag = _nullmap[offset];
+if (get_nullmap()) {
+null_flag = get_nullmap()[offset];
}
return null_flag ? nullptr : values_ + offset;
}
---
[doris] 02/03: [fix](merge-on-write) fix that missed rows don't match merged rows for base compaction (#18262)
This is an automated email from the ASF dual-hosted git repository.
zhangchen pushed a commit to branch branch-1.2-lts
in repository https://gitbox.apache.org/repos/asf/doris.git
commit 4c22234520fd0418a17158c78105f38cbaa1a177
Author: Xin Liao
AuthorDate: Fri Mar 31 15:06:51 2023 +0800
[fix](merge-on-write) fix that missed rows don't match merged rows for base
compaction (#18262)
---
be/src/olap/compaction.cpp | 10 ++
1 file changed, 6 insertions(+), 4 deletions(-)
diff --git a/be/src/olap/compaction.cpp b/be/src/olap/compaction.cpp
index 48ef55c347..c154667e6e 100644
--- a/be/src/olap/compaction.cpp
+++ b/be/src/olap/compaction.cpp
@@ -473,10 +473,12 @@ Status Compaction::modify_rowsets(const
Merger::Statistics* stats) {
_tablet->calc_compaction_output_rowset_delete_bitmap(
_input_rowsets, _rowid_conversion, version.second,
UINT64_MAX, &missed_rows,
&location_map, &output_rowset_delete_bitmap);
-DCHECK_EQ(missed_rows.size(), missed_rows_size);
-if (missed_rows.size() != missed_rows_size) {
-LOG(WARNING) << "missed rows don't match, before: " <<
missed_rows_size
- << " after: " << missed_rows.size();
+if (compaction_type() == READER_CUMULATIVE_COMPACTION) {
+DCHECK_EQ(missed_rows.size(), missed_rows_size);
+if (missed_rows.size() != missed_rows_size) {
+LOG(WARNING) << "missed rows don't match, before: " <<
missed_rows_size
+ << " after: " << missed_rows.size();
+}
}
RETURN_IF_ERROR(_tablet->check_rowid_conversion(_output_rowset,
location_map));
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[doris] 01/03: [bugfix](publish) fix TabletLoadInfo may released by delete txn (#17986)
This is an automated email from the ASF dual-hosted git repository.
zhangchen pushed a commit to branch branch-1.2-lts
in repository https://gitbox.apache.org/repos/asf/doris.git
commit 058fb4e420cf38034d421b55811347a3e739c153
Author: yixiutt <102007456+yixi...@users.noreply.github.com>
AuthorDate: Fri Mar 24 20:14:34 2023 +0800
[bugfix](publish) fix TabletLoadInfo may released by delete txn (#17986)
---
be/src/olap/task/engine_publish_version_task.cpp | 10 +++---
be/src/olap/txn_manager.cpp | 10 +-
2 files changed, 12 insertions(+), 8 deletions(-)
diff --git a/be/src/olap/task/engine_publish_version_task.cpp
b/be/src/olap/task/engine_publish_version_task.cpp
index 0422912944..f8847c173c 100644
--- a/be/src/olap/task/engine_publish_version_task.cpp
+++ b/be/src/olap/task/engine_publish_version_task.cpp
@@ -183,9 +183,13 @@ Status EnginePublishVersionTask::finish() {
}
}
-LOG(INFO) << "finish to publish version on transaction."
- << "transaction_id=" << transaction_id << ", cost(us): " <<
watch.get_elapse_time_us()
- << ", error_tablet_size=" << _error_tablet_ids->size() << ",
res=" << res.to_string();
+if (!res.is()) {
+LOG(INFO) << "finish to publish version on transaction."
+ << "transaction_id=" << transaction_id
+ << ", cost(us): " << watch.get_elapse_time_us()
+ << ", error_tablet_size=" << _error_tablet_ids->size()
+ << ", res=" << res.to_string();
+}
return res;
}
diff --git a/be/src/olap/txn_manager.cpp b/be/src/olap/txn_manager.cpp
index e953ed87e2..474b4c8936 100644
--- a/be/src/olap/txn_manager.cpp
+++ b/be/src/olap/txn_manager.cpp
@@ -298,7 +298,7 @@ Status TxnManager::publish_txn(OlapMeta* meta, TPartitionId
partition_id,
pair key(partition_id, transaction_id);
TabletInfo tablet_info(tablet_id, schema_hash, tablet_uid);
RowsetSharedPtr rowset_ptr = nullptr;
-TabletTxnInfo* load_info = nullptr;
+TabletTxnInfo load_info;
{
{
std::unique_lock
txn_rlock(_get_txn_lock(transaction_id));
@@ -310,8 +310,8 @@ Status TxnManager::publish_txn(OlapMeta* meta, TPartitionId
partition_id,
if (load_itr != it->second.end()) {
// found load for txn,tablet
// case 1: user commit rowset, then the load id must be
equal
-load_info = &load_itr->second;
-rowset_ptr = load_info->rowset;
+load_info = load_itr->second;
+rowset_ptr = load_info.rowset;
}
}
}
@@ -323,13 +323,13 @@ Status TxnManager::publish_txn(OlapMeta* meta,
TPartitionId partition_id,
rowset_ptr->make_visible(version);
// update delete_bitmap
{
-if (load_info != nullptr &&
load_info->unique_key_merge_on_write) {
+if (load_info.unique_key_merge_on_write) {
auto tablet =
StorageEngine::instance()->tablet_manager()->get_tablet(tablet_id);
if (tablet == nullptr) {
return Status::OK();
}
-RETURN_IF_ERROR(tablet->update_delete_bitmap(rowset_ptr,
load_info));
+RETURN_IF_ERROR(tablet->update_delete_bitmap(rowset_ptr,
&load_info));
std::shared_lock rlock(tablet->get_header_lock());
tablet->save_meta();
}
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch branch-1.2-lts updated (54228425b3 -> f4456fc0e9)
This is an automated email from the ASF dual-hosted git repository. zhangchen pushed a change to branch branch-1.2-lts in repository https://gitbox.apache.org/repos/asf/doris.git from 54228425b3 [cherry-pick](query exec) should print error status when query meet error #18247 new 058fb4e420 [bugfix](publish) fix TabletLoadInfo may released by delete txn (#17986) new 4c22234520 [fix](merge-on-write) fix that missed rows don't match merged rows for base compaction (#18262) new f4456fc0e9 [fix](load) handle null map right in OlapDataConvertor (#18236) The 3 revisions listed above as "new" are entirely new to this repository and will be described in separate emails. The revisions listed as "add" were already present in the repository and have only been added to this reference. Summary of changes: be/src/olap/compaction.cpp | 10 ++ be/src/olap/task/engine_publish_version_task.cpp | 10 +++--- be/src/olap/txn_manager.cpp | 10 +- be/src/vec/olap/olap_data_convertor.cpp | 17 + be/src/vec/olap/olap_data_convertor.h| 16 5 files changed, 35 insertions(+), 28 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] gitccl commented on pull request #18291: [fix](profile) fix problem on stream load profile log
gitccl commented on PR #18291: URL: https://github.com/apache/doris/pull/18291#issuecomment-1493880708 > > FYI @gitccl #18015 > > @morningman This is a fix for #18015 , which would cause stream load failed. @hust-hhb it's a good idea to add a new header to control this behavior, you can try to implement in that way. It's failed when be reportExecStatus to fe: `runtime error, query xxx does not exist`. Why fe cannot find the query_id in `coordinatorMap` ? https://github.com/apache/doris/blob/master/fe/fe-core/src/main/java/org/apache/doris/qe/QeProcessorImpl.java#L181-L195 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #18239: [DATEV2](test) enable datev2 by default
github-actions[bot] commented on PR #18239: URL: https://github.com/apache/doris/pull/18239#issuecomment-1493882174 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] GronwD commented on issue #18338: [Bug] BE crashes with bvar::detail::SamplerCollector::sampling_thread
GronwD commented on issue #18338: URL: https://github.com/apache/doris/issues/18338#issuecomment-1493884783 after investigation we found this happens when we're querying a very large table without partition key in where conditions. hope this helps. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] Gabriel39 commented on pull request #18239: [DATEV2](test) enable datev2 by default
Gabriel39 commented on PR #18239: URL: https://github.com/apache/doris/pull/18239#issuecomment-1493885292 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] zhangstar333 commented on pull request #16563: [Only-Test-P0](fold_constant) add defer in fold constant to close
zhangstar333 commented on PR #16563: URL: https://github.com/apache/doris/pull/16563#issuecomment-1493889140 run p0 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #16563: [Only-Test-P0](fold_constant) add defer in fold constant to close
github-actions[bot] commented on PR #16563: URL: https://github.com/apache/doris/pull/16563#issuecomment-1493891940 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #18332: [fix](scan) Avoid using incorrect cache code in ComparisonPredicate
github-actions[bot] commented on PR #18332: URL: https://github.com/apache/doris/pull/18332#issuecomment-1493897402 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #18332: [fix](scan) Avoid using incorrect cache code in ComparisonPredicate
github-actions[bot] commented on PR #18332: URL: https://github.com/apache/doris/pull/18332#issuecomment-1493897450 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] czzmmc opened a new issue, #18343: [Feature] java udf/udaf support columns as function parameters like builtin function PERCENTILE
czzmmc opened a new issue, #18343: URL: https://github.com/apache/doris/issues/18343 ### Search before asking - [X] I had searched in the [issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no similar issues. ### Description Sometimes we want to make a function with extra parameters during running time. In order not to create multiple functions which it's really tedious, we usually make a general function with an extra parameter. When we use it, we could pass different value to do different work. ### Use case For example, think about we create a java udaf to copy the function of builtin function PERCENTILE. We do need this new feature to finish the job. ### Related issues None ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #18328: [refactor](typesystem) using typeindex to create column instead of type name because type name is not stable
github-actions[bot] commented on PR #18328: URL: https://github.com/apache/doris/pull/18328#issuecomment-1493900064 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] xiaokang commented on pull request #18302: [bugfix](inverted index) fix inverted index to support NULL value filter
xiaokang commented on PR #18302: URL: https://github.com/apache/doris/pull/18302#issuecomment-1493900066 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] BiteTheDDDDt opened a new pull request, #18344: [Chore](build) enchancement for backend build time usage
BiteThet opened a new pull request, #18344: URL: https://github.com/apache/doris/pull/18344 # Proposed changes 1. add time usage display on build https://user-images.githubusercontent.com/7939630/229455453-c2bc6ca2-f254-4f5d-922a-c289c1b7fb12.png";> 2. remove some unused datetime function registe. 3. split max_by/min_by instantiation. ## Problem summary Describe your changes. ## Checklist(Required) * [ ] Does it affect the original behavior * [ ] Has unit tests been added * [ ] Has document been added or modified * [ ] Does it need to update dependencies * [ ] Is this PR support rollback (If NO, please explain WHY) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #18302: [bugfix](inverted index) fix inverted index to support NULL value filter
github-actions[bot] commented on PR #18302: URL: https://github.com/apache/doris/pull/18302#issuecomment-1493907652 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #18331: [improve](nereids)compute statsRange.length() according to the column datatype
hello-stephen commented on PR #18331: URL: https://github.com/apache/doris/pull/18331#issuecomment-1493908789 TeamCity pipeline, clickbench performance test result: the sum of best hot time: 33.52 seconds stream load tsv: 450 seconds loaded 74807831229 Bytes, about 158 MB/s stream load json: 21 seconds loaded 2358488459 Bytes, about 107 MB/s stream load orc: 72 seconds loaded 1101869774 Bytes, about 14 MB/s stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230403083652_clickbench_pr_124075.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] BiteTheDDDDt commented on pull request #18344: [Chore](build) enchancement for backend build time usage
BiteThet commented on PR #18344: URL: https://github.com/apache/doris/pull/18344#issuecomment-1493909808 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #18344: [Chore](build) enchancement for backend build time usage
github-actions[bot] commented on PR #18344: URL: https://github.com/apache/doris/pull/18344#issuecomment-1493912356 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #18336: [Bug](DECIMALV3) report failure if a decimal value is overflow
github-actions[bot] commented on PR #18336: URL: https://github.com/apache/doris/pull/18336#issuecomment-1493923637 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] jackwener commented on pull request #18316: [feature](Nereids): when cost time > 10s, throw timeout Exception
jackwener commented on PR #18316: URL: https://github.com/apache/doris/pull/18316#issuecomment-1493927434 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] Gabriel39 commented on pull request #18336: [Bug](DECIMALV3) report failure if a decimal value is overflow
Gabriel39 commented on PR #18336: URL: https://github.com/apache/doris/pull/18336#issuecomment-1493938058 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] jacktengg opened a new pull request, #18345: [fix](grouping) fix coredump of grouping function for outer join
jacktengg opened a new pull request, #18345: URL: https://github.com/apache/doris/pull/18345 # Proposed changes Issue Number: close #xxx ## Problem summary Pick from master https://github.com/apache/doris/pull/18292. ## Checklist(Required) * [ ] Does it affect the original behavior * [ ] Has unit tests been added * [ ] Has document been added or modified * [ ] Does it need to update dependencies * [ ] Is this PR support rollback (If NO, please explain WHY) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] liaoxin01 opened a new pull request, #18346: [fix](merge-on-write) fix that failed to capture_consistent_rowsets when full clone
liaoxin01 opened a new pull request, #18346: URL: https://github.com/apache/doris/pull/18346 # Proposed changes Issue Number: close #xxx ## Problem summary When full clone, if the max version of the local table is less than or equal to the max version of the clone table, there is no need to calculate the delete bitmap again. ## Checklist(Required) * [ ] Does it affect the original behavior * [ ] Has unit tests been added * [ ] Has document been added or modified * [ ] Does it need to update dependencies * [ ] Is this PR support rollback (If NO, please explain WHY) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #18346: [fix](merge-on-write) fix that failed to capture_consistent_rowsets when full clone
github-actions[bot] commented on PR #18346: URL: https://github.com/apache/doris/pull/18346#issuecomment-1493958297 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] yiguolei merged pull request #18311: [opt](array) optimize array_sortby using qsort instead of bubble sort
yiguolei merged PR #18311: URL: https://github.com/apache/doris/pull/18311 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated (96a64dc9e8 -> d4688620e9)
This is an automated email from the ASF dual-hosted git repository. yiguolei pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/doris.git from 96a64dc9e8 [Improvement](pipeline) Use bloom runtime filter by default for pipeline engine (#18177) add d4688620e9 [opt](array) optimize array_sortby using qsort instead of bubble sort #18311 No new revisions were added by this update. Summary of changes: .../vec/functions/array/function_array_sortby.cpp | 27 +++--- 1 file changed, 14 insertions(+), 13 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] liaoxin01 commented on pull request #18346: [fix](merge-on-write) fix that failed to capture_consistent_rowsets when full clone
liaoxin01 commented on PR #18346: URL: https://github.com/apache/doris/pull/18346#issuecomment-1493963697 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] Gabriel39 commented on pull request #18239: [DATEV2](test) enable datev2 by default
Gabriel39 commented on PR #18239: URL: https://github.com/apache/doris/pull/18239#issuecomment-1493969412 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #18239: [DATEV2](test) enable datev2 by default
github-actions[bot] commented on PR #18239: URL: https://github.com/apache/doris/pull/18239#issuecomment-1493974099 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #18309: [Optimization](String) Optimize q20 q21 q22 q23 LIKE_SUBSTRING (like '%xxx%')
github-actions[bot] commented on PR #18309: URL: https://github.com/apache/doris/pull/18309#issuecomment-1493984766 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #18309: [Optimization](String) Optimize q20 q21 q22 q23 LIKE_SUBSTRING (like '%xxx%')
github-actions[bot] commented on PR #18309: URL: https://github.com/apache/doris/pull/18309#issuecomment-1493984856 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] gitccl commented on pull request #18291: [fix](profile) fix problem on stream load profile log
gitccl commented on PR #18291: URL: https://github.com/apache/doris/pull/18291#issuecomment-1493988407 > > FYI @gitccl #18015 > > @morningman This is a fix for #18015 , which would cause stream load failed. @hust-hhb it's a good idea to add a new header to control this behavior, you can try to implement in that way. I will fix it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] zhangstar333 opened a new issue, #18348: [Bug] array_map function should be compared offset rows one by one
zhangstar333 opened a new issue, #18348: URL: https://github.com/apache/doris/issues/18348 ### Search before asking - [X] I had searched in the [issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no similar issues. ### Version master ### What's Wrong? array_map function it's needs the input array column have same offset, so should be compared offset rows one by one ### What You Expected? running ok ### How to Reproduce? _No response_ ### Anything Else? _No response_ ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] luzhijing merged pull request #18329: [typo](docs) add regression test doc & fix api doc
luzhijing merged PR #18329: URL: https://github.com/apache/doris/pull/18329 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated: [typo](docs) add regression test doc & fix api doc (#18329)
This is an automated email from the ASF dual-hosted git repository. luzhijing pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris.git The following commit(s) were added to refs/heads/master by this push: new eb6dbc03e0 [typo](docs) add regression test doc & fix api doc (#18329) eb6dbc03e0 is described below commit eb6dbc03e0b19dcf223fdf349735bc91c65aa4db Author: yongkang.zhong AuthorDate: Mon Apr 3 17:40:41 2023 +0800 [typo](docs) add regression test doc & fix api doc (#18329) --- .../en/docs/admin-manual/http-actions/fe/query-profile-action.md | 2 +- docs/zh-CN/community/developer-guide/regression-testing.md | 9 - .../docs/admin-manual/http-actions/fe/query-profile-action.md| 2 +- 3 files changed, 10 insertions(+), 3 deletions(-) diff --git a/docs/en/docs/admin-manual/http-actions/fe/query-profile-action.md b/docs/en/docs/admin-manual/http-actions/fe/query-profile-action.md index efce4a26b8..fcc5e3ab7d 100644 --- a/docs/en/docs/admin-manual/http-actions/fe/query-profile-action.md +++ b/docs/en/docs/admin-manual/http-actions/fe/query-profile-action.md @@ -153,7 +153,7 @@ Get query id by trance id. Before executing a Query, set a unique trace id: -`set set session_context="trace_id:your_trace_id";` +`set session_context="trace_id:your_trace_id";` After executing the Query within the same Session, the query id can be obtained through the trace id. diff --git a/docs/zh-CN/community/developer-guide/regression-testing.md b/docs/zh-CN/community/developer-guide/regression-testing.md index 5d8a64caab..48c6de8f7c 100644 --- a/docs/zh-CN/community/developer-guide/regression-testing.md +++ b/docs/zh-CN/community/developer-guide/regression-testing.md @@ -685,7 +685,14 @@ Doris 支持一些外部署数据源的查询。所以回归框架也提供了 * `sqlserver.yaml.tpl`：Docker compose 文件模板。无需修改。 * `sqlserver.env`：配置 SQLServer 对外端口，默认为 1433。 -7. Iceberg + 7. ClickHouse + + 提供 ClickHouse 22 镜像，存放在 docker/thirdparties/docker-compose/clickhouse/ 下。 + + * `clickhouse.yaml.tpl`：Docker compose 文件模板。无需修改。 + * `clickhouse.env`：配置 ClickHouse 对外端口，默认为 8123。 + +8. Iceberg 提供 Iceberg + Spark + Minio 镜像组合。存放在 docker/thirdparties/docker-compose/iceberg/ 下。 diff --git a/docs/zh-CN/docs/admin-manual/http-actions/fe/query-profile-action.md b/docs/zh-CN/docs/admin-manual/http-actions/fe/query-profile-action.md index 3954d129cc..e02324bc75 100644 --- a/docs/zh-CN/docs/admin-manual/http-actions/fe/query-profile-action.md +++ b/docs/zh-CN/docs/admin-manual/http-actions/fe/query-profile-action.md @@ -153,7 +153,7 @@ GET /rest/v2/manager/query/query_info 在执行一个 Query 前，先设置一个唯一的 trace id: -`set set session_context="trace_id:your_trace_id";` +`set session_context="trace_id:your_trace_id";` 在同一个 Session 链接内执行 Query 后，可以通过 trace id 获取 query id。 - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] wanqiang2046 opened a new issue, #18349: 小批量更新的时候，BE节点CPU200%以上
wanqiang2046 opened a new issue, #18349:
URL: https://github.com/apache/doris/issues/18349
### Search before asking
- [X] I had searched in the
[issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no
similar issues.
### Version
doris-1.2.3-rc02-Unknown
### What's Wrong?
在用UNIQUE模型做业务处理，进行小批量（几千条数据）更新的情况下，CPU一直在200%以上，更新语句执行超级慢，另外日志打印也不正常。
这是我建表语句：
CREATE TABLE `dwd_bi_waybill_scan` (
`id` varchar(64) NOT NULL COMMENT '组合ID',
`waybill_time` datetime NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT
'运单tt_waybill创建时间',
`bu_id` bigint(20) NOT NULL COMMENT '业务ID',
`tb_id` smallint(6) NOT NULL DEFAULT "1" COMMENT '汇聚表类型，如1-扫描表，2-异常登记表',
`country_code` varchar(32) NOT NULL DEFAULT "" COMMENT '国家编码',
`waybill_code` varchar(64) NOT NULL DEFAULT "" COMMENT '运单编号',
`main_waybill_code` varchar(64) NULL DEFAULT "" COMMENT '主运单号',
`order_no` varchar(64) NULL DEFAULT "" COMMENT '订单号',
`order_time` datetime NULL DEFAULT CURRENT_TIMESTAMP COMMENT '下单时间',
`scan_type` int(11) NOT NULL COMMENT '扫描类型
1、揽收。2、发件。3、到件。4、派件。5、签收。-1、问题件。-2、异常件',
`sub_type` varchar(32) NULL DEFAULT "" COMMENT
'子类型：无分类时同主扫描类型，异常签收、问题件有子类型',
`abnormal_sub_type` varchar(32) NULL DEFAULT "" COMMENT '异常子类明细:目前问题问题件有',
`operation_time` datetime NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT
'操作时间',
`operation_date` date NOT NULL COMMENT '操作时间',
`operator_code` varchar(64) NOT NULL DEFAULT "" COMMENT '操作人编号',
`site_code` varchar(64) NOT NULL DEFAULT "" COMMENT '操作网点编号',
`site_type` varchar(64) NULL DEFAULT "" COMMENT
'操作网点类型:SL01-总部,SL02-财务中心,SL03-分拨中心,SL04-一级网点,SL05-二级网点',
`current_status` varchar(16) NULL COMMENT '运单状态',
`data_source` smallint(6) NOT NULL DEFAULT "6" COMMENT '数据源:1-K9, 6-订单中心',
`bl_return` tinyint(4) NOT NULL DEFAULT "0" COMMENT '是否退件:0-未退件, 1-退件',
`bl_return_sign` tinyint(4) NOT NULL DEFAULT "0" COMMENT '是否退件签收:0-未退件签收,
1-退件签收',
`bl_sign` tinyint(4) NOT NULL DEFAULT "0" COMMENT '是否签收:0-未签收, 1-已签收',
`operation_no` smallint(6) NULL DEFAULT "0" COMMENT '扫描历史操作次数',
`customer_code` varchar(100) NOT NULL DEFAULT "" COMMENT '客户代码',
`cod` decimal(15, 2) NULL DEFAULT "0.00" COMMENT '代收货款',
`order_type_code` varchar(8) NOT NULL DEFAULT "" COMMENT
'订单类型:1快递,2零担,3整车',
`transport_type` varchar(8) NULL DEFAULT "" COMMENT
'运输方式:TT01汽运,TT02空运,TT03海运,TT04海空运',
`package_type` tinyint(4) NULL DEFAULT "-1" COMMENT '包裹类型:-1默认 0经济 1标准
2标准特货 3无忧简易 4经济特货 5速达非埃邮泛非标准专线',
`order_product_type_code` varchar(8) NULL DEFAULT "" COMMENT
'产品类型:ST01本地件,ST02国内直发,ST03海外仓发,ST04国际快递,ST05海外仓头程',
`picked_time` datetime NULL COMMENT '揽收时间',
`picked_site_code` varchar(64) NULL DEFAULT "" COMMENT '揽收网点编号',
`picked_site_type` varchar(64) NULL DEFAULT "" COMMENT '揽收网点类型',
`pre_site_code` varchar(64) NULL DEFAULT "" COMMENT '上一站网点编号',
`pre_site_type` varchar(64) NULL DEFAULT "" COMMENT '上一站网点类型',
`next_site_code` varchar(64) NULL DEFAULT "" COMMENT '下一站网点编号',
`next_site_type` varchar(64) NULL DEFAULT "" COMMENT '下一站网点类型',
`actor_code` varchar(100) NULL DEFAULT "" COMMENT '签收人/派件员',
`sign_time` datetime NULL COMMENT '正常签收时间',
`remark` varchar(128) NULL DEFAULT "" COMMENT '备注',
`enable_flag` tinyint(4) NOT NULL DEFAULT "0" COMMENT '有效标识:有效0, 删除1, 取消2',
`status` tinyint(4) NOT NULL DEFAULT "0" COMMENT '状态:0正常 1取消',
`bl_self` tinyint(4) NULL DEFAULT "0" COMMENT '是否自收自派:0否 1是',
`bl_last_return` tinyint(4) NULL DEFAULT "0" COMMENT '是否最后退件网点:0否 1是',
`bl_last_sign` tinyint(4) NULL DEFAULT "0" COMMENT '是否最后签收网点:0否 1是',
`bl_day_first_come` tinyint(4) NULL DEFAULT "0" COMMENT '是否当天第一次到件:0否 1是',
`bl_abnormal_before_delivery` tinyint(4) NULL DEFAULT "0" COMMENT
'派件前是否登记了异常件:0否 1是',
`bl_last_dispatching` tinyint(4) NULL DEFAULT "0" COMMENT '是否是最新派件中的记录:0否
1是',
`duty_site` varchar(64) NULL DEFAULT "" COMMENT '责任网点',
`create_by` varchar(64) NULL DEFAULT "" COMMENT '创建者',
`create_time` datetime NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT '创建时间',
`update_by` varchar(64) NULL DEFAULT "" COMMENT '更新者',
`update_time` datetime NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT '更新时间',
INDEX id_idx (`id`) USING BITMAP COMMENT 'id index'
) ENGINE=OLAP
UNIQUE KEY(`id`, `waybill_time`)
COMMENT '扫描信息'
PARTITION BY RANGE(`waybill_time`)
(PARTITION p202201 VALUES [('-01-01 00:00:00'), ('2022-02-01 00:00:00')),
PARTITION p202202 VALUES [('2022-02-01 00:00:00'), ('2022-03-01 00:00:00')),
PARTITION p202203 VALUES [('2022-03-01 00:00:00'), ('2022-04-01 00:00:00')),
PARTITION p202204 VALUES [('2022-04-01 00:00:00'), ('2022-05-01 00:00:00')),
PARTITION p202205 VALUES [('2022-05-01 00:00:00'), ('2022-06-01 00:00:00')),
PARTITION p202206 VALUES [('2022-06-01 00:00:00'), ('2022-07-01 00:00:00')),
PARTITION p202207 VALUES [('2022-07-01 00:00:00'), ('2022-08-01
[GitHub] [doris] zhangstar333 commented on issue #18348: [Bug] array_map function should be compared offset rows one by one
zhangstar333 commented on issue #18348: URL: https://github.com/apache/doris/issues/18348#issuecomment-1494006025 @herry2038 hi, when you resolve the problem, and link this issue in your PR -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] wanqiang2046 commented on issue #18349: 小批量更新的时候，BE节点CPU200%以上
wanqiang2046 commented on issue #18349: URL: https://github.com/apache/doris/issues/18349#issuecomment-1494006629 BE日志一直打印类似于这些日志： r_tablet_size=2, res=[E-3115] I0403 09:32:13.431612 639 engine_publish_version_task.cpp:186] finish to publish version on transaction.transaction_id=3324162, cost(us): 3, error_tablet_size=1, res=[E-3115] I0403 09:32:13.431615 641 engine_publish_version_task.cpp:186] finish to publish version on transaction.transaction_id=3324179, cost(us): 3, error_tablet_size=1, res=[E-3115] I0403 09:32:13.431620 639 engine_publish_version_task.cpp:186] finish to publish version on transaction.transaction_id=3324189, cost(us): 3, error_tablet_size=1, res=[E-3115] I0403 09:32:13.431623 641 engine_publish_version_task.cpp:186] finish to publish version on transaction.transaction_id=3324163, cost(us): 3, error_tablet_size=1, res=[E-3115] I0403 09:32:13.431635 639 engine_publish_version_task.cpp:186] finish to publish version on transaction.transaction_id=3324158, cost(us): 10, error_tablet_size=1, res=[E-3115] I0403 09:32:13.431648 641 engine_publish_version_task.cpp:186] finish to publish version on transaction.transaction_id=3324185, cost(us): 3, error_tablet_size=1, res=[E-3115] I0403 09:32:13.431656 641 engine_publish_version_task.cpp:186] finish to publish version on transaction.transaction_id=3324159, cost(us): 4, error_tablet_size=1, res=[E-3115] I0403 09:32:13.431660 639 engine_publish_version_task.cpp:186] finish to publish version on transaction.transaction_id=3324194, cost(us): 4, error_tablet_size=1, res=[E-3115] -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] herry2038 commented on issue #18348: [Bug] array_map function should be compared offset rows one by one
herry2038 commented on issue #18348: URL: https://github.com/apache/doris/issues/18348#issuecomment-1494006879 > @herry2038 hi, when you resolve the problem, and link this issue in your PR ok -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] zhangstar333 commented on issue #18266: [Feature] support array_last function, get the last element of the array
zhangstar333 commented on issue #18266: URL: https://github.com/apache/doris/issues/18266#issuecomment-1494011425 @slience-club you could see here first, and this implement maybe have a help for you -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] wanqiang2046 commented on issue #18349: 小批量更新的时候，BE节点CPU200%以上
wanqiang2046 commented on issue #18349: URL: https://github.com/apache/doris/issues/18349#issuecomment-1494017379 BE的warn日志大量打印： failed to publish version|signature=3326621|transaction_id=3326621|error_tablets_num=1|error=[E-3115] W0403 09:48:50.326066 641 task_worker_pool.cpp:723] failed to publish version|signature=3326619|transaction_id=3326619|error_tablets_num=1|error=[E-3115] W0403 09:48:51.318729 642 task_worker_pool.cpp:723] failed to publish version|signature=3326622|transaction_id=3326622|error_tablets_num=1|error=[E-3115] W0403 09:48:51.321202 642 task_worker_pool.cpp:723] failed to publish version|signature=3326623|transaction_id=3326623|error_tablets_num=1|error=[E-3115] W0403 09:48:52.322594 639 task_worker_pool.cpp:723] failed to publish version|signature=3326624|transaction_id=3326624|error_tablets_num=1|error=[E-3115] W0403 09:48:53.320883 639 task_worker_pool.cpp:723] failed to publish version|signature=3326626|transaction_id=3326626|error_tablets_num=1|error=[E-3115] W0403 09:48:54.325161 641 task_worker_pool.cpp:723] failed to publish version|signature=3326629|transaction_id=3326629|error_tablets_num=1|error=[E-3115] W0403 09:48:55.318825 640 task_worker_pool.cpp:723] failed to publish version|signature=3326635|transaction_id=3326635|error_tablets_num=1|error=[E-3115] W0403 09:48:55.338490 638 task_worker_pool.cpp:723] failed to publish version|signature=3326634|transaction_id=3326634|error_tablets_num=1|error=[E-3115] W0403 09:48:55.339264 638 task_worker_pool.cpp:723] failed to publish version|signature=3326636|transaction_id=3326636|error_tablets_num=1|error=[E-3115] -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] zhangstar333 commented on issue #18343: [Feature] java udf/udaf support columns as function parameters like builtin function PERCENTILE
zhangstar333 commented on issue #18343: URL: https://github.com/apache/doris/issues/18343#issuecomment-1494019222 could you be more detailed or give a simple demo? thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] yiguolei commented on pull request #18332: [fix](scan) Avoid using incorrect cache code in ComparisonPredicate
yiguolei commented on PR #18332: URL: https://github.com/apache/doris/pull/18332#issuecomment-1494033822 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] yiguolei commented on pull request #18182: [fix](bitmap) fix bug: orthogonal_bitmap_union_count coredump when arg is nullable
yiguolei commented on PR #18182: URL: https://github.com/apache/doris/pull/18182#issuecomment-1494035477 branch 1.2-lts 是没问题的 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] yiguolei merged pull request #18341: [fix](scan) Avoid using incorrect cache code in ComparisonPredicate
yiguolei merged PR #18341: URL: https://github.com/apache/doris/pull/18341 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch branch-1.2-lts updated: [fix](scan) Avoid using incorrect cache code in ComparisonPredicate (#18341)
This is an automated email from the ASF dual-hosted git repository.
yiguolei pushed a commit to branch branch-1.2-lts
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-1.2-lts by this push:
new 13b055d296 [fix](scan) Avoid using incorrect cache code in
ComparisonPredicate (#18341)
13b055d296 is described below
commit 13b055d296cd0350f7a03c2631077c166326b6cb
Author: Jerry Hu
AuthorDate: Mon Apr 3 18:07:10 2023 +0800
[fix](scan) Avoid using incorrect cache code in ComparisonPredicate (#18341)
---
be/src/olap/comparison_predicate.h | 8 ++--
be/src/olap/rowset/segment_v2/segment_iterator.cpp | 1 +
2 files changed, 7 insertions(+), 2 deletions(-)
diff --git a/be/src/olap/comparison_predicate.h
b/be/src/olap/comparison_predicate.h
index 599bbd50ab..6898eeb1c1 100644
--- a/be/src/olap/comparison_predicate.h
+++ b/be/src/olap/comparison_predicate.h
@@ -36,7 +36,9 @@ public:
_value(value) {}
void clone(ColumnPredicate** to) const override {
-*to = new ComparisonPredicateBase(_column_id, _value, _opposite);
+auto* cloned = new ComparisonPredicateBase(_column_id, _value,
_opposite);
+cloned->_cache_code_enabled = true;
+*to = cloned;
}
bool need_to_clone() const override { return true; }
@@ -628,7 +630,8 @@ private:
__attribute__((flatten)) int32_t _find_code_from_dictionary_column(
const vectorized::ColumnDictI32& column) const {
-if (UNLIKELY(_cached_code == _InvalidateCodeValue)) {
+/// if _cache_code_enabled is false, always find the code from dict.
+if (UNLIKELY(!_cache_code_enabled || _cached_code ==
_InvalidateCodeValue)) {
_cached_code = _is_range() ? column.find_code_by_bound(_value,
_is_greater(), _is_eq())
: column.find_code(_value);
}
@@ -643,6 +646,7 @@ private:
static constexpr int32_t _InvalidateCodeValue =
std::numeric_limits::max();
mutable int32_t _cached_code;
+bool _cache_code_enabled = false;
T _value;
};
diff --git a/be/src/olap/rowset/segment_v2/segment_iterator.cpp
b/be/src/olap/rowset/segment_v2/segment_iterator.cpp
index d9cac145c8..9fded3c31d 100644
--- a/be/src/olap/rowset/segment_v2/segment_iterator.cpp
+++ b/be/src/olap/rowset/segment_v2/segment_iterator.cpp
@@ -166,6 +166,7 @@ SegmentIterator::~SegmentIterator() {
Status SegmentIterator::init(const StorageReadOptions& opts) {
_opts = opts;
+_col_predicates.clear();
for (auto& predicate : opts.column_predicates) {
if (predicate->need_to_clone()) {
ColumnPredicate* cloned;
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on a diff in pull request #18278: [Improvement](predicate) optimize short-circuit predicates
github-actions[bot] commented on code in PR #18278:
URL: https://github.com/apache/doris/pull/18278#discussion_r1155755222
##
be/src/olap/column_predicate.h:
##
@@ -121,6 +121,34 @@ struct PredicateTypeTraits {
}
};
+#define EVALUATE_BY_SELECTOR(EVALUATE_IMPL_WITH_NULL_MAP,
EVALUATE_IMPL_WITHOUT_NULL_MAP) \
Review Comment:
warning: macro is not used [clang-diagnostic-unused-macros]
```cpp
#define EVALUATE_BY_SELECTOR(EVALUATE_IMPL_WITH_NULL_MAP,
EVALUATE_IMPL_WITHOUT_NULL_MAP) \
^
```
##
be/src/olap/comparison_predicate.h:
##
@@ -504,20 +512,36 @@ class ComparisonPredicateBase : public ColumnPredicate {
template
uint16_t _base_loop(uint16_t* sel, uint16_t size, const uint8_t*
__restrict null_map,
-const TArray* __restrict data_array, const TValue&
value) const {
+const TArray* __restrict data_array, const TValue&
value,
+const bool is_dense) const {
uint16_t new_size = 0;
-for (uint16_t i = 0; i < size; ++i) {
-uint16_t idx = sel[i];
-if constexpr (is_nullable) {
-if (_opposite ^ (!null_map[idx] && _operator(data_array[idx],
value))) {
-sel[new_size++] = idx;
+if (is_dense) {
+for (uint16_t i = 0; i < size; ++i) {
+if constexpr (is_nullable) {
+if (_opposite ^ (!null_map[i] && _operator(data_array[i],
value))) {
+sel[new_size++] = i;
+}
+} else {
+if (_opposite ^ _operator(data_array[i], value)) {
+sel[new_size++] = i;
+}
}
-} else {
-if (_opposite ^ _operator(data_array[idx], value)) {
-sel[new_size++] = idx;
+}
+} else {
+for (uint16_t i = 0; i < size; ++i) {
+uint16_t idx = sel[i];
+if constexpr (is_nullable) {
+if () {
Review Comment:
warning: expected expression [clang-diagnostic-error]
```cpp
if () {
^
```
##
be/src/olap/comparison_predicate.h:
##
@@ -504,20 +512,36 @@
template
uint16_t _base_loop(uint16_t* sel, uint16_t size, const uint8_t*
__restrict null_map,
-const TArray* __restrict data_array, const TValue&
value) const {
+const TArray* __restrict data_array, const TValue&
value,
+const bool is_dense) const {
uint16_t new_size = 0;
-for (uint16_t i = 0; i < size; ++i) {
-uint16_t idx = sel[i];
-if constexpr (is_nullable) {
-if (_opposite ^ (!null_map[idx] && _operator(data_array[idx],
value))) {
-sel[new_size++] = idx;
+if (is_dense) {
+for (uint16_t i = 0; i < size; ++i) {
+if constexpr (is_nullable) {
+if (_opposite ^ (!null_map[i] && _operator(data_array[i],
value))) {
+sel[new_size++] = i;
+}
+} else {
+if (_opposite ^ _operator(data_array[i], value)) {
+sel[new_size++] = i;
+}
}
-} else {
-if (_opposite ^ _operator(data_array[idx], value)) {
-sel[new_size++] = idx;
+}
+} else {
+for (uint16_t i = 0; i < size; ++i) {
+uint16_t idx = sel[i];
+if constexpr (is_nullable) {
+if () {
+sel[new_size++] = idx;
+}
+} else {
+if () {
Review Comment:
warning: expected expression [clang-diagnostic-error]
```cpp
if () {
^
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] yiguolei merged pull request #18309: [Optimization](String) Optimize q20 q21 q22 q23 LIKE_SUBSTRING (like '%xxx%')
yiguolei merged PR #18309: URL: https://github.com/apache/doris/pull/18309 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated: [Optimization](String) Optimize q20 q21 q22 q23 LIKE_SUBSTRING (like '%xxx%') (#18309)
This is an automated email from the ASF dual-hosted git repository.
yiguolei pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new b627088e8c [Optimization](String) Optimize q20 q21 q22 q23
LIKE_SUBSTRING (like '%xxx%') (#18309)
b627088e8c is described below
commit b627088e8c89da2e8cdacb439f0608df4ee734df
Author: ZhangYu0123 <67053339+zhangyu0...@users.noreply.github.com>
AuthorDate: Mon Apr 3 18:09:15 2023 +0800
[Optimization](String) Optimize q20 q21 q22 q23 LIKE_SUBSTRING (like
'%xxx%') (#18309)
Optimize q20, q21, q22, q23 LIKE_SUBSTRING (like '%%'). Idea is from
clickhouse stringsearcher:
Stringsearcher is about 10%~20% faster than volnitsky algorithm when needle
size is less than 10 using two chars at beginning search in SIMD .
Stringsearcher is faster than volnitsky algorithm, when needle size is less
than 21.
The changes are as follows:
Using first two chars of needle at beginning search. We can compare two
chars of needle and [n:n+17) chars in haystack in SIMD in one loop. Filter
efficiency will be higher.
When env support SIMD, we use stringsearcher.
Test result in clickbench:
q20 is about 15% up.
q20: SELECT COUNT(*) FROM hits WHERE URL LIKE '%google%';
q21, q22 is about 1%~5% up.
q21: SELECT SearchPhrase, MIN(URL), COUNT(*) AS c FROM hits WHERE URL LIKE
'%google%' AND SearchPhrase <> '' GROUP BY SearchPhrase ORDER BY c DESC LIMIT
10;
q22: SELECT SearchPhrase, MIN(URL), MIN(Title), COUNT(*) AS c,
COUNT(DISTINCT UserID) FROM hits WHERE Title LIKE '%Google%' AND URL NOT LIKE
'%.google.%' AND SearchPhrase <> '' GROUP BY SearchPhrase ORDER BY c DESC LIMIT
10;
q23 is about 30%~40% up and not stable.
q23: SELECT * FROM hits WHERE URL LIKE '%google%' ORDER BY EventTime LIMIT
10;
---
be/src/vec/common/string_searcher.h | 68 +++--
be/src/vec/common/volnitsky.h | 4 +++
2 files changed, 61 insertions(+), 11 deletions(-)
diff --git a/be/src/vec/common/string_searcher.h
b/be/src/vec/common/string_searcher.h
index 895ba538a9..19fd3d2c86 100644
--- a/be/src/vec/common/string_searcher.h
+++ b/be/src/vec/common/string_searcher.h
@@ -77,8 +77,10 @@ private:
uint8_t first {};
#ifdef __SSE4_1__
-/// vector filled `first` for determining leftmost position of the first
symbol
-__m128i pattern;
+uint8_t second {};
+/// vector filled `first` or `second` for determining leftmost position of
the first and second symbols
+__m128i first_pattern;
+__m128i second_pattern;
/// vector of first 16 characters of `needle`
__m128i cache = _mm_setzero_si128();
int cachemask {};
@@ -95,8 +97,11 @@ public:
first = *needle;
#ifdef __SSE4_1__
-pattern = _mm_set1_epi8(first);
-
+first_pattern = _mm_set1_epi8(first);
+if (needle + 1 < needle_end) {
+second = *(needle + 1);
+second_pattern = _mm_set1_epi8(second);
+}
const auto* needle_pos = needle;
//for (const auto i : collections::range(0, n))
@@ -155,16 +160,57 @@ public:
const CharT* search(const CharT* haystack, const CharT* const
haystack_end) const {
if (needle == needle_end) return haystack;
-while (haystack < haystack_end) {
+const auto needle_size = needle_end - needle;
#ifdef __SSE4_1__
-if (haystack + n <= haystack_end && page_safe(haystack)) {
-/// find first character
-const auto v_haystack = _mm_loadu_si128(reinterpret_cast(haystack));
-const auto v_against_pattern = _mm_cmpeq_epi8(v_haystack,
pattern);
+/// Here is the quick path when needle_size is 1.
+if (needle_size == 1) {
+while (haystack < haystack_end) {
+if (haystack + n <= haystack_end && page_safe(haystack)) {
+const auto v_haystack =
+_mm_loadu_si128(reinterpret_cast(haystack));
+const auto v_against_pattern = _mm_cmpeq_epi8(v_haystack,
first_pattern);
+const auto mask = _mm_movemask_epi8(v_against_pattern);
+if (mask == 0) {
+haystack += n;
+continue;
+}
+
+const auto offset = __builtin_ctz(mask);
+haystack += offset;
+
+return haystack;
+}
-const auto mask = _mm_movemask_epi8(v_against_pattern);
+if (haystack == haystack_end) {
+return haystack_end;
+}
-/// first character not present in 16 octets starting at
`haystack`
+if (*haystack == first) {
+return haystack;
+}
+++haysta
[GitHub] [doris] yiguolei commented on pull request #18306: [improvement](executor) Reduce ScannnerCtx Scheduling times
yiguolei commented on PR #18306: URL: https://github.com/apache/doris/pull/18306#issuecomment-1494046707 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] yiguolei merged pull request #18345: [fix](grouping) fix coredump of grouping function for outer join
yiguolei merged PR #18345: URL: https://github.com/apache/doris/pull/18345 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch branch-1.2-lts updated: [fix](grouping) fix coredump of grouping function for outer join (#18345)
This is an automated email from the ASF dual-hosted git repository.
yiguolei pushed a commit to branch branch-1.2-lts
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-1.2-lts by this push:
new 20ce60c326 [fix](grouping) fix coredump of grouping function for outer
join (#18345)
20ce60c326 is described below
commit 20ce60c32664506845eba0e8471d4a567c5a77d9
Author: TengJianPing <18241664+jackte...@users.noreply.github.com>
AuthorDate: Mon Apr 3 18:10:52 2023 +0800
[fix](grouping) fix coredump of grouping function for outer join (#18345)
---
be/src/vec/functions/function_grouping.h | 5 +-
.../query_p0/grouping_sets/test_grouping_sets1.out | 5 ++
.../grouping_sets/test_grouping_sets1.groovy | 84 ++
3 files changed, 93 insertions(+), 1 deletion(-)
diff --git a/be/src/vec/functions/function_grouping.h
b/be/src/vec/functions/function_grouping.h
index 17aa1fdfe6..ae96a50a76 100644
--- a/be/src/vec/functions/function_grouping.h
+++ b/be/src/vec/functions/function_grouping.h
@@ -38,7 +38,10 @@ public:
size_t result, size_t input_rows_count) override {
const ColumnWithTypeAndName& src_column =
block.get_by_position(arguments[0]);
DCHECK(src_column.column->size() == input_rows_count);
-block.get_by_position(result).column = src_column.column;
+// result of functions grouping and grouping_id is always not nullable,
+// but outer join will convert the column to nullable when necessary,
+// so need to remove nullable here when functions grouping and
grouping_id are executed
+block.get_by_position(result).column =
remove_nullable(src_column.column);
return Status::OK();
}
};
diff --git
a/regression-test/data/query_p0/grouping_sets/test_grouping_sets1.out
b/regression-test/data/query_p0/grouping_sets/test_grouping_sets1.out
index a717f57bf4..31973c7bb9 100644
--- a/regression-test/data/query_p0/grouping_sets/test_grouping_sets1.out
+++ b/regression-test/data/query_p0/grouping_sets/test_grouping_sets1.out
@@ -29,3 +29,8 @@ a \N a -1 0 0 0 0
0 1
a \N a -1 0 1 0 1 1 1
\N \N all -1 1 1 1 1 3 2
+-- !sql_grouping_nullable --
+2019-05-04 2019-05-04 2019-05-04 2019-05-04
+2019-05-05 2019-05-05 2019-05-05 2019-05-05
+\N empty \N empty
+
diff --git
a/regression-test/suites/query_p0/grouping_sets/test_grouping_sets1.groovy
b/regression-test/suites/query_p0/grouping_sets/test_grouping_sets1.groovy
index 477f607a76..e3808f55f0 100644
--- a/regression-test/suites/query_p0/grouping_sets/test_grouping_sets1.groovy
+++ b/regression-test/suites/query_p0/grouping_sets/test_grouping_sets1.groovy
@@ -109,4 +109,88 @@ suite("test_grouping_sets1") {
grouping_col1,grouping_col2,col1,col2
;
"""
+
+
+sql """ DROP TABLE IF EXISTS `grouping_t1`; """
+sql """
+CREATE TABLE `grouping_t1` (
+ `p_date` date NULL,
+ `entry_id` varchar(200) NULL,
+ `publish_date` text NULL
+) ENGINE=OLAP
+UNIQUE KEY(`p_date`, `entry_id`)
+COMMENT 'OLAP'
+DISTRIBUTED BY HASH(`entry_id`) BUCKETS 1
+PROPERTIES (
+"replication_allocation" = "tag.location.default: 1"
+);
+"""
+
+sql """
+insert into grouping_t1 values ("2023-03-29", "aaa", "2019-05-04"),
+ ("2023-03-29", "bbb", "2019-05-04"),
+ ("2023-03-30", "aaa", "2019-05-05");
+"""
+
+sql """ DROP TABLE IF EXISTS `grouping_t2`; """
+sql """
+CREATE TABLE `grouping_t2` (
+ `p_date` date NULL,
+ `entry_id` varchar(64) NULL,
+ `entry_date` varchar(64) NULL
+) ENGINE=OLAP
+DUPLICATE KEY(`p_date`, `entry_id`)
+DISTRIBUTED BY HASH(`entry_id`) BUCKETS 2
+PROPERTIES (
+"replication_allocation" = "tag.location.default: 1"
+);
+"""
+
+sql """
+insert into grouping_t2 values ("2023-03-29", "aaa", "2019-05-04"),
+ ("2023-03-29", "bbb", "2019-05-04"),
+ ("2023-03-30", "aaa", "2019-05-05");
+"""
+
+qt_sql_grouping_nullable """
+select
+ *
+from
+ (
+select
+ idt_335.publish_date,
+ if(
+grouping(idt_335.publish_date) = 0,
+idt_335.publish_date,
+'empty'
+ ) as dim_207
+from
+ (
+select
+ *
+from
+ grouping_t1
+ ) idt_335
+group by
+ GROUPING
[GitHub] [doris] CalvinKirs commented on issue #18304: [Enhancement] Spring Framework 身份认证绕过漏洞(CVE-2023-20860)
CalvinKirs commented on issue #18304: URL: https://github.com/apache/doris/issues/18304#issuecomment-1494051778 I checked the code of Doris, and using Doris does not trigger such a situation. Of course, we should upgrade to the appropriate version as soon as possible. BTW, any questions about security vulnerabilities should not be discussed in public, -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on a diff in pull request #18324: [Fix](orc-reader) Fix the scale of decimal column is incorrect when query orc tables.
github-actions[bot] commented on code in PR #18324:
URL: https://github.com/apache/doris/pull/18324#discussion_r1155768368
##
be/src/vec/exec/format/orc/vorc_reader.h:
##
@@ -147,13 +147,19 @@ class OrcReader : public GenericReader {
Status _decode_explicit_decimal_column(const std::string& col_name,
const MutableColumnPtr& data_column,
const DataTypePtr& data_type,
- DecimalScaleParams& scale_params,
orc::ColumnVectorBatch* cvb, size_t
num_values) {
OrcColumnType* data = dynamic_cast(cvb);
if (data == nullptr) {
return Status::InternalError("Wrong data type for colum '{}'",
col_name);
}
-_init_decimal_converter(data_type, scale_params,
data->scale);
+if (_decimal_scale_params_index >= _decimal_scale_params.size()) {
+DecimalScaleParams temp_scale_params;
+_init_decimal_converter(data_type,
temp_scale_params,
+ data->scale);
+_decimal_scale_params.emplace_back(std::move(temp_scale_params));
Review Comment:
warning: std::move of the variable 'temp_scale_params' of the
trivially-copyable type 'doris::vectorized::DecimalScaleParams' has no effect;
remove std::move() [performance-move-const-arg]
```suggestion
_decimal_scale_params.emplace_back(temp_scale_params);
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] xiaokang opened a new pull request, #18350: [Enhencement](like) fallback to re2 if hyperscan failed
xiaokang opened a new pull request, #18350: URL: https://github.com/apache/doris/pull/18350 # Proposed changes Issue Number: close #xxx ## Problem summary Describe your changes. ## Checklist(Required) * [ ] Does it affect the original behavior * [ ] Has unit tests been added * [ ] Has document been added or modified * [ ] Does it need to update dependencies * [ ] Is this PR support rollback (If NO, please explain WHY) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] zy-kkk closed issue #18304: [Enhancement] Spring Framework 身份认证绕过漏洞(CVE-2023-20860)
zy-kkk closed issue #18304: [Enhancement] Spring Framework 身份认证绕过漏洞(CVE-2023-20860) URL: https://github.com/apache/doris/issues/18304 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] HappenLee commented on a diff in pull request #18321: [Optimization](string) optimize constant empty string compare ( column='', column!='')
HappenLee commented on code in PR #18321:
URL: https://github.com/apache/doris/pull/18321#discussion_r1155789374
##
be/src/vec/functions/functions_comparison.h:
##
@@ -211,15 +211,28 @@ struct StringEqualsImpl {
ColumnString::Offset b_size,
PaddedPODArray& c) {
size_t size = a_offsets.size();
-ColumnString::Offset prev_a_offset = 0;
-
-for (size_t i = 0; i < size; ++i) {
-auto a_size = a_offsets[i] - prev_a_offset;
-
-c[i] = positive == memequal_small_allow_overflow15(a_data.data() +
prev_a_offset,
- a_size,
b_data.data(), b_size);
-
-prev_a_offset = a_offsets[i];
+if (b_size == 0) {
+auto* __restrict data = c.data();
+auto* __restrict offsets = a_offsets.data();
+if (positive) {
+for (size_t i = 0; i < size; ++i) {
+data[i] = (offsets[i] == offsets[i - 1]);
Review Comment:
`data[i] = positive ? (offsets[i] == offsets[i - 1]) : (offsets[i] !=
offsets[i - 1]);`
The code can auto simd
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
