[GitHub] [doris] github-actions[bot] commented on pull request #17886: [typo](doc)Replace invalid urls
github-actions[bot] commented on PR #17886: URL: https://github.com/apache/doris/pull/17886#issuecomment-1473270327 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #17886: [typo](doc)Replace invalid urls
github-actions[bot] commented on PR #17886: URL: https://github.com/apache/doris/pull/17886#issuecomment-1473270404 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] GoGoWen commented on issue #17876: [Bug] 相同的join查询，当join的键(字段名)大写和小写的时候，系统匹配的join方式不同
GoGoWen commented on issue #17876: URL: https://github.com/apache/doris/issues/17876#issuecomment-1473270684 seems an issue. i will take it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] CalvinKirs opened a new pull request, #17894: [fe](test)Add MicroBench module to make it easier for developers to write JMH Test
CalvinKirs opened a new pull request, #17894:
URL: https://github.com/apache/doris/pull/17894
Writing the benchmark itself is not hard, but getting it right is.
Thankfully, the JMH suite provides helpful annotations and features to mitigate
most of them. To get started, you need to make your benchmark extend the
AbstractMicrobenchmark, which makes sure the test gets run through JUnit and
configures some defaults:
@BenchmarkMode({Mode.Throughput})
public class LinkedArrayBenchmark extends AbstractMicrobenchmark {
}
For most benchmark tests, you can write it as a normal JUnit.
We could also choose Customizing Runtime Conditions.
At present, you can run all JMH Tests by running the jar directly after
packaging.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] Tanya-W commented on pull request #17825: [fix](schema change) Fix fe restart failed because of replay schema change alter job failed
Tanya-W commented on PR #17825: URL: https://github.com/apache/doris/pull/17825#issuecomment-1473280133 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] GoGoWen opened a new issue, #17895: [Feature] (Doris External Table) support HLL/BITMAP for Doris External Table with JDBC
GoGoWen opened a new issue, #17895: URL: https://github.com/apache/doris/issues/17895 ### Search before asking - [X] I had searched in the [issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no similar issues. ### Description Currently, when we query a Doris table with JDBC external table, the HLL/BITMAP column is not supported. ### Use case _No response_ ### Related issues _No response_ ### Are you willing to submit PR? - [X] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] HappenLee opened a new pull request, #17896: [Bug][Fix] pipeline exec engine get wrong result when run regression test
HappenLee opened a new pull request, #17896: URL: https://github.com/apache/doris/pull/17896 # Proposed changes Fix regression p1：regression-test/suites/datev2/tpcds_sf1_p1/sql/pipeline case ## Problem summary Describe your changes. ## Checklist(Required) * [ ] Does it affect the original behavior * [ ] Has unit tests been added * [ ] Has document been added or modified * [ ] Does it need to update dependencies * [ ] Is this PR support rollback (If NO, please explain WHY) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] DarvenDuan opened a new pull request, #17897: [Enhancement] (auditLog) Support recording table names in fe.audit.log directly
DarvenDuan opened a new pull request, #17897: URL: https://github.com/apache/doris/pull/17897 # Proposed changes Issue Number: close #17891 ## Problem summary Describe your changes. ## Checklist(Required) * [ ] Does it affect the original behavior * [ ] Has unit tests been added * [ ] Has document been added or modified * [ ] Does it need to update dependencies * [X] Is this PR support rollback (If NO, please explain WHY) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] yagagagaga opened a new pull request, #17899: [typo](docs) Add a hyperlink to facilitate user redirect.
yagagagaga opened a new pull request, #17899: URL: https://github.com/apache/doris/pull/17899 # Proposed changes Issue Number: close #xxx ## Problem summary Describe your changes. ## Checklist(Required) * [ ] Does it affect the original behavior * [ ] Has unit tests been added * [x] Has document been added or modified * [ ] Does it need to update dependencies * [x] Is this PR support rollback (If NO, please explain WHY) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] morrySnow opened a new pull request, #17900: [fix](regression-test) fix unstable regression test cases found in p0
morrySnow opened a new pull request, #17900: URL: https://github.com/apache/doris/pull/17900 # Proposed changes Issue Number: close #xxx ## Problem summary fix below cases: 1. nereids_p0/sql_functions/json_function/test_query_json_object 2. nereids_p0/sql_functions/math_functions/test_multiply 3. nereids_syntax_p0/lateral_view 4. nereids_syntax_p0/window_function 5. query_p0/sql_functions/array_functions/test_array_functions_of_array_difference 6. query_p0/sql_functions/json_function/test_query_json_array 7. query_p0/sql_functions/json_function/test_query_json_object ## Checklist(Required) * [ ] Does it affect the original behavior * [ ] Has unit tests been added * [ ] Has document been added or modified * [ ] Does it need to update dependencies * [ ] Is this PR support rollback (If NO, please explain WHY) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] Yulei-Yang commented on pull request #17108: [Enhancement](spark load)Support spark time out config
Yulei-Yang commented on PR #17108: URL: https://github.com/apache/doris/pull/17108#issuecomment-1473324537 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] morrySnow commented on pull request #17872: [Feature](Nereids) add funtion sum with multi parameters in nereids
morrySnow commented on PR #17872: URL: https://github.com/apache/doris/pull/17872#issuecomment-1473349590 we should not support sum(a, b, c) and forbid it on legacy planner -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] morrySnow closed pull request #17872: [Feature](Nereids) add funtion sum with multi parameters in nereids
morrySnow closed pull request #17872: [Feature](Nereids) add funtion sum with multi parameters in nereids URL: https://github.com/apache/doris/pull/17872 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #17615: Init task Group func
github-actions[bot] commented on PR #17615: URL: https://github.com/apache/doris/pull/17615#issuecomment-1473352011 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] Gabriel39 opened a new pull request, #17901: [Bug](float) fix wrong value when enable fold constant by BE
Gabriel39 opened a new pull request, #17901: URL: https://github.com/apache/doris/pull/17901 # Proposed changes Issue Number: close #xxx ## Problem summary Describe your changes. ## Checklist(Required) * [ ] Does it affect the original behavior * [ ] Has unit tests been added * [ ] Has document been added or modified * [ ] Does it need to update dependencies * [ ] Is this PR support rollback (If NO, please explain WHY) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] TangSiyang2001 commented on pull request #17837: [fix](timeout) fix set query_timeout and insert_timeout failure
TangSiyang2001 commented on PR #17837: URL: https://github.com/apache/doris/pull/17837#issuecomment-1473360363 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #17901: [Bug](float) fix wrong value when enable fold constant by BE
github-actions[bot] commented on PR #17901: URL: https://github.com/apache/doris/pull/17901#issuecomment-1473361543 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #17901: [Bug](float) fix wrong value when enable fold constant by BE
github-actions[bot] commented on PR #17901: URL: https://github.com/apache/doris/pull/17901#issuecomment-1473361783 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #17837: [fix](timeout) fix set query_timeout and insert_timeout failure
github-actions[bot] commented on PR #17837: URL: https://github.com/apache/doris/pull/17837#issuecomment-1473364861 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] freemandealer opened a new issue, #17902: [Enhancement] inaccurate estimation of the load time for low compression rate data
freemandealer opened a new issue, #17902: URL: https://github.com/apache/doris/issues/17902 ### Search before asking - [X] I had searched in the [issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no similar issues. ### Description ORC/parquet files can have very low compression rates, such as 3%. This means a 3GB file can expand to 100GB during loading. Users may wonder why loading such ‘small’ files takes hours. We need to explain the data size to them. ### Solution In a loading job, there are three types of data size: 1) the original size which we need to read as input. It may be compressed and encoded compactly. 2) the size during process. Each record is decompressed and decoded so that Doris can deal with it. 3) the size we write to disk. Again, compressed and encoded. We should explain the three sizes individually to the user during & after the load, by: 1) enhancing `show load`/`show stream load` statements 2) improving profile facilities 3) other places that can show the user intuitively ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] sohardforaname commented on pull request #17537: [Enhancement](Planner)fix unclear exception msg when create mv
sohardforaname commented on PR #17537: URL: https://github.com/apache/doris/pull/17537#issuecomment-1473366195 run p0 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] sohardforaname commented on pull request #17815: [fix](planner)fix failed to create view when use window function
sohardforaname commented on PR #17815: URL: https://github.com/apache/doris/pull/17815#issuecomment-1473367502 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] Gabriel39 commented on pull request #17901: [Bug](float) fix wrong value when enable fold constant by BE
Gabriel39 commented on PR #17901: URL: https://github.com/apache/doris/pull/17901#issuecomment-1473388851 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] Gabriel39 commented on pull request #17901: [Bug](float) fix wrong value when enable fold constant by BE
Gabriel39 commented on PR #17901: URL: https://github.com/apache/doris/pull/17901#issuecomment-1473390463 We will set enable fold constant by default after this PR so we do not need to add a regression test now -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] zenoyang commented on pull request #17828: [improvement](fetch data) Merge result into batch to reduce rpc times
zenoyang commented on PR #17828: URL: https://github.com/apache/doris/pull/17828#issuecomment-1473392713 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #17896: [Bug][Fix] pipeline exec engine get wrong result when run regression test
github-actions[bot] commented on PR #17896: URL: https://github.com/apache/doris/pull/17896#issuecomment-1473397159 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #17896: [Bug][Fix] pipeline exec engine get wrong result when run regression test
github-actions[bot] commented on PR #17896: URL: https://github.com/apache/doris/pull/17896#issuecomment-1473397259 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #17828: [improvement](fetch data) Merge result into batch to reduce rpc times
github-actions[bot] commented on PR #17828: URL: https://github.com/apache/doris/pull/17828#issuecomment-1473398574 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] BiteTheDDDDt commented on pull request #17850: [Bug](materialized-view) fix select mv rollback fail on left join
BiteThet commented on PR #17850: URL: https://github.com/apache/doris/pull/17850#issuecomment-1473418084 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #17850: [Bug](materialized-view) fix select mv rollback fail on left join
github-actions[bot] commented on PR #17850: URL: https://github.com/apache/doris/pull/17850#issuecomment-1473418750 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #17850: [Bug](materialized-view) fix select mv rollback fail on left join
github-actions[bot] commented on PR #17850: URL: https://github.com/apache/doris/pull/17850#issuecomment-1473423533 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] nextdreamblue opened a new issue, #17903: [Bug] routine load strict mode error
nextdreamblue opened a new issue, #17903: URL: https://github.com/apache/doris/issues/17903 ### Search before asking - [X] I had searched in the [issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no similar issues. ### Version master ### What's Wrong? CREATE TABLE `test` ( `k1` datetime NOT NULL COMMENT "", `v1` datetime NOT NULL DEFAULT CURRENT_TIMESTAMP COMMENT "", `v2` varchar(8) NOT NULL COMMENT "", `v3` largeint(40) NOT NULL COMMENT "", `v4` varchar(18) NULL COMMENT "", `v5` varchar(20) NULL COMMENT "", `v6` int NULL COMMENT "", `v7` varchar(8) NOT NULL COMMENT "", `v8` varchar(8) NOT NULL COMMENT "", `v9` varchar(320) NOT NULL COMMENT "" ) ENGINE=OLAP DUPLICATE KEY(`k1`) COMMENT "OLAP" DISTRIBUTED BY HASH(`k1`) BUCKETS 1 PROPERTIES ( "replication_allocation" = "tag.location.default: 1" ); CREATE ROUTINE LOAD test123123 ON test COLUMNS TERMINATED BY "|", COLUMNS(k1, v2, v3, v4, v5, v6, v7, v8, v9) PROPERTIES ( "max_batch_interval" = "30", "max_batch_rows" = "30", "max_batch_size" = "11100", "strict_mode" = "true" ) FROM KAFKA ( "kafka_broker_list" = "", "kafka_topic" = "", "kafka_partitions" = "*, "property.group.id" = "**" ); 数据内容如下： ``` 2023-03-02|test|345634235887||132213123||AF|af|x...@gmail.com ``` 报错： ``` Reason: column(v6) value is incorrect while strict mode is true, src value is af. src line [2023-03-02 test 345634235887 11 132213123 AF af x...@gmail.com]; ``` 相同的数据和表使用stream load导入就可以正确导入，routine load就会失败 ### What You Expected? 正确导入 ### How to Reproduce? _No response_ ### Anything Else? _No response_ ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] GoGoWen opened a new pull request, #17904: [Fix](plan)fix issue 17876
GoGoWen opened a new pull request, #17904: URL: https://github.com/apache/doris/pull/17904 # Proposed changes Issue Number: close #17876 ## Problem summary Describe your changes. ## Checklist(Required) * [ ] Does it affect the original behavior * [ ] Has unit tests been added * [ ] Has document been added or modified * [ ] Does it need to update dependencies * [ ] Is this PR support rollback (If NO, please explain WHY) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] qidaye merged pull request #17842: [Bug](dynamic-table) Fix column alignment logic and support filtering…
qidaye merged PR #17842: URL: https://github.com/apache/doris/pull/17842 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #17842: [Bug](dynamic-table) Fix column alignment logic and support filtering…
github-actions[bot] commented on PR #17842: URL: https://github.com/apache/doris/pull/17842#issuecomment-1473433368 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #17842: [Bug](dynamic-table) Fix column alignment logic and support filtering…
github-actions[bot] commented on PR #17842: URL: https://github.com/apache/doris/pull/17842#issuecomment-1473433465 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated (5bd5402378 -> 043f77200f)
This is an automated email from the ASF dual-hosted git repository. jianliangqi pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/doris.git from 5bd5402378 [bug](udf) add synchronized to test resolve error of zip file closed (#17812) add 043f77200f [Bug](dynamic-table) Fix column alignment logic and support filtering null values when slot is not null (#17842) No new revisions were added by this update. Summary of changes: be/src/vec/columns/column_object.cpp | 60 +--- be/src/vec/columns/column_object.h | 12 +- be/src/vec/common/schema_util.cpp | 64 ++--- be/src/vec/common/schema_util.h| 4 +- be/src/vec/data_types/data_type_object.cpp | 10 +- be/src/vec/exec/format/json/new_json_reader.cpp| 151 ++--- be/src/vec/exec/format/json/new_json_reader.h | 45 +++--- regression-test/data/dynamic_table_p0/sql/q05.out | 32 + .../suites/dynamic_table_p0/load.groovy| 51 +++ .../suites/dynamic_table_p0/sql/q05.sql| 9 ++ 10 files changed, 287 insertions(+), 151 deletions(-) create mode 100644 regression-test/data/dynamic_table_p0/sql/q05.out create mode 100644 regression-test/suites/dynamic_table_p0/sql/q05.sql - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] erDogLi commented on issue #10909: have no queryable replicas. err: 15606's backend 10002 does not exist or not alive
erDogLi commented on issue #10909: URL: https://github.com/apache/doris/issues/10909#issuecomment-1473435318 解决了吗 我是加入就报错 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] erDogLi opened a new issue, #17905: [Feature]
erDogLi opened a new issue, #17905: URL: https://github.com/apache/doris/issues/17905 ### Search before asking - [X] I had searched in the [issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no similar issues. ### Description heartbeatmgr.runaftercatalogready():139 ### Use case heartbeatmgr.runaftercatalogready():139 ### Related issues heartbeatmgr.runaftercatalogready():139 添加了be之后就报这个错误连不上 端口是通的 ### Are you willing to submit PR? - [X] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #17857: [fix](merge-on-write) fix that failed to update delete bitmap when revise tablet meta for clone
github-actions[bot] commented on PR #17857: URL: https://github.com/apache/doris/pull/17857#issuecomment-1473439792 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] zhannngchen commented on pull request #17857: [fix](merge-on-write) fix that failed to update delete bitmap when revise tablet meta for clone
zhannngchen commented on PR #17857: URL: https://github.com/apache/doris/pull/17857#issuecomment-1473444024 The patch is testing, will merge it tomorrow morning if everything goes well. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] nextdreamblue commented on pull request #17803: [feature](multi-catalog) add specified_database_list PROPERTY for jdbc catalog
nextdreamblue commented on PR #17803: URL: https://github.com/apache/doris/pull/17803#issuecomment-1473446896 run clickbench p0 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] eldenmoon opened a new pull request, #17906: [regression-test](array) Fix array case, add order by
eldenmoon opened a new pull request, #17906: URL: https://github.com/apache/doris/pull/17906 # Proposed changes Issue Number: close #xxx ## Problem summary Describe your changes. ## Checklist(Required) * [ ] Does it affect the original behavior * [ ] Has unit tests been added * [ ] Has document been added or modified * [ ] Does it need to update dependencies * [ ] Is this PR support rollback (If NO, please explain WHY) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] eldenmoon commented on pull request #17906: [regression-test](array) Fix array case, add order by
eldenmoon commented on PR #17906: URL: https://github.com/apache/doris/pull/17906#issuecomment-1473461300 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] BiteTheDDDDt commented on pull request #17850: [Bug](materialized-view) fix select mv rollback fail on left join
BiteThet commented on PR #17850: URL: https://github.com/apache/doris/pull/17850#issuecomment-1473467159 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] xinyiZzz opened a new pull request, #17907: [fix](function) resubmit-fix AES/SM3/SM4 encrypt/ decrypt algorithm initialization vector bug
xinyiZzz opened a new pull request, #17907: URL: https://github.com/apache/doris/pull/17907 # Proposed changes Issue Number: close #xxx ## Problem summary The first fix: #17420, there is a compatibility problem, So add aes/sm4 V2 function to support correct behavior. The original aes/sm4 still maintains the previous behavior. —— 1. ECB algorithm, `block_encryption_mode` does not take effect, it only takes effect when init vector is provided. Solved: 192/256 supports calculation without init vector 2. For other algorithms, an error should be reported when there is no init vector Initialization Vector. The default value for the block_encryption_mode system variable is aes-128-ecb, or ECB mode, which does not require an initialization vector. The alternative permitted block encryption modes CBC, CFB1, CFB8, CFB128, and OFB all require an initialization vector. Reference: https://dev.mysql.com/doc/refman/8.0/en/encryption-functions.html#function_aes-decrypt ## Checklist(Required) * [ ] Does it affect the original behavior * [ ] Has unit tests been added * [ ] Has document been added or modified * [ ] Does it need to update dependencies * [ ] Is this PR support rollback (If NO, please explain WHY) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] eldenmoon commented on pull request #17906: [regression-test](array) Fix array case, add order by
eldenmoon commented on PR #17906: URL: https://github.com/apache/doris/pull/17906#issuecomment-1473475883 run p0 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] levy5307 opened a new pull request, #17909: fix: fix the bug that g_fragmentmgr_prepare_latency update uncorrectly
levy5307 opened a new pull request, #17909:
URL: https://github.com/apache/doris/pull/17909
# Proposed changes
Issue Number: close #xxx
## Problem summary
In the scope of `{}`, the value of `duration_ns` is 0. So this will make
`g_fragmentmgr_prepare_latency` uncorrectly.
## Checklist(Required)
* [-] Does it affect the original behavior
* [-] Has unit tests been added
I wanted to add a unit test to this function, but found it difficult to
implement.
* [-] Has document been added or modified
* [-] Does it need to update dependencies
* [x] Is this PR support rollback (If NO, please explain WHY)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #17909: [fix]fix the bug that g_fragmentmgr_prepare_latency update uncorrectly
github-actions[bot] commented on PR #17909: URL: https://github.com/apache/doris/pull/17909#issuecomment-1473499942 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] jacktengg opened a new pull request, #17910: Revert "[enhancement](memory) PODArray replaces MemPool in PredicateColumn (#17800)"
jacktengg opened a new pull request, #17910: URL: https://github.com/apache/doris/pull/17910 This reverts commit 17d1c1bc7f6cc95eecd224eaa219c976b60fa17e. Since it introduces bugs. # Proposed changes Issue Number: close #xxx ## Problem summary Describe your changes. ## Checklist(Required) * [ ] Does it affect the original behavior * [ ] Has unit tests been added * [ ] Has document been added or modified * [ ] Does it need to update dependencies * [ ] Is this PR support rollback (If NO, please explain WHY) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #17910: Revert "[enhancement](memory) PODArray replaces MemPool in PredicateColumn (#17800)"
github-actions[bot] commented on PR #17910: URL: https://github.com/apache/doris/pull/17910#issuecomment-1473510882 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] Gabriel39 commented on pull request #17670: [refactor](date) unify DateTimeValue and VecDateTimeValue
Gabriel39 commented on PR #17670: URL: https://github.com/apache/doris/pull/17670#issuecomment-1473528236 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] xinyiZzz closed pull request #17869: [fix](memory) Fix PredicateColumn wrong usage after arena expansion
xinyiZzz closed pull request #17869: [fix](memory) Fix PredicateColumn wrong usage after arena expansion URL: https://github.com/apache/doris/pull/17869 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #17790: [enhancement](nereids) update FilterEstimation and Agg in stats derive
github-actions[bot] commented on PR #17790: URL: https://github.com/apache/doris/pull/17790#issuecomment-1473533641 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] yiguolei commented on pull request #17910: Revert "[enhancement](memory) PODArray replaces MemPool in PredicateColumn (#17800)"
yiguolei commented on PR #17910: URL: https://github.com/apache/doris/pull/17910#issuecomment-1473533685 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] sohardforaname commented on pull request #17815: [fix](planner)fix failed to create view when use window function
sohardforaname commented on PR #17815: URL: https://github.com/apache/doris/pull/17815#issuecomment-1473535008 run p0 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] sohardforaname commented on pull request #17549: [regression-test](Nereids)add regression-test of arithmetic expressions of decimalv3 for nereids.
sohardforaname commented on PR #17549: URL: https://github.com/apache/doris/pull/17549#issuecomment-1473534682 run p0 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #17910: Revert "[enhancement](memory) PODArray replaces MemPool in PredicateColumn (#17800)"
github-actions[bot] commented on PR #17910: URL: https://github.com/apache/doris/pull/17910#issuecomment-1473535071 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #17910: Revert "[enhancement](memory) PODArray replaces MemPool in PredicateColumn (#17800)"
github-actions[bot] commented on PR #17910: URL: https://github.com/apache/doris/pull/17910#issuecomment-1473535198 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #17670: [refactor](date) unify DateTimeValue and VecDateTimeValue
github-actions[bot] commented on PR #17670: URL: https://github.com/apache/doris/pull/17670#issuecomment-1473548414 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] starocean999 commented on pull request #17831: [fix](planner)project should be done inside inlineview
starocean999 commented on PR #17831: URL: https://github.com/apache/doris/pull/17831#issuecomment-1473560556 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated: [enhancement](nereids) update FilterEstimation and Agg in stats derive (#17790)
This is an automated email from the ASF dual-hosted git repository.
englefly pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 8debc96d74 [enhancement](nereids) update FilterEstimation and Agg in
stats derive (#17790)
8debc96d74 is described below
commit 8debc96d74b2a5c2c4d24d47f8df47554096333d
Author: minghong
AuthorDate: Fri Mar 17 18:01:50 2023 +0800
[enhancement](nereids) update FilterEstimation and Agg in stats derive
(#17790)
* 1. update ndv in Stats,
2. skip __DORIS_DELETE_SIGN__=0 in stats derive,
3. equalTo in stats derive
4. update agg stats derive, support the case: all column_stats are unknown
* computeSize
* fix ut
---
.../apache/doris/nereids/memo/GroupExpression.java | 16 ++---
.../doris/nereids/stats/FilterEstimation.java | 69 +-
.../apache/doris/nereids/stats/JoinEstimation.java | 6 ++
.../doris/nereids/stats/StatsCalculator.java | 35 ---
.../plans/physical/PhysicalHashAggregate.java | 2 +-
.../plans/physical/PhysicalNestedLoopJoin.java | 3 +-
.../org/apache/doris/statistics/Statistics.java| 31 +++---
.../org/apache/doris/nereids/memo/RankTest.java| 3 +-
.../doris/nereids/stats/FilterEstimationTest.java | 37 ++--
.../doris/nereids/stats/StatsCalculatorTest.java | 12 ++--
.../suites/nereids_syntax_p0/join.groovy | 27 -
11 files changed, 132 insertions(+), 109 deletions(-)
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/nereids/memo/GroupExpression.java
b/fe/fe-core/src/main/java/org/apache/doris/nereids/memo/GroupExpression.java
index 5b4774284a..12e9816f8e 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/nereids/memo/GroupExpression.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/nereids/memo/GroupExpression.java
@@ -57,7 +57,7 @@ public class GroupExpression {
private final BitSet ruleMasks;
private boolean statDerived;
-private long estOutputRowCount = -1;
+private double estOutputRowCount = -1;
//Record the rule that generate this plan. It's used for debugging
private Rule fromRule;
@@ -301,16 +301,13 @@ public class GroupExpression {
return new Statistics(child(idx).getStatistics());
}
-public void setEstOutputRowCount(long estOutputRowCount) {
+public void setEstOutputRowCount(double estOutputRowCount) {
this.estOutputRowCount = estOutputRowCount;
}
-public long getEstOutputRowCount() {
-return estOutputRowCount;
-}
-
@Override
public String toString() {
+DecimalFormat format = new DecimalFormat("#,###.##");
StringBuilder builder = new StringBuilder("id:");
builder.append(id.asInt());
if (ownerGroup == null) {
@@ -318,11 +315,8 @@ public class GroupExpression {
} else {
builder.append("#").append(ownerGroup.getGroupId().asInt());
}
-
-DecimalFormat decimalFormat = new DecimalFormat();
-decimalFormat.setGroupingSize(3);
-builder.append(" cost=").append(decimalFormat.format((long) cost));
-builder.append(" estRows=").append(estOutputRowCount);
+builder.append(" cost=").append(format.format((long) cost));
+builder.append(" estRows=").append(format.format(estOutputRowCount));
builder.append(" (plan=").append(plan.toString()).append(")
children=[");
for (Group group : children) {
builder.append(group.getGroupId()).append(" ");
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/nereids/stats/FilterEstimation.java
b/fe/fe-core/src/main/java/org/apache/doris/nereids/stats/FilterEstimation.java
index 2aa55114d8..e2159f1040 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/nereids/stats/FilterEstimation.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/nereids/stats/FilterEstimation.java
@@ -31,10 +31,10 @@ import
org.apache.doris.nereids.trees.expressions.LessThanEqual;
import org.apache.doris.nereids.trees.expressions.Not;
import org.apache.doris.nereids.trees.expressions.NullSafeEqual;
import org.apache.doris.nereids.trees.expressions.Or;
+import org.apache.doris.nereids.trees.expressions.Slot;
import org.apache.doris.nereids.trees.expressions.SlotReference;
import org.apache.doris.nereids.trees.expressions.literal.Literal;
import org.apache.doris.nereids.trees.expressions.visitor.ExpressionVisitor;
-import org.apache.doris.nereids.types.coercion.NumericType;
import org.apache.doris.statistics.Bucket;
import org.apache.doris.statistics.ColumnStatistic;
import org.apache.doris.statistics.ColumnStatisticBuilder;
@@ -48,6 +48,7 @@ import com.google.common.base.Preconditions;
import java.util.ArrayList;
import java.util.List;
import java.util.Map;
+import java.util.Set;
/**
* Calculate selectivity of expression that produces bo
[GitHub] [doris] englefly merged pull request #17790: [enhancement](nereids) update FilterEstimation and Agg in stats derive
englefly merged PR #17790: URL: https://github.com/apache/doris/pull/17790 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] liutang123 commented on pull request #17909: [fix](fragment mgr)fix the bug that g_fragmentmgr_prepare_latency update uncorrectly
liutang123 commented on PR #17909: URL: https://github.com/apache/doris/pull/17909#issuecomment-1473617437 LGTM -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] liujiwen-up opened a new pull request, #17911: Fix a few character errors
liujiwen-up opened a new pull request, #17911: URL: https://github.com/apache/doris/pull/17911 # Proposed changes Issue Number: close #xxx ## Problem summary Describe your changes. ## Checklist(Required) * [ ] Does it affect the original behavior * [ ] Has unit tests been added * [ ] Has document been added or modified * [ ] Does it need to update dependencies * [ ] Is this PR support rollback (If NO, please explain WHY) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] BiteTheDDDDt opened a new pull request, #17912: [Bug](planner) fix core dump when lateral view above union node and have predicate
BiteThet opened a new pull request, #17912:
URL: https://github.com/apache/doris/pull/17912
# Proposed changes
fix core dump when lateral view above union node and have predicate
```cpp
0# doris::signal::(anonymous namespace)::FailureSignalHandler(int,
siginfo_t*, void*) at
/mnt/disk1/compile/doris-1.2-lts/be/src/common/signal_handler.h:420
1# os::Linux::chained_handler(int, siginfo_t*, void*) in
/usr/lib/jvm/java-11-openjdk-11.0.12.0.7-0.1.al8.x86_64/lib/server/libjvm.so
2# JVM_handle_linux_signal in
/usr/lib/jvm/java-11-openjdk-11.0.12.0.7-0.1.al8.x86_64/lib/server/libjvm.so
3# signalHandler(int, siginfo_t*, void*) in
/usr/lib/jvm/java-11-openjdk-11.0.12.0.7-0.1.al8.x86_64/lib/server/libjvm.so
4# 0x7FD9104ACC40 in /lib64/libc.so.6
5# doris::vectorized::DataTypeNullable::equals(doris::vectorized::IDataType
const&) const at
/mnt/disk1/compile/doris-1.2-lts/be/src/vec/data_types/data_type_nullable.cpp
:147
6# void
doris::vectorized::MutableBlock::merge(doris::vectorized::Block&&)
at /mnt/disk1/compile/doris-1.2-lts/be/src/vec/core/block.h:471
7#
doris::vectorized::VUnionNode::get_next_materialized(doris::RuntimeState*,
doris::vectorized::Block*) at
/mnt/disk1/compile/doris-1.2-lts/be/src/vec/exec/vunion_node.c
pp:156
8# doris::vectorized::VUnionNode::get_next(doris::RuntimeState*,
doris::vectorized::Block*, bool*) at
/mnt/disk1/compile/doris-1.2-lts/be/src/vec/exec/vunion_node.cpp:241
9# doris::ExecNode::get_next_after_projects(doris::RuntimeState*,
doris::vectorized::Block*, bool*) at
/mnt/disk1/compile/doris-1.2-lts/be/src/exec/exec_node.cpp:852
10#
doris::vectorized::VTableFunctionNode::get_expanded_block(doris::RuntimeState*,
doris::vectorized::Block*, bool*) at
/mnt/disk1/compile/doris-1.2-lts/be/src/vec/exec/vtable_function_node.cpp:117
11# doris::vectorized::VTableFunctionNode::get_next(doris::RuntimeState*,
doris::vectorized::Block*, bool*) at
/mnt/disk1/compile/doris-1.2-lts/be/src/vec/exec/vtable_function_node.cpp:86
12# doris::ExecNode::get_next_after_projects(doris::RuntimeState*,
doris::vectorized::Block*, bool*) at
/mnt/disk1/compile/doris-1.2-lts/be/src/exec/exec_node.cpp:852
13# doris::vectorized::AggregationNode::get_next(doris::RuntimeState*,
doris::vectorized::Block*, bool*) at
/mnt/disk1/compile/doris-1.2-lts/be/src/vec/exec/vaggregation_node.cpp:500
14# doris::ExecNode::get_next_after_projects(doris::RuntimeState*,
doris::vectorized::Block*, bool*) at
/mnt/disk1/compile/doris-1.2-lts/be/src/exec/exec_node.cpp:852
15#
doris::PlanFragmentExecutor::get_vectorized_internal(doris::vectorized::Block**)
at
/mnt/disk1/compile/doris-1.2-lts/be/src/runtime/plan_fragment_executor.cpp:358
16# doris::PlanFragmentExecutor::open_vectorized_internal() at
/mnt/disk1/compile/doris-1.2-lts/be/src/runtime/plan_fragment_executor.cpp:306
17# doris::PlanFragmentExecutor::open() at
/mnt/disk1/compile/doris-1.2-lts/be/src/runtime/plan_fragment_executor.cpp:259
18# doris::FragmentExecState::execute() at
/mnt/disk1/compile/doris-1.2-lts/be/src/runtime/fragment_mgr.cpp:253
19#
doris::FragmentMgr::_exec_actual(std::shared_ptr,
std::function) at
/mnt/disk1/compile/doris-1.2-lts/be/src/runtime/fragment_mgr.cpp:500
20# std::_Function_handler)::{lambda()#1}>::_M_invoke(std::_Any_data
const&) at /opt/ldb_toolchain/include/c++/11/bits/std_function.h:291
21# doris::ThreadPool::dispatch_thread() at
/mnt/disk1/compile/doris-1.2-lts/be/src/util/threadpool.cpp:543
22# doris::Thread::supervise_thread(void*) at
/mnt/disk1/compile/doris-1.2-lts/be/src/util/thread.cpp:455
23# start_thread in /lib64/libpthread.so.0
24# clone in /lib64/libc.so.6
```
## Problem summary
Describe your changes.
## Checklist(Required)
* [ ] Does it affect the original behavior
* [ ] Has unit tests been added
* [ ] Has document been added or modified
* [ ] Does it need to update dependencies
* [ ] Is this PR support rollback (If NO, please explain WHY)
## Further comments
If this is a relatively large or complex change, kick off the discussion at
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you
chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] BiteTheDDDDt commented on pull request #17912: [Bug](planner) fix core dump when lateral view above union node and have predicate
BiteThet commented on PR #17912: URL: https://github.com/apache/doris/pull/17912#issuecomment-1473622667 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #17912: [Bug](planner) fix core dump when lateral view above union node and have predicate
github-actions[bot] commented on PR #17912: URL: https://github.com/apache/doris/pull/17912#issuecomment-1473626492 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] starocean999 opened a new pull request, #17913: [fix](nereids)add a project node above sort node to eliminate unused order by keys
starocean999 opened a new pull request, #17913: URL: https://github.com/apache/doris/pull/17913 # Proposed changes Issue Number: close #xxx ## Problem summary if the order by keys are not simple slot in sort node, the order by exprs have to been added to sort node's output tuple. In that case, we need add a project node above sort node to eliminate the unused order by exprs. for example: WITH t0 AS (SELECT DATE_FORMAT(date, '%Y%m%d') AS date FROM cir_1756_t1 ), t3 AS (SELECT date_format(date, '%Y%m%d') AS `date` FROM `cir_1756_t2` GROUP BY date_format(date, '%Y%m%d') **ORDER BY date_format(date, '%Y%m%d')** ) SELECT t0.date FROM t0 LEFT JOIN t3 ON t0.date = t3.date; before: ``` +--+ | Explain String | +--+ | LogicalProject[159] ( distinct=false, projects=[date#1], excepts=[], canEliminate=true ) | | +--LogicalJoin[158] ( type=LEFT_OUTER_JOIN, markJoinSlotReference=Optional.empty, hashJoinConjuncts=[(date#1 = date#3)], otherJoinConjuncts=[] ) | ||--LogicalProject[151] ( distinct=false, projects=[date_format(date#0, '%Y%m%d') AS `date`#1], excepts=[], canEliminate=true )| || +--LogicalOlapScan ( qualified=default_cluster:bugfix.cir_1756_t1, indexName=cir_1756_t1, selectedIndexId=412339, preAgg=ON ) | |+--LogicalSort[157] ( orderKeys=[date_format(cast(date#3 as DATETIME), '%Y%m%d') asc null first] )| | +--LogicalAggregate[156] ( groupByExpr=[date#3], outputExpr=[date#3], hasRepeat=false ) | | +--LogicalProject[155] ( distinct=false, projects=[date_format(date#2, '%Y%m%d') AS `date`#3], excepts=[], canEliminate=true ) | | +--LogicalOlapScan ( qualified=default_cluster:bugfix.cir_1756_t2, indexName=cir_1756_t2, selectedIndexId=412352, preAgg=ON )| +--+ ``` after: ``` +--+ | Explain String | +--+ | LogicalProject[171] ( distinct=false, projects=[date#2], excepts=[], canEliminate=true ) | | +--LogicalJoin[170] ( type=LEFT_OUTER_JOIN, markJoinSlotReference=Optional.empty, hashJoinConjuncts=[(date#2 = date#4)], otherJoinConjuncts=[] ) | ||--LogicalProject[162] ( distinct=false, projects=[date_format(date#0, '%Y%m%d') AS `date`#2], excepts=[], canEliminate=true )| || +--LogicalOlapScan ( qualified=default_cluster:bugfix.cir_1756_t1, indexName=cir_1756_t1, selectedIndexId=1049812, preAgg=ON ) | |+--LogicalProject[169] ( distinct=false, projects=[date#4], excepts=[], canEliminate=false ) | | +--LogicalSort[168] ( orderKeys=[date_format(cast(date#4 as DATETIME), '%Y%m%d') asc null first] ) | | +--LogicalAggregate[167] ( groupByExpr=[date#4], outputExpr=[date#4], hasRepeat=false ) | | +--LogicalProject[166] ( distinct=false, projects=[date_format(date#3, '%Y%m%d') AS `date`#4], excepts=[], canEliminate=true ) | |+--LogicalOlapScan ( qualified=default_cluster:bugfix.cir_1756_t2, indexName=cir_1756_t2, selectedIndexId=1049825, preAgg=ON )| +--+ ``` ## Checklist(Required) * [ ] Does it affect the original behavior * [ ] Has unit tests been added * [ ] Has document been added or modified * [ ] Does it need to update dependencies * [ ] Is this PR support rollback (If NO, please explain WHY) ## Further comments If this
[GitHub] [doris] morrySnow opened a new pull request, #17914: [enhencement](planner) update and delete support use alias for target table
morrySnow opened a new pull request, #17914: URL: https://github.com/apache/doris/pull/17914 # Proposed changes Issue Number: close #xxx ## Problem summary Describe your changes. ## Checklist(Required) * [ ] Does it affect the original behavior * [ ] Has unit tests been added * [ ] Has document been added or modified * [ ] Does it need to update dependencies * [ ] Is this PR support rollback (If NO, please explain WHY) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] zhangstar333 commented on pull request #16563: [fix](fold_constant) add defer in fold constant to close
zhangstar333 commented on PR #16563: URL: https://github.com/apache/doris/pull/16563#issuecomment-1473637928 run p0 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] zenoyang commented on pull request #17828: [improvement](fetch data) Merge result into batch to reduce rpc times
zenoyang commented on PR #17828: URL: https://github.com/apache/doris/pull/17828#issuecomment-1473640334 run clickbench -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #16563: [fix](fold_constant) add defer in fold constant to close
github-actions[bot] commented on PR #16563: URL: https://github.com/apache/doris/pull/16563#issuecomment-1473640561 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] zddr opened a new pull request, #17915: [Enhancement](auth)support ranger col priv
zddr opened a new pull request, #17915: URL: https://github.com/apache/doris/pull/17915 # Proposed changes Issue Number: close #xxx ## Problem summary Describe your changes. 1.When querying data, it is no longer necessary to verify the permissions of the entire table, but rather to verify the permissions of the queried columns. Currently, the 'ranger' already supports column permissions, and the internal catalog provides the implementation of dummy column permissions (the actual verified permissions are still table permissions) 2.delete roles in userIdentity 3.Change trigger logic of initAccessController ## Checklist(Required) * [ ] Does it affect the original behavior * [ ] Has unit tests been added * [ ] Has document been added or modified * [ ] Does it need to update dependencies * [ ] Is this PR support rollback (If NO, please explain WHY) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] zddr commented on pull request #17877: [fix](multi-catalog) when checkProperties failed,will have dirty data
zddr commented on PR #17877: URL: https://github.com/apache/doris/pull/17877#issuecomment-1473646419 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] zddr commented on pull request #17915: [Enhancement](auth)support ranger col priv
zddr commented on PR #17915: URL: https://github.com/apache/doris/pull/17915#issuecomment-1473646505 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] morrySnow commented on pull request #17900: [fix](regression-test) fix unstable regression test cases found in p0
morrySnow commented on PR #17900: URL: https://github.com/apache/doris/pull/17900#issuecomment-1473656781 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] SaintBacchus commented on pull request #17745: [BugFix](DOE) Add http prefix when it's not set in hosts properties.
SaintBacchus commented on PR #17745: URL: https://github.com/apache/doris/pull/17745#issuecomment-1473659846 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] SaintBacchus commented on pull request #17775: [Feature-Wip](MySQL LOAD)Add trim quotes property for mysql load
SaintBacchus commented on PR #17775: URL: https://github.com/apache/doris/pull/17775#issuecomment-1473660261 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #17910: Revert "[enhancement](memory) PODArray replaces MemPool in PredicateColumn (#17800)"
hello-stephen commented on PR #17910: URL: https://github.com/apache/doris/pull/17910#issuecomment-1473667451 TeamCity pipeline, clickbench performance test result: the sum of best hot time: 34.29 seconds stream load tsv: 463 seconds loaded 74807831229 Bytes, about 154 MB/s stream load json: 23 seconds loaded 2358488459 Bytes, about 97 MB/s stream load orc: 74 seconds loaded 1101869774 Bytes, about 14 MB/s stream load parquet: 31 seconds loaded 861443392 Bytes, about 26 MB/s https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230317110529_clickbench_pr_116014.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #17915: [Enhancement](auth)support ranger col priv
hello-stephen commented on PR #17915: URL: https://github.com/apache/doris/pull/17915#issuecomment-1473710451 TeamCity pipeline, clickbench performance test result: the sum of best hot time: 34.51 seconds stream load tsv: 473 seconds loaded 74807831229 Bytes, about 150 MB/s stream load json: 22 seconds loaded 2358488459 Bytes, about 102 MB/s stream load orc: 75 seconds loaded 1101869774 Bytes, about 14 MB/s stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230317114514_clickbench_pr_116046.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] morningman merged pull request #17590: [Refactor](Metadata tvf) Reconstruct Metadata table-value function into a more general framework.
morningman merged PR #17590: URL: https://github.com/apache/doris/pull/17590 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated: [Refactor](Metadata tvf) Reconstruct Metadata table-value function into a more general framework. (#17590)
This is an automated email from the ASF dual-hosted git repository.
morningman pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 46d88ede02 [Refactor](Metadata tvf) Reconstruct Metadata table-value
function into a more general framework. (#17590)
46d88ede02 is described below
commit 46d88ede024d1cb8c31837c54c1ac721b35e6966
Author: Tiewei Fang <43782773+bepppo...@users.noreply.github.com>
AuthorDate: Fri Mar 17 19:54:50 2023 +0800
[Refactor](Metadata tvf) Reconstruct Metadata table-value function into a
more general framework. (#17590)
---
be/src/vec/exec/scan/vmeta_scanner.cpp | 62 +++---
be/src/vec/exec/scan/vmeta_scanner.h | 5 +-
.../org/apache/doris/planner/DataGenScanNode.java | 5 +-
.../doris/planner/external/MetadataScanNode.java | 30 +--
.../apache/doris/service/FrontendServiceImpl.java | 211 +-
.../tablefunction/DataGenTableValuedFunction.java | 8 +
.../tablefunction/IcebergTableValuedFunction.java | 57 +++--
.../doris/tablefunction/MetadataGenerator.java | 248 +
.../tablefunction/MetadataTableValuedFunction.java | 18 +-
.../tablefunction/NumbersTableValuedFunction.java | 9 -
gensrc/thrift/FrontendService.thrift | 10 +-
gensrc/thrift/PlanNodes.thrift | 15 +-
gensrc/thrift/Types.thrift | 8 +
13 files changed, 368 insertions(+), 318 deletions(-)
diff --git a/be/src/vec/exec/scan/vmeta_scanner.cpp
b/be/src/vec/exec/scan/vmeta_scanner.cpp
index 86e60edf11..50877d5558 100644
--- a/be/src/vec/exec/scan/vmeta_scanner.cpp
+++ b/be/src/vec/exec/scan/vmeta_scanner.cpp
@@ -31,7 +31,6 @@ VMetaScanner::VMetaScanner(RuntimeState* state,
VMetaScanNode* parent, int64_t t
const TScanRangeParams& scan_range, int64_t limit,
RuntimeProfile* profile)
: VScanner(state, static_cast(parent), limit, profile),
- _parent(parent),
_meta_eos(false),
_tuple_id(tuple_id),
_scan_range(scan_range.scan_range) {}
@@ -46,11 +45,7 @@ Status VMetaScanner::prepare(RuntimeState* state,
VExprContext** vconjunct_ctx_p
VLOG_CRITICAL << "VMetaScanner::prepare";
RETURN_IF_ERROR(VScanner::prepare(_state, vconjunct_ctx_ptr));
_tuple_desc = state->desc_tbl().get_tuple_descriptor(_tuple_id);
-if (_scan_range.meta_scan_range.__isset.iceberg_params) {
-RETURN_IF_ERROR(_fetch_iceberg_metadata_batch());
-} else {
-_meta_eos = true;
-}
+RETURN_IF_ERROR(_fetch_metadata(_scan_range.meta_scan_range));
return Status::OK();
}
@@ -158,37 +153,28 @@ Status VMetaScanner::_fill_block_with_remote_data(const
std::vectorscan_params();
-TMetadataTableRequestParams meta_table_params =
TMetadataTableRequestParams();
-meta_table_params.catalog = scan_params.catalog;
-meta_table_params.__isset.catalog = true;
-meta_table_params.database = scan_params.database;
-meta_table_params.__isset.database = true;
-meta_table_params.table = scan_params.table;
-meta_table_params.__isset.table = true;
-
-meta_table_params.iceberg_metadata_params =
_scan_range.meta_scan_range.iceberg_params;
-meta_table_params.__isset.iceberg_metadata_params = true;
-
-request.metada_table_params = meta_table_params;
-request.__isset.metada_table_params = true;
+switch (meta_scan_range.metadata_type) {
+case TMetadataType::ICEBERG:
+RETURN_IF_ERROR(_build_iceberg_metadata_request(meta_scan_range,
&request));
+break;
+default:
+_meta_eos = true;
+return Status::OK();
+}
+// _state->query_timeout() is seconds, change to milliseconds
+int time_out = _state->query_timeout() * 1000;
TNetworkAddress master_addr =
ExecEnv::GetInstance()->master_info()->network_address;
TFetchSchemaTableDataResult result;
-
RETURN_IF_ERROR(ThriftRpcHelper::rpc(
master_addr.hostname, master_addr.port,
[&request, &result](FrontendServiceConnection& client) {
client->fetchSchemaTableData(result, request);
},
-config::txn_commit_rpc_timeout_ms));
+time_out));
Status status(result.status);
if (!status.ok()) {
@@ -199,6 +185,26 @@ Status VMetaScanner::_fetch_iceberg_metadata_batch() {
return Status::OK();
}
+Status VMetaScanner::_build_iceberg_metadata_request(const TMetaScanRange&
meta_scan_range,
+
TFetchSchemaTableDataRequest* request) {
+VLOG_CRITICAL << "VMetaScanner::_build_iceberg_metadata_request";
+if (!meta_scan_range.__isset.iceberg_params) {
+return Status::InternalError("Can not find TIcebergMetadataParams from
meta_scan_range.");
+}
+
+// create request
+request->_
[GitHub] [doris] github-actions[bot] commented on pull request #17837: [fix](timeout) fix set query_timeout and insert_timeout failure
github-actions[bot] commented on PR #17837: URL: https://github.com/apache/doris/pull/17837#issuecomment-1473734614 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] TangSiyang2001 commented on pull request #17837: [fix](timeout) fix set timeout failure and simplify timeout logic
TangSiyang2001 commented on PR #17837: URL: https://github.com/apache/doris/pull/17837#issuecomment-1473739604 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] SaintBacchus commented on pull request #17745: [BugFix](DOE) Add http prefix when it's not set in hosts properties.
SaintBacchus commented on PR #17745: URL: https://github.com/apache/doris/pull/17745#issuecomment-1473741766 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] SaintBacchus commented on pull request #17775: [Feature-Wip](MySQL LOAD)Add trim quotes property for mysql load
SaintBacchus commented on PR #17775: URL: https://github.com/apache/doris/pull/17775#issuecomment-1473742013 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #17837: [fix](timeout) fix set timeout failure and simplify timeout logic
github-actions[bot] commented on PR #17837: URL: https://github.com/apache/doris/pull/17837#issuecomment-1473743308 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] TangSiyang2001 commented on pull request #17837: [fix](timeout) fix set timeout failure and simplify timeout logic
TangSiyang2001 commented on PR #17837: URL: https://github.com/apache/doris/pull/17837#issuecomment-1473754346 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] Tanya-W commented on pull request #17825: [fix](schema change) Fix fe restart failed because of replay schema change alter job failed
Tanya-W commented on PR #17825: URL: https://github.com/apache/doris/pull/17825#issuecomment-1473756935 run p0 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #17837: [fix](timeout) fix set timeout failure and simplify timeout logic
github-actions[bot] commented on PR #17837: URL: https://github.com/apache/doris/pull/17837#issuecomment-1473758002 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] englefly opened a new pull request, #17916: [improve](nereids)adjust distribution stats derive and fix bug in join estimation
englefly opened a new pull request, #17916: URL: https://github.com/apache/doris/pull/17916 # Proposed changes ## adjust distribution stats derive replica distribution: estimation of replica distribution is too high. The cost of replica a table should be smaller than scan a table in cpuCost and memCost. ## fix bug in join estimation if there is more than one equal-condition in hash join, the dominate one should be the one whose sel is smallest. Issue Number: close #xxx ## Problem summary Describe your changes. ## Checklist(Required) * [ ] Does it affect the original behavior * [ ] Has unit tests been added * [ ] Has document been added or modified * [ ] Does it need to update dependencies * [ ] Is this PR support rollback (If NO, please explain WHY) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] morningman opened a new pull request, #17917: [fix](truncate) fix unable to truncate table due to wrong storage medium
morningman opened a new pull request, #17917: URL: https://github.com/apache/doris/pull/17917 # Proposed changes Issue Number: close #xxx ## Problem summary When setting FE config `default_storage_medium` to SSD, and set all BE storage path as SSD. And table will be stored with storage medium SSD. But there is a FE config `storage_cooldown_second` and its default value is 30 days. So after 30 days, the storage medium of table will be changed to HDD, which is unexpected. This PR removes the `storage_cooldown_second`, and use a max value to set the cooldown time of SSD storage medium when the `default_storage_medium` is SSD. ## Checklist(Required) * [ ] Does it affect the original behavior * [ ] Has unit tests been added * [ ] Has document been added or modified * [ ] Does it need to update dependencies * [ ] Is this PR support rollback (If NO, please explain WHY) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] dataroaring merged pull request #17910: Revert "[enhancement](memory) PODArray replaces MemPool in PredicateColumn (#17800)"
dataroaring merged PR #17910: URL: https://github.com/apache/doris/pull/17910 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated: Revert "[enhancement](memory) PODArray replaces MemPool in PredicateColumn (#17800)" (#17910)
This is an automated email from the ASF dual-hosted git repository.
dataroaring pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 5c5dcfda78 Revert "[enhancement](memory) PODArray replaces MemPool in
PredicateColumn (#17800)" (#17910)
5c5dcfda78 is described below
commit 5c5dcfda78d33f98816ba43d6190f361442d32d4
Author: TengJianPing <18241664+jackte...@users.noreply.github.com>
AuthorDate: Fri Mar 17 20:50:01 2023 +0800
Revert "[enhancement](memory) PODArray replaces MemPool in PredicateColumn
(#17800)" (#17910)
This reverts commit 17d1c1bc7f6cc95eecd224eaa219c976b60fa17e.
---
be/src/vec/columns/predicate_column.h | 20 ++--
1 file changed, 14 insertions(+), 6 deletions(-)
diff --git a/be/src/vec/columns/predicate_column.h
b/be/src/vec/columns/predicate_column.h
index aa381b61bd..f67909889f 100644
--- a/be/src/vec/columns/predicate_column.h
+++ b/be/src/vec/columns/predicate_column.h
@@ -19,6 +19,7 @@
#include "olap/decimal12.h"
#include "olap/uint24.h"
+#include "runtime/mem_pool.h"
#include "runtime/primitive_type.h"
#include "vec/columns/column.h"
#include "vec/columns/column_decimal.h"
@@ -275,9 +276,11 @@ public:
return;
}
if constexpr (std::is_same_v) {
+if (_pool == nullptr) {
+_pool.reset(new MemPool());
+}
const auto total_mem_size = offsets[num] - offsets[0];
-strings.resize(strings.size() + total_mem_size);
-char* destination = reinterpret_cast(strings.data()) +
strings.size();
+char* destination = (char*)_pool->allocate(total_mem_size);
memcpy(destination, data_ + offsets[0], total_mem_size);
size_t org_elem_num = data.size();
data.resize(org_elem_num + num);
@@ -297,13 +300,16 @@ public:
return;
}
if constexpr (std::is_same_v) {
+if (_pool == nullptr) {
+_pool.reset(new MemPool());
+}
+
size_t total_mem_size = 0;
for (size_t i = 0; i < num; i++) {
total_mem_size += len_array[i];
}
-strings.resize(strings.size() + total_mem_size);
-char* destination = reinterpret_cast(strings.data()) +
strings.size();
+char* destination = (char*)_pool->allocate(total_mem_size);
char* org_dst = destination;
size_t org_elem_num = data.size();
data.resize(org_elem_num + num);
@@ -334,7 +340,9 @@ public:
void clear() override {
data.clear();
-strings.clear();
+if (_pool != nullptr) {
+_pool->clear();
+}
}
size_t byte_size() const override { return data.size() * sizeof(T); }
@@ -539,7 +547,7 @@ public:
private:
Container data;
// manages the memory for slice's data(For string type)
-ColumnString::Chars strings;
+std::unique_ptr _pool;
};
} // namespace doris::vectorized
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] dataroaring merged pull request #17825: [fix](schema change) Fix fe restart failed because of replay schema change alter job failed
dataroaring merged PR #17825: URL: https://github.com/apache/doris/pull/17825 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated: [fix](schema change) Fix fe restart failed because of replay schema change alter job failed (#17825)
This is an automated email from the ASF dual-hosted git repository.
dataroaring pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 3593b82498 [fix](schema change) Fix fe restart failed because of
replay schema change alter job failed (#17825)
3593b82498 is described below
commit 3593b82498e83cb2b16febe5723c858dd4133715
Author: YueW <45946325+tany...@users.noreply.github.com>
AuthorDate: Fri Mar 17 20:54:50 2023 +0800
[fix](schema change) Fix fe restart failed because of replay schema change
alter job failed (#17825)
---
.../apache/doris/alter/SchemaChangeHandler.java| 40 ++
.../org/apache/doris/alter/SchemaChangeJobV2.java | 5 ---
.../java/org/apache/doris/master/MasterImpl.java | 4 ++-
.../test_dytable_complex_data.groovy | 4 ++-
.../suites/index_p0/test_index_meta.groovy | 4 ++-
.../inverted_index_p0/test_add_drop_index.groovy | 4 ++-
.../test_add_drop_index_ignore_case_column.groovy | 4 ++-
.../test_add_drop_index_with_data.groovy | 4 ++-
.../test_add_drop_index_with_delete_data.groovy| 4 ++-
.../test_index_like_select.groovy | 4 ++-
.../test_index_match_term_and_phrase_select.groovy | 4 ++-
.../test_index_range_between_select.groovy | 4 ++-
.../test_index_range_in_select.groovy | 2 ++
.../test_index_range_not_in_select.groovy | 2 ++
.../test_alter_table_column_nullable.groovy| 1 +
15 files changed, 75 insertions(+), 15 deletions(-)
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/alter/SchemaChangeHandler.java
b/fe/fe-core/src/main/java/org/apache/doris/alter/SchemaChangeHandler.java
index ddc72f3ad6..5dfdb6c55f 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/alter/SchemaChangeHandler.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/alter/SchemaChangeHandler.java
@@ -1596,9 +1596,35 @@ public class SchemaChangeHandler extends AlterHandler {
});
}
}
+
+if (alterJobsV2.isDone()) {
+changeTableState(alterJobsV2.getDbId(),
alterJobsV2.getTableId(), OlapTableState.NORMAL);
+runnableSchemaChangeJobV2.remove(alterJobsV2.getJobId());
+LOG.info("set table's state to NORMAL, table id: {}, job id:
{}", alterJobsV2.getTableId(),
+alterJobsV2.getJobId());
+}
});
}
+private void changeTableState(long dbId, long tableId, OlapTableState
olapTableState) {
+try {
+Database db =
Env.getCurrentInternalCatalog().getDbOrMetaException(dbId);
+OlapTable olapTable = (OlapTable)
db.getTableOrMetaException(tableId, Table.TableType.OLAP);
+olapTable.writeLockOrMetaException();
+try {
+if (olapTable.getState() == olapTableState) {
+return;
+} else if (olapTable.getState() ==
OlapTableState.SCHEMA_CHANGE) {
+olapTable.setState(olapTableState);
+}
+} finally {
+olapTable.writeUnlock();
+}
+} catch (MetaNotFoundException e) {
+LOG.warn("[INCONSISTENT META] changing table status failed after
schema change job done", e);
+}
+}
+
public List> getAllAlterJobInfos() {
List> schemaChangeJobInfos = new LinkedList<>();
for (AlterJobV2 alterJob : ImmutableList.copyOf(alterJobsV2.values()))
{
@@ -2132,6 +2158,11 @@ public class SchemaChangeHandler extends AlterHandler {
if (!schemaChangeJobV2.cancel("user cancelled")) {
throw new DdlException("Job can not be cancelled. State: " +
schemaChangeJobV2.getJobState());
}
+if (schemaChangeJobV2.isDone()) {
+changeTableState(schemaChangeJobV2.getDbId(),
schemaChangeJobV2.getTableId(), OlapTableState.NORMAL);
+LOG.info("set table's state to NORMAL when cancel job, table
id: {}, job id: {}",
+schemaChangeJobV2.getTableId(),
schemaChangeJobV2.getJobId());
+}
return;
}
}
@@ -2235,6 +2266,9 @@ public class SchemaChangeHandler extends AlterHandler {
while (iterator.hasNext()) {
AlterJobV2 alterJobV2 = iterator.next().getValue();
if (alterJobV2.isDone()) {
+changeTableState(alterJobV2.getDbId(),
alterJobV2.getTableId(), OlapTableState.NORMAL);
+LOG.info("set table's state to NORMAL, table id: {}, job id:
{}", alterJobV2.getTableId(),
+alterJobV2.getJobId());
iterator.remove();
}
}
@@ -2254,6 +2288,12 @@ public class SchemaChangeHandler extends AlterHandler {
runnableSchemaChangeJobV2.put(alterJob.getJobId(), alt
[GitHub] [doris] SaintBacchus commented on pull request #17775: [Feature-Wip](MySQL LOAD)Add trim quotes property for mysql load
SaintBacchus commented on PR #17775: URL: https://github.com/apache/doris/pull/17775#issuecomment-1473807549 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] SaintBacchus commented on pull request #17745: [BugFix](DOE) Add http prefix when it's not set in hosts properties.
SaintBacchus commented on PR #17745: URL: https://github.com/apache/doris/pull/17745#issuecomment-1473807855 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] qidaye commented on pull request #17827: [improvement](inverted index)UNIQUE_KEYS table only supports inverted…
qidaye commented on PR #17827: URL: https://github.com/apache/doris/pull/17827#issuecomment-1473808626 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
