Re: [PR] [Refactor](SchemaCache) remove redundant Schema cache [doris]
github-actions[bot] commented on PR #40091: URL: https://github.com/apache/doris/pull/40091#issuecomment-2323152251 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](spill) should call set_ready after changing the status [doris]
github-actions[bot] commented on PR #40075: URL: https://github.com/apache/doris/pull/40075#issuecomment-2322810198 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [I] [Feature] Support protobuf format for routine load [doris]
Authorlove commented on issue #9302: URL: https://github.com/apache/doris/issues/9302#issuecomment-2322834245 @xy720 Has this feature been completed? Is there any user documentation? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [case](array) fix unsort array case [doris]
HappenLee merged PR #39941: URL: https://github.com/apache/doris/pull/39941 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](recycler) Implement missing delete_prefix() of hdfs accessor [doris]
doris-robot commented on PR #40215: URL: https://github.com/apache/doris/pull/40215#issuecomment-2322910434 TPC-DS: Total hot run time: 192593 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 15ce85a75d46fbdcb423e5bd86bec302bd8bbbf8, data reload: false query1 1241890 868 868 query2 6242204520142014 query3 10646 415637863786 query4 59295 23746 23293 23293 query5 5424497 502 497 query6 416 168 161 161 query7 5781298 298 298 query8 298 225 207 207 query9 9049250725152507 query10 498 320 267 267 query11 17295 14985 15066 14985 query12 154 110 103 103 query13 1565393 378 378 query14 11180 673172486731 query15 247 179 179 179 query16 7606479 473 473 query17 1129574 581 574 query18 2053296 300 296 query19 285 149 144 144 query20 126 113 112 112 query21 206 103 104 103 query22 47064634 query23 34177 33485 33428 33428 query24 5985291228122812 query25 523 385 385 385 query26 691 153 152 152 query27 1799282 277 277 query28 3552212121012101 query29 702 414 398 398 query30 231 158 149 149 query31 945 733 755 733 query32 84 58 61 58 query33 457 319 290 290 query34 865 481 503 481 query35 811 733 716 716 query36 1087982 943 943 query37 146 88 90 88 query38 4049386239133862 query39 1426139314191393 query40 197 114 114 114 query41 46 47 44 44 query42 117 101 92 92 query43 531 494 479 479 query44 1101747 749 747 query45 200 169 168 168 query46 1097742 757 742 query47 1868178818001788 query48 375 306 292 292 query49 758 424 444 424 query50 810 420 409 409 query51 7183713770707070 query52 98 87 87 87 query53 249 182 177 177 query54 556 477 443 443 query55 77 78 78 78 query56 283 260 255 255 query57 1183108410571057 query58 241 238 288 238 query59 3246275228712752 query60 300 291 292 291 query61 127 119 117 117 query62 739 650 633 633 query63 216 185 191 185 query64 2939756 747 747 query65 3191316731513151 query66 701 355 357 355 query67 15348 15206 15026 15026 query68 3008579 593 579 query69 408 291 283 283 query70 1199115111431143 query71 373 283 288 283 query72 6431398840353988 query73 757 334 327 327 query74 9173879988208799 query75 3350270326992699 query76 1414105710141014 query77 534 320 320 320 query78 11700 967291169116 query79 1158555 547 547 query80 875 498 513 498 query81 572 238 238 238 query82 311 150 144 144 query83 174 148 148 148 query84 268 76 79 76 query85 721 287 319 287 query86 366 304 302 302 query87 4454433944074339 query88 3178230923072307 query89 386 290 293 290 query90 1864192 189 189 query91 123 96 98 96 query92 61 50 48 48 query93 1073527 530 527 query94 754 301 290 290 query95 349 254 259 254 query96 586 275 264 264 query97 3236310131003100 query98 222 212 207 207 query99 1569126812441244 Total cold run time: 307396 ms Total hot run time: 192593 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to
Re: [PR] [opt](function)Some geo functions incorrectly used get. [doris]
github-actions[bot] commented on PR #40107: URL: https://github.com/apache/doris/pull/40107#issuecomment-2322805263 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](regression) fix regression failure of window_funnel caused by duplicated table name [doris]
yiguolei merged PR #40216: URL: https://github.com/apache/doris/pull/40216 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](regression) fix regression failure of window_funnel caused by duplicated table name [doris]
github-actions[bot] commented on PR #40217: URL: https://github.com/apache/doris/pull/40217#issuecomment-2322890299 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [opt](function)Some geo functions incorrectly used get. [doris]
github-actions[bot] commented on PR #40107: URL: https://github.com/apache/doris/pull/40107#issuecomment-2322805314 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [fix](recycler) Implement missing delete_prefix() of hdfs accessor [doris]
gavinchou opened a new pull request, #40215: URL: https://github.com/apache/doris/pull/40215 To test it, 1. enable commit_rowset fail with injection point 2. issue a streamload 3. check the recycler log or storage vault the uncommitted rowset recycled ``` curl 'be_ip:http_port/api/injection_point/enable' curl 'be_ip:http_port/api/injection_point/clear' curl "be_ip:http_port/api/injection_point/set?name=CloudMetaMgr::commit_rowset&behavior=return_error" curl -XPUT --location-trusted -vv -T tmp/1.csv -H'format:csv' -H 'column_separator:,' -H'timeout:60' -uroot: be_ip:http_port/api/db1/t2/_stream_load curl 'be_ip:http_port/api/injection_point/clear' curl 'be_ip:http_port/api/injection_point/disable' ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [Bug](expr) execute expr should use local states instead of operators (#40189) [doris]
zhangstar333 opened a new pull request, #40219: URL: https://github.com/apache/doris/pull/40219 ## Proposed changes cherry-pick from master #40189 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [branch-2.1][Feat](Job)After a job is paused, it can be manually triggered to execute. [doris]
yiguolei merged PR #40180: URL: https://github.com/apache/doris/pull/40180 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Bug](expr) execute expr should use local states instead of operators (#40189) [doris]
yiguolei merged PR #40219: URL: https://github.com/apache/doris/pull/40219 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Enhancement] Add E as default value function for doris [doris]
doris-robot commented on PR #40209: URL: https://github.com/apache/doris/pull/40209#issuecomment-2322806231 ClickBench: Total hot run time: 32.46 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 7172513e1a6d9397329fdab5dce20b21f0b15516, data reload: false query1 0.050.040.04 query2 0.070.040.04 query3 0.220.050.05 query4 1.680.070.08 query5 0.500.510.51 query6 1.140.730.73 query7 0.020.010.02 query8 0.050.040.04 query9 0.550.510.49 query10 0.540.530.52 query11 0.150.110.11 query12 0.150.120.12 query13 0.610.590.59 query14 2.042.042.03 query15 0.890.810.83 query16 0.380.370.38 query17 1.011.041.05 query18 0.210.200.19 query19 1.821.831.75 query20 0.010.020.01 query21 15.40 0.650.64 query22 4.606.912.38 query23 18.26 1.371.26 query24 2.100.230.21 query25 0.140.080.08 query26 0.270.180.17 query27 0.080.080.07 query28 13.22 1.011.02 query29 12.59 3.353.33 query30 0.250.060.05 query31 2.880.410.40 query32 3.230.470.48 query33 3.023.023.02 query34 17.14 4.394.38 query35 4.454.414.40 query36 0.660.470.52 query37 0.180.150.16 query38 0.160.150.15 query39 0.050.040.04 query40 0.160.120.13 query41 0.100.050.05 query42 0.060.060.05 query43 0.050.040.04 Total cold run time: 111.14 s Total hot run time: 32.46 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris-website) branch dependabot/npm_and_yarn/webpack-5.94.0 created (now 1e6d99179ab)
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a change to branch dependabot/npm_and_yarn/webpack-5.94.0 in repository https://gitbox.apache.org/repos/asf/doris-website.git at 1e6d99179ab Bump webpack from 5.93.0 to 5.94.0 No new revisions were added by this update. - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [test](memory) find large memory allocation not catch exception [doris]
doris-robot commented on PR #35223: URL: https://github.com/apache/doris/pull/35223#issuecomment-2322831212 TPC-H: Total hot run time: 37928 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 0bc9aa558788665bbfe96285bbe78d26eb49c226, data reload: false -- Round 1 -- q1 17606 445343014301 q2 2013181 210 181 q3 11943 949 1153949 q4 10524 733 699 699 q5 7744282927712771 q6 230 140 137 137 q7 979 620 618 618 q8 9356205720572057 q9 6944653966046539 q10 6980226922392239 q11 446 258 242 242 q12 393 228 234 228 q13 17753 312130483048 q14 290 243 244 243 q15 540 500 502 500 q16 581 504 510 504 q17 991 686 710 686 q18 7286681668106810 q19 1389995 1054995 q20 676 348 332 332 q21 4021285031572850 q22 1100999 1022999 Total cold run time: 109785 ms Total hot run time: 37928 ms - Round 2, with runtime_filter_mode=off - q1 4382437242904290 q2 384 270 269 269 q3 2886267227182672 q4 1923166516351635 q5 5640569556775677 q6 231 138 140 138 q7 2262184118681841 q8 3336348934403440 q9 8925884488138813 q10 3591338933813381 q11 623 519 529 519 q12 809 657 664 657 q13 15346 319132833191 q14 327 288 291 288 q15 563 494 499 494 q16 647 600 567 567 q17 1838155015551550 q18 8179787677557755 q19 1724172616971697 q20 2161192318861886 q21 5754553556605535 q22 1127105210031003 Total cold run time: 72658 ms Total hot run time: 57298 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[I] [Enhancement] 存算分离架构部署有问题,这个ky数据库FoundationDB 能不能换成redis等常用的,维护成本高,还有学习成本 [doris]
zhaomingcheng01 opened a new issue, #40214:
URL: https://github.com/apache/doris/issues/40214
### Search before asking
- [X] I had searched in the
[issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no
similar issues.
### Description
apiVersion: v1
data:
instance.conf: |
{
"instance_id": "evo-rw-doris",
"user_id": "root",
"vault": {
"obj_info": {
"ak": "admin",
"sk": "",
"bucket": "doris",
"endpoint": "minio.minio:9000",
"external_endpoint": "minio.minio:9000",
"region": "us-east1",
"provider": "S3",
}
}
}
kind: ConfigMap
metadata:
name: doris-store
namespace: doris
---
apiVersion: disaggregated.metaservice.doris.com/v1
kind: DorisDisaggregatedMetaService
metadata:
name: doris-meta-service
namespace: doris
spec:
fdb:
image: docker.rainbond.cc/selectdb/foundationdb:7.1.38
sidecarImage:
docker.rainbond.cc/selectdb/foundationdb-kubernetes-sidecar:7.1.36-1
nodeSelector:
node.kubernetes.io/node-type: doris
volumeClaimTemplate:
spec:
resources:
requests:
storage: "200Gi"
requests:
cpu: 1
memory: 2Gi
limits:
cpu: 1
memory: 4Gi
ms:
image: docker.rainbond.cc/selectdb/doris.ms-ubuntu:3.0.0
nodeSelector:
node.kubernetes.io/node-type: doris
requests:
cpu: 2
memory: 4Gi
limits:
cpu: 2
memory: 4Gi
persistentVolume:
persistentVolumeClaimSpec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 200Gi
recycler:
image: docker.rainbond.cc/selectdb/doris.ms-ubuntu:3.0.0
nodeSelector:
node.kubernetes.io/node-type: doris
requests:
cpu: 2
memory: 4Gi
limits:
cpu: 2
memory: 4Gi
persistentVolume:
persistentVolumeClaimSpec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 200Gi
---
apiVersion: disaggregated.cluster.doris.com/v1
kind: DorisDisaggregatedCluster
metadata:
name: doris-disaggregated-cluster
namespace: doris
spec:
instanceConfigMap: doris-store
metaService:
name: doris-meta-service
namespace: doris
feSpec:
image: 'docker.rainbond.cc/selectdb/doris.fe-ubuntu:3.0.0'
replicas: 1
requests:
cpu: 1
memory: 8Gi
limits:
cpu: 4
memory: 32Gi
nodeSelector:
node.kubernetes.io/node-type: doris
service:
type: ClusterIP
persistentVolume:
mountPaths:
- /opt/apache-doris/fe/doris-meta
- /opt/apache-doris/fe/log
persistentVolumeClaimSpec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 400Gi
storageClassName: ontap-nas
computeClusters:
- name: evo-rw-doris
image: 'docker.rainbond.cc/selectdb/doris.be-ubuntu:3.0.0'
replicas: 3
requests:
cpu: 1
memory: 8Gi
limits:
cpu: 4
memory: 32Gi
nodeSelector:
node.kubernetes.io/node-type: doris
service:
type: ClusterIP
persistentVolume:
mountPaths:
- /opt/apache-doris/be/storage
- /opt/apache-doris/be/log
persistentVolumeClaimSpec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 400Gi
storageClassName: ontap-nas
---
官方文档老旧没有更新,按照上边的yaml部署启动不了,这个ky数据库FoundationDB 能不能换成redis等常用的,维护成本高,还有学习成本
### Solution
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of
Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Bug](expr) execute expr should use local states instead of operators (#40189) [doris]
github-actions[bot] commented on PR #40219: URL: https://github.com/apache/doris/pull/40219#issuecomment-2322911475 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [opt](inverted index) cache result sign is initialized only once [doris]
github-actions[bot] commented on PR #40212: URL: https://github.com/apache/doris/pull/40212#issuecomment-2323157634 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch master updated: [fix](regression) fix regression failure of window_funnel caused by duplicated table name (#40217)
This is an automated email from the ASF dual-hosted git repository.
airborne pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 83e12a84b48 [fix](regression) fix regression failure of window_funnel
caused by duplicated table name (#40217)
83e12a84b48 is described below
commit 83e12a84b485a6da92180ce706941bd6245c3d1f
Author: TengJianPing <18241664+jackte...@users.noreply.github.com>
AuthorDate: Sun Sep 1 09:25:52 2024 +0800
[fix](regression) fix regression failure of window_funnel caused by
duplicated table name (#40217)
## Proposed changes
Issue Number: close #xxx
---
.../data/nereids_p0/aggregate/window_funnel.out| 3 +
.../window_functions/window_funnel.out | 84 -
.../data/query_p0/aggregate/window_funnel.out | 31 --
.../nereids_p0/aggregate/window_funnel.groovy | 67 +++-
.../window_functions/window_funnel.sql | 92 --
.../suites/query_p0/aggregate/window_funnel.groovy | 360 -
6 files changed, 69 insertions(+), 568 deletions(-)
diff --git a/regression-test/data/nereids_p0/aggregate/window_funnel.out
b/regression-test/data/nereids_p0/aggregate/window_funnel.out
index f16f050c8e6..0bbe397fe3e 100644
--- a/regression-test/data/nereids_p0/aggregate/window_funnel.out
+++ b/regression-test/data/nereids_p0/aggregate/window_funnel.out
@@ -11,6 +11,9 @@
-- !window_funnel --
2
+-- !window_funnel_deduplication_compat --
+4
+
-- !window_funnel_deduplication --
2
diff --git
a/regression-test/data/nereids_p0/sql_functions/window_functions/window_funnel.out
b/regression-test/data/nereids_p0/sql_functions/window_functions/window_funnel.out
deleted file mode 100644
index 43948dc7951..000
---
a/regression-test/data/nereids_p0/sql_functions/window_functions/window_funnel.out
+++ /dev/null
@@ -1,84 +0,0 @@
--- This file is automatically generated. You should know what you did if you
want to edit this
--- !window_funnel --
-0
-
--- !window_funnel_2 --
-0
-
--- !window_funnel_3 --
-4
-
--- !window_funnel_4 --
-1 2022-03-12T19:05:04 4
-1 2022-03-12T16:15:01 3
-1 2022-03-12T13:28:02 2
-1 2022-03-12T10:411
-
--- !window_funnel_5 --
-1
-
--- !window_funnel_6 --
-2
-
--- !window_funnel_7 --
-0
-
--- !window_funnel_8 --
-1
-
--- !window_funnel_9 --
-1
-
--- !window_funnel_10 --
-1
-
--- !window_funnel_11 --
-1
-
--- !window_funnel_12 --
-1
-
--- !window_funnel_13 --
-1
-
--- !window_funnel_14 --
-1
-
--- !window_funnel_15 --
-1
-
--- !window_funnel_16 --
-1
-
--- !window_funnel_17 --
-1
-
--- !window_funnel_18 --
-1
-
--- !window_funnel_19 --
-1
-
--- !window_funnel_20 --
-1
-
--- !window_funnel_21 --
-1
-
--- !window_funnel_22 --
-1
-
--- !window_funnel_23 --
-1
-
--- !window_funnel_24 --
-1
-
--- !window_funnel_25 --
-1
-
--- !window_funnel_26 --
-users 13
-browser10
-buy1
-
diff --git a/regression-test/data/query_p0/aggregate/window_funnel.out
b/regression-test/data/query_p0/aggregate/window_funnel.out
deleted file mode 100644
index fa239a9e719..000
--- a/regression-test/data/query_p0/aggregate/window_funnel.out
+++ /dev/null
@@ -1,31 +0,0 @@
--- This file is automatically generated. You should know what you did if you
want to edit this
--- !window_funnel --
-1
-
--- !window_funnel --
-2
-
--- !window_funnel --
-1
-
--- !window_funnel --
-2
-
--- !window_funnel_deduplication_compat --
-4
-
--- !window_funnel_deduplication --
-2
-
--- !window_funnel_fixed --
-2
-
--- !window_funnel_fixed --
-1
-
--- !window_funnel_increase --
-4
-
--- !window_funnel_increase --
-2
-
diff --git a/regression-test/suites/nereids_p0/aggregate/window_funnel.groovy
b/regression-test/suites/nereids_p0/aggregate/window_funnel.groovy
index 7b551c28ac2..378311a575f 100644
--- a/regression-test/suites/nereids_p0/aggregate/window_funnel.groovy
+++ b/regression-test/suites/nereids_p0/aggregate/window_funnel.groovy
@@ -111,7 +111,7 @@ suite("window_funnel") {
strBuilder.append("curl --location-trusted -u " + context.config.jdbcUser
+ ":" + context.config.jdbcPassword)
strBuilder.append(" http://"; + context.config.feHttpAddress +
"/rest/v1/config/fe?conf_item=be_exec_version")
-String command = strBuilder.toString()
+def command = strBuilder.toString()
def process = command.toString().execute()
def code = process.waitFor()
def err = IOGroovyMethods.getText(new BufferedReader(new
InputStreamReader(process.getErrorStream(;
@@ -130,8 +130,73 @@ suite("window_funnel") {
}
}
if (beExecVersion < 3) {
+sql """ DROP TABLE IF EXISTS ${tableName} """
+sql """
+CREATE TABLE IF NOT EXISTS ${tableName} (
+xwho varchar(50) NULL COMMENT 'xwho',
+xwhen datetimev2(3) COMMENT 'xwhen',
+xwhat int NULL COMMENT 'xwhat'
+
Re: [PR] [fix](regression) fix regression failure of window_funnel caused by duplicated table name [doris]
doris-robot commented on PR #40217: URL: https://github.com/apache/doris/pull/40217#issuecomment-2322861013 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website). See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](spill) should call set_ready after changing the status [doris]
github-actions[bot] commented on PR #40075: URL: https://github.com/apache/doris/pull/40075#issuecomment-2322810181 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [chore](recycler) Change tablet_rowset_stale_sweep_time_sec=600 to reduce possibility failed to find rowset [doris]
github-actions[bot] commented on PR #40076: URL: https://github.com/apache/doris/pull/40076#issuecomment-2323162140 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](inverted index) Fix Session Variable Compatibility [doris]
xiaokang merged PR #39940: URL: https://github.com/apache/doris/pull/39940 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](regression) fix regression failure of window_funnel caused by duplicated table name [doris]
jacktengg commented on PR #40216: URL: https://github.com/apache/doris/pull/40216#issuecomment-2322856973 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [opt](inverted index) cache result sign is initialized only once [doris]
xiaokang commented on code in PR #40212: URL: https://github.com/apache/doris/pull/40212#discussion_r1739969955 ## be/src/vec/exprs/vexpr.cpp: ## @@ -617,7 +617,14 @@ bool VExpr::fast_execute(Block& block, const ColumnNumbers& arguments, size_t re return false; } -std::string result_column_name = gen_predicate_result_sign(block, arguments, function_name); +// cache value is initialized only once. Review Comment: It can be done in open without lock. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [refactor](opt) improve BE code readability of multi_match_any function [doris]
doris-robot commented on PR #39354: URL: https://github.com/apache/doris/pull/39354#issuecomment-2323159084 ClickBench: Total hot run time: 32.43 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit b299e958fe235bf8c87aee40dfa43c793e96fd6d, data reload: false query1 0.040.040.04 query2 0.090.040.05 query3 0.230.050.05 query4 1.680.070.07 query5 0.510.490.50 query6 1.130.730.74 query7 0.010.010.01 query8 0.060.050.05 query9 0.550.480.49 query10 0.550.540.52 query11 0.160.120.11 query12 0.150.120.13 query13 0.610.580.60 query14 2.032.112.10 query15 0.850.820.82 query16 0.380.380.37 query17 1.011.021.06 query18 0.220.220.21 query19 1.951.881.70 query20 0.010.010.01 query21 15.41 0.660.66 query22 4.297.282.25 query23 18.26 1.431.30 query24 2.070.220.21 query25 0.150.080.07 query26 0.260.190.18 query27 0.070.080.07 query28 13.27 1.030.99 query29 12.62 3.283.31 query30 0.240.060.06 query31 2.880.400.41 query32 3.240.500.48 query33 3.062.993.05 query34 17.07 4.434.44 query35 4.474.514.49 query36 0.660.480.51 query37 0.190.160.15 query38 0.160.150.14 query39 0.050.030.04 query40 0.150.130.12 query41 0.090.040.04 query42 0.060.050.05 query43 0.050.040.04 Total cold run time: 110.99 s Total hot run time: 32.43 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](vcompound-or)fix array_contains in vcompound-or [doris]
xiaokang closed pull request #38113: [fix](vcompound-or)fix array_contains in vcompound-or URL: https://github.com/apache/doris/pull/38113 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [opt](scan) merge small tablets into one scanner [doris]
doris-robot commented on PR #40133: URL: https://github.com/apache/doris/pull/40133#issuecomment-2322886718 TPC-DS: Total hot run time: 187338 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 41227be09db6d40a025cd4bbf85bdd56445f3098, data reload: false query1 917 376 366 366 query2 6497209720462046 query3 6684209 222 209 query4 26756 23146 23076 23076 query5 4152498 483 483 query6 268 164 169 164 query7 4592284 298 284 query8 250 206 202 202 query9 9326248524622462 query10 415 267 269 267 query11 18032 14924 14873 14873 query12 145 99 98 98 query13 1618390 357 357 query14 10280 715470517051 query15 297 168 176 168 query16 7989468 474 468 query17 1603588 567 567 query18 2100283 282 282 query19 256 137 146 137 query20 113 105 107 105 query21 206 105 104 104 query22 4572421441744174 query23 34072 33258 33567 33258 query24 11235 290727372737 query25 631 393 397 393 query26 1094160 168 160 query27 2424288 287 287 query28 7554217621252125 query29 831 401 396 396 query30 298 157 152 152 query31 1030747 774 747 query32 104 55 60 55 query33 771 281 277 277 query34 999 488 477 477 query35 858 742 704 704 query36 1076946 934 934 query37 147 89 90 89 query38 4081385738073807 query39 1431143213801380 query40 195 115 111 111 query41 44 46 46 46 query42 109 102 95 95 query43 502 461 469 461 query44 1233785 781 781 query45 190 157 161 157 query46 1124722 743 722 query47 1881182618101810 query48 391 300 294 294 query49 1072414 418 414 query50 802 408 398 398 query51 7176706170887061 query52 99 86 87 86 query53 260 183 177 177 query54 1091439 448 439 query55 79 76 77 76 query56 270 242 270 242 query57 1151111810531053 query58 236 218 251 218 query59 3134287028082808 query60 287 253 255 253 query61 100 102 103 102 query62 838 634 668 634 query63 215 183 176 176 query64 4159665 684 665 query65 3187312631553126 query66 1038336 338 336 query67 15526 15236 15303 15236 query68 3146593 575 575 query69 403 274 265 265 query70 1153111311321113 query71 329 270 263 263 query72 6345407939713971 query73 731 333 331 331 query74 9013874688138746 query75 3361269326462646 query76 2147943 877 877 query77 459 307 316 307 query78 10705 933491229122 query79 1050531 538 531 query80 657 476 477 476 query81 502 236 232 232 query82 239 151 142 142 query83 229 154 155 154 query84 226 80 75 75 query85 710 292 279 279 query86 317 284 289 284 query87 4355430142744274 query88 3170231122982298 query89 391 288 287 287 query90 1959196 199 196 query91 128 112 104 104 query92 64 53 52 52 query93 1014535 538 535 query94 863 290 299 290 query95 362 267 254 254 query96 599 276 271 271 query97 3182304530483045 query98 221 201 260 201 query99 1449127712441244 Total cold run time: 281572 ms Total hot run time: 187338 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t
Re: [PR] [Feat](Nereids) add use mv hint [doris]
LiBinfeng-01 commented on PR #40167: URL: https://github.com/apache/doris/pull/40167#issuecomment-2323130432 run external -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](recycler) Implement missing delete_prefix() of hdfs accessor [doris]
github-actions[bot] commented on PR #40215: URL: https://github.com/apache/doris/pull/40215#issuecomment-2323162275 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [test](memory) find large memory allocation not catch exception [doris]
yiguolei commented on PR #35223: URL: https://github.com/apache/doris/pull/35223#issuecomment-2322821046 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Refactor](inverted index) refactor inverted index compound predicates evaluate logic [doris]
airborne12 commented on PR #38908: URL: https://github.com/apache/doris/pull/38908#issuecomment-2323142167 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [debug](load) find invalid load memory allocation [doris]
doris-robot commented on PR #40220: URL: https://github.com/apache/doris/pull/40220#issuecomment-2322929873 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website). See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [chore](cloud) Support starting both meta-service and recycler within single process [doris]
github-actions[bot] commented on PR #40223: URL: https://github.com/apache/doris/pull/40223#issuecomment-2323103324 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Refactor](inverted index) refactor inverted index compound predicates evaluate logic [doris]
doris-robot commented on PR #38908: URL: https://github.com/apache/doris/pull/38908#issuecomment-2323159370 ClickBench: Total hot run time: 32.08 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit f62d014352a19a7beda85922fde3b034c1fb886c, data reload: false query1 0.050.040.04 query2 0.090.040.04 query3 0.230.050.05 query4 1.680.080.08 query5 0.520.500.49 query6 1.130.730.73 query7 0.010.010.01 query8 0.040.040.04 query9 0.540.480.50 query10 0.550.550.56 query11 0.140.120.11 query12 0.140.120.12 query13 0.610.580.59 query14 2.062.072.07 query15 0.890.820.81 query16 0.420.380.38 query17 0.991.011.06 query18 0.200.210.20 query19 1.871.791.76 query20 0.020.010.01 query21 15.41 0.680.68 query22 4.287.921.86 query23 18.25 1.331.28 query24 2.070.230.23 query25 0.150.080.07 query26 0.280.180.18 query27 0.080.080.07 query28 13.14 1.021.02 query29 12.84 3.353.37 query30 0.240.060.05 query31 3.010.400.38 query32 3.250.480.48 query33 2.932.983.03 query34 16.89 4.414.41 query35 4.394.434.44 query36 0.660.470.48 query37 0.180.150.15 query38 0.150.140.15 query39 0.040.050.04 query40 0.170.130.13 query41 0.090.050.06 query42 0.060.040.04 query43 0.040.040.03 Total cold run time: 110.78 s Total hot run time: 32.08 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[I] 外部表导入数据报错，不知道原因。外部表和doris表的元数据结构确认一致 [doris-website]
13057537919 opened a new issue, #1071: URL: https://github.com/apache/doris-website/issues/1071 Path:/zh-CN/docs/data-operate/import/insert-into-manual   -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [chore](cancel) Print query id to client when query_ctx not found. [doris]
doris-robot commented on PR #39961: URL: https://github.com/apache/doris/pull/39961#issuecomment-2323138056 ClickBench: Total hot run time: 32.4 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 825f80a46ae326d65dfa1297fd067e961fa233ff, data reload: false query1 0.050.040.04 query2 0.080.040.04 query3 0.230.040.05 query4 1.670.080.08 query5 0.510.500.51 query6 1.120.730.72 query7 0.020.010.01 query8 0.040.040.04 query9 0.550.500.49 query10 0.530.540.54 query11 0.150.110.12 query12 0.150.130.12 query13 0.600.590.59 query14 2.032.052.09 query15 0.860.820.82 query16 0.350.380.40 query17 1.041.050.99 query18 0.220.190.21 query19 1.881.791.78 query20 0.010.010.01 query21 15.38 0.660.64 query22 4.666.522.26 query23 18.23 1.321.30 query24 2.050.220.22 query25 0.150.090.08 query26 0.270.190.18 query27 0.090.080.08 query28 13.33 1.011.01 query29 12.61 3.333.29 query30 0.240.050.06 query31 2.860.400.40 query32 3.250.500.48 query33 2.943.003.03 query34 16.88 4.384.37 query35 4.504.454.43 query36 0.650.490.50 query37 0.190.160.15 query38 0.160.150.14 query39 0.050.040.04 query40 0.150.120.13 query41 0.100.040.04 query42 0.060.050.05 query43 0.050.040.04 Total cold run time: 110.94 s Total hot run time: 32.4 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [test](memory) find large memory allocation not catch exception [doris]
yiguolei opened a new pull request, #35223: URL: https://github.com/apache/doris/pull/35223 ## Proposed changes Issue Number: close #xxx ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch branch-2.1 updated: Revert "[Improvement](sort) Free sort blocks if this block is exhausted (#39306)" (#40211)
This is an automated email from the ASF dual-hosted git repository.
yiguolei pushed a commit to branch branch-2.1
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-2.1 by this push:
new 1768169b9a0 Revert "[Improvement](sort) Free sort blocks if this block
is exhausted (#39306)" (#40211)
1768169b9a0 is described below
commit 1768169b9a034204491cbc3368d101977d55ec7a
Author: yiguolei <676222...@qq.com>
AuthorDate: Sat Aug 31 15:58:55 2024 +0800
Revert "[Improvement](sort) Free sort blocks if this block is exhausted
(#39306)" (#40211)
Reverts apache/doris#39956
---
be/src/vec/common/sort/partition_sorter.cpp | 42 +
be/src/vec/common/sort/partition_sorter.h | 4 +-
be/src/vec/common/sort/sorter.cpp | 71 ++---
be/src/vec/common/sort/sorter.h | 12 +++--
be/src/vec/common/sort/topn_sorter.cpp | 17 +++
be/src/vec/core/sort_cursor.h | 68 ---
be/src/vec/runtime/vsorted_run_merger.cpp | 34 ++
be/src/vec/runtime/vsorted_run_merger.h | 13 --
8 files changed, 149 insertions(+), 112 deletions(-)
diff --git a/be/src/vec/common/sort/partition_sorter.cpp
b/be/src/vec/common/sort/partition_sorter.cpp
index c363a41d1c7..1ea7c6de6a8 100644
--- a/be/src/vec/common/sort/partition_sorter.cpp
+++ b/be/src/vec/common/sort/partition_sorter.cpp
@@ -58,17 +58,20 @@ Status PartitionSorter::append_block(Block* input_block) {
Block sorted_block =
VectorizedUtils::create_empty_columnswithtypename(_row_desc);
DCHECK(input_block->columns() == sorted_block.columns());
RETURN_IF_ERROR(partial_sort(*input_block, sorted_block));
-_state->add_sorted_block(Block::create_shared(std::move(sorted_block)));
+RETURN_IF_ERROR(_state->add_sorted_block(sorted_block));
return Status::OK();
}
Status PartitionSorter::prepare_for_read() {
+auto& cursors = _state->get_cursors();
auto& blocks = _state->get_sorted_block();
auto& priority_queue = _state->get_priority_queue();
for (auto& block : blocks) {
-priority_queue.push(MergeSortCursorImpl::create_shared(block,
_sort_description));
+cursors.emplace_back(block, _sort_description);
+}
+for (auto& cursor : cursors) {
+priority_queue.push(MergeSortCursor(&cursor));
}
-blocks.clear();
return Status::OK();
}
@@ -81,30 +84,29 @@ void PartitionSorter::reset_sorter_state(RuntimeState*
runtime_state) {
}
Status PartitionSorter::get_next(RuntimeState* state, Block* block, bool* eos)
{
-if (_state->get_priority_queue().empty()) {
-*eos = true;
-} else if (_state->get_priority_queue().size() == 1 && _has_global_limit) {
-block->swap(*_state->get_priority_queue().top().impl->block);
-block->set_num_rows(_partition_inner_limit);
+if (_state->get_sorted_block().empty()) {
*eos = true;
} else {
-RETURN_IF_ERROR(partition_sort_read(block, eos, state->batch_size()));
+if (_state->get_sorted_block().size() == 1 && _has_global_limit) {
+auto& sorted_block = _state->get_sorted_block()[0];
+block->swap(sorted_block);
+block->set_num_rows(_partition_inner_limit);
+*eos = true;
+} else {
+RETURN_IF_ERROR(partition_sort_read(block, eos,
state->batch_size()));
+}
}
return Status::OK();
}
Status PartitionSorter::partition_sort_read(Block* output_block, bool* eos,
int batch_size) {
-auto& priority_queue = _state->get_priority_queue();
-if (priority_queue.empty()) {
-*eos = true;
-return Status::OK();
-}
-const auto& sorted_block = priority_queue.top().impl->block;
-size_t num_columns = sorted_block->columns();
+const auto& sorted_block = _state->get_sorted_block()[0];
+size_t num_columns = sorted_block.columns();
MutableBlock m_block =
-VectorizedUtils::build_mutable_mem_reuse_block(output_block,
*sorted_block);
+VectorizedUtils::build_mutable_mem_reuse_block(output_block,
sorted_block);
MutableColumns& merged_columns = m_block.mutable_columns();
size_t current_output_rows = 0;
+auto& priority_queue = _state->get_priority_queue();
bool get_enough_data = false;
while (!priority_queue.empty()) {
@@ -119,7 +121,7 @@ Status PartitionSorter::partition_sort_read(Block*
output_block, bool* eos, int
//1 row_number no need to check distinct, just output
partition_inner_limit row
if ((current_output_rows + _output_total_rows) <
_partition_inner_limit) {
for (size_t i = 0; i < num_columns; ++i) {
-
merged_columns[i]->insert_from(*current->block->get_columns()[i], current->pos);
+merged_columns[i]->insert_from(*current->all_columns[i],
current->pos);
}
Re: [PR] [chore](cancel) Print query id to client when query_ctx not found. [doris]
doris-robot commented on PR #39961: URL: https://github.com/apache/doris/pull/39961#issuecomment-2323136739 TPC-DS: Total hot run time: 193696 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 825f80a46ae326d65dfa1297fd067e961fa233ff, data reload: false query1 1263923 882 882 query2 6423207820072007 query3 10592 393239793932 query4 59662 24911 23346 23346 query5 5327504 501 501 query6 418 159 166 159 query7 5787299 296 296 query8 294 212 216 212 query9 8756252224922492 query10 495 278 258 258 query11 17854 15132 15397 15132 query12 171 111 106 106 query13 1524439 398 398 query14 11059 743672137213 query15 236 171 195 171 query16 7453483 468 468 query17 1101592 569 569 query18 1999312 318 312 query19 281 157 146 146 query20 117 111 130 111 query21 206 103 101 101 query22 4528461744354435 query23 34345 33750 33202 33202 query24 6000285028272827 query25 528 365 381 365 query26 680 156 155 155 query27 1775281 278 278 query28 3945211221002100 query29 664 407 406 406 query30 226 150 152 150 query31 952 786 777 777 query32 74 56 57 56 query33 411 282 278 278 query34 863 477 474 474 query35 833 730 691 691 query36 1059955 952 952 query37 150 92 85 85 query38 3955385440193854 query39 1450139513791379 query40 196 119 119 119 query41 47 52 47 47 query42 120 98 131 98 query43 540 492 485 485 query44 1114740 736 736 query45 200 169 170 169 query46 1092761 743 743 query47 1913180518551805 query48 367 293 301 293 query49 759 417 431 417 query50 814 412 412 412 query51 7199707771507077 query52 98 86 86 86 query53 249 176 179 176 query54 578 457 444 444 query55 75 76 76 76 query56 272 259 263 259 query57 1202105610831056 query58 223 227 231 227 query59 3127281028032803 query60 295 277 269 269 query61 139 98 100 98 query62 719 645 670 645 query63 214 184 182 182 query64 2698696 675 675 query65 3217315331553153 query66 660 326 340 326 query67 15381 15166 15154 15154 query68 2817578 570 570 query69 401 291 282 282 query70 1191110911191109 query71 348 276 271 271 query72 5783403841134038 query73 745 335 340 335 query74 9264890089318900 query75 3369267426712671 query76 1387968 1046968 query77 558 348 324 324 query78 11752 959699229596 query79 1107548 533 533 query80 693 515 510 510 query81 549 232 232 232 query82 284 152 139 139 query83 177 155 153 153 query84 260 81 83 81 query85 762 293 290 290 query86 307 275 299 275 query87 4400433243924332 query88 3155230823052305 query89 379 290 294 290 query90 1757192 187 187 query91 128 103 98 98 query92 61 52 53 52 query93 1053532 525 525 query94 553 303 303 303 query95 338 263 259 259 query96 598 261 269 261 query97 3244305530713055 query98 224 198 200 198 query99 1523126612481248 Total cold run time: 306131 ms Total hot run time: 193696 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t
Re: [PR] [chore] Reduce range of nextId when calling advanceNextId() [doris]
doris-robot commented on PR #40160: URL: https://github.com/apache/doris/pull/40160#issuecomment-2323021039 TPC-DS: Total hot run time: 192750 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit e4942f8cc3e207151e0d875072a6add305ffa738, data reload: false query1 1250917 860 860 query2 6156200620382006 query3 10627 413040534053 query4 60388 24735 23168 23168 query5 5378503 501 501 query6 394 157 152 152 query7 5857299 296 296 query8 282 213 229 213 query9 9015251325032503 query10 491 272 268 268 query11 18371 15107 15468 15107 query12 149 103 106 103 query13 1574394 385 385 query14 11025 742270907090 query15 248 183 199 183 query16 7590496 434 434 query17 1156587 551 551 query18 1929300 291 291 query19 287 155 156 155 query20 123 124 116 116 query21 210 107 113 107 query22 4656457141714171 query23 34013 33744 33441 33441 query24 5987287129082871 query25 529 379 374 374 query26 687 157 159 157 query27 1799281 279 279 query28 3845212520992099 query29 679 419 399 399 query30 188 149 149 149 query31 928 759 796 759 query32 79 55 54 54 query33 459 285 289 285 query34 873 483 486 483 query35 835 718 727 718 query36 1080965 939 939 query37 149 100 87 87 query38 3978392640063926 query39 1496139414121394 query40 202 116 117 116 query41 48 45 45 45 query42 112 98 94 94 query43 518 485 490 485 query44 1091772 751 751 query45 205 172 187 172 query46 1118722 737 722 query47 1893183518641835 query48 370 290 294 290 query49 743 411 420 411 query50 807 431 421 421 query51 7221702270467022 query52 99 86 87 86 query53 257 177 176 176 query54 572 441 443 441 query55 79 74 76 74 query56 268 245 256 245 query57 1213107810981078 query58 229 217 226 217 query59 3161286328882863 query60 283 265 268 265 query61 103 102 131 102 query62 748 653 624 624 query63 225 177 188 177 query64 2843682 645 645 query65 3200316631443144 query66 634 330 326 326 query67 15427 15198 15304 15198 query68 2942586 565 565 query69 400 298 320 298 query70 1150104511531045 query71 344 279 276 276 query72 5304398640433986 query73 758 331 329 329 query74 9299885089308850 query75 3374264226932642 query76 1429954 954 954 query77 499 322 317 317 query78 9721941390649064 query79 1046551 536 536 query80 701 495 511 495 query81 524 237 232 232 query82 249 143 145 143 query83 171 146 149 146 query84 267 74 72 72 query85 678 306 290 290 query86 332 294 308 294 query87 4415425043094250 query88 2950233223242324 query89 381 290 291 290 query90 1999192 193 192 query91 127 99 94 94 query92 59 50 51 50 query93 1042547 537 537 query94 683 298 306 298 query95 321 262 265 262 query96 591 273 267 267 query97 3188305130553051 query98 222 207 196 196 query99 1730128412531253 Total cold run time: 305325 ms Total hot run time: 192750 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to
Re: [PR] [debug](load) find invalid load memory allocation [doris]
doris-robot commented on PR #40220: URL: https://github.com/apache/doris/pull/40220#issuecomment-2322949361 TPC-H: Total hot run time: 38611 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 9e6a71e519f9f101dba64909968ffabf3ee01060, data reload: false -- Round 1 -- q1 17612 462844134413 q2 2012186 178 178 q3 11771 975 1123975 q4 10508 642 742 642 q5 7795287028842870 q6 232 141 141 141 q7 994 629 620 620 q8 9332212621422126 q9 7106655365926553 q10 6987223422392234 q11 457 241 245 241 q12 409 226 233 226 q13 18022 308430473047 q14 281 233 236 233 q15 520 490 481 481 q16 593 495 518 495 q17 1001648 692 648 q18 7397703269186918 q19 1412115211621152 q20 673 344 331 331 q21 4388337230913091 q22 11361002996 996 Total cold run time: 110638 ms Total hot run time: 38611 ms - Round 2, with runtime_filter_mode=off - q1 4374442945294429 q2 382 263 280 263 q3 2939263526592635 q4 1973170817161708 q5 5717564957765649 q6 237 133 135 133 q7 2251186618301830 q8 3322344635323446 q9 8881888589108885 q10 3643345133573357 q11 617 508 508 508 q12 882 669 679 669 q13 13505 322232553222 q14 312 295 286 286 q15 553 507 524 507 q16 602 595 602 595 q17 1839153915331533 q18 8188795579167916 q19 1773166915441544 q20 2175190619341906 q21 5847546754585458 q22 1100103110281028 Total cold run time: 71112 ms Total hot run time: 57507 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Enhancement](vec) Remove duplicated code in BlockReader [doris]
doris-robot commented on PR #39784: URL: https://github.com/apache/doris/pull/39784#issuecomment-2322853400 ClickBench: Total hot run time: 32.91 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit a1609296a0bca768894e0fd60c8d4ac090340ad1, data reload: false query1 0.040.040.04 query2 0.080.040.04 query3 0.220.050.05 query4 1.670.090.09 query5 0.490.500.50 query6 1.140.730.73 query7 0.020.010.02 query8 0.050.040.04 query9 0.550.480.49 query10 0.540.550.54 query11 0.170.120.11 query12 0.160.120.12 query13 0.620.600.61 query14 2.072.042.08 query15 0.920.840.85 query16 0.380.370.38 query17 0.961.001.04 query18 0.220.210.22 query19 1.881.741.73 query20 0.010.010.02 query21 15.39 0.650.65 query22 4.147.062.61 query23 18.26 1.431.33 query24 2.180.210.22 query25 0.170.090.08 query26 0.270.180.19 query27 0.090.070.08 query28 13.19 1.041.03 query29 12.65 3.383.34 query30 0.240.060.06 query31 2.900.410.40 query32 3.240.500.50 query33 3.003.003.04 query34 16.92 4.424.39 query35 4.424.454.42 query36 0.660.490.49 query37 0.180.150.15 query38 0.170.150.15 query39 0.050.040.03 query40 0.160.130.13 query41 0.090.050.06 query42 0.060.050.06 query43 0.050.040.04 Total cold run time: 110.67 s Total hot run time: 32.91 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [debug](load) find invalid load memory allocation [doris]
github-actions[bot] commented on PR #40220: URL: https://github.com/apache/doris/pull/40220#issuecomment-2322932509 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Enhancement]use awaitility.await() [doris]
Vallishp commented on PR #40112: URL: https://github.com/apache/doris/pull/40112#issuecomment-2322819077 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](recycler) Implement missing delete_prefix() of hdfs accessor [doris]
github-actions[bot] commented on PR #40215: URL: https://github.com/apache/doris/pull/40215#issuecomment-2322834900 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [chore](recycler) Change tablet_rowset_stale_sweep_time_sec=600 to reduce possibility failed to find rowset [doris]
doris-robot commented on PR #40076: URL: https://github.com/apache/doris/pull/40076#issuecomment-2323167953 TPC-DS: Total hot run time: 188123 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 4d0d206877736afee39790119c580231df6f89f8, data reload: false query1 916 378 360 360 query2 6467205519501950 query3 6645209 247 209 query4 34190 23150 23113 23113 query5 4203504 492 492 query6 249 166 163 163 query7 4576299 290 290 query8 253 206 209 206 query9 8605250025032500 query10 447 265 269 265 query11 17837 14995 14985 14985 query12 147 99 97 97 query13 1638410 365 365 query14 9817744772917291 query15 272 172 178 172 query16 7985462 506 462 query17 1596581 560 560 query18 1936294 288 288 query19 279 150 150 150 query20 115 114 113 113 query21 211 107 105 105 query22 4465443341714171 query23 34045 33283 33426 33283 query24 11187 287828252825 query25 645 406 403 403 query26 1139158 159 158 query27 2748284 279 279 query28 7433216321322132 query29 829 431 422 422 query30 290 154 155 154 query31 987 764 816 764 query32 94 60 73 60 query33 765 294 295 294 query34 947 466 496 466 query35 872 776 693 693 query36 1099911 950 911 query37 166 103 97 97 query38 3977380338913803 query39 1531141713881388 query40 209 120 119 119 query41 51 49 48 48 query42 120 97 99 97 query43 525 484 477 477 query44 1208769 747 747 query45 203 174 172 172 query46 1085766 736 736 query47 1887182217911791 query48 393 302 299 299 query49 1173434 425 425 query50 807 404 409 404 query51 7277704070207020 query52 95 89 88 88 query53 252 189 185 185 query54 910 468 453 453 query55 77 78 79 78 query56 274 250 257 250 query57 1185108110561056 query58 247 222 231 222 query59 3086288328332833 query60 289 267 262 262 query61 100 100 98 98 query62 844 674 657 657 query63 214 184 182 182 query64 4199673 671 671 query65 3209315131573151 query66 966 333 340 333 query67 15717 15544 15197 15197 query68 4627564 557 557 query69 428 280 272 272 query70 1203109410821082 query71 339 282 267 267 query72 6655409540954095 query73 741 331 330 330 query74 9183893188348834 query75 3403261527002615 query76 2705987 966 966 query77 497 316 312 312 query78 10552 911595879115 query79 1501536 547 536 query80 1048510 501 501 query81 562 234 241 234 query82 1131139 146 139 query83 225 150 151 150 query84 223 82 77 77 query85 1213336 275 275 query86 376 289 294 289 query87 4419422142624221 query88 2959230722752275 query89 385 290 293 290 query90 1810197 195 195 query91 124 101 101 101 query92 65 49 53 49 query93 1059552 534 534 query94 928 279 295 279 query95 351 261 254 254 query96 579 273 271 271 query97 3236308030653065 query98 214 208 198 198 query99 1476128812551255 Total cold run time: 292422 ms Total hot run time: 188123 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to th
Re: [PR] [test](memory) find large memory allocation not catch exception [doris]
doris-robot commented on PR #35223: URL: https://github.com/apache/doris/pull/35223#issuecomment-2322835227 ClickBench: Total hot run time: 32.52 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 0bc9aa558788665bbfe96285bbe78d26eb49c226, data reload: false query1 0.050.040.04 query2 0.080.040.04 query3 0.230.040.05 query4 1.680.080.07 query5 0.500.490.49 query6 1.140.730.73 query7 0.020.010.02 query8 0.050.050.04 query9 0.560.490.49 query10 0.550.550.53 query11 0.160.120.12 query12 0.160.120.12 query13 0.610.590.58 query14 2.132.062.04 query15 0.900.820.83 query16 0.380.390.38 query17 1.001.031.00 query18 0.220.200.21 query19 1.961.681.85 query20 0.010.010.01 query21 15.39 0.660.66 query22 3.576.932.40 query23 18.28 1.451.37 query24 2.090.230.21 query25 0.160.080.09 query26 0.270.180.19 query27 0.080.080.07 query28 13.28 1.041.01 query29 12.63 3.333.29 query30 0.240.050.06 query31 3.210.410.40 query32 3.550.480.49 query33 3.023.043.06 query34 17.05 4.344.44 query35 4.444.474.44 query36 0.650.480.50 query37 0.190.160.15 query38 0.160.140.16 query39 0.050.050.04 query40 0.160.130.12 query41 0.090.050.05 query42 0.060.050.05 query43 0.040.040.04 Total cold run time: 111.05 s Total hot run time: 32.52 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [chore](cloud) Support starting both meta-service and recycler within single process [doris]
github-actions[bot] commented on PR #40223: URL: https://github.com/apache/doris/pull/40223#issuecomment-2323103340 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [chore](cloud) Support starting both meta-service and recycler within single process [doris]
gavinchou opened a new pull request, #40223: URL: https://github.com/apache/doris/pull/40223 e.g. ``` ./bin/start.sh --recycler --meta-service --daemon ``` the log file will be meta_service.INFO* -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improvement](fdb) add a tool which deploys and starts fdb [doris]
github-actions[bot] commented on PR #39803:
URL: https://github.com/apache/doris/pull/39803#issuecomment-2322929921
`sh-checker report`
To get the full details, please check in the
[job]("https://github.com/apache/doris/actions/runs/10646306291";) output.
shellcheck errors
```
'shellcheck ' returned error 1 finding the following syntactical issues:
--
In tools/fdb/fdb_vars.sh line 1:
# Licensed to the Apache Software Foundation (ASF) under one
^-- SC2148 (error): Tips depend on target shell and yours is unknown. Add a
shebang or a 'shell' directive.
In tools/fdb/fdb_vars.sh line 25:
DATA_DIRS="/mnt/foundationdb/data1,/mnt/foundationdb/data2,/mnt/foundationdb/data3"
^---^ SC2034 (warning): DATA_DIRS appears unused. Verify use (or export
if used externally).
In tools/fdb/fdb_vars.sh line 33:
FDB_CLUSTER_IPS="172.200.0.2,172.200.0.3,172.200.0.4"
^-^ SC2034 (warning): FDB_CLUSTER_IPS appears unused. Verify use
(or export if used externally).
In tools/fdb/fdb_vars.sh line 38:
FDB_HOME="/fdbhome"
^--^ SC2034 (warning): FDB_HOME appears unused. Verify use (or export if
used externally).
In tools/fdb/fdb_vars.sh line 42:
FDB_CLUSTER_ID="SAQESzbh"
^^ SC2034 (warning): FDB_CLUSTER_ID appears unused. Verify use
(or export if used externally).
In tools/fdb/fdb_vars.sh line 45:
FDB_CLUSTER_DESC="mycluster"
^--^ SC2034 (warning): FDB_CLUSTER_DESC appears unused. Verify
use (or export if used externally).
In tools/fdb/fdb_vars.sh line 50:
MEMORY_LIMIT_GB=16
^-^ SC2034 (warning): MEMORY_LIMIT_GB appears unused. Verify use
(or export if used externally).
In tools/fdb/fdb_vars.sh line 53:
CPU_CORES_LIMIT=8
^-^ SC2034 (warning): CPU_CORES_LIMIT appears unused. Verify use
(or export if used externally).
In tools/fdb/fdb_vars.sh line 58:
FDB_PORT=4500
^--^ SC2034 (warning): FDB_PORT appears unused. Verify use (or export if
used externally).
In tools/fdb/fdb_vars.sh line 61:
FDB_VERSION="7.1.38"
^-^ SC2034 (warning): FDB_VERSION appears unused. Verify use (or
export if used externally).
In tools/fdb/fdb_vars.sh line 64:
USER=`whoami`
^--^ SC2006 (style): Use $(...) notation instead of legacy
backticks `...`.
Did you mean:
USER=$(whoami)
For more information:
https://www.shellcheck.net/wiki/SC2148 -- Tips depend on target shell and
y...
https://www.shellcheck.net/wiki/SC2034 -- CPU_CORES_LIMIT appears unused.
V...
https://www.shellcheck.net/wiki/SC2006 -- Use $(...) notation instead of
le...
--
You can address the above issues in one of three ways:
1. Manually correct the issue in the offending shell script;
2. Disable specific issues by adding the comment:
# shellcheck disable=
above the line that contains the issue, where is the error code;
3. Add '-e ' to the SHELLCHECK_OPTS setting in your .yml action file.
```
shfmt errors
```
'shfmt ' returned error 1 finding the following formatting issues:
--
--- tools/fdb/fdb_ctr.sh.orig
+++ tools/fdb/fdb_ctr.sh
@@ -142,7 +142,7 @@
local data_dir_count
# Convert comma-separated DATA_DIRS into an array
-IFS=',' read -r -a DATA_DIR_ARRAY <<< "${DATA_DIRS}"
+IFS=',' read -r -a DATA_DIR_ARRAY <<<"${DATA_DIRS}"
data_dir_count=${#DATA_DIR_ARRAY[@]}
# Stateless processes (at least 1, up to 1/4 of CPU cores)
@@ -152,13 +152,13 @@
# Storage processes (must be a multiple of the number of data
directories)
local storage_processes=$((cpu_cores / 4))
[[ ${storage_processes} -lt ${data_dir_count} ]] &&
storage_processes=${data_dir_count}
-storage_processes=$(( (storage_processes / data_dir_count) *
data_dir_count ))
+storage_processes=$(((storage_processes / data_dir_count) *
data_dir_count))
# Transaction processes (must be a multiple of the number of data
directories)
local transaction_processes=$((cpu_cores / 8))
[[ ${transaction_processes} -lt ${min_processes} ]] &&
transaction_processes=${min_processes}
[[ ${transaction_processes} -lt ${data_dir_count} ]] &&
transaction_processes=${data_dir_count}
-transaction_processes=$(( (transaction_processes / data_dir_count) *
data_dir_count ))
+transaction_processes=$(((transaction_processes / data_dir_count) *
data_dir_count))
# Return the values
echo "${stateless_processes} ${storage_processes}
${transaction_processes}"
@@ -175,9 +175,9 @@
CLUSTER_DESC="${FDB_CLUSTER_DESC:-${FDB_CLUSTER_ID}}"
# Convert comma-separated DATA_DIRS into an array
-IFS=',' read -r -a
Re: [PR] [debug](load) find invalid load memory allocation [doris]
doris-robot commented on PR #40220: URL: https://github.com/apache/doris/pull/40220#issuecomment-2322954098 TPC-DS: Total hot run time: 193206 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 9e6a71e519f9f101dba64909968ffabf3ee01060, data reload: false query1 1263904 868 868 query2 6461202320182018 query3 10627 394239773942 query4 60195 25592 23355 23355 query5 5356516 517 516 query6 393 167 177 167 query7 5780298 304 298 query8 292 195 215 195 query9 8875252024652465 query10 481 289 263 263 query11 16897 15029 15292 15029 query12 149 105 110 105 query13 1576398 383 383 query14 11063 711774007117 query15 212 182 187 182 query16 7394472 537 472 query17 1103576 600 576 query18 1919290 293 290 query19 280 146 145 145 query20 124 114 114 114 query21 215 113 110 110 query22 4633442346014423 query23 34336 33738 33257 33257 query24 5993298728722872 query25 531 400 410 400 query26 659 166 162 162 query27 1713288 297 288 query28 3845215521172117 query29 681 419 432 419 query30 247 152 162 152 query31 940 737 748 737 query32 71 56 56 56 query33 475 303 309 303 query34 877 488 486 486 query35 852 727 728 727 query36 1089962 948 948 query37 153 95 97 95 query38 3987392638553855 query39 1446138014241380 query40 203 124 122 122 query41 50 48 46 46 query42 115 100 98 98 query43 527 492 480 480 query44 1112757 756 756 query45 201 173 171 171 query46 1114767 751 751 query47 1891179718361797 query48 393 305 296 296 query49 780 453 453 453 query50 839 435 421 421 query51 7297702770797027 query52 101 89 91 89 query53 259 183 192 183 query54 582 460 468 460 query55 82 85 85 85 query56 295 277 280 277 query57 1216106010681060 query58 235 235 238 235 query59 3035296228762876 query60 308 284 279 279 query61 121 192 103 103 query62 762 655 645 645 query63 220 188 195 188 query64 2329697 672 672 query65 3184310031383100 query66 665 356 345 345 query67 15552 15271 15258 15258 query68 4134576 579 576 query69 405 289 283 283 query70 1238109211401092 query71 350 282 284 282 query72 6618405840824058 query73 765 333 336 333 query74 9193882688568826 query75 3357269226912691 query76 15871015972 972 query77 553 344 331 331 query78 9757916589658965 query79 1075546 540 540 query80 834 528 520 520 query81 479 233 235 233 query82 369 155 146 146 query83 181 148 150 148 query84 253 80 75 75 query85 799 346 279 279 query86 329 298 295 295 query87 4427426041844184 query88 3397229923102299 query89 396 287 292 287 query90 1878198 194 194 query91 129 97 100 97 query92 66 54 55 54 query93 544 547 544 query94 784 303 308 303 query95 365 267 268 267 query96 610 265 273 265 query97 3246308130293029 query98 216 203 201 201 query99 1557127712791277 Total cold run time: 307109 ms Total hot run time: 193206 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to
Re: [PR] [Compatibility](agg-state) add be_exec_version check to avoid aggregation data format changed le… [doris]
BiteThet commented on PR #40193: URL: https://github.com/apache/doris/pull/40193#issuecomment-2323190993 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [test](inverted index)Add upgrade and downgrade test cases for inverted_index (#37783) [doris]
doris-robot commented on PR #40221: URL: https://github.com/apache/doris/pull/40221#issuecomment-2322932629 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website). See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [refactor](opt) improve BE code readability of multi_match_any function [doris]
superdiaodiao commented on PR #39354: URL: https://github.com/apache/doris/pull/39354#issuecomment-2322851114 run beut -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [chore](recycler) Change tablet_rowset_stale_sweep_time_sec=600 to reduce possibility failed to find rowset [doris]
doris-robot commented on PR #40076: URL: https://github.com/apache/doris/pull/40076#issuecomment-2323169161 ClickBench: Total hot run time: 32.37 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 4d0d206877736afee39790119c580231df6f89f8, data reload: false query1 0.050.040.03 query2 0.070.030.04 query3 0.220.050.05 query4 1.670.070.07 query5 0.490.480.50 query6 1.130.740.73 query7 0.020.020.01 query8 0.050.050.04 query9 0.560.490.48 query10 0.520.530.54 query11 0.150.120.12 query12 0.160.120.12 query13 0.610.590.58 query14 2.052.072.06 query15 0.860.820.81 query16 0.360.370.36 query17 1.010.960.97 query18 0.210.210.20 query19 1.831.861.76 query20 0.010.010.01 query21 15.42 0.660.65 query22 4.097.182.39 query23 18.29 1.361.22 query24 1.870.260.22 query25 0.160.090.08 query26 0.270.180.18 query27 0.080.080.07 query28 13.26 1.021.00 query29 12.89 3.343.33 query30 0.250.060.05 query31 2.890.410.40 query32 3.220.480.48 query33 2.992.983.05 query34 16.93 4.414.38 query35 4.464.464.50 query36 0.660.470.47 query37 0.180.170.15 query38 0.150.140.15 query39 0.060.040.04 query40 0.160.130.12 query41 0.100.040.05 query42 0.060.050.05 query43 0.050.040.05 Total cold run time: 110.52 s Total hot run time: 32.37 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [refactor](opt) improve BE code readability of multi_match_any function [doris]
doris-robot commented on PR #39354: URL: https://github.com/apache/doris/pull/39354#issuecomment-2323153829 TPC-H: Total hot run time: 37889 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit b299e958fe235bf8c87aee40dfa43c793e96fd6d, data reload: false -- Round 1 -- q1 17609 438142844284 q2 2014179 177 177 q3 11800 945 1147945 q4 10522 724 691 691 q5 7740282528182818 q6 230 141 137 137 q7 955 613 599 599 q8 9327209820522052 q9 7134656165616561 q10 6998220021852185 q11 450 234 240 234 q12 398 230 229 229 q13 17754 301430473014 q14 273 227 232 227 q15 529 497 481 481 q16 582 521 512 512 q17 961 655 656 655 q18 7323683467736773 q19 1401102610721026 q20 671 342 334 334 q21 4087294530012945 q22 1120104910101010 Total cold run time: 109878 ms Total hot run time: 37889 ms - Round 2, with runtime_filter_mode=off - q1 4316428742434243 q2 383 271 274 271 q3 2869270726642664 q4 1916163316341633 q5 5542568456965684 q6 229 134 135 134 q7 2279185518521852 q8 3309339833893389 q9 8851889189078891 q10 3592337733623362 q11 614 501 551 501 q12 820 669 653 653 q13 13789 325430973097 q14 324 286 289 286 q15 530 505 488 488 q16 598 580 562 562 q17 1820154515281528 q18 8062766778577667 q19 1724155115141514 q20 2170191119281911 q21 5782549054055405 q22 1186110810311031 Total cold run time: 70705 ms Total hot run time: 56766 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Enhancement](vec) Remove duplicated code in BlockReader [doris]
doris-robot commented on PR #39784: URL: https://github.com/apache/doris/pull/39784#issuecomment-2322849183 TPC-H: Total hot run time: 38476 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit a1609296a0bca768894e0fd60c8d4ac090340ad1, data reload: false -- Round 1 -- q1 18322 443044494430 q2 2680184 173 173 q3 11754 113511461135 q4 10554 749 886 749 q5 8093290427802780 q6 226 137 140 137 q7 981 610 615 610 q8 9337204020662040 q9 7170654165246524 q10 7011224022022202 q11 459 237 240 237 q12 399 224 220 220 q13 18037 307530713071 q14 292 246 246 246 q15 514 485 486 485 q16 571 519 506 506 q17 984 678 720 678 q18 7369676567976765 q19 1391109110431043 q20 687 335 344 335 q21 3997312130923092 q22 102710181018 Total cold run time: 111939 ms Total hot run time: 38476 ms - Round 2, with runtime_filter_mode=off - q1 4317433742804280 q2 378 274 281 274 q3 2880261626812616 q4 1952163316471633 q5 5379536453905364 q6 226 132 129 129 q7 2079173817811738 q8 3215334233463342 q9 8412837084068370 q10 3463320231863186 q11 588 496 505 496 q12 788 640 609 609 q13 11502 303030313030 q14 310 293 282 282 q15 521 485 491 485 q16 603 534 555 534 q17 1779149614971496 q18 7756754873337333 q19 1659164015851585 q20 2079180418341804 q21 5386514152815141 q22 1122101110451011 Total cold run time: 66394 ms Total hot run time: 54738 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [opt](inverted index) cache result sign is initialized only once [doris]
github-actions[bot] commented on PR #40212: URL: https://github.com/apache/doris/pull/40212#issuecomment-2323157643 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](recycler) Implement missing delete_prefix() of hdfs accessor [doris]
gavinchou commented on PR #40215: URL: https://github.com/apache/doris/pull/40215#issuecomment-2322836257 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [cloud] Simplify meta-service deployment by integrating recycler [doris-website]
gavinchou opened a new pull request, #1073: URL: https://github.com/apache/doris-website/pull/1073 # Versions - [x] dev - [x] 3.0 - [ ] 2.1 - [ ] 2.0 # Languages - [x] Chinese - [ ] English -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [chore](cloud) Support starting both meta-service and recycler within single process [doris]
gavinchou commented on code in PR #40223:
URL: https://github.com/apache/doris/pull/40223#discussion_r1739852747
##
cloud/script/start.sh:
##
@@ -131,14 +131,22 @@ fi
mkdir -p "${DORIS_HOME}/log"
echo "starts ${process} with args: $*"
+out_file=${DORIS_HOME}/log/${process}.out
if [[ "${RUN_DAEMON}" -eq 1 ]]; then
-date >>"${DORIS_HOME}/log/${process}.out"
-nohup "${bin}" "$@" >>"${DORIS_HOME}/log/${process}.out" 2>&1 &
-# wait for log flush
-sleep 1.5
-tail -n10 "${DORIS_HOME}/log/${process}.out" | grep 'working directory'
-B1 -A10
-echo "please check process log for more details"
-echo ""
+# append 10 blank lines to ensure the following tail -n10 works correctly
+printf "\n\n\n\n\n\n\n\n\n\n" >> "${out_file}"
+echo "$(date +'%F %T') try to start ${process}" >> "${out_file}"
+nohup "${bin}" "$@" >> "${out_file}" 2>&1 &
Review Comment:
```suggestion
printf "\n\n\n\n\n\n\n\n\n\n" >>"${out_file}"
echo "$(date +'%F %T') try to start ${process}" >>"${out_file}"
nohup "${bin}" "$@" >>"${out_file}" 2>&1 &
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](recycler) Implement missing delete_prefix() of hdfs accessor [doris]
gavinchou commented on PR #40215: URL: https://github.com/apache/doris/pull/40215#issuecomment-2322896930 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] Revert "[Improvement](sort) Free sort blocks if this block is exhausted (#39306)" [doris]
yiguolei merged PR #40211: URL: https://github.com/apache/doris/pull/40211 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [chore](cloud) Support starting both meta-service and recycler within single process [doris]
github-actions[bot] commented on PR #40223: URL: https://github.com/apache/doris/pull/40223#issuecomment-2322980905 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [chore](cancel) Print query id to client when query_ctx not found. [doris]
github-actions[bot] commented on PR #39961: URL: https://github.com/apache/doris/pull/39961#issuecomment-2323116629 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [debug](load) find invalid load memory allocation [doris]
yiguolei commented on PR #40220: URL: https://github.com/apache/doris/pull/40220#issuecomment-2323110534 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](recycler) Implement missing delete_prefix() of hdfs accessor [doris]
doris-robot commented on PR #40215: URL: https://github.com/apache/doris/pull/40215#issuecomment-2322906858 TPC-H: Total hot run time: 38280 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 15ce85a75d46fbdcb423e5bd86bec302bd8bbbf8, data reload: false -- Round 1 -- q1 17630 459043434343 q2 2030186 177 177 q3 11489 997 1121997 q4 10448 727 700 700 q5 7749284327642764 q6 226 143 143 143 q7 964 622 603 603 q8 9315206120932061 q9 7247654565466545 q10 7058216622412166 q11 451 234 239 234 q12 402 227 225 225 q13 18647 302430333024 q14 289 246 246 246 q15 519 507 516 507 q16 626 524 485 485 q17 1002727 684 684 q18 7354697168606860 q19 1398102710231023 q20 703 338 325 325 q21 4252314132603141 q22 1123103210271027 Total cold run time: 110922 ms Total hot run time: 38280 ms - Round 2, with runtime_filter_mode=off - q1 4435437843784378 q2 390 266 272 266 q3 2912272726962696 q4 1957172116551655 q5 5727576258165762 q6 242 137 141 137 q7 2203187718191819 q8 3344349435433494 q9 9061908391199083 q10 3713346133423342 q11 643 513 533 513 q12 913 718 708 708 q13 15325 325931083108 q14 315 291 289 289 q15 546 479 498 479 q16 598 607 571 571 q17 1845163615421542 q18 8301794677927792 q19 1770161515721572 q20 2154190419011901 q21 5673555055795550 q22 1170107610891076 Total cold run time: 73237 ms Total hot run time: 57733 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [refactor](opt) improve BE code readability of multi_match_any function [doris]
doris-robot commented on PR #39354: URL: https://github.com/apache/doris/pull/39354#issuecomment-2323157565 TPC-DS: Total hot run time: 193469 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit b299e958fe235bf8c87aee40dfa43c793e96fd6d, data reload: false query1 1269890 862 862 query2 6294210619981998 query3 10749 404739933993 query4 60115 26878 23200 23200 query5 5417505 486 486 query6 396 164 162 162 query7 5781291 299 291 query8 292 217 212 212 query9 9200250524922492 query10 499 280 264 264 query11 17997 15158 15468 15158 query12 156 104 99 99 query13 1596392 371 371 query14 11027 744973067306 query15 242 176 181 176 query16 7526490 500 490 query17 1115564 568 564 query18 1892294 295 294 query19 285 148 175 148 query20 129 119 109 109 query21 210 104 103 103 query22 4748450844524452 query23 34460 33725 33580 33580 query24 5955284328262826 query25 522 378 383 378 query26 685 151 151 151 query27 1801277 276 276 query28 3810210721162107 query29 664 399 402 399 query30 239 148 147 147 query31 945 763 773 763 query32 86 54 63 54 query33 469 279 281 279 query34 850 473 468 468 query35 830 726 704 704 query36 1034950 940 940 query37 140 92 93 92 query38 3923385139373851 query39 1447141213701370 query40 200 115 113 113 query41 47 46 44 44 query42 121 96 99 96 query43 529 466 472 466 query44 1077738 740 738 query45 199 167 166 166 query46 1079747 754 747 query47 1922183017891789 query48 362 308 291 291 query49 753 421 440 421 query50 811 417 410 410 query51 7186692970586929 query52 98 86 88 86 query53 256 178 181 178 query54 564 455 458 455 query55 78 74 74 74 query56 273 274 273 273 query57 1200105610921056 query58 237 259 230 230 query59 2975280828802808 query60 302 280 283 280 query61 125 166 97 97 query62 766 662 666 662 query63 217 180 191 180 query64 2813665 658 658 query65 3201316431853164 query66 687 343 340 340 query67 15325 15268 15317 15268 query68 2965574 571 571 query69 398 277 277 277 query70 1181110411491104 query71 356 272 275 272 query72 5637413140514051 query73 744 324 321 321 query74 9036890287998799 query75 3340270326992699 query76 1424100310031003 query77 550 313 307 307 query78 10879 965291469146 query79 1060533 532 532 query80 683 500 495 495 query81 525 242 234 234 query82 255 165 155 155 query83 166 146 148 146 query84 262 75 76 75 query85 680 286 283 283 query86 315 303 309 303 query87 4353432542744274 query88 3041231822942294 query89 385 282 280 280 query90 1748191 188 188 query91 122 98 100 98 query92 58 52 50 50 query93 1044529 529 529 query94 554 305 300 300 query95 340 256 254 254 query96 580 271 267 267 query97 3184309930613061 query98 214 200 223 200 query99 1692128312671267 Total cold run time: 305969 ms Total hot run time: 193469 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t
Re: [PR] [chore] Reduce range of nextId when calling advanceNextId() [doris]
doris-robot commented on PR #40160: URL: https://github.com/apache/doris/pull/40160#issuecomment-2323018036 TPC-H: Total hot run time: 37849 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit e4942f8cc3e207151e0d875072a6add305ffa738, data reload: false -- Round 1 -- q1 17612 440943304330 q2 2740179 182 179 q3 11619 961 1066961 q4 10506 812 731 731 q5 7774279828142798 q6 222 137 137 137 q7 971 619 594 594 q8 9372203320932033 q9 7759650365596503 q10 7095219422002194 q11 442 240 240 240 q12 410 223 225 223 q13 18760 305530373037 q14 280 236 245 236 q15 508 479 492 479 q16 583 509 496 496 q17 997 672 671 671 q18 7329669369346693 q19 1391102910801029 q20 680 329 333 329 q21 3902294229772942 q22 1091101410231014 Total cold run time: 112043 ms Total hot run time: 37849 ms - Round 2, with runtime_filter_mode=off - q1 4394435143174317 q2 373 268 271 268 q3 2870266626962666 q4 1912167017571670 q5 5679562358185623 q6 234 134 135 134 q7 2184185118151815 q8 3252343634993436 q9 8871887988338833 q10 3645342133433343 q11 594 515 514 514 q12 856 663 670 663 q13 13259 301530513015 q14 313 277 266 266 q15 518 469 482 469 q16 609 581 565 565 q17 1814157215231523 q18 7920787377837783 q19 1736153516321535 q20 2129191019271910 q21 577254985498 q22 1140110010941094 Total cold run time: 70074 ms Total hot run time: 56940 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [opt](scan) merge small tablets into one scanner [doris]
doris-robot commented on PR #40133: URL: https://github.com/apache/doris/pull/40133#issuecomment-2322888213 ClickBench: Total hot run time: 32.26 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit 41227be09db6d40a025cd4bbf85bdd56445f3098, data reload: false query1 0.040.040.05 query2 0.070.040.04 query3 0.230.050.05 query4 1.670.080.07 query5 0.530.480.48 query6 1.130.740.72 query7 0.010.010.01 query8 0.050.050.05 query9 0.560.480.49 query10 0.540.550.53 query11 0.150.120.12 query12 0.160.120.12 query13 0.610.580.59 query14 2.042.082.06 query15 0.890.820.83 query16 0.380.370.38 query17 1.021.061.06 query18 0.210.200.20 query19 1.811.791.73 query20 0.010.010.01 query21 15.42 0.650.65 query22 4.657.212.07 query23 18.23 1.371.28 query24 2.050.230.22 query25 0.150.080.08 query26 0.270.170.18 query27 0.080.080.08 query28 13.34 1.020.98 query29 12.61 3.363.32 query30 0.240.060.06 query31 2.870.410.40 query32 3.230.480.47 query33 2.962.993.02 query34 16.90 4.414.44 query35 4.464.504.47 query36 0.670.500.47 query37 0.180.150.16 query38 0.160.150.15 query39 0.050.040.04 query40 0.170.130.13 query41 0.100.050.04 query42 0.060.050.05 query43 0.040.040.04 Total cold run time: 111 s Total hot run time: 32.26 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improvement](fdb) add a tool which deploys and starts fdb [doris]
github-actions[bot] commented on PR #39803:
URL: https://github.com/apache/doris/pull/39803#issuecomment-2322934720
`sh-checker report`
To get the full details, please check in the
[job]("https://github.com/apache/doris/actions/runs/10646383192";) output.
shellcheck errors
```
'shellcheck ' returned error 1 finding the following syntactical issues:
--
In tools/fdb/fdb_vars.sh line 1:
# Licensed to the Apache Software Foundation (ASF) under one
^-- SC2148 (error): Tips depend on target shell and yours is unknown. Add a
shebang or a 'shell' directive.
In tools/fdb/fdb_vars.sh line 25:
DATA_DIRS="/mnt/foundationdb/data1,/mnt/foundationdb/data2,/mnt/foundationdb/data3"
^---^ SC2034 (warning): DATA_DIRS appears unused. Verify use (or export
if used externally).
In tools/fdb/fdb_vars.sh line 33:
FDB_CLUSTER_IPS="172.200.0.2,172.200.0.3,172.200.0.4"
^-^ SC2034 (warning): FDB_CLUSTER_IPS appears unused. Verify use
(or export if used externally).
In tools/fdb/fdb_vars.sh line 38:
FDB_HOME="/fdbhome"
^--^ SC2034 (warning): FDB_HOME appears unused. Verify use (or export if
used externally).
In tools/fdb/fdb_vars.sh line 42:
FDB_CLUSTER_ID="SAQESzbh"
^^ SC2034 (warning): FDB_CLUSTER_ID appears unused. Verify use
(or export if used externally).
In tools/fdb/fdb_vars.sh line 45:
FDB_CLUSTER_DESC="mycluster"
^--^ SC2034 (warning): FDB_CLUSTER_DESC appears unused. Verify
use (or export if used externally).
In tools/fdb/fdb_vars.sh line 50:
MEMORY_LIMIT_GB=16
^-^ SC2034 (warning): MEMORY_LIMIT_GB appears unused. Verify use
(or export if used externally).
In tools/fdb/fdb_vars.sh line 53:
CPU_CORES_LIMIT=8
^-^ SC2034 (warning): CPU_CORES_LIMIT appears unused. Verify use
(or export if used externally).
In tools/fdb/fdb_vars.sh line 58:
FDB_PORT=4500
^--^ SC2034 (warning): FDB_PORT appears unused. Verify use (or export if
used externally).
In tools/fdb/fdb_vars.sh line 61:
FDB_VERSION="7.1.38"
^-^ SC2034 (warning): FDB_VERSION appears unused. Verify use (or
export if used externally).
In tools/fdb/fdb_vars.sh line 64:
USER=`whoami`
^--^ SC2006 (style): Use $(...) notation instead of legacy
backticks `...`.
Did you mean:
USER=$(whoami)
For more information:
https://www.shellcheck.net/wiki/SC2148 -- Tips depend on target shell and
y...
https://www.shellcheck.net/wiki/SC2034 -- CPU_CORES_LIMIT appears unused.
V...
https://www.shellcheck.net/wiki/SC2006 -- Use $(...) notation instead of
le...
--
You can address the above issues in one of three ways:
1. Manually correct the issue in the offending shell script;
2. Disable specific issues by adding the comment:
# shellcheck disable=
above the line that contains the issue, where is the error code;
3. Add '-e ' to the SHELLCHECK_OPTS setting in your .yml action file.
```
shfmt errors
```
'shfmt ' returned error 1 finding the following formatting issues:
--
--- tools/fdb/fdb_ctr.sh.orig
+++ tools/fdb/fdb_ctr.sh
@@ -142,7 +142,7 @@
local data_dir_count
# Convert comma-separated DATA_DIRS into an array
-IFS=',' read -r -a DATA_DIR_ARRAY <<< "${DATA_DIRS}"
+IFS=',' read -r -a DATA_DIR_ARRAY <<<"${DATA_DIRS}"
data_dir_count=${#DATA_DIR_ARRAY[@]}
# Stateless processes (at least 1, up to 1/4 of CPU cores)
@@ -152,13 +152,13 @@
# Storage processes (must be a multiple of the number of data
directories)
local storage_processes=$((cpu_cores / 4))
[[ ${storage_processes} -lt ${data_dir_count} ]] &&
storage_processes=${data_dir_count}
-storage_processes=$(( (storage_processes / data_dir_count) *
data_dir_count ))
+storage_processes=$(((storage_processes / data_dir_count) *
data_dir_count))
# Transaction processes (must be a multiple of the number of data
directories)
local transaction_processes=$((cpu_cores / 8))
[[ ${transaction_processes} -lt ${min_processes} ]] &&
transaction_processes=${min_processes}
[[ ${transaction_processes} -lt ${data_dir_count} ]] &&
transaction_processes=${data_dir_count}
-transaction_processes=$(( (transaction_processes / data_dir_count) *
data_dir_count ))
+transaction_processes=$(((transaction_processes / data_dir_count) *
data_dir_count))
# Return the values
echo "${stateless_processes} ${storage_processes}
${transaction_processes}"
@@ -175,9 +175,9 @@
CLUSTER_DESC="${FDB_CLUSTER_DESC:-${FDB_CLUSTER_ID}}"
# Convert comma-separated DATA_DIRS into an array
-IFS=',' read -r -a
[PR] [fix](regression) fix regression failure of window_funnel caused by duplicated table name [doris]
jacktengg opened a new pull request, #40217: URL: https://github.com/apache/doris/pull/40217 ## Proposed changes Issue Number: close #xxx -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](regression) fix regression failure of window_funnel caused by duplicated table name [doris]
github-actions[bot] commented on PR #40217: URL: https://github.com/apache/doris/pull/40217#issuecomment-2322890308 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Refactor](inverted index) refactor inverted index compound predicates evaluate logic [doris]
github-actions[bot] commented on code in PR #38908:
URL: https://github.com/apache/doris/pull/38908#discussion_r1739965507
##
be/src/vec/exprs/vexpr.cpp:
##
@@ -611,80 +611,134 @@ Status VExpr::get_result_from_const(vectorized::Block*
block, const std::string&
return Status::OK();
}
-bool VExpr::fast_execute(Block& block, const ColumnNumbers& arguments, size_t
result,
- size_t input_rows_count, const std::string&
function_name) {
-if (!_enable_inverted_index_query) {
-return false;
+Status VExpr::_evaluate_inverted_index(VExprContext* context, const
FunctionBasePtr& function,
Review Comment:
warning: function '_evaluate_inverted_index' exceeds recommended
size/complexity thresholds [readability-function-size]
```cpp
Status VExpr::_evaluate_inverted_index(VExprContext* context, const
FunctionBasePtr& function,
^
```
Additional context
**be/src/vec/exprs/vexpr.cpp:613:** 106 lines including whitespace and
comments (threshold 80)
```cpp
Status VExpr::_evaluate_inverted_index(VExprContext* context, const
FunctionBasePtr& function,
^
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Enhancement](vec) Remove duplicated code in BlockReader [doris]
yiguolei commented on PR #39784: URL: https://github.com/apache/doris/pull/39784#issuecomment-2322845243 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [chore](cloud) Support starting both meta-service and recycler within single process [doris]
doris-robot commented on PR #40223: URL: https://github.com/apache/doris/pull/40223#issuecomment-2322966654 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+your+PR) Since 2024-03-18, the Document has been moved to [doris-website](https://github.com/apache/doris-website). See [Doris Document](https://cwiki.apache.org/confluence/display/DORIS/Doris+Document). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [refactor](opt) improve BE code readability of multi_match_any function [doris]
superdiaodiao commented on PR #39354: URL: https://github.com/apache/doris/pull/39354#issuecomment-2322842476 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [debug](load) find invalid load memory allocation [doris]
yiguolei commented on PR #40220: URL: https://github.com/apache/doris/pull/40220#issuecomment-2322929915 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [I] 采用with 写法，不能用insert，只能将源表select语句拼接在insert后面才能执行。脚本写的非常不优雅 [doris-website]
13057537919 commented on issue #1071: URL: https://github.com/apache/doris-website/issues/1071#issuecomment-2322845795 这么写才能成功，测试出with写法不支持与insert联用 INSERT into dexpdb.ods_asystem_mysql_goods_baseinfo select * from titandb_32.titandb.goods_baseinfo where id BETWEEN (select breakpoint from dexpdb.dexp_access_breakpoint where source_table_name = 'titandb_32.titandb.goods_baseinfo' and sink_table_name ='dexpdb.ods_asystem_mysql_goods_baseinfo' order by ts desc limit 1) and (select max(id) as currentpoint from titandb_32.titandb.goods_baseinfo ); -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](regression) fix regression failure of window_funnel caused by duplicated table name [doris]
jacktengg commented on PR #40217: URL: https://github.com/apache/doris/pull/40217#issuecomment-2322861029 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [chore] Reduce range of nextId when calling advanceNextId() [doris]
doris-robot commented on PR #40160: URL: https://github.com/apache/doris/pull/40160#issuecomment-2323022195 ClickBench: Total hot run time: 32.3 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools ClickBench test result on commit e4942f8cc3e207151e0d875072a6add305ffa738, data reload: false query1 0.050.040.04 query2 0.080.040.04 query3 0.230.050.04 query4 1.660.080.07 query5 0.510.500.49 query6 1.130.740.73 query7 0.020.010.01 query8 0.050.050.05 query9 0.550.490.49 query10 0.550.540.53 query11 0.160.110.12 query12 0.160.130.13 query13 0.620.580.58 query14 2.042.052.10 query15 0.840.810.82 query16 0.380.370.37 query17 1.010.960.98 query18 0.220.200.20 query19 1.811.821.73 query20 0.010.000.01 query21 15.38 0.650.64 query22 3.686.832.37 query23 18.26 1.441.37 query24 2.090.230.21 query25 0.140.080.08 query26 0.260.180.19 query27 0.080.070.07 query28 13.24 1.020.99 query29 12.68 3.353.32 query30 0.240.050.05 query31 2.870.420.39 query32 3.230.490.48 query33 2.992.963.01 query34 17.03 4.414.31 query35 4.424.394.46 query36 0.660.480.47 query37 0.180.160.15 query38 0.160.150.14 query39 0.050.040.05 query40 0.150.130.12 query41 0.090.050.05 query42 0.050.050.05 query43 0.050.040.04 Total cold run time: 110.06 s Total hot run time: 32.3 s ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [chore](recycler) Change tablet_rowset_stale_sweep_time_sec=600 to reduce possibility failed to find rowset [doris]
gavinchou commented on PR #40076: URL: https://github.com/apache/doris/pull/40076#issuecomment-2323160797 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [fix](recycler) Implement missing delete_prefix() of hdfs accessor [doris]
doris-robot commented on PR #40215: URL: https://github.com/apache/doris/pull/40215#issuecomment-2322843768 TPC-DS: Total hot run time: 188676 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit 21db926afb4516aa8ef28db219082598ace4e50e, data reload: false query1 934 375 364 364 query2 6467210219941994 query3 6651206 215 206 query4 34489 23151 23250 23151 query5 4180510 484 484 query6 253 172 177 172 query7 4590314 292 292 query8 258 217 215 215 query9 8484249924922492 query10 434 275 260 260 query11 15423 15309 15295 15295 query12 150 101 98 98 query13 1628378 370 370 query14 10168 725271927192 query15 262 169 189 169 query16 7913442 451 442 query17 1581593 541 541 query18 1971284 284 284 query19 229 143 142 142 query20 116 110 112 110 query21 212 105 102 102 query22 4348411743284117 query23 34180 33719 33454 33454 query24 11131 281828332818 query25 625 378 382 378 query26 1201161 160 160 query27 2347283 281 281 query28 7421214521102110 query29 808 415 407 407 query30 310 151 153 151 query31 987 757 790 757 query32 104 58 61 58 query33 788 286 284 284 query34 991 468 500 468 query35 855 760 718 718 query36 1093920 933 920 query37 154 90 100 90 query38 3984392839543928 query39 1481136513781365 query40 205 117 117 117 query41 48 44 44 44 query42 113 94 99 94 query43 513 488 489 488 query44 1272762 741 741 query45 210 169 169 169 query46 1101718 737 718 query47 1888179217911791 query48 384 283 285 283 query49 1085438 430 430 query50 805 405 419 405 query51 7189705569916991 query52 100 88 87 87 query53 272 191 185 185 query54 995 494 466 466 query55 77 76 76 76 query56 276 267 262 262 query57 1198103710881037 query58 230 226 253 226 query59 3044286728182818 query60 296 266 259 259 query61 100 98 104 98 query62 836 667 678 667 query63 229 198 181 181 query64 4175746 666 666 query65 3243315431613154 query66 1416337 339 337 query67 15748 15241 15197 15197 query68 3119579 587 579 query69 404 280 270 270 query70 1193113611421136 query71 334 273 268 268 query72 6337402639743974 query73 745 326 337 326 query74 9122887088948870 query75 3403268627072686 query76 1882103510031003 query77 472 321 319 319 query78 9626908295389082 query79 1043540 594 540 query80 715 509 509 509 query81 520 247 241 241 query82 247 158 148 148 query83 174 151 154 151 query84 226 77 75 75 query85 887 290 282 282 query86 308 295 300 295 query87 4413432742914291 query88 3186231422992299 query89 375 299 288 288 query90 1816197 196 196 query91 128 102 157 102 query92 59 51 52 51 query93 1029547 532 532 query94 590 292 295 292 query95 350 264 259 259 query96 588 269 271 269 query97 3200310830313031 query98 222 202 203 202 query99 1480128412951284 Total cold run time: 284442 ms Total hot run time: 188676 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t
Re: [PR] [chore](cloud) Support starting both meta-service and recycler within single process [doris]
github-actions[bot] commented on PR #40223:
URL: https://github.com/apache/doris/pull/40223#issuecomment-2322966909
`sh-checker report`
To get the full details, please check in the
[job]("https://github.com/apache/doris/actions/runs/10646853238";) output.
shellcheck errors
```
'shellcheck ' found no issues.
```
shfmt errors
```
'shfmt ' returned error 1 finding the following formatting issues:
--
--- cloud/script/start.sh.orig
+++ cloud/script/start.sh
@@ -134,9 +134,9 @@
out_file=${DORIS_HOME}/log/${process}.out
if [[ "${RUN_DAEMON}" -eq 1 ]]; then
# append 10 blank lines to ensure the following tail -n10 works
correctly
-printf "\n\n\n\n\n\n\n\n\n\n" >> "${out_file}"
-echo "$(date +'%F %T') try to start ${process}" >> "${out_file}"
-nohup "${bin}" "$@" >> "${out_file}" 2>&1 &
+printf "\n\n\n\n\n\n\n\n\n\n" >>"${out_file}"
+echo "$(date +'%F %T') try to start ${process}" >>"${out_file}"
+nohup "${bin}" "$@" >>"${out_file}" 2>&1 &
echo "wait and check ${process} start successfully"
sleep 3
tail -n10 "${out_file}" | grep 'successfully started brpc'
--
You can reformat the above files to meet shfmt's requirements by typing:
shfmt -w filename
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [Refactor](inverted index) refactor inverted index compound predicates evaluate logic [doris]
doris-robot commented on PR #38908: URL: https://github.com/apache/doris/pull/38908#issuecomment-2323157797 TPC-DS: Total hot run time: 192663 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS sf100 test result on commit f62d014352a19a7beda85922fde3b034c1fb886c, data reload: false query1 1237900 885 885 query2 6343188318741874 query3 10620 392140833921 query4 60062 25652 23259 23259 query5 5429522 501 501 query6 397 163 159 159 query7 5781308 300 300 query8 284 200 211 200 query9 9072252725112511 query10 486 277 261 261 query11 17638 15022 15413 15022 query12 160 111 109 109 query13 1574410 377 377 query14 11466 716075757160 query15 242 181 201 181 query16 7504491 506 491 query17 1117585 560 560 query18 2013298 304 298 query19 279 150 149 149 query20 119 108 117 108 query21 205 108 103 103 query22 4517429345504293 query23 34502 33621 33249 33249 query24 5962282128132813 query25 526 366 377 366 query26 691 161 158 158 query27 1771277 281 277 query28 3784212320742074 query29 674 403 409 403 query30 230 155 158 155 query31 946 766 789 766 query32 78 54 56 54 query33 426 291 283 283 query34 875 483 467 467 query35 836 733 698 698 query36 1049948 910 910 query37 142 98 90 90 query38 3925380839333808 query39 1447138413851384 query40 199 117 112 112 query41 49 43 45 43 query42 110 96 93 93 query43 514 471 466 466 query44 1128750 750 750 query45 193 162 164 162 query46 1085737 736 736 query47 1935179418131794 query48 383 309 296 296 query49 784 440 442 440 query50 816 437 429 429 query51 7241715271067106 query52 103 93 95 93 query53 259 189 190 189 query54 602 473 468 468 query55 82 76 76 76 query56 288 275 271 271 query57 1201110110811081 query58 231 249 265 249 query59 3085282029432820 query60 301 274 300 274 query61 121 124 119 119 query62 766 638 637 637 query63 222 194 189 189 query64 2912741 802 741 query65 3182314631583146 query66 678 339 336 336 query67 15380 15319 15389 15319 query68 3354579 560 560 query69 393 280 280 280 query70 1181105010661050 query71 363 278 273 273 query72 6351402440374024 query73 744 330 332 330 query74 9240889088768876 query75 3377268227422682 query76 14881071964 964 query77 532 328 325 325 query78 9785894189958941 query79 1050541 529 529 query80 725 505 501 501 query81 555 232 231 231 query82 298 146 144 144 query83 172 152 151 151 query84 271 78 78 78 query85 720 285 284 284 query86 307 303 300 300 query87 4443423842194219 query88 3061239923312331 query89 377 284 286 284 query90 1950191 191 191 query91 124 98 95 95 query92 59 51 51 51 query93 1051535 536 535 query94 707 312 288 288 query95 345 252 255 252 query96 588 272 265 265 query97 3234306730643064 query98 216 206 202 202 query99 1576128312511251 Total cold run time: 306896 ms Total hot run time: 192663 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to
Re: [PR] [refactor](opt) improve BE code readability of multi_match_any function [doris]
superdiaodiao commented on code in PR #39354:
URL: https://github.com/apache/doris/pull/39354#discussion_r1739645930
##
be/src/vec/functions/functions_multi_string_search.cpp:
##
@@ -287,46 +311,30 @@ struct FunctionMultiMatchAnyImpl {
continue;
}
-multiregexps::DeferredConstructedRegexpsPtr
deferred_constructed_regexps =
-multiregexps::getOrSet(
-needles, std::nullopt);
-multiregexps::Regexps* regexps =
deferred_constructed_regexps->get();
-
-hs_scratch_t* scratch = nullptr;
-hs_error_t err = hs_clone_scratch(regexps->getScratch(), &scratch);
-
-if (err != HS_SUCCESS)
-return Status::InternalError("could not clone scratch space
for vectorscan");
-
-multiregexps::ScratchPtr smart_scratch(scratch);
-
-auto on_match = []([[maybe_unused]] unsigned int id,
- unsigned long long /* from */, // NOLINT
- unsigned long long /* to */, // NOLINT
- unsigned int /* flags */, void* context) -> int
{
-if constexpr (FindAnyIndex)
-*reinterpret_cast(context) = id;
-else if constexpr (FindAny)
-*reinterpret_cast(context) = 1;
-/// Once we hit the callback, there is no need to search for
others.
-return 1;
-};
+multiregexps::Regexps* regexps = nullptr;
+multiregexps::ScratchPtr smart_scratch;
+Status status = prepare_regexps_and_scratch(needles, regexps,
smart_scratch);
Review Comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [opt](scan) merge small tablets into one scanner [doris]
doris-robot commented on PR #40133: URL: https://github.com/apache/doris/pull/40133#issuecomment-2322884118 TPC-H: Total hot run time: 38486 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 41227be09db6d40a025cd4bbf85bdd56445f3098, data reload: false -- Round 1 -- q1 16725 513243824382 q2 1977179 176 176 q3 10427 112411761124 q4 10108 757 661 661 q5 7385294628232823 q6 228 138 137 137 q7 988 614 609 609 q8 7556212120702070 q9 6674660665756575 q10 6360222322112211 q11 401 245 248 245 q12 403 223 225 223 q13 13162 304430803044 q14 287 243 233 233 q15 535 482 475 475 q16 589 505 527 505 q17 1006676 730 676 q18 7475685768436843 q19 1267102010501020 q20 633 333 330 330 q21 3843312731023102 q22 1119103210221022 Total cold run time: 99148 ms Total hot run time: 38486 ms - Round 2, with runtime_filter_mode=off - q1 4454432143104310 q2 374 265 272 265 q3 2879265226872652 q4 1952171116851685 q5 5423544254305430 q6 226 133 134 133 q7 2081172317691723 q8 3210337833533353 q9 8415843483878387 q10 3416320231883188 q11 612 496 495 495 q12 786 624 603 603 q13 12582 300030353000 q14 310 266 283 266 q15 540 471 485 471 q16 605 574 563 563 q17 1818151514981498 q18 7703749875227498 q19 1687156914901490 q20 2061183518121812 q21 5427512352735123 q22 1130105510491049 Total cold run time: 67691 ms Total hot run time: 54994 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[PR] [debug](load) find invalid load memory allocation [doris]
yiguolei opened a new pull request, #40220: URL: https://github.com/apache/doris/pull/40220 ## Proposed changes Issue Number: close #xxx -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [refactor](opt) improve BE code readability of multi_match_any function [doris]
github-actions[bot] commented on code in PR #39354: URL: https://github.com/apache/doris/pull/39354#discussion_r1739646709 ## be/src/vec/functions/functions_multi_string_search.cpp: ## @@ -20,10 +20,10 @@ #include Review Comment: warning: 'hs/hs_common.h' file not found [clang-diagnostic-error] ```cpp #include ^ ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [chore](cloud) Improve cloud start stop script: wait for quit synchronously [doris]
gavinchou commented on PR #40218: URL: https://github.com/apache/doris/pull/40218#issuecomment-2322901933 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [debug](load) find invalid load memory allocation [doris]
github-actions[bot] commented on code in PR #40220:
URL: https://github.com/apache/doris/pull/40220#discussion_r1739948925
##
be/src/vec/exprs/vliteral.h:
##
@@ -43,6 +43,10 @@ class VLiteral : public VExpr {
}
}
+~VLiteral() {
Review Comment:
warning: use '= default' to define a trivial destructor
[modernize-use-equals-default]
```cpp
~VLiteral() {
^
```
##
be/src/vec/exprs/vliteral.h:
##
@@ -43,6 +43,10 @@
}
}
+~VLiteral() {
Review Comment:
warning: annotate this function with 'override' or (rarely) 'final'
[modernize-use-override]
```suggestion
~VLiteral() override {
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
(doris) branch branch-2.1 updated: [Bug](expr) execute expr should use local states instead of operators (#40189) (#40219)
This is an automated email from the ASF dual-hosted git repository.
yiguolei pushed a commit to branch branch-2.1
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-2.1 by this push:
new b104e933cd4 [Bug](expr) execute expr should use local states instead
of operators (#40189) (#40219)
b104e933cd4 is described below
commit b104e933cd46a742f7ccc380c1470735c7e5f2ed
Author: zhangstar333 <87313068+zhangstar...@users.noreply.github.com>
AuthorDate: Sun Sep 1 00:41:10 2024 +0800
[Bug](expr) execute expr should use local states instead of operators
(#40189) (#40219)
## Proposed changes
cherry-pick from master #40189
---
be/src/pipeline/exec/aggregation_source_operator.cpp| 3 ++-
be/src/pipeline/exec/assert_num_rows_operator.cpp | 3 ++-
be/src/pipeline/exec/assert_num_rows_operator.h | 3 +++
.../exec/distinct_streaming_aggregation_operator.cpp| 4 ++--
be/src/pipeline/exec/multi_cast_data_stream_source.h| 7 ---
be/src/pipeline/exec/repeat_operator.cpp| 2 +-
be/src/pipeline/exec/streaming_aggregation_operator.cpp | 4 ++--
regression-test/data/javaudf_p0/test_javaudf_string.out | 3 +++
.../suites/javaudf_p0/test_javaudf_string.groovy| 13 +
9 files changed, 32 insertions(+), 10 deletions(-)
diff --git a/be/src/pipeline/exec/aggregation_source_operator.cpp
b/be/src/pipeline/exec/aggregation_source_operator.cpp
index fadddee9034..8f19c24589b 100644
--- a/be/src/pipeline/exec/aggregation_source_operator.cpp
+++ b/be/src/pipeline/exec/aggregation_source_operator.cpp
@@ -435,7 +435,8 @@ Status AggSourceOperatorX::get_block(RuntimeState* state,
vectorized::Block* blo
RETURN_IF_ERROR(local_state._executor.get_result(state, block, eos));
local_state.make_nullable_output_key(block);
// dispose the having clause, should not be execute in prestreaming agg
-RETURN_IF_ERROR(vectorized::VExprContext::filter_block(_conjuncts, block,
block->columns()));
+
RETURN_IF_ERROR(vectorized::VExprContext::filter_block(local_state._conjuncts,
block,
+ block->columns()));
local_state.reached_limit(block, eos);
return Status::OK();
}
diff --git a/be/src/pipeline/exec/assert_num_rows_operator.cpp
b/be/src/pipeline/exec/assert_num_rows_operator.cpp
index ef0efd3f86b..8f86e3ecb87 100644
--- a/be/src/pipeline/exec/assert_num_rows_operator.cpp
+++ b/be/src/pipeline/exec/assert_num_rows_operator.cpp
@@ -120,7 +120,8 @@ Status AssertNumRowsOperatorX::pull(doris::RuntimeState*
state, vectorized::Bloc
}
COUNTER_SET(local_state.rows_returned_counter(),
local_state.num_rows_returned());
COUNTER_UPDATE(local_state.blocks_returned_counter(), 1);
-RETURN_IF_ERROR(vectorized::VExprContext::filter_block(_conjuncts, block,
block->columns()));
+
RETURN_IF_ERROR(vectorized::VExprContext::filter_block(local_state._conjuncts,
block,
+ block->columns()));
return Status::OK();
}
diff --git a/be/src/pipeline/exec/assert_num_rows_operator.h
b/be/src/pipeline/exec/assert_num_rows_operator.h
index 4d6d835f815..3874c1fc771 100644
--- a/be/src/pipeline/exec/assert_num_rows_operator.h
+++ b/be/src/pipeline/exec/assert_num_rows_operator.h
@@ -46,6 +46,9 @@ public:
AssertNumRowsLocalState(RuntimeState* state, OperatorXBase* parent)
: PipelineXLocalState(state, parent) {}
~AssertNumRowsLocalState() = default;
+
+private:
+friend class AssertNumRowsOperatorX;
};
class AssertNumRowsOperatorX final : public
StreamingOperatorX {
diff --git a/be/src/pipeline/exec/distinct_streaming_aggregation_operator.cpp
b/be/src/pipeline/exec/distinct_streaming_aggregation_operator.cpp
index 16c0df07b49..1635d927b6d 100644
--- a/be/src/pipeline/exec/distinct_streaming_aggregation_operator.cpp
+++ b/be/src/pipeline/exec/distinct_streaming_aggregation_operator.cpp
@@ -515,8 +515,8 @@ Status DistinctStreamingAggOperatorX::pull(RuntimeState*
state, vectorized::Bloc
local_state._make_nullable_output_key(block);
if (!_is_streaming_preagg) {
// dispose the having clause, should not be execute in prestreaming agg
-RETURN_IF_ERROR(
-vectorized::VExprContext::filter_block(_conjuncts, block,
block->columns()));
+
RETURN_IF_ERROR(vectorized::VExprContext::filter_block(local_state._conjuncts,
block,
+
block->columns()));
}
local_state.add_num_rows_returned(block->rows());
COUNTER_UPDATE(local_state.blocks_returned_counter(), 1);
diff --git a/be/src/pipeline/exec/multi_cast_data_stream_source.h
b/be/src/pipeline/exec/multi_cast_data_stream_source.h
index 761e899c3d1..aab5cb96dbe 100644
--- a/be/src/pipeline/exec/multi_cast_data_stream
(doris) branch master updated: [enhancement](memory) add exception logic in write page and schema scanner (#40168)
This is an automated email from the ASF dual-hosted git repository.
yiguolei pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 352be298ac0 [enhancement](memory) add exception logic in write page
and schema scanner (#40168)
352be298ac0 is described below
commit 352be298ac048e96cb336ce917f03f1202579d6e
Author: yiguolei <676222...@qq.com>
AuthorDate: Sat Aug 31 15:10:41 2024 +0800
[enhancement](memory) add exception logic in write page and schema scanner
(#40168)
## Proposed changes
Issue Number: close #xxx
-
Co-authored-by: yiguolei
---
be/src/exec/schema_scanner.cpp | 6 +-
be/src/olap/rowset/segment_v2/column_writer.cpp | 3 ++-
be/src/olap/rowset/segment_v2/column_writer.h | 8 +++-
3 files changed, 14 insertions(+), 3 deletions(-)
diff --git a/be/src/exec/schema_scanner.cpp b/be/src/exec/schema_scanner.cpp
index be0bd8eff72..b5cd484b650 100644
--- a/be/src/exec/schema_scanner.cpp
+++ b/be/src/exec/schema_scanner.cpp
@@ -134,7 +134,11 @@ Status SchemaScanner::get_next_block_async(RuntimeState*
state) {
_opened = true;
}
bool eos = false;
-
_scanner_status.update(get_next_block_internal(_data_block.get(), &eos));
+auto call_next_block_internal = [&]() -> Status {
+RETURN_IF_CATCH_EXCEPTION(
+{ return
get_next_block_internal(_data_block.get(), &eos); });
+};
+_scanner_status.update(call_next_block_internal());
_eos = eos;
_async_thread_running = false;
_dependency->set_ready();
diff --git a/be/src/olap/rowset/segment_v2/column_writer.cpp
b/be/src/olap/rowset/segment_v2/column_writer.cpp
index 75acd9f1f95..2637017b78d 100644
--- a/be/src/olap/rowset/segment_v2/column_writer.cpp
+++ b/be/src/olap/rowset/segment_v2/column_writer.cpp
@@ -534,7 +534,8 @@ Status ScalarColumnWriter::append_data(const uint8_t** ptr,
size_t num_rows) {
return Status::OK();
}
-Status ScalarColumnWriter::append_data_in_current_page(const uint8_t* data,
size_t* num_written) {
+Status ScalarColumnWriter::_internal_append_data_in_current_page(const
uint8_t* data,
+ size_t*
num_written) {
RETURN_IF_ERROR(_page_builder->add(data, num_written));
if (_opts.need_zone_map) {
_zone_map_index_builder->add_values(data, *num_written);
diff --git a/be/src/olap/rowset/segment_v2/column_writer.h
b/be/src/olap/rowset/segment_v2/column_writer.h
index acda2b75fb2..62f209db5ad 100644
--- a/be/src/olap/rowset/segment_v2/column_writer.h
+++ b/be/src/olap/rowset/segment_v2/column_writer.h
@@ -217,10 +217,16 @@ public:
// used for append not null data. When page is full, will append data not
reach num_rows.
Status append_data_in_current_page(const uint8_t** ptr, size_t*
num_written);
-Status append_data_in_current_page(const uint8_t* ptr, size_t*
num_written);
+Status append_data_in_current_page(const uint8_t* ptr, size_t*
num_written) {
+RETURN_IF_CATCH_EXCEPTION(
+{ return _internal_append_data_in_current_page(ptr,
num_written); });
+}
friend class ArrayColumnWriter;
friend class OffsetColumnWriter;
+private:
+Status _internal_append_data_in_current_page(const uint8_t* ptr, size_t*
num_written);
+
private:
std::unique_ptr _page_builder;
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [debug](load) find invalid load memory allocation [doris]
doris-robot commented on PR #40220: URL: https://github.com/apache/doris/pull/40220#issuecomment-2323129355 TPC-H: Total hot run time: 38010 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100 test result on commit 9113c64f10b2edae861fcdb054194243ee129899, data reload: false -- Round 1 -- q1 17633 439143274327 q2 2021188 179 179 q3 11758 950 1099950 q4 10510 741 761 741 q5 7745288527932793 q6 227 136 136 136 q7 955 614 594 594 q8 9313205720662057 q9 7306662865836583 q10 7005219721642164 q11 444 243 242 242 q12 398 218 219 218 q13 18088 300530563005 q14 268 231 233 231 q15 530 500 493 493 q16 594 508 516 508 q17 966 738 725 725 q18 7352679768416797 q19 13921090981 981 q20 676 338 344 338 q21 3932302529272927 q22 1133104110211021 Total cold run time: 110246 ms Total hot run time: 38010 ms - Round 2, with runtime_filter_mode=off - q1 4351435242804280 q2 385 269 272 269 q3 2881268026712671 q4 1917163416741634 q5 5571569156925691 q6 231 143 143 143 q7 2202180218251802 q8 342234073407 q9 8796878988658789 q10 3591338833733373 q11 615 510 518 510 q12 831 654 689 654 q13 15034 313732203137 q14 327 284 310 284 q15 541 497 486 486 q16 640 563 596 563 q17 1829155515421542 q18 8173784178777841 q19 1718160916011601 q20 2166189418861886 q21 5706549455065494 q22 1109105010611050 Total cold run time: 71947 ms Total hot run time: 57107 ms ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [improvement](fdb) add a tool which deploys and starts fdb [doris]
github-actions[bot] commented on PR #39803:
URL: https://github.com/apache/doris/pull/39803#issuecomment-2322928974
`sh-checker report`
To get the full details, please check in the
[job]("https://github.com/apache/doris/actions/runs/10646279310";) output.
shellcheck errors
```
'shellcheck ' returned error 1 finding the following syntactical issues:
--
In tools/fdb/fdb_ctr.sh line 305:
rm -rf "${FDB_HOME}"/*
^-^ SC2115 (warning): Use "${var:?}" to ensure this
never expands to /* .
In tools/fdb/fdb_ctr.sh line 379:
echo "Unknown cmd: ${cmd}\n"
^-^ SC2028 (info): echo may not expand escape
sequences. Use printf.
In tools/fdb/fdb_vars.sh line 1:
# Licensed to the Apache Software Foundation (ASF) under one
^-- SC2148 (error): Tips depend on target shell and yours is unknown. Add a
shebang or a 'shell' directive.
In tools/fdb/fdb_vars.sh line 25:
DATA_DIRS="/mnt/foundationdb/data1,/mnt/foundationdb/data2,/mnt/foundationdb/data3"
^---^ SC2034 (warning): DATA_DIRS appears unused. Verify use (or export
if used externally).
In tools/fdb/fdb_vars.sh line 33:
FDB_CLUSTER_IPS="172.200.0.2,172.200.0.3,172.200.0.4"
^-^ SC2034 (warning): FDB_CLUSTER_IPS appears unused. Verify use
(or export if used externally).
In tools/fdb/fdb_vars.sh line 38:
FDB_HOME="/fdbhome"
^--^ SC2034 (warning): FDB_HOME appears unused. Verify use (or export if
used externally).
In tools/fdb/fdb_vars.sh line 42:
FDB_CLUSTER_ID="SAQESzbh"
^^ SC2034 (warning): FDB_CLUSTER_ID appears unused. Verify use
(or export if used externally).
In tools/fdb/fdb_vars.sh line 45:
FDB_CLUSTER_DESC="mycluster"
^--^ SC2034 (warning): FDB_CLUSTER_DESC appears unused. Verify
use (or export if used externally).
In tools/fdb/fdb_vars.sh line 50:
MEMORY_LIMIT_GB=16
^-^ SC2034 (warning): MEMORY_LIMIT_GB appears unused. Verify use
(or export if used externally).
In tools/fdb/fdb_vars.sh line 53:
CPU_CORES_LIMIT=8
^-^ SC2034 (warning): CPU_CORES_LIMIT appears unused. Verify use
(or export if used externally).
In tools/fdb/fdb_vars.sh line 58:
FDB_PORT=4500
^--^ SC2034 (warning): FDB_PORT appears unused. Verify use (or export if
used externally).
In tools/fdb/fdb_vars.sh line 61:
FDB_VERSION="7.1.38"
^-^ SC2034 (warning): FDB_VERSION appears unused. Verify use (or
export if used externally).
In tools/fdb/fdb_vars.sh line 64:
USER=`whoami`
^--^ SC2006 (style): Use $(...) notation instead of legacy
backticks `...`.
Did you mean:
USER=$(whoami)
For more information:
https://www.shellcheck.net/wiki/SC2148 -- Tips depend on target shell and
y...
https://www.shellcheck.net/wiki/SC2034 -- CPU_CORES_LIMIT appears unused.
V...
https://www.shellcheck.net/wiki/SC2115 -- Use "${var:?}" to ensure this
nev...
--
You can address the above issues in one of three ways:
1. Manually correct the issue in the offending shell script;
2. Disable specific issues by adding the comment:
# shellcheck disable=
above the line that contains the issue, where is the error code;
3. Add '-e ' to the SHELLCHECK_OPTS setting in your .yml action file.
```
shfmt errors
```
'shfmt ' returned error 1 finding the following formatting issues:
--
--- tools/fdb/fdb_ctr.sh.orig
+++ tools/fdb/fdb_ctr.sh
@@ -142,7 +142,7 @@
local data_dir_count
# Convert comma-separated DATA_DIRS into an array
-IFS=',' read -r -a DATA_DIR_ARRAY <<< "${DATA_DIRS}"
+IFS=',' read -r -a DATA_DIR_ARRAY <<<"${DATA_DIRS}"
data_dir_count=${#DATA_DIR_ARRAY[@]}
# Stateless processes (at least 1, up to 1/4 of CPU cores)
@@ -152,13 +152,13 @@
# Storage processes (must be a multiple of the number of data
directories)
local storage_processes=$((cpu_cores / 4))
[[ ${storage_processes} -lt ${data_dir_count} ]] &&
storage_processes=${data_dir_count}
-storage_processes=$(( (storage_processes / data_dir_count) *
data_dir_count ))
+storage_processes=$(((storage_processes / data_dir_count) *
data_dir_count))
# Transaction processes (must be a multiple of the number of data
directories)
local transaction_processes=$((cpu_cores / 8))
[[ ${transaction_processes} -lt ${min_processes} ]] &&
transaction_processes=${min_processes}
[[ ${transaction_processes} -lt ${data_dir_count} ]] &&
transaction_processes=${data_dir_count}
-transaction_processes=$(( (transaction_processes / data_dir_count) *
data_dir_count ))
+transaction_processe
Re: [PR] [test](mtmv) SSB mv rewrite test use little data set for test performance [doris]
github-actions[bot] commented on PR #40188: URL: https://github.com/apache/doris/pull/40188#issuecomment-2322931043 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [chore](cloud) Support starting both meta-service and recycler within single process [doris]
gavinchou commented on PR #40223: URL: https://github.com/apache/doris/pull/40223#issuecomment-2322978073 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
Re: [PR] [chore] Reduce range of nextId when calling advanceNextId() [doris]
gavinchou commented on PR #40160: URL: https://github.com/apache/doris/pull/40160#issuecomment-2323006484 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
