Re: [PR] [refactor](scanner) refactoring and optimizing scanner scheduling [doris]

2024-02-07 Thread via GitHub
yiguolei merged PR #30746: URL: https://github.com/apache/doris/pull/30746 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apa

Re: [PR] [refactor](scanner) refactoring and optimizing scanner scheduling [doris]

2024-02-07 Thread via GitHub
github-actions[bot] commented on PR #30746: URL: https://github.com/apache/doris/pull/30746#issuecomment-1931524836 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

Re: [PR] [refactor](scanner) refactoring and optimizing scanner scheduling [doris]

2024-02-06 Thread via GitHub
doris-robot commented on PR #30746: URL: https://github.com/apache/doris/pull/30746#issuecomment-1931405387 Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G' ``` Load test result on commit fbb7db4a66cc1d5990a824bf18d13d3a5ac57c1f with default session variables Stream

Re: [PR] [refactor](scanner) refactoring and optimizing scanner scheduling [doris]

2024-02-06 Thread via GitHub
doris-robot commented on PR #30746: URL: https://github.com/apache/doris/pull/30746#issuecomment-1931402265 ClickBench: Total hot run time: 31.13 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools

Re: [PR] [refactor](scanner) refactoring and optimizing scanner scheduling [doris]

2024-02-06 Thread via GitHub
doris-robot commented on PR #30746: URL: https://github.com/apache/doris/pull/30746#issuecomment-1931396986 TPC-DS: Total hot run time: 181015 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS

Re: [PR] [refactor](scanner) refactoring and optimizing scanner scheduling [doris]

2024-02-06 Thread via GitHub
doris-robot commented on PR #30746: URL: https://github.com/apache/doris/pull/30746#issuecomment-1931386524 TeamCity be ut coverage result: Function Coverage: 36.19% (8638/23871) Line Coverage: 28.23% (70616/250171) Region Coverage: 27.20% (36424/133899) Branch Coverage: 23.9

Re: [PR] [refactor](scanner) refactoring and optimizing scanner scheduling [doris]

2024-02-06 Thread via GitHub
doris-robot commented on PR #30746: URL: https://github.com/apache/doris/pull/30746#issuecomment-1931385481 TPC-H: Total hot run time: 41186 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100

Re: [PR] [refactor](scanner) refactoring and optimizing scanner scheduling [doris]

2024-02-06 Thread via GitHub
github-actions[bot] commented on code in PR #30746: URL: https://github.com/apache/doris/pull/30746#discussion_r1480947982 ## be/src/vec/exec/scan/scanner_context.h: ## @@ -81,88 +122,49 @@ virtual Status init(); vectorized::BlockUPtr get_free_block(); -void retu

Re: [PR] [refactor](scanner) refactoring and optimizing scanner scheduling [doris]

2024-02-06 Thread via GitHub
AshinGau commented on PR #30746: URL: https://github.com/apache/doris/pull/30746#issuecomment-1931357391 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To uns

Re: [PR] [refactor](scanner) refactoring and optimizing scanner scheduling [doris]

2024-02-06 Thread via GitHub
doris-robot commented on PR #30746: URL: https://github.com/apache/doris/pull/30746#issuecomment-1931264305 Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G' ``` Load test result on commit 41a298b8f5ca722d7e38e2ccf0eff6265218b168 with default session variables Stream

Re: [PR] [refactor](scanner) refactoring and optimizing scanner scheduling [doris]

2024-02-06 Thread via GitHub
doris-robot commented on PR #30746: URL: https://github.com/apache/doris/pull/30746#issuecomment-1931261943 ClickBench: Total hot run time: 31.19 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools

Re: [PR] [refactor](scanner) refactoring and optimizing scanner scheduling [doris]

2024-02-06 Thread via GitHub
doris-robot commented on PR #30746: URL: https://github.com/apache/doris/pull/30746#issuecomment-1931257586 TPC-DS: Total hot run time: 180061 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS

Re: [PR] [refactor](scanner) refactoring and optimizing scanner scheduling [doris]

2024-02-06 Thread via GitHub
doris-robot commented on PR #30746: URL: https://github.com/apache/doris/pull/30746#issuecomment-1931252386 TeamCity be ut coverage result: Function Coverage: 36.19% (8637/23864) Line Coverage: 28.24% (70613/250088) Region Coverage: 27.22% (36436/133851) Branch Coverage: 24.0

Re: [PR] [refactor](scanner) refactoring and optimizing scanner scheduling [doris]

2024-02-06 Thread via GitHub
doris-robot commented on PR #30746: URL: https://github.com/apache/doris/pull/30746#issuecomment-1931248932 TPC-H: Total hot run time: 41441 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100

Re: [PR] [refactor](scanner) refactoring and optimizing scanner scheduling [doris]

2024-02-06 Thread via GitHub
AshinGau commented on PR #30746: URL: https://github.com/apache/doris/pull/30746#issuecomment-1931218522 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To uns

Re: [PR] [refactor](scanner) refactoring and optimizing scanner scheduling [doris]

2024-02-06 Thread via GitHub
doris-robot commented on PR #30746: URL: https://github.com/apache/doris/pull/30746#issuecomment-1931186108 TeamCity be ut coverage result: Function Coverage: 36.18% (8636/23871) Line Coverage: 28.22% (70585/250161) Region Coverage: 27.20% (36419/133893) Branch Coverage: 23.9

Re: [PR] [refactor](scanner) refactoring and optimizing scanner scheduling [doris]

2024-02-06 Thread via GitHub
github-actions[bot] commented on code in PR #30746: URL: https://github.com/apache/doris/pull/30746#discussion_r1480819025 ## be/src/vec/exec/scan/scanner_scheduler.cpp: ## @@ -139,155 +117,97 @@ Status ScannerScheduler::init(ExecEnv* env) { return Status::OK(); } -Statu

Re: [PR] [refactor](scanner) refactoring and optimizing scanner scheduling [doris]

2024-02-06 Thread via GitHub
AshinGau commented on PR #30746: URL: https://github.com/apache/doris/pull/30746#issuecomment-1931146109 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To uns

Re: [PR] [refactor](scanner) refactoring and optimizing scanner scheduling [doris]

2024-02-06 Thread via GitHub
doris-robot commented on PR #30746: URL: https://github.com/apache/doris/pull/30746#issuecomment-1930431536 TeamCity be ut coverage result: Function Coverage: 36.21% (8636/23852) Line Coverage: 28.23% (70604/250074) Region Coverage: 27.23% (36432/133780) Branch Coverage: 24.0

Re: [PR] [refactor](scanner) refactoring and optimizing scanner scheduling [doris]

2024-02-06 Thread via GitHub
doris-robot commented on PR #30746: URL: https://github.com/apache/doris/pull/30746#issuecomment-1930400793 Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G' ``` Load test result on commit b162fea709fc72215e7b50ddc8683e4edf342015 with default session variables Stream

Re: [PR] [refactor](scanner) refactoring and optimizing scanner scheduling [doris]

2024-02-06 Thread via GitHub
doris-robot commented on PR #30746: URL: https://github.com/apache/doris/pull/30746#issuecomment-1930394704 ClickBench: Total hot run time: 30.09 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools

Re: [PR] [refactor](scanner) refactoring and optimizing scanner scheduling [doris]

2024-02-06 Thread via GitHub
doris-robot commented on PR #30746: URL: https://github.com/apache/doris/pull/30746#issuecomment-1930384795 TPC-DS: Total hot run time: 180988 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS

Re: [PR] [refactor](scanner) refactoring and optimizing scanner scheduling [doris]

2024-02-06 Thread via GitHub
doris-robot commented on PR #30746: URL: https://github.com/apache/doris/pull/30746#issuecomment-1930358906 TPC-H: Total hot run time: 41760 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100

Re: [PR] [refactor](scanner) refactoring and optimizing scanner scheduling [doris]

2024-02-06 Thread via GitHub
AshinGau commented on PR #30746: URL: https://github.com/apache/doris/pull/30746#issuecomment-1930150852 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To uns

Re: [PR] [refactor](scanner) refactoring and optimizing scanner scheduling [doris]

2024-02-06 Thread via GitHub
doris-robot commented on PR #30746: URL: https://github.com/apache/doris/pull/30746#issuecomment-1929375237 Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G' ``` Load test result on commit 26ece75481afd3b81d4b2617ad6c1b3a5589efc2 with default session variables Stream

Re: [PR] [refactor](scanner) refactoring and optimizing scanner scheduling [doris]

2024-02-06 Thread via GitHub
doris-robot commented on PR #30746: URL: https://github.com/apache/doris/pull/30746#issuecomment-1929370309 ClickBench: Total hot run time: 31.15 s ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools

Re: [PR] [refactor](scanner) refactoring and optimizing scanner scheduling [doris]

2024-02-06 Thread via GitHub
doris-robot commented on PR #30746: URL: https://github.com/apache/doris/pull/30746#issuecomment-1929361934 TPC-DS: Total hot run time: 181024 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools TPC-DS

Re: [PR] [refactor](scanner) refactoring and optimizing scanner scheduling [doris]

2024-02-06 Thread via GitHub
doris-robot commented on PR #30746: URL: https://github.com/apache/doris/pull/30746#issuecomment-1929345356 TPC-H: Total hot run time: 41574 ms ``` machine: 'aliyun_ecs.c7a.8xlarge_32C64G' scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools Tpch sf100

Re: [PR] [refactor](scanner) refactoring and optimizing scanner scheduling [doris]

2024-02-06 Thread via GitHub
doris-robot commented on PR #30746: URL: https://github.com/apache/doris/pull/30746#issuecomment-1929345245 TeamCity be ut coverage result: Function Coverage: 36.22% (8635/23841) Line Coverage: 28.23% (70574/249960) Region Coverage: 27.24% (36428/133713) Branch Coverage: 24.0

Re: [PR] [refactor](scanner) refactoring and optimizing scanner scheduling [doris]

2024-02-06 Thread via GitHub
AshinGau commented on PR #30746: URL: https://github.com/apache/doris/pull/30746#issuecomment-1929246126 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To uns

Re: [PR] [refactor](scanner) refactoring and optimizing scanner scheduling [doris]

2024-02-06 Thread via GitHub
AshinGau commented on PR #30746: URL: https://github.com/apache/doris/pull/30746#issuecomment-1929074596 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To uns

Re: [PR] [refactor](scanner) refactoring and optimizing scanner scheduling [doris]

2024-02-05 Thread via GitHub
doris-robot commented on PR #30746: URL: https://github.com/apache/doris/pull/30746#issuecomment-1928859492 TeamCity be ut coverage result: Function Coverage: 36.20% (8629/23834) Line Coverage: 28.24% (70572/249876) Region Coverage: 27.25% (36422/133675) Branch Coverage: 24.0

Re: [PR] [refactor](scanner) refactoring and optimizing scanner scheduling [doris]

2024-02-05 Thread via GitHub
github-actions[bot] commented on PR #30746: URL: https://github.com/apache/doris/pull/30746#issuecomment-1928840369 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the UR

Re: [PR] [refactor](scanner) refactoring and optimizing scanner scheduling [doris]

2024-02-05 Thread via GitHub
github-actions[bot] commented on PR #30746: URL: https://github.com/apache/doris/pull/30746#issuecomment-1928840320 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

Re: [PR] [refactor](scanner) refactoring and optimizing scanner scheduling [doris]

2024-02-05 Thread via GitHub
AshinGau commented on PR #30746: URL: https://github.com/apache/doris/pull/30746#issuecomment-1928783923 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To uns

Re: [PR] [refactor](scanner) refactoring and optimizing scanner scheduling [doris]

2024-02-05 Thread via GitHub
yiguolei commented on code in PR #30746: URL: https://github.com/apache/doris/pull/30746#discussion_r1479106081 ## be/src/vec/exec/scan/scanner_context.cpp: ## @@ -220,138 +162,224 @@ std::string ScannerContext::parent_name() { vectorized::BlockUPtr ScannerContext::get_free_blo

Re: [PR] [refactor](scanner) refactoring and optimizing scanner scheduling [doris]

2024-02-05 Thread via GitHub
yiguolei commented on code in PR #30746: URL: https://github.com/apache/doris/pull/30746#discussion_r1479106239 ## be/src/vec/exec/scan/scanner_context.cpp: ## @@ -220,138 +162,224 @@ std::string ScannerContext::parent_name() { vectorized::BlockUPtr ScannerContext::get_free_blo

Re: [PR] [refactor](scanner) refactoring and optimizing scanner scheduling [doris]

2024-02-05 Thread via GitHub
yiguolei commented on code in PR #30746: URL: https://github.com/apache/doris/pull/30746#discussion_r1479105367 ## be/src/vec/exec/scan/scanner_context.cpp: ## @@ -220,138 +162,224 @@ std::string ScannerContext::parent_name() { vectorized::BlockUPtr ScannerContext::get_free_blo

Re: [PR] [refactor](scanner) refactoring and optimizing scanner scheduling [doris]

2024-02-05 Thread via GitHub
AshinGau commented on code in PR #30746: URL: https://github.com/apache/doris/pull/30746#discussion_r1477979657 ## be/src/vec/exec/scan/scanner_context.h: ## @@ -58,6 +58,47 @@ class VScanNode; class ScannerScheduler; class SimplifiedScanScheduler; +class ScanTask { +public:

Re: [PR] [refactor](scanner) refactoring and optimizing scanner scheduling [doris]

2024-02-05 Thread via GitHub
AshinGau commented on code in PR #30746: URL: https://github.com/apache/doris/pull/30746#discussion_r1477978094 ## be/src/vec/exec/scan/scanner_scheduler.cpp: ## @@ -139,155 +117,107 @@ Status ScannerScheduler::init(ExecEnv* env) { return Status::OK(); } -Status ScannerS

Re: [PR] [refactor](scanner) refactoring and optimizing scanner scheduling [doris]

2024-02-05 Thread via GitHub
AshinGau commented on code in PR #30746: URL: https://github.com/apache/doris/pull/30746#discussion_r1477977157 ## be/src/vec/exec/scan/scanner_scheduler.cpp: ## @@ -139,155 +117,107 @@ Status ScannerScheduler::init(ExecEnv* env) { return Status::OK(); } -Status ScannerS

Re: [PR] [refactor](scanner) refactoring and optimizing scanner scheduling [doris]

2024-02-05 Thread via GitHub
AshinGau commented on code in PR #30746: URL: https://github.com/apache/doris/pull/30746#discussion_r1477976873 ## be/src/vec/exec/scan/scanner_scheduler.cpp: ## @@ -139,155 +117,107 @@ Status ScannerScheduler::init(ExecEnv* env) { return Status::OK(); } -Status ScannerS

Re: [PR] [refactor](scanner) refactoring and optimizing scanner scheduling [doris]

2024-02-05 Thread via GitHub
AshinGau commented on code in PR #30746: URL: https://github.com/apache/doris/pull/30746#discussion_r1477973805 ## be/src/vec/exec/scan/scanner_scheduler.cpp: ## @@ -139,155 +117,107 @@ Status ScannerScheduler::init(ExecEnv* env) { return Status::OK(); } -Status ScannerS

Re: [PR] [refactor](scanner) refactoring and optimizing scanner scheduling [doris]

2024-02-05 Thread via GitHub
AshinGau commented on code in PR #30746: URL: https://github.com/apache/doris/pull/30746#discussion_r1477968497 ## be/src/vec/exec/scan/scanner_context.h: ## @@ -58,6 +58,47 @@ class VScanNode; class ScannerScheduler; class SimplifiedScanScheduler; +class ScanTask { +public:

Re: [PR] [refactor](scanner) refactoring and optimizing scanner scheduling [doris]

2024-02-05 Thread via GitHub
yiguolei commented on code in PR #30746: URL: https://github.com/apache/doris/pull/30746#discussion_r1477953751 ## be/src/vec/exec/scan/scanner_context.h: ## @@ -58,6 +58,47 @@ class VScanNode; class ScannerScheduler; class SimplifiedScanScheduler; +class ScanTask { +public:

Re: [PR] [refactor](scanner) refactoring and optimizing scanner scheduling [doris]

2024-02-05 Thread via GitHub
yiguolei commented on code in PR #30746: URL: https://github.com/apache/doris/pull/30746#discussion_r1477950319 ## be/src/vec/exec/scan/scanner_scheduler.cpp: ## @@ -139,155 +117,107 @@ Status ScannerScheduler::init(ExecEnv* env) { return Status::OK(); } -Status ScannerS

Re: [PR] [refactor](scanner) refactoring and optimizing scanner scheduling [doris]

2024-02-05 Thread via GitHub
yiguolei commented on code in PR #30746: URL: https://github.com/apache/doris/pull/30746#discussion_r1477949753 ## be/src/vec/exec/scan/scanner_scheduler.cpp: ## @@ -139,155 +117,107 @@ Status ScannerScheduler::init(ExecEnv* env) { return Status::OK(); } -Status ScannerS

Re: [PR] [refactor](scanner) refactoring and optimizing scanner scheduling [doris]

2024-02-05 Thread via GitHub
yiguolei commented on code in PR #30746: URL: https://github.com/apache/doris/pull/30746#discussion_r1477949208 ## be/src/vec/exec/scan/scanner_scheduler.cpp: ## @@ -139,155 +117,107 @@ Status ScannerScheduler::init(ExecEnv* env) { return Status::OK(); } -Status ScannerS

Re: [PR] [refactor](scanner) refactoring and optimizing scanner scheduling [doris]

2024-02-05 Thread via GitHub
yiguolei commented on code in PR #30746: URL: https://github.com/apache/doris/pull/30746#discussion_r1477948575 ## be/src/vec/exec/scan/scanner_scheduler.cpp: ## @@ -139,155 +117,107 @@ Status ScannerScheduler::init(ExecEnv* env) { return Status::OK(); } -Status ScannerS

Re: [PR] [refactor](scanner) refactoring and optimizing scanner scheduling [doris]

2024-02-05 Thread via GitHub
doris-robot commented on PR #30746: URL: https://github.com/apache/doris/pull/30746#issuecomment-1926612393 TeamCity be ut coverage result: Function Coverage: 36.18% (8627/23846) Line Coverage: 28.25% (70545/249733) Region Coverage: 27.24% (36402/133631) Branch Coverage: 24.0

Re: [PR] [refactor](scanner) refactoring and optimizing scanner scheduling [doris]

2024-02-05 Thread via GitHub
yiguolei commented on code in PR #30746: URL: https://github.com/apache/doris/pull/30746#discussion_r1477918124 ## be/src/vec/exec/scan/vscan_node.cpp: ## @@ -246,31 +244,16 @@ Status VScanNode::get_next(RuntimeState* state, vectorized::Block* block, bool* }}; if (s

Re: [PR] [refactor](scanner) refactoring and optimizing scanner scheduling [doris]

2024-02-05 Thread via GitHub
yiguolei commented on code in PR #30746: URL: https://github.com/apache/doris/pull/30746#discussion_r1477913625 ## be/src/vec/exec/scan/scanner_context.h: ## @@ -58,6 +58,47 @@ class VScanNode; class ScannerScheduler; class SimplifiedScanScheduler; +class ScanTask { +public:

Re: [PR] [refactor](scanner) refactoring and optimizing scanner scheduling [doris]

2024-02-05 Thread via GitHub
yiguolei commented on code in PR #30746: URL: https://github.com/apache/doris/pull/30746#discussion_r1477909755 ## be/src/vec/exec/scan/scanner_context.cpp: ## @@ -380,168 +400,28 @@ Status ScannerContext::validate_block_schema(Block* block) { return Status::OK(); } -vo

Re: [PR] [refactor](scanner) refactoring and optimizing scanner scheduling [doris]

2024-02-05 Thread via GitHub
github-actions[bot] commented on code in PR #30746: URL: https://github.com/apache/doris/pull/30746#discussion_r1477883433 ## be/src/vec/exec/scan/scanner_scheduler.cpp: ## @@ -139,155 +117,107 @@ Status ScannerScheduler::init(ExecEnv* env) { return Status::OK(); } -Stat

Re: [PR] [refactor](scanner) refactoring and optimizing scanner scheduling [doris]

2024-02-05 Thread via GitHub
AshinGau commented on PR #30746: URL: https://github.com/apache/doris/pull/30746#issuecomment-1926535291 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To uns

Re: [PR] [refactor](scanner) refactoring and optimizing scanner scheduling [doris]

2024-02-05 Thread via GitHub
yiguolei commented on code in PR #30746: URL: https://github.com/apache/doris/pull/30746#discussion_r1477851532 ## be/src/vec/exec/scan/scanner_context.cpp: ## @@ -210,148 +141,237 @@ Status ScannerContext::init() { thread

Re: [PR] [refactor](scanner) refactoring and optimizing scanner scheduling [doris]

2024-02-05 Thread via GitHub
yiguolei commented on code in PR #30746: URL: https://github.com/apache/doris/pull/30746#discussion_r1477851532 ## be/src/vec/exec/scan/scanner_context.cpp: ## @@ -210,148 +141,237 @@ Status ScannerContext::init() { thread

Re: [PR] [refactor](scanner) refactoring and optimizing scanner scheduling [doris]

2024-02-05 Thread via GitHub
AshinGau commented on PR #30746: URL: https://github.com/apache/doris/pull/30746#issuecomment-1926440382 run buildall -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To uns

Re: [PR] [refactor](scanner) refactoring and optimizing scanner scheduling [doris]

2024-02-04 Thread via GitHub
github-actions[bot] commented on code in PR #30746: URL: https://github.com/apache/doris/pull/30746#discussion_r1477713431 ## be/src/vec/exec/scan/scanner_scheduler.cpp: ## @@ -139,155 +117,108 @@ Status ScannerScheduler::init(ExecEnv* env) { return Status::OK(); } -Stat

Re: [PR] [refactor](scanner) refactoring and optimizing scanner scheduling [doris]

2024-02-04 Thread via GitHub
AshinGau commented on code in PR #30746: URL: https://github.com/apache/doris/pull/30746#discussion_r1477312581 ## be/src/vec/exec/scan/scanner_context.cpp: ## @@ -210,148 +136,238 @@ Status ScannerContext::init() { thread

Re: [PR] [refactor](scanner) refactoring and optimizing scanner scheduling [doris]

2024-02-04 Thread via GitHub
AshinGau commented on code in PR #30746: URL: https://github.com/apache/doris/pull/30746#discussion_r1477312359 ## be/src/vec/exec/scan/scanner_context.cpp: ## @@ -210,148 +136,238 @@ Status ScannerContext::init() { thread

Re: [PR] [refactor](scanner) refactoring and optimizing scanner scheduling [doris]

2024-02-04 Thread via GitHub
AshinGau commented on code in PR #30746: URL: https://github.com/apache/doris/pull/30746#discussion_r1477312014 ## be/src/pipeline/exec/scan_operator.cpp: ## @@ -1456,36 +1450,19 @@ Status ScanOperatorX::get_block(RuntimeState* state, vectorized: }}; if (state->is_c

Re: [PR] [refactor](scanner) refactoring and optimizing scanner scheduling [doris]

2024-02-04 Thread via GitHub
AshinGau commented on code in PR #30746: URL: https://github.com/apache/doris/pull/30746#discussion_r1477311455 ## be/src/vec/exec/scan/scanner_context.cpp: ## @@ -31,64 +29,56 @@ #include "common/status.h" #include "pipeline/exec/scan_operator.h" #include "runtime/descriptor

Re: [PR] [refactor](scanner) refactoring and optimizing scanner scheduling [doris]

2024-02-04 Thread via GitHub
AshinGau commented on code in PR #30746: URL: https://github.com/apache/doris/pull/30746#discussion_r1477308418 ## be/src/vec/exec/scan/scanner_context.cpp: ## @@ -31,64 +29,56 @@ #include "common/status.h" #include "pipeline/exec/scan_operator.h" #include "runtime/descriptor

Re: [PR] [refactor](scanner) refactoring and optimizing scanner scheduling [doris]

2024-02-04 Thread via GitHub
AshinGau commented on code in PR #30746: URL: https://github.com/apache/doris/pull/30746#discussion_r1477308048 ## be/src/pipeline/exec/file_scan_operator.cpp: ## @@ -62,7 +64,7 @@ void FileScanLocalState::set_scan_ranges(RuntimeState* state,

Re: [PR] [refactor](scanner) refactoring and optimizing scanner scheduling [doris]

2024-02-04 Thread via GitHub
AshinGau commented on code in PR #30746: URL: https://github.com/apache/doris/pull/30746#discussion_r1477307640 ## be/src/vec/exec/scan/scanner_scheduler.cpp: ## @@ -310,61 +244,43 @@ void ScannerScheduler::_scanner_scan(ScannerScheduler* scheduler, if (!scanner->is_init()

Re: [PR] [refactor](scanner) refactoring and optimizing scanner scheduling [doris]

2024-02-04 Thread via GitHub
AshinGau commented on code in PR #30746: URL: https://github.com/apache/doris/pull/30746#discussion_r1477305078 ## be/src/vec/exec/scan/scanner_scheduler.cpp: ## @@ -139,155 +117,111 @@ Status ScannerScheduler::init(ExecEnv* env) { return Status::OK(); } -Status ScannerS

Re: [PR] [refactor](scanner) refactoring and optimizing scanner scheduling [doris]

2024-02-04 Thread via GitHub
yiguolei commented on code in PR #30746: URL: https://github.com/apache/doris/pull/30746#discussion_r1477235038 ## be/src/vec/exec/scan/new_file_scan_node.cpp: ## @@ -62,7 +62,7 @@ void NewFileScanNode::set_scan_ranges(RuntimeState* state,

Re: [PR] [refactor](scanner) refactoring and optimizing scanner scheduling [doris]

2024-02-04 Thread via GitHub
yiguolei commented on code in PR #30746: URL: https://github.com/apache/doris/pull/30746#discussion_r1477234921 ## be/src/pipeline/exec/scan_operator.cpp: ## @@ -1456,36 +1450,19 @@ Status ScanOperatorX::get_block(RuntimeState* state, vectorized: }}; if (state->is_c

Re: [PR] [refactor](scanner) refactoring and optimizing scanner scheduling [doris]

2024-02-04 Thread via GitHub
yiguolei commented on code in PR #30746: URL: https://github.com/apache/doris/pull/30746#discussion_r1477234648 ## be/src/pipeline/exec/file_scan_operator.cpp: ## @@ -62,7 +64,7 @@ void FileScanLocalState::set_scan_ranges(RuntimeState* state,

Re: [PR] [refactor](scanner) refactoring and optimizing scanner scheduling [doris]

2024-02-04 Thread via GitHub
yiguolei commented on code in PR #30746: URL: https://github.com/apache/doris/pull/30746#discussion_r1477233912 ## be/src/vec/exec/scan/scanner_scheduler.cpp: ## @@ -310,61 +244,43 @@ void ScannerScheduler::_scanner_scan(ScannerScheduler* scheduler, if (!scanner->is_init()

Re: [PR] [refactor](scanner) refactoring and optimizing scanner scheduling [doris]

2024-02-04 Thread via GitHub
yiguolei commented on code in PR #30746: URL: https://github.com/apache/doris/pull/30746#discussion_r1477233710 ## be/src/vec/exec/scan/scanner_scheduler.cpp: ## @@ -310,61 +244,43 @@ void ScannerScheduler::_scanner_scan(ScannerScheduler* scheduler, if (!scanner->is_init()

Re: [PR] [refactor](scanner) refactoring and optimizing scanner scheduling [doris]

2024-02-04 Thread via GitHub
yiguolei commented on code in PR #30746: URL: https://github.com/apache/doris/pull/30746#discussion_r1477233123 ## be/src/vec/exec/scan/scanner_scheduler.cpp: ## @@ -139,155 +117,111 @@ Status ScannerScheduler::init(ExecEnv* env) { return Status::OK(); } -Status ScannerS

Re: [PR] [refactor](scanner) refactoring and optimizing scanner scheduling [doris]

2024-02-04 Thread via GitHub
yiguolei commented on code in PR #30746: URL: https://github.com/apache/doris/pull/30746#discussion_r1477232992 ## be/src/vec/exec/scan/scanner_scheduler.cpp: ## @@ -139,155 +117,111 @@ Status ScannerScheduler::init(ExecEnv* env) { return Status::OK(); } -Status ScannerS

Re: [PR] [refactor](scanner) refactoring and optimizing scanner scheduling [doris]

2024-02-04 Thread via GitHub
yiguolei commented on code in PR #30746: URL: https://github.com/apache/doris/pull/30746#discussion_r1477231222 ## be/src/vec/exec/scan/scanner_scheduler.cpp: ## @@ -139,155 +117,111 @@ Status ScannerScheduler::init(ExecEnv* env) { return Status::OK(); } -Status ScannerS

Re: [PR] [refactor](scanner) refactoring and optimizing scanner scheduling [doris]

2024-02-04 Thread via GitHub
yiguolei commented on code in PR #30746: URL: https://github.com/apache/doris/pull/30746#discussion_r1477223228 ## be/src/vec/exec/scan/scanner_context.cpp: ## @@ -31,64 +29,56 @@ #include "common/status.h" #include "pipeline/exec/scan_operator.h" #include "runtime/descriptor

Re: [PR] [refactor](scanner) refactoring and optimizing scanner scheduling [doris]

2024-02-04 Thread via GitHub
yiguolei commented on code in PR #30746: URL: https://github.com/apache/doris/pull/30746#discussion_r1477222646 ## be/src/vec/exec/scan/scanner_context.cpp: ## @@ -210,148 +136,238 @@ Status ScannerContext::init() { thread

Re: [PR] [refactor](scanner) refactoring and optimizing scanner scheduling [doris]

2024-02-04 Thread via GitHub
yiguolei commented on code in PR #30746: URL: https://github.com/apache/doris/pull/30746#discussion_r1477222416 ## be/src/vec/exec/scan/scanner_context.cpp: ## @@ -31,64 +29,56 @@ #include "common/status.h" #include "pipeline/exec/scan_operator.h" #include "runtime/descriptor

Re: [PR] [refactor](scanner) refactoring and optimizing scanner scheduling [doris]

2024-02-04 Thread via GitHub
yiguolei commented on code in PR #30746: URL: https://github.com/apache/doris/pull/30746#discussion_r1477222377 ## be/src/vec/exec/scan/scanner_context.cpp: ## @@ -31,64 +29,56 @@ #include "common/status.h" #include "pipeline/exec/scan_operator.h" #include "runtime/descriptor

Re: [PR] [refactor](scanner) refactoring and optimizing scanner scheduling [doris]

2024-02-04 Thread via GitHub
yiguolei commented on code in PR #30746: URL: https://github.com/apache/doris/pull/30746#discussion_r1477221457 ## be/src/pipeline/exec/scan_operator.cpp: ## @@ -1456,36 +1450,19 @@ Status ScanOperatorX::get_block(RuntimeState* state, vectorized: }}; if (state->is_c

Re: [PR] [refactor](scanner) refactoring and optimizing scanner scheduling [doris]

2024-02-04 Thread via GitHub
yiguolei commented on code in PR #30746: URL: https://github.com/apache/doris/pull/30746#discussion_r1477217929 ## be/src/pipeline/exec/scan_operator.cpp: ## @@ -1456,36 +1450,19 @@ Status ScanOperatorX::get_block(RuntimeState* state, vectorized: }}; if (state->is_c

Re: [PR] [refactor](scanner) refactoring and optimizing scanner scheduling [doris]

2024-02-04 Thread via GitHub
yiguolei commented on code in PR #30746: URL: https://github.com/apache/doris/pull/30746#discussion_r1477217437 ## be/src/pipeline/exec/scan_operator.cpp: ## @@ -1288,16 +1287,11 @@ Status ScanLocalState::_init_profile() { profile()->add_child(_scanner_profile.get(), true,

Re: [PR] [refactor](scanner) refactoring and optimizing scanner scheduling [doris]

2024-02-04 Thread via GitHub
morningman commented on code in PR #30746: URL: https://github.com/apache/doris/pull/30746#discussion_r1477173233 ## be/src/vec/exec/scan/scanner_context.cpp: ## @@ -210,148 +136,238 @@ Status ScannerContext::init() { thre

Re: [PR] [refactor](scanner) refactoring and optimizing scanner scheduling [doris]

2024-02-04 Thread via GitHub
yiguolei commented on code in PR #30746: URL: https://github.com/apache/doris/pull/30746#discussion_r1477217240 ## be/src/pipeline/exec/scan_operator.cpp: ## @@ -1288,16 +1287,11 @@ Status ScanLocalState::_init_profile() { profile()->add_child(_scanner_profile.get(), true,

Re: [PR] [refactor](scanner) refactoring and optimizing scanner scheduling [doris]

2024-02-04 Thread via GitHub
yiguolei commented on code in PR #30746: URL: https://github.com/apache/doris/pull/30746#discussion_r1477217240 ## be/src/pipeline/exec/scan_operator.cpp: ## @@ -1288,16 +1287,11 @@ Status ScanLocalState::_init_profile() { profile()->add_child(_scanner_profile.get(), true,

Re: [PR] [refactor](scanner) refactoring and optimizing scanner scheduling [doris]

2024-02-03 Thread via GitHub
yiguolei commented on code in PR #30746: URL: https://github.com/apache/doris/pull/30746#discussion_r1477194770 ## be/src/vec/exec/scan/vscan_node.h: ## @@ -102,6 +102,35 @@ class ScannerDelegate { ScannerDelegate(ScannerDelegate&&) = delete; }; +class RunningScanner { +

Re: [PR] [refactor](scanner) refactoring and optimizing scanner scheduling [doris]

2024-02-03 Thread via GitHub
AshinGau commented on code in PR #30746: URL: https://github.com/apache/doris/pull/30746#discussion_r1477193009 ## be/src/pipeline/exec/scan_operator.cpp: ## @@ -1456,36 +1450,19 @@ Status ScanOperatorX::get_block(RuntimeState* state, vectorized: }}; if (state->is_c

Re: [PR] [refactor](scanner) refactoring and optimizing scanner scheduling [doris]

2024-02-03 Thread via GitHub
yiguolei commented on code in PR #30746: URL: https://github.com/apache/doris/pull/30746#discussion_r1477191632 ## be/src/vec/exec/scan/vscan_node.h: ## @@ -102,6 +102,35 @@ class ScannerDelegate { ScannerDelegate(ScannerDelegate&&) = delete; }; +class RunningScanner {

Re: [PR] [refactor](scanner) refactoring and optimizing scanner scheduling [doris]

2024-02-03 Thread via GitHub
yiguolei commented on code in PR #30746: URL: https://github.com/apache/doris/pull/30746#discussion_r1477191550 ## be/src/vec/exec/scan/scanner_context.cpp: ## @@ -210,148 +136,238 @@ Status ScannerContext::init() { thread

Re: [PR] [refactor](scanner) refactoring and optimizing scanner scheduling [doris]

2024-02-03 Thread via GitHub
yiguolei commented on code in PR #30746: URL: https://github.com/apache/doris/pull/30746#discussion_r1477191521 ## be/src/vec/exec/scan/scanner_context.cpp: ## @@ -210,148 +136,238 @@ Status ScannerContext::init() { thread

Re: [PR] [refactor](scanner) refactoring and optimizing scanner scheduling [doris]

2024-02-03 Thread via GitHub
wangbo commented on PR #30746: URL: https://github.com/apache/doris/pull/30746#issuecomment-1925601213 I will test workload group for sacn after this pr is merged. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the UR

Re: [PR] [refactor](scanner) refactoring and optimizing scanner scheduling [doris]

2024-02-01 Thread via GitHub
github-actions[bot] commented on code in PR #30746: URL: https://github.com/apache/doris/pull/30746#discussion_r1475590585 ## be/src/vec/exec/scan/scanner_scheduler.cpp: ## @@ -139,155 +117,111 @@ Status ScannerScheduler::init(ExecEnv* env) { return Status::OK(); } -Stat

Re: [PR] [refactor](scanner) refactoring and optimizing scanner scheduling [doris]

2024-02-01 Thread via GitHub
doris-robot commented on PR #30746: URL: https://github.com/apache/doris/pull/30746#issuecomment-1922921644 Thank you for your contribution to Apache Doris. Don't know what should be done next? See [How to process your PR](https://cwiki.apache.org/confluence/display/DORIS/How+to+process+y

[PR] [refactor](scanner) refactoring and optimizing scanner scheduling [doris]

2024-02-01 Thread via GitHub
AshinGau opened a new pull request, #30746: URL: https://github.com/apache/doris/pull/30746 ## Proposed changes Issue Number: close #xxx ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto