[GitHub] [doris] hello-stephen commented on pull request #16152: [Bug](decimalv3) fix BE crash for function `if`
hello-stephen commented on PR #16152: URL: https://github.com/apache/doris/pull/16152#issuecomment-1407325087 TeamCity pipeline, clickbench performance test result: the sum of best hot time: 33.27 seconds load time: 491 seconds storage size: 17120625313 Bytes https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230128080018_clickbench_pr_85473.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] SaintBacchus closed pull request #16149: [MTMV](WIP)Support master and follow change in multi fe for mtmv
SaintBacchus closed pull request #16149: [MTMV](WIP)Support master and follow change in multi fe for mtmv URL: https://github.com/apache/doris/pull/16149 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] BiteTheDDDDt opened a new pull request, #16157: [Bug](exec) enable warning on ignoring function return value for vctx
BiteThet opened a new pull request, #16157: URL: https://github.com/apache/doris/pull/16157 # Proposed changes enable warning on ignoring function return value for vctx ## Problem summary Describe your changes. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [ ] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [ ] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [ ] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] yiguolei merged pull request #16083: [improvement](memory) load support overcommit memory
yiguolei merged PR #16083: URL: https://github.com/apache/doris/pull/16083 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated (49395390be -> 949a065f22)
This is an automated email from the ASF dual-hosted git repository. yiguolei pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/doris.git from 49395390be [bugfix](metareader) meta reader could not load image (#16148) add 949a065f22 [improvement](memory) load support overcommit memory (#16083) No new revisions were added by this update. Summary of changes: be/src/runtime/memory/mem_tracker_limiter.cpp | 37 +++ be/src/runtime/memory/mem_tracker_limiter.h | 17 be/src/runtime/runtime_state.cpp | 3 ++- be/src/util/mem_info.cpp | 17 4 files changed, 51 insertions(+), 23 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #16157: [Bug](exec) enable warning on ignoring function return value for vctx
github-actions[bot] commented on PR #16157: URL: https://github.com/apache/doris/pull/16157#issuecomment-1407328157 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hf200012 opened a new pull request, #16158: Upgrade springboot version to 2.7.8
hf200012 opened a new pull request, #16158: URL: https://github.com/apache/doris/pull/16158 # Proposed changes Upgrade the springboot version to 2.7.8, and fix other problems that springboot depends on Issue Number: close #xxx ## Problem summary Describe your changes. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [ ] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [ ] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [ ] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] Gabriel39 opened a new pull request, #16159: [datev2](function) make function nullable DEPEND_ON_ARGUMENT
Gabriel39 opened a new pull request, #16159: URL: https://github.com/apache/doris/pull/16159 # Proposed changes Set all functions' nullable mode for `datev2` to `DEPEND_ON_ARGUMENT` ## Problem summary Describe your changes. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [ ] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [ ] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [ ] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] httpshirley opened a new pull request, #16160: [docs]update en docs
httpshirley opened a new pull request, #16160: URL: https://github.com/apache/doris/pull/16160 # Proposed changes Issue Number: close #xxx ## Problem summary Describe your changes. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [x] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [x] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [x] No Need 4. Does it need to update dependencies: - [ ] Yes - [x] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [x] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #16132: [fix](datatype) fix some bugs about data type array datetimev2 and decimalv3
github-actions[bot] commented on PR #16132: URL: https://github.com/apache/doris/pull/16132#issuecomment-1407340190 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #16153: [bug](cooldown)add config for Cooldown Job
hello-stephen commented on PR #16153: URL: https://github.com/apache/doris/pull/16153#issuecomment-1407342007 TeamCity pipeline, clickbench performance test result: the sum of best hot time: 34.96 seconds load time: 526 seconds storage size: 17123103742 Bytes https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230128084211_clickbench_pr_85484.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] luzhijing commented on pull request #16160: [docs](docs)update en docs
luzhijing commented on PR #16160: URL: https://github.com/apache/doris/pull/16160#issuecomment-1407343066 LGTM -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] Tanya-W opened a new pull request, #16161: [fix](inverted index) fulltext match query should not prune by zone map
Tanya-W opened a new pull request, #16161: URL: https://github.com/apache/doris/pull/16161 # Proposed changes zone map prune data by min/max value, should not prune by zone map when fulltext match term is substring in a string type column's value, otherwise, some data meeting the conditions will be pruned. such like: - test data: | col_1 | | | | hello world | | say hello world | | zero | - match query: ``` select * from tb where col_1 match 'world'; ``` `hello world` and `say hello world` should be returned to the user instead of being pruned by zone map. ## Problem summary Describe your changes. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [ ] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [ ] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [ ] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] morningman commented on a diff in pull request #16160: [docs](docs)update en docs
morningman commented on code in PR #16160: URL: https://github.com/apache/doris/pull/16160#discussion_r1089690101 ## docs/en/docs/lakehouse/multi-catalog/multi-catalog.md: ## @@ -261,7 +261,7 @@ See [Hudi](./hudi) ### Connect to Elasticsearch -See [Elasticsearch](./elasticsearch) +See [Elasticsearch](./es) Review Comment: Modify the Chinese version too ## docs/en/docs/lakehouse/multi-catalog/dlf.md: ## @@ -25,7 +25,79 @@ under the License. --> -# Aliyun DLF +# Alibaba Cloud DLF + +Data Lake Formation (DLF) is the unified metadata management service of Alibaba Cloud. It is compatible with the Hive Metastore protocol. + +> [What is DLF](https://www.alibabacloud.com/product/datalake-formation) + +Doris can access DLF the same way as it accesses Hive Metastore. + +## Connect to DLF + +1. Create `hive-site.xml` + + Create the `hive-site.xml` file, and put it in the `fe/conf` directory. + + ``` + + + + + hive.metastore.type + dlf + + + dlf.catalog.endpoint + dlf-vpc.cn-beijing.aliyuncs.com + + + dlf.catalog.region + cn-beijing + + + dlf.catalog.proxyMode + DLF_ONLY + + + dlf.catalog.uid + 2 + + + dlf.catalog.accessKeyId + XXX + + + dlf.catalog.accessKeySecret + X + + + ``` + + * `dlf.catalog.endpoint`: DLF Endpoint. See [Regions and Endpoints of DLF](https://www.alibabacloud.com/help/en/data-lake-formation/latest/regions-and-endpoints). + * `dlf.catalog.region`: DLF Region. See [Regions and Endpoints of DLF](https://www.alibabacloud.com/help/en/data-lake-formation/latest/regions-and-endpoints). + * `dlf.catalog.uid`: Alibaba Cloud account. You can find the "Account ID" in the upper right corner on the Alibaba Cloud console. + * `dlf.catalog.accessKeyId`：AccessKey, which you can create and manage on the [Alibaba Cloud console](https://ram.console.aliyun.com/manage/ak). + * `dlf.catalog.accessKeySecret`：SecretKey, which you can create and manage on the [Alibaba Cloud console](https://ram.console.aliyun.com/manage/ak). + + Other configuration items are fixed and require no modifications. Review Comment: ```suggestion Other configuration items are fixed and require no modifications. ``` ## docs/en/docs/lakehouse/multi-catalog/hive.md: ## @@ -26,4 +26,149 @@ under the License. # Hive -TODO: translate +Once Doris is connected to Hive Metastore or made compatible with Hive Metastore metadata service, it can access databases and tables in Hive and conduct queries. + +Besides Hive, many other systems, such as Iceberg and Hudi, use Hive Metastore to keep their metadata. Thus, Doris can also access these systems via Hive Catalog. + +## Usage + +When connnecting to Hive, Doris: + +1. Supports Hive version 1/2/3; +2. Supports both Managed Table and External Table; +3. Can identify metadata of Hive, Iceberg, and Hudi stored in Hive Metastore; +4. Supports Hive tables with data stored in JuiceFS, which can be used the same way as normal Hive tables (put `juicefs-hadoop-x.x.x.jar` in `fe/lib/` and `apache_hdfs_broker/lib/`). + +## Create Catalog + +```sql +CREATE CATALOG hive PROPERTIES ( +'type'='hms', +'hive.metastore.uris' = 'thrift://172.21.0.1:7004', +'hadoop.username' = 'hive', +'dfs.nameservices'='your-nameservice', +'dfs.ha.namenodes.your-nameservice'='nn1,nn2', +'dfs.namenode.rpc-address.your-nameservice.nn1'='172.21.0.2:4007', +'dfs.namenode.rpc-address.your-nameservice.nn2'='172.21.0.3:4007', + 'dfs.client.failover.proxy.provider.your-nameservice'='org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider' +); +``` + + In addition to `type` and `hive.metastore.uris` , which are required, you can specify other parameters regarding the connection. + +For example, to specify HDFS HA: + +```sql +CREATE CATALOG hive PROPERTIES ( +'type'='hms', +'hive.metastore.uris' = 'thrift://172.21.0.1:7004', +'hadoop.username' = 'hive', +'dfs.nameservices'='your-nameservice', +'dfs.ha.namenodes.your-nameservice'='nn1,nn2', +'dfs.namenode.rpc-address.your-nameservice.nn1'='172.21.0.2:4007', +'dfs.namenode.rpc-address.your-nameservice.nn2'='172.21.0.3:4007', + 'dfs.client.failover.proxy.provider.your-nameservice'='org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider' +); +``` + +To specify HDFS HA and Kerberos authentication information: + +```sql +CREATE CATALOG hive PROPERTIES ( +'type'='hms', +'hive.metastore.uris' = 'thrift://172.21.0.1:7004', +'hive.metastore.sasl.enabled' = 'true', +'dfs.nameservices'='your-nameservice', +'dfs.namenode.rpc-address.your-nameservice.nn1'='172.21.0.2:4007',
[GitHub] [doris] morningman commented on a diff in pull request #16160: [docs](docs)update en docs
morningman commented on code in PR #16160: URL: https://github.com/apache/doris/pull/16160#discussion_r1089690156 ## docs/en/docs/lakehouse/multi-catalog/hive.md: ## @@ -26,4 +26,149 @@ under the License. # Hive -TODO: translate +Once Doris is connected to Hive Metastore or made compatible with Hive Metastore metadata service, it can access databases and tables in Hive and conduct queries. + +Besides Hive, many other systems, such as Iceberg and Hudi, use Hive Metastore to keep their metadata. Thus, Doris can also access these systems via Hive Catalog. + +## Usage + +When connnecting to Hive, Doris: + +1. Supports Hive version 1/2/3; +2. Supports both Managed Table and External Table; +3. Can identify metadata of Hive, Iceberg, and Hudi stored in Hive Metastore; +4. Supports Hive tables with data stored in JuiceFS, which can be used the same way as normal Hive tables (put `juicefs-hadoop-x.x.x.jar` in `fe/lib/` and `apache_hdfs_broker/lib/`). + +## Create Catalog + +```sql +CREATE CATALOG hive PROPERTIES ( +'type'='hms', +'hive.metastore.uris' = 'thrift://172.21.0.1:7004', +'hadoop.username' = 'hive', +'dfs.nameservices'='your-nameservice', +'dfs.ha.namenodes.your-nameservice'='nn1,nn2', +'dfs.namenode.rpc-address.your-nameservice.nn1'='172.21.0.2:4007', +'dfs.namenode.rpc-address.your-nameservice.nn2'='172.21.0.3:4007', + 'dfs.client.failover.proxy.provider.your-nameservice'='org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider' +); +``` + + In addition to `type` and `hive.metastore.uris` , which are required, you can specify other parameters regarding the connection. + +For example, to specify HDFS HA: + +```sql +CREATE CATALOG hive PROPERTIES ( +'type'='hms', +'hive.metastore.uris' = 'thrift://172.21.0.1:7004', +'hadoop.username' = 'hive', +'dfs.nameservices'='your-nameservice', +'dfs.ha.namenodes.your-nameservice'='nn1,nn2', +'dfs.namenode.rpc-address.your-nameservice.nn1'='172.21.0.2:4007', +'dfs.namenode.rpc-address.your-nameservice.nn2'='172.21.0.3:4007', + 'dfs.client.failover.proxy.provider.your-nameservice'='org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider' +); +``` + +To specify HDFS HA and Kerberos authentication information: + +```sql +CREATE CATALOG hive PROPERTIES ( +'type'='hms', +'hive.metastore.uris' = 'thrift://172.21.0.1:7004', +'hive.metastore.sasl.enabled' = 'true', +'dfs.nameservices'='your-nameservice', +'dfs.namenode.rpc-address.your-nameservice.nn1'='172.21.0.2:4007', +'dfs.namenode.rpc-address.your-nameservice.nn2'='172.21.0.3:4007', + 'dfs.client.failover.proxy.provider.your-nameservice'='org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider', +'hadoop.security.authentication' = 'kerberos', +'hadoop.kerberos.keytab' = '/your-keytab-filepath/your.keytab', +'hadoop.kerberos.principal' = 'your-princi...@your.com', +'yarn.resourcemanager.address' = 'your-rm-address:your-rm-port', +'yarn.resourcemanager.principal' = 'your-rm-principal/_h...@your.com' +); +``` + +To provide Hadoop KMS encrypted transmission information: + +```sql +CREATE CATALOG hive PROPERTIES ( +'type'='hms', +'hive.metastore.uris' = 'thrift://172.21.0.1:7004', +'dfs.encryption.key.provider.uri' = 'kms://http@kms_host:kms_port/kms' +); +``` + +Or to connect to Hive data stored in JuiceFS: + +```sql +CREATE CATALOG hive PROPERTIES ( +'type'='hms', +'hive.metastore.uris' = 'thrift://172.21.0.1:7004', +'hadoop.username' = 'root', +'fs.jfs.impl' = 'io.juicefs.JuiceFileSystem', +'fs.AbstractFileSystem.jfs.impl' = 'io.juicefs.JuiceFS', +'juicefs.meta' = 'xxx' +); +``` + +In Doris 1.2.1 and newer, you can create a Resource that contains all these parameters, and reuse the Resource when creating new Catalogs. Here is an example: + +```sql +# 1. Create Resource +CREATE RESOURCE hms_resource PROPERTIES ( +'type'='hms', +'hive.metastore.uris' = 'thrift://172.21.0.1:7004', +'hadoop.username' = 'hive', +'dfs.nameservices'='your-nameservice', +'dfs.ha.namenodes.your-nameservice'='nn1,nn2', +'dfs.namenode.rpc-address.your-nameservice.nn1'='172.21.0.2:4007', +'dfs.namenode.rpc-address.your-nameservice.nn2'='172.21.0.3:4007', + 'dfs.client.failover.proxy.provider.your-nameservice'='org.apache.hadoop.hdfs.server.namenode.ha.ConfiguredFailoverProxyProvider' +); + +# 2. Create Catalog and use an existing Resource. The key and value information in the followings will overwrite the corresponding information in the Resource. +CREATE CATALOG hive WITH RESOURCE hms_resource PROPERTIES( + 'key' = 'value' Review Comment: Please use 4 spaces to replace all tabs ## docs/en/docs/lakehouse/multi-catalog/hive.md: ## @@ -26,4 +26,149 @@ under the License. # Hive -TODO: translate +Once Doris is connected to Hive Meta
[GitHub] [doris] github-actions[bot] commented on pull request #16161: [fix](inverted index) fulltext match query should not prune by zone map
github-actions[bot] commented on PR #16161: URL: https://github.com/apache/doris/pull/16161#issuecomment-1407346107 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #16161: [fix](inverted index) fulltext match query should not prune by zone map
github-actions[bot] commented on PR #16161: URL: https://github.com/apache/doris/pull/16161#issuecomment-1407346437 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #16158: [deps](http)Upgrade springboot version to 2.7.8
github-actions[bot] commented on PR #16158: URL: https://github.com/apache/doris/pull/16158#issuecomment-1407347233 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #16158: [deps](http)Upgrade springboot version to 2.7.8
github-actions[bot] commented on PR #16158: URL: https://github.com/apache/doris/pull/16158#issuecomment-1407347238 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] yiguolei opened a new pull request, #16162: [refactor](remove unused code) remove partition info from datastream sender
yiguolei opened a new pull request, #16162: URL: https://github.com/apache/doris/pull/16162 # Proposed changes Issue Number: close #xxx ## Problem summary Describe your changes. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [ ] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [ ] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [ ] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #16162: [refactor](remove unused code) remove partition info from datastream sender
github-actions[bot] commented on PR #16162: URL: https://github.com/apache/doris/pull/16162#issuecomment-1407355573 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] sohardforaname opened a new pull request, #16163: [Fix](Nereids)changing test data for more effectively testing for nereids_function_p0.
sohardforaname opened a new pull request, #16163: URL: https://github.com/apache/doris/pull/16163 # Proposed changes Issue Number: close #xxx ## Problem summary changing test data for more effectively testing for nereids_function_p0. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [x] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [x] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [x] No Need 4. Does it need to update dependencies: - [ ] Yes - [x] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [x] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #16158: [deps](http)Upgrade springboot version to 2.7.8
hello-stephen commented on PR #16158: URL: https://github.com/apache/doris/pull/16158#issuecomment-1407360132 TeamCity pipeline, clickbench performance test result: the sum of best hot time: 35.31 seconds load time: 493 seconds storage size: 17122289820 Bytes https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230128094701_clickbench_pr_85549.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #16157: [Bug](exec) enable warning on ignoring function return value for vctx
hello-stephen commented on PR #16157: URL: https://github.com/apache/doris/pull/16157#issuecomment-1407363069 TeamCity pipeline, clickbench performance test result: the sum of best hot time: 35.56 seconds load time: 543 seconds storage size: 17122534818 Bytes https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230128095756_clickbench_pr_85521.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] Gabriel39 merged pull request #16102: [Bug](runtimefilter) Fix wrong runtime filter on datetime
Gabriel39 merged PR #16102: URL: https://github.com/apache/doris/pull/16102 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated: [Bug](runtimefilter) Fix wrong runtime filter on datetime (#16102)
This is an automated email from the ASF dual-hosted git repository.
gabriellee pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 7cf7706eb1 [Bug](runtimefilter) Fix wrong runtime filter on datetime
(#16102)
7cf7706eb1 is described below
commit 7cf7706eb1b6bb0c0361e85cef9944e887a05ebb
Author: Gabriel
AuthorDate: Sat Jan 28 18:16:06 2023 +0800
[Bug](runtimefilter) Fix wrong runtime filter on datetime (#16102)
---
be/src/exprs/bloom_filter_func.h | 19 +
be/src/exprs/minmax_predicate.h| 5 ++-
be/src/runtime/datetime_value.h| 2 +-
be/src/vec/runtime/vdatetime_value.cpp | 2 +-
be/src/vec/runtime/vdatetime_value.h | 2 +-
.../datatype_p0/date/test_date_runtime_filter.out | 26
.../date/test_date_runtime_filter.groovy | 49 ++
7 files changed, 94 insertions(+), 11 deletions(-)
diff --git a/be/src/exprs/bloom_filter_func.h b/be/src/exprs/bloom_filter_func.h
index 84a61ef2e2..3d8f565aa3 100644
--- a/be/src/exprs/bloom_filter_func.h
+++ b/be/src/exprs/bloom_filter_func.h
@@ -314,9 +314,13 @@ struct FixedStringFindOp : public StringFindOp {
struct DateTimeFindOp : public CommonFindOp {
bool find_olap_engine(const BloomFilterAdaptor& bloom_filter, const void*
data) const {
-DateTimeValue value;
+vectorized::VecDateTimeValue value;
value.from_olap_datetime(*reinterpret_cast(data));
-return bloom_filter.test(Slice((char*)&value, sizeof(DateTimeValue)));
+return bloom_filter.test(Slice((char*)&value,
sizeof(vectorized::VecDateTimeValue)));
+}
+
+void insert(BloomFilterAdaptor& bloom_filter, const void* data) const {
+bloom_filter.add_bytes((char*)data,
sizeof(vectorized::VecDateTimeValue));
}
};
@@ -328,13 +332,14 @@ struct DateFindOp : public CommonFindOp {
uint24_t date = *static_cast(data);
uint64_t value = uint32_t(date);
-DateTimeValue date_value;
+vectorized::VecDateTimeValue date_value;
date_value.from_olap_date(value);
-date_value.to_datetime();
-char data_bytes[sizeof(date_value)];
-memcpy(&data_bytes, &date_value, sizeof(date_value));
-return bloom_filter.test(Slice(data_bytes, sizeof(DateTimeValue)));
+return bloom_filter.test(Slice((char*)&date_value,
sizeof(vectorized::VecDateTimeValue)));
+}
+
+void insert(BloomFilterAdaptor& bloom_filter, const void* data) const {
+bloom_filter.add_bytes((char*)data,
sizeof(vectorized::VecDateTimeValue));
}
};
diff --git a/be/src/exprs/minmax_predicate.h b/be/src/exprs/minmax_predicate.h
index 4394c36d69..65d9ee005f 100644
--- a/be/src/exprs/minmax_predicate.h
+++ b/be/src/exprs/minmax_predicate.h
@@ -49,7 +49,10 @@ public:
}
T val_data;
-if constexpr (sizeof(T) >= sizeof(int128_t)) {
+if constexpr (std::is_same_v) {
+reinterpret_cast(data)->convert_vec_dt_to_dt(
+&val_data);
+} else if constexpr (sizeof(T) >= sizeof(int128_t)) {
// use dereference operator on unalign address maybe lead
segmentation fault
memcpy(&val_data, data, sizeof(T));
} else {
diff --git a/be/src/runtime/datetime_value.h b/be/src/runtime/datetime_value.h
index 3486692ddd..856bd3f854 100644
--- a/be/src/runtime/datetime_value.h
+++ b/be/src/runtime/datetime_value.h
@@ -615,7 +615,7 @@ public:
private:
// Used to make sure sizeof DateTimeValue
friend class UnusedClass;
-friend void
doris::vectorized::VecDateTimeValue::convert_vec_dt_to_dt(DateTimeValue* dt);
+friend void
doris::vectorized::VecDateTimeValue::convert_vec_dt_to_dt(DateTimeValue* dt)
const;
friend void
doris::vectorized::VecDateTimeValue::convert_dt_to_vec_dt(DateTimeValue* dt);
void from_packed_time(int64_t packed_time) {
diff --git a/be/src/vec/runtime/vdatetime_value.cpp
b/be/src/vec/runtime/vdatetime_value.cpp
index 8111569c6b..69eae65d99 100644
--- a/be/src/vec/runtime/vdatetime_value.cpp
+++ b/be/src/vec/runtime/vdatetime_value.cpp
@@ -1749,7 +1749,7 @@ void
VecDateTimeValue::create_from_date_v2(DateV2Value& value, TimeType type)
}
void VecDateTimeValue::convert_vec_dt_to_dt(
-doris::DateTimeValue* dt) { //use convert VecDateTimeValue to
DateTimeValue
+doris::DateTimeValue* dt) const { //use convert VecDateTimeValue to
DateTimeValue
dt->_neg = this->_neg;
dt->_type = this->_type;
dt->_hour = this->_hour;
diff --git a/be/src/vec/runtime/vdatetime_value.h
b/be/src/vec/runtime/vdatetime_value.h
index 93438e179c..a49209492a 100644
--- a/be/src/vec/runtime/vdatetime_value.h
+++ b/be/src/vec/runtime/vdatetime_value.h
@@ -629,7 +629,7 @@ public:
_day > 0;
}
-void convert_v
[GitHub] [doris] github-actions[bot] commented on pull request #16162: [refactor](remove unused code) remove partition info from datastream sender
github-actions[bot] commented on PR #16162: URL: https://github.com/apache/doris/pull/16162#issuecomment-1407366751 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #16162: [refactor](remove unused code) remove partition info from datastream sender
github-actions[bot] commented on PR #16162: URL: https://github.com/apache/doris/pull/16162#issuecomment-1407366761 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] jackwener opened a new pull request, #16164: [fix](Nereids): fix Ceiling.
jackwener opened a new pull request, #16164: URL: https://github.com/apache/doris/pull/16164 # Proposed changes Issue Number: close #xxx ## Problem summary Describe your changes. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [ ] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [ ] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [ ] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #16129: [cleanup](fe) cleanup useless code.
github-actions[bot] commented on PR #16129: URL: https://github.com/apache/doris/pull/16129#issuecomment-1407371127 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated: [cleanup](fe) cleanup useless code. (#16129)
This is an automated email from the ASF dual-hosted git repository.
yiguolei pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 7e7fd5d049 [cleanup](fe) cleanup useless code. (#16129)
7e7fd5d049 is described below
commit 7e7fd5d0496dc54cc8353ea6138d92bb99b5bdbe
Author: jakevin
AuthorDate: Sat Jan 28 18:44:43 2023 +0800
[cleanup](fe) cleanup useless code. (#16129)
* [cleanup](Nereids): cleanup useless code.
* revert ErrorCode.java
---
.../java/org/apache/doris/catalog/ScalarType.java | 45 --
.../main/java/org/apache/doris/catalog/Type.java | 6 +--
.../java/org/apache/doris/analysis/ExprId.java | 5 ---
.../java/org/apache/doris/analysis/SlotId.java | 5 ---
.../java/org/apache/doris/analysis/TupleId.java| 5 ---
.../java/org/apache/doris/common/ErrorReport.java | 4 --
.../java/org/apache/doris/common/IdGenerator.java | 2 -
.../main/java/org/apache/doris/common/Status.java | 1 -
.../java/org/apache/doris/common/ThriftServer.java | 7 +---
.../org/apache/doris/cooldown/CooldownHandler.java | 2 +-
.../org/apache/doris/cooldown/CooldownJob.java | 4 +-
.../org/apache/doris/datasource/MetaObjCache.java | 32 ---
.../org/apache/doris/nereids/memo/GroupId.java | 5 ---
.../doris/nereids/trees/expressions/ExprId.java| 5 ---
.../doris/nereids/trees/plans/RelationId.java | 5 ---
.../org/apache/doris/planner/PlanFragmentId.java | 5 ---
.../java/org/apache/doris/planner/PlanNodeId.java | 5 ---
.../org/apache/doris/planner/RuntimeFilterId.java | 5 ---
18 files changed, 5 insertions(+), 143 deletions(-)
diff --git
a/fe/fe-common/src/main/java/org/apache/doris/catalog/ScalarType.java
b/fe/fe-common/src/main/java/org/apache/doris/catalog/ScalarType.java
index 9015099f8c..9c18f02eab 100644
--- a/fe/fe-common/src/main/java/org/apache/doris/catalog/ScalarType.java
+++ b/fe/fe-common/src/main/java/org/apache/doris/catalog/ScalarType.java
@@ -1096,51 +1096,6 @@ public class ScalarType extends Type {
return PrimitiveType.isImplicitCast(type.getPrimitiveType(),
targetType.getPrimitiveType());
}
-@Override
-public int getStorageLayoutBytes() {
-switch (type) {
-case BOOLEAN:
-case TINYINT:
-return 1;
-case SMALLINT:
-return 2;
-case INT:
-case FLOAT:
-case DECIMAL32:
-return 4;
-case BIGINT:
-case TIME:
-case DATETIME:
-// TODO(Gabriel): unify execution engine and storage engine
-case TIMEV2:
-case DATETIMEV2:
-case DECIMAL64:
-return 8;
-case LARGEINT:
-case DECIMALV2:
-case DECIMAL128:
-return 16;
-case DOUBLE:
-return 12;
-case DATEV2:
-case DATE:
-return 3;
-case CHAR:
-case VARCHAR:
-return len;
-case HLL:
-return 16385;
-case BITMAP:
-return 1024; // this is a estimated value
-case QUANTILE_STATE:
-return 1024; // TODO(weixiang): no used in FE, figure out
whether can delete this funcion?
-case STRING:
-return 1024;
-default:
-return 0;
-}
-}
-
@Override
public TColumnType toColumnTypeThrift() {
TColumnType thrift = new TColumnType();
diff --git a/fe/fe-common/src/main/java/org/apache/doris/catalog/Type.java
b/fe/fe-common/src/main/java/org/apache/doris/catalog/Type.java
index 0fb6a779b6..e6c2e3a4cd 100644
--- a/fe/fe-common/src/main/java/org/apache/doris/catalog/Type.java
+++ b/fe/fe-common/src/main/java/org/apache/doris/catalog/Type.java
@@ -1704,13 +1704,9 @@ public abstract class Type {
}
}
-public int getStorageLayoutBytes() {
-return 0;
-}
-
public int getIndexSize() {
if (this.getPrimitiveType() == PrimitiveType.CHAR) {
-return ((ScalarType) this).getLength();
+return this.getLength();
} else {
return this.getPrimitiveType().getOlapColumnIndexSize();
}
diff --git a/fe/fe-core/src/main/java/org/apache/doris/analysis/ExprId.java
b/fe/fe-core/src/main/java/org/apache/doris/analysis/ExprId.java
index 303d2fbb9d..2c3e64941d 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/analysis/ExprId.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/analysis/ExprId.java
@@ -35,11 +35,6 @@ public class ExprId extends Id {
public ExprId getNextId() {
return new ExprId(nextId++);
}
-
-@Override
-public ExprId getMaxId() {
-retu
[GitHub] [doris] github-actions[bot] commented on pull request #16129: [cleanup](fe) cleanup useless code.
github-actions[bot] commented on PR #16129: URL: https://github.com/apache/doris/pull/16129#issuecomment-1407371121 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] yiguolei merged pull request #16129: [cleanup](fe) cleanup useless code.
yiguolei merged PR #16129: URL: https://github.com/apache/doris/pull/16129 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] jacktengg opened a new issue, #16165: [Bug] join produces too many rows causing be coredump
jacktengg opened a new issue, #16165: URL: https://github.com/apache/doris/issues/16165 ### Search before asking - [X] I had searched in the [issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no similar issues. ### Version master 7e7fd5d0496dc54cc8353ea6138d92bb99b5bdbe ### What's Wrong? sql causing be coredump: ``` SELECT /*+ SET_VAR(query_timeout = 600) */ CASE WHEN CASE WHEN ref_2.`p02` IS NOT NULL THEN ref_2.`sid` ELSE ref_2.`sid` END IS NULL THEN ref_2.`p08` ELSE ref_2.`p08` END AS c0, ref_16.`object_size` AS c1, ref_1.`wr_net_loss` AS c2, ref_2.`p12` AS c3, bitmap_empty() AS c4, ref_16.`request` AS c5 FROM regression_test_tpcds_sf1_unique_p1.web_returns AS ref_1 LEFT JOIN regression_test_37Wan.ods_register AS ref_2 ON (ref_1.`wr_returned_date_sk` = ref_2.`time`) LEFT JOIN regression_test_brown_p2.logs2 AS ref_16 ON (ref_1.`wr_returning_addr_sk` = ref_16.`object_size`) WHERE ref_2.`p12` IS NULL ``` coredump callstack: ``` ==2031262==ERROR: AddressSanitizer: stack-buffer-underflow on address 0x7f2d23eebaf0 at pc 0x55654a887e53 bp 0x7f2d23d41cd0 sp 0x7f2d23d41478 WRITE of size 10218101 at 0x7f2d23eebaf0 thread T275 (FragmentMgrThre) #0 0x55654a887e52 in memset (/mnt/disk1/yuejing/projects/doris/output/be/lib/doris_be+0x11f97e52) #1 0x55655415bf74 in doris::Status doris::vectorized::FunctionCase::execute_impl, false, true>(std::shared_ptr const&, doris::vectorized::Block&, unsigned long, doris::vectorized::CaseWhenColumnHolder) /mnt/disk1/yuejing/projects/doris/be/src/vec/functions/function_case.h:190 #2 0x5565540f66b5 in doris::Status doris::vectorized::FunctionCase::execute_get_then_null, false>(std::shared_ptr const&, doris::vectorized::Block&, std::vector > const&, unsigned long, unsigned long) /mnt/disk1/yuejing/projects/doris/be/src/vec/functions/function_case.h:322 #3 0x5565540d238f in doris::Status doris::vectorized::FunctionCase::execute_get_when_null >(std::shared_ptr const&, doris::vectorized::Block&, std::vector > const&, unsigned long, unsigned long) /mnt/disk1/yuejing/projects/doris/be/src/vec/functions/function_case.h:352 #4 0x5565540c607e in doris::vectorized::FunctionCase::execute_get_type(std::shared_ptr const&, doris::vectorized::Block&, std::vector > const&, unsigned long, unsigned long) /mnt/disk1/yuejing/projects/doris/be/src/vec/functions/function_case.h:367 #5 0x5565540c3682 in doris::vectorized::FunctionCase::execute_impl(doris_udf::FunctionContext*, doris::vectorized::Block&, std::vector > const&, unsigned long, unsigned long) /mnt/disk1/yuejing/projects/doris/be/src/vec/functions/function_case.h:375 #6 0x55655266d3b7 in doris::vectorized::DefaultExecutable::execute_impl(doris_udf::FunctionContext*, doris::vectorized::Block&, std::vector > const&, unsigned long, unsigned long) /mnt/disk1/yuejing/projects/doris/be/src/vec/functions/function.h:467 #7 0x556553fd213a in doris::vectorized::PreparedFunctionImpl::execute_without_low_cardinality_columns(doris_udf::FunctionContext*, doris::vectorized::Block&, std: :vector > const&, unsigned long, unsigned long, bool) /mnt/disk1/yuejing/projects/doris/be/src/vec/functions/function.cpp :251 #8 0x556553fd223a in doris::vectorized::PreparedFunctionImpl::execute(doris_udf::FunctionContext*, doris::vectorized::Block&, std::vector > const&, unsigned long, unsigned long, bool) /mnt/disk1/yuejing/projects/doris/be/src/vec/functions/function.cpp:273 #9 0x556552669fe2 in doris::vectorized::IFunctionBase::execute(doris_udf::FunctionContext*, doris::vectorized::Block&, std::vector > const&, unsigned long, unsigned long, bool) /mnt/disk1/yuejing/projects/doris/be/src/vec/functions/function.h:136 #10 0x5565525f867f in doris::vectorized::VCaseExpr::execute(doris::vectorized::VExprContext*, doris::vectorized::Block*, int*) /mnt/disk1/yuejing/projects/doris/b e/src/vec/exprs/vcase_expr.cpp:89 #11 0x5565525544a7 in doris::vectorized::VectorizedFnCall::execute(doris::vectorized::VExprContext*, doris::vectorized::Block*, int*) /mnt/disk1/yuejing/projects/ doris/be/src/vec/exprs/vectorized_fn_call.cpp:101 #12 0x5565525f82a5 in doris::vectorized::VCaseExpr::execute(doris::vectorized::VExprContext*, doris::vectorized::Block*, int*) /mnt/disk1/yuejing/projects/doris/b e/src/vec/exprs/vcase_expr.cpp:80 #13 0x55655256677c in
[GitHub] [doris] jacktengg opened a new pull request, #16166: [fix](hashjoin) join produce blocks with rows larger than batch size
jacktengg opened a new pull request, #16166: URL: https://github.com/apache/doris/pull/16166 # Proposed changes Issue Number: close #16165 ## Problem summary Describe your changes. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [ ] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [ ] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [ ] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on a diff in pull request #16166: [fix](hashjoin) join produce blocks with rows larger than batch size
github-actions[bot] commented on code in PR #16166:
URL: https://github.com/apache/doris/pull/16166#discussion_r1089717648
##
be/src/vec/exec/join/process_hash_table_probe_impl.h:
##
@@ -208,123 +207,180 @@ Status
ProcessHashTableProbe::do_process(HashTableType& hash_table_c
bool all_match_one = true;
int last_probe_index = probe_index;
+size_t probe_size = 0;
+auto& probe_row_match_iter = hash_table_ctx.probe_row_match_iter;
{
SCOPED_TIMER(_search_hashtable_timer);
-while (probe_index < probe_rows) {
-if constexpr (ignore_null && need_null_map_for_probe) {
-if ((*null_map)[probe_index]) {
-if constexpr (probe_all) {
-_items_counts[probe_index++] = (uint32_t)1;
-// only full outer / left outer need insert the data
of right table
-if (LIKELY(current_offset < _build_block_rows.size()))
{
-_build_block_offsets[current_offset] = -1;
-_build_block_rows[current_offset] = -1;
-} else {
-_build_block_offsets.emplace_back(-1);
-_build_block_rows.emplace_back(-1);
-}
-++current_offset;
+if constexpr (!is_right_semi_anti_join) {
+if (probe_row_match_iter.ok()) {
+for (; probe_row_match_iter.ok(); ++probe_row_match_iter) {
+if (LIKELY(current_offset < _build_block_rows.size())) {
+_build_block_offsets[current_offset] =
probe_row_match_iter->block_offset;
+_build_block_rows[current_offset] =
probe_row_match_iter->row_num;
} else {
-_items_counts[probe_index++] = (uint32_t)0;
+
_build_block_offsets.emplace_back(probe_row_match_iter->block_offset);
+
_build_block_rows.emplace_back(probe_row_match_iter->row_num);
}
-all_match_one = false;
-continue;
-}
-}
-int last_offset = current_offset;
-auto find_result =
-!need_null_map_for_probe
-? key_getter.find_key(hash_table_ctx.hash_table,
probe_index, *_arena)
-: (*null_map)[probe_index]
-?
decltype(key_getter.find_key(hash_table_ctx.hash_table, probe_index,
- *_arena)) {nullptr,
false}
-: key_getter.find_key(hash_table_ctx.hash_table,
probe_index, *_arena);
-if (probe_index + PREFETCH_STEP < probe_rows)
-key_getter.template prefetch(hash_table_ctx.hash_table,
- probe_index +
PREFETCH_STEP, *_arena);
-
-if constexpr (JoinOpType == TJoinOp::LEFT_ANTI_JOIN ||
- JoinOpType == TJoinOp::NULL_AWARE_LEFT_ANTI_JOIN) {
-if (is_mark_join) {
-++current_offset;
-
assert_cast&>(*mcol[mcol.size() - 1])
-.get_data()
-.template push_back(!find_result.is_found());
-} else {
-if (!find_result.is_found()) {
-++current_offset;
+if (++current_offset >= _batch_size) {
+break;
}
}
-} else if constexpr (JoinOpType == TJoinOp::LEFT_SEMI_JOIN) {
-if (is_mark_join) {
-++current_offset;
-
assert_cast&>(*mcol[mcol.size() - 1])
-.get_data()
-.template push_back(find_result.is_found());
-} else {
-if (find_result.is_found()) {
-++current_offset;
-}
+all_match_one &= (current_offset == 1);
+_items_counts[probe_index] = current_offset;
+if (!probe_row_match_iter.ok()) {
+++probe_index;
}
-} else {
-DCHECK(!is_mark_join);
-if (find_result.is_found()) {
-auto& mapped = find_result.get_mapped();
-// TODO: Iterators are currently considered to be a heavy
operation and have a certain impact on performance.
-// We should rethink whether to use this iterator mode in
the future. Now just opt the one row case
-if (mapped.get_row_count() == 1) {
-if constexpr (std::is_same_v) {
-mapped.visited = true;
-}
+pro
[GitHub] [doris] Tanya-W opened a new pull request, #16167: [Feature-WIP] (inverted index) Filter out and remain predicates that do not support apply by inverted index, and add inverted index regressi
Tanya-W opened a new pull request, #16167: URL: https://github.com/apache/doris/pull/16167 # Proposed changes 1. Filter out and remain predicates that do not support apply by inverted index, like `BF` predicate, `IS_NULL` predicate, `IS_NOT_NULL` predicate. 2. Add inverted index regression case that based on tpcds_sf1 data set. ## Problem summary Describe your changes. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [ ] No - [ ] I don't know 3. Has unit tests been added: - [ ] Yes - [ ] No - [ ] No Need 4. Has document been added or modified: - [ ] Yes - [ ] No - [ ] No Need 5. Does it need to update dependencies: - [ ] Yes - [ ] No 6. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [ ] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #16161: [fix](inverted index) fulltext match query should not prune by zone map
hello-stephen commented on PR #16161: URL: https://github.com/apache/doris/pull/16161#issuecomment-1407376959 TeamCity pipeline, clickbench performance test result: the sum of best hot time: 35.29 seconds load time: 539 seconds storage size: 17122430817 Bytes https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230128111713_clickbench_pr_85584.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #16167: [Feature-WIP](inverted index) Filter out and remain predicates that do not support apply by inverted index, and add inverted inde
github-actions[bot] commented on PR #16167: URL: https://github.com/apache/doris/pull/16167#issuecomment-1407377037 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #16161: [fix](inverted index) fulltext match query should not prune by zone map
github-actions[bot] commented on PR #16161: URL: https://github.com/apache/doris/pull/16161#issuecomment-1407377323 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #16167: [Feature-WIP](inverted index) Filter out and remain predicates that do not support apply by inverted index, and add inverted inde
github-actions[bot] commented on PR #16167: URL: https://github.com/apache/doris/pull/16167#issuecomment-1407377423 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #16162: [refactor](remove unused code) remove partition info from datastream sender
hello-stephen commented on PR #16162: URL: https://github.com/apache/doris/pull/16162#issuecomment-1407379910 TeamCity pipeline, clickbench performance test result: the sum of best hot time: 35.08 seconds load time: 489 seconds storage size: 17122304340 Bytes https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230128113623_clickbench_pr_85600.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #16156: [refactor](Nereids) bind slot, function and do type coercion in one rule
hello-stephen commented on PR #16156: URL: https://github.com/apache/doris/pull/16156#issuecomment-1407380117 TeamCity pipeline, clickbench performance test result: the sum of best hot time: 34.17 seconds load time: 524 seconds storage size: 17122260141 Bytes https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230128113743_clickbench_pr_85630.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] Gabriel39 closed issue #15986: [Bug] if 函数后计算结果发生错误
Gabriel39 closed issue #15986: [Bug] if 函数后计算结果发生错误 URL: https://github.com/apache/doris/issues/15986 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] Gabriel39 merged pull request #16152: [Bug](decimalv3) fix BE crash for function `if`
Gabriel39 merged PR #16152: URL: https://github.com/apache/doris/pull/16152 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated (7e7fd5d049 -> 26fc7c8196)
This is an automated email from the ASF dual-hosted git repository.
gabriellee pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
from 7e7fd5d049 [cleanup](fe) cleanup useless code. (#16129)
add 26fc7c8196 [Bug](decimalv3) fix BE crash for function `if` (#16152)
No new revisions were added by this update.
Summary of changes:
.../apache/doris/analysis/FunctionCallExpr.java| 10
.../decimalv3/test_functions.out} | 3 ++-
.../{test_load.groovy => test_functions.groovy}| 27 +-
3 files changed, 23 insertions(+), 17 deletions(-)
copy regression-test/data/{correctness_p0/test_bitmap_intersect.out =>
datatype_p0/decimalv3/test_functions.out} (75%)
copy regression-test/suites/datatype_p0/decimalv3/{test_load.groovy =>
test_functions.groovy} (59%)
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #16167: [Feature-WIP](inverted index) Filter out and remain predicates that do not support apply by inverted index, and add inverted inde
github-actions[bot] commented on PR #16167: URL: https://github.com/apache/doris/pull/16167#issuecomment-1407380183 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #16164: [fix](Nereids): fix Ceiling.
hello-stephen commented on PR #16164: URL: https://github.com/apache/doris/pull/16164#issuecomment-1407383197 TeamCity pipeline, clickbench performance test result: the sum of best hot time: 34.34 seconds load time: 490 seconds storage size: 17122375691 Bytes https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230128115628_clickbench_pr_85621.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] yiguolei merged pull request #16162: [refactor](remove unused code) remove partition info from datastream sender
yiguolei merged PR #16162: URL: https://github.com/apache/doris/pull/16162 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated (26fc7c8196 -> 241a956b20)
This is an automated email from the ASF dual-hosted git repository. yiguolei pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/doris.git from 26fc7c8196 [Bug](decimalv3) fix BE crash for function `if` (#16152) add 241a956b20 [refactor](remove unused code) remove partition info from datastream sender (#16162) No new revisions were added by this update. Summary of changes: be/src/pipeline/exec/exchange_sink_operator.cpp | 1 - be/src/runtime/CMakeLists.txt | 2 - be/src/runtime/dpp_sink_internal.cpp| 137 - be/src/runtime/dpp_sink_internal.h | 186 be/src/runtime/raw_value.cpp| 161 be/src/runtime/raw_value.h | 146 --- be/src/vec/CMakeLists.txt | 1 - be/src/vec/runtime/vpartition_info.cpp | 55 --- be/src/vec/runtime/vpartition_info.h| 59 be/src/vec/sink/vdata_stream_sender.cpp | 32 +--- be/src/vec/sink/vdata_stream_sender.h | 7 +- be/test/CMakeLists.txt | 2 - be/test/runtime/data_spliter_test.cpp | 176 -- be/test/runtime/dpp_sink_internal_test.cpp | 96 be/test/runtime/raw_value_test.cpp | 45 -- 15 files changed, 2 insertions(+), 1104 deletions(-) delete mode 100644 be/src/runtime/dpp_sink_internal.cpp delete mode 100644 be/src/runtime/dpp_sink_internal.h delete mode 100644 be/src/runtime/raw_value.cpp delete mode 100644 be/src/vec/runtime/vpartition_info.cpp delete mode 100644 be/src/vec/runtime/vpartition_info.h delete mode 100644 be/test/runtime/data_spliter_test.cpp delete mode 100644 be/test/runtime/dpp_sink_internal_test.cpp delete mode 100644 be/test/runtime/raw_value_test.cpp - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #16153: [bug](cooldown)add config for Cooldown Job
github-actions[bot] commented on PR #16153: URL: https://github.com/apache/doris/pull/16153#issuecomment-1407383521 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #16153: [bug](cooldown)add config for Cooldown Job
github-actions[bot] commented on PR #16153: URL: https://github.com/apache/doris/pull/16153#issuecomment-1407383534 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] yiguolei merged pull request #16153: [bug](cooldown)add config for Cooldown Job
yiguolei merged PR #16153: URL: https://github.com/apache/doris/pull/16153 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated: [bug](cooldown)add config for Cooldown Job
This is an automated email from the ASF dual-hosted git repository.
yiguolei pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new c506b4a1e3 [bug](cooldown)add config for Cooldown Job
c506b4a1e3 is described below
commit c506b4a1e35d49566ddcbeb06d13d08b4bc441f9
Author: pengxiangyu
AuthorDate: Sat Jan 28 19:58:50 2023 +0800
[bug](cooldown)add config for Cooldown Job
---
.../org/apache/doris/cooldown/CooldownHandler.java | 20
.../main/java/org/apache/doris/persist/EditLog.java | 6 --
2 files changed, 16 insertions(+), 10 deletions(-)
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/cooldown/CooldownHandler.java
b/fe/fe-core/src/main/java/org/apache/doris/cooldown/CooldownHandler.java
index fe0815c370..4ad601e5f4 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/cooldown/CooldownHandler.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/cooldown/CooldownHandler.java
@@ -136,18 +136,22 @@ public class CooldownHandler extends MasterDaemon {
}
public void write(DataOutput out) throws IOException {
-out.writeInt(runableCooldownJobs.size());
-for (CooldownJob cooldownJob : runableCooldownJobs.values()) {
-cooldownJob.write(out);
+if (Config.cooldown_single_remote_file) {
+out.writeInt(runableCooldownJobs.size());
+for (CooldownJob cooldownJob : runableCooldownJobs.values()) {
+cooldownJob.write(out);
+}
}
}
public void readField(DataInput in) throws IOException {
-if (Env.getCurrentEnvJournalVersion() >= FeMetaVersion.VERSION_115) {
-int size = in.readInt();
-for (int i = 0; i < size; i++) {
-CooldownJob cooldownJob = CooldownJob.read(in);
-replayCooldownJob(cooldownJob);
+if (Config.cooldown_single_remote_file) {
+if (Env.getCurrentEnvJournalVersion() >=
FeMetaVersion.VERSION_115) {
+int size = in.readInt();
+for (int i = 0; i < size; i++) {
+CooldownJob cooldownJob = CooldownJob.read(in);
+replayCooldownJob(cooldownJob);
+}
}
}
}
diff --git a/fe/fe-core/src/main/java/org/apache/doris/persist/EditLog.java
b/fe/fe-core/src/main/java/org/apache/doris/persist/EditLog.java
index 83315e4dd7..4c359c72d7 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/persist/EditLog.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/persist/EditLog.java
@@ -721,8 +721,10 @@ public class EditLog {
break;
}
case OperationType.OP_PUSH_COOLDOWN_CONF:
-CooldownJob cooldownJob = (CooldownJob) journal.getData();
-env.getCooldownHandler().replayCooldownJob(cooldownJob);
+if (Config.cooldown_single_remote_file) {
+CooldownJob cooldownJob = (CooldownJob)
journal.getData();
+
env.getCooldownHandler().replayCooldownJob(cooldownJob);
+}
break;
case OperationType.OP_BATCH_ADD_ROLLUP: {
BatchAlterJobPersistInfo batchAlterJobV2 =
(BatchAlterJobPersistInfo) journal.getData();
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] BiteTheDDDDt opened a new pull request, #16168: [Bug](function) remove datev2 signature of hour_ceil/hour_floor
BiteThet opened a new pull request, #16168: URL: https://github.com/apache/doris/pull/16168 # Proposed changes remove datev2 signature of hour_ceil/hour_floor ## Problem summary Describe your changes. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [ ] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [ ] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [ ] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on a diff in pull request #16159: [datev2](function) make function nullable DEPEND_ON_ARGUMENT
github-actions[bot] commented on code in PR #16159:
URL: https://github.com/apache/doris/pull/16159#discussion_r1089723206
##
be/src/vec/functions/function_timestamp.cpp:
##
@@ -721,37 +837,59 @@
static Status execute_impl(FunctionContext* context, Block& block,
const ColumnNumbers& arguments, size_t result,
size_t input_rows_count) {
-auto null_map = ColumnUInt8::create(input_rows_count, 0);
-ColumnPtr res_column;
+const auto is_nullable =
block.get_by_position(result).type->is_nullable();
ColumnPtr argument_column =
block.get_by_position(arguments[0]).column->convert_to_full_column_if_const();
-if constexpr (std::is_same_v ||
- std::is_same_v) {
-auto data_col = assert_cast*>(argument_column.get());
-res_column = ColumnInt64::create(input_rows_count);
-execute_straight(
-input_rows_count, null_map->get_data(),
data_col->get_data(),
-
static_cast*>(res_column->assume_mutable().get())
-->get_data());
-
-} else if constexpr (std::is_same_v) {
-auto data_col = assert_cast*>(argument_column.get());
-res_column = ColumnVector::create(input_rows_count);
-execute_straight, UInt32, UInt32>(
-input_rows_count, null_map->get_data(),
data_col->get_data(),
-
static_cast*>(res_column->assume_mutable().get())
-->get_data());
+ColumnPtr res_column;
+if (is_nullable) {
+auto null_map = ColumnUInt8::create(input_rows_count, 0);
+if constexpr (std::is_same_v ||
+ std::is_same_v) {
+auto data_col = assert_cast*>(argument_column.get());
+res_column = ColumnInt64::create(input_rows_count);
+execute_straight(
+input_rows_count, null_map->get_data(),
data_col->get_data(),
+
static_cast*>(res_column->assume_mutable().get())
+->get_data());
-} else if constexpr (std::is_same_v) {
-auto data_col = assert_cast*>(argument_column.get());
-res_column = ColumnVector::create(input_rows_count);
-execute_straight, UInt32, UInt64>(
-input_rows_count, null_map->get_data(),
data_col->get_data(),
-
static_cast*>(res_column->assume_mutable().get())
-->get_data());
+} else if constexpr (std::is_same_v) {
+auto data_col = assert_cast*>(argument_column.get());
+res_column = ColumnVector::create(input_rows_count);
+execute_straight, UInt32, UInt32>(
+input_rows_count, null_map->get_data(),
data_col->get_data(),
+
static_cast*>(res_column->assume_mutable().get())
+->get_data());
+
+} else if constexpr (std::is_same_v)
{
+auto data_col = assert_cast*>(argument_column.get());
+res_column = ColumnVector::create(input_rows_count);
+execute_straight, UInt32,
UInt64>(
+input_rows_count, null_map->get_data(),
data_col->get_data(),
+
static_cast*>(res_column->assume_mutable().get())
+->get_data());
+}
+block.replace_by_position(
+result, ColumnNullable::create(std::move(res_column),
std::move(null_map)));
Review Comment:
warning: passing result of std::move() as a const reference argument; no
move will actually happen [performance-move-const-arg]
```suggestion
result, ColumnNullable::create(res_column,
std::move(null_map)));
```
##
be/src/vec/functions/function_timestamp.cpp:
##
@@ -620,38 +690,59 @@ struct LastDayImpl {
static Status execute_impl(FunctionContext* context, Block& block,
const ColumnNumbers& arguments, size_t result,
size_t input_rows_count) {
-auto null_map = ColumnUInt8::create(input_rows_count, 0);
+const auto is_nullable =
block.get_by_position(result).type->is_nullable();
+auto null_map = ColumnUInt8::create();
ColumnPtr res_column;
ColumnPtr argument_column =
block.get_by_position(arguments[0]).column->convert_to_full_column_if_const();
-if constexpr (std::is_same_v ||
- std::is_same_v) {
-auto data_col = assert_cast*>(argument_column.get());
-res_column = ColumnInt64::create(input_rows_count);
-execute_straight(
-input_rows_count, null_map->get_data(),
da
[GitHub] [doris] github-actions[bot] commented on pull request #16168: [Bug](function) remove datev2 signature of hour_ceil/hour_floor
github-actions[bot] commented on PR #16168: URL: https://github.com/apache/doris/pull/16168#issuecomment-1407384603 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #16168: [Bug](function) remove datev2 signature of hour_ceil/hour_floor
github-actions[bot] commented on PR #16168: URL: https://github.com/apache/doris/pull/16168#issuecomment-1407384757 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #16168: [Bug](function) remove datev2 signature of hour_ceil/hour_floor
github-actions[bot] commented on PR #16168: URL: https://github.com/apache/doris/pull/16168#issuecomment-1407384767 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #16159: [datev2](function) make function nullable DEPEND_ON_ARGUMENT
github-actions[bot] commented on PR #16159: URL: https://github.com/apache/doris/pull/16159#issuecomment-1407385770 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on a diff in pull request #16159: [datev2](function) make function nullable DEPEND_ON_ARGUMENT
github-actions[bot] commented on code in PR #16159:
URL: https://github.com/apache/doris/pull/16159#discussion_r1089723868
##
be/src/vec/functions/function_timestamp.cpp:
##
@@ -721,37 +837,59 @@ struct MondayImpl {
static Status execute_impl(FunctionContext* context, Block& block,
const ColumnNumbers& arguments, size_t result,
size_t input_rows_count) {
-auto null_map = ColumnUInt8::create(input_rows_count, 0);
-ColumnPtr res_column;
+const auto is_nullable =
block.get_by_position(result).type->is_nullable();
ColumnPtr argument_column =
block.get_by_position(arguments[0]).column->convert_to_full_column_if_const();
-if constexpr (std::is_same_v ||
- std::is_same_v) {
-auto data_col = assert_cast*>(argument_column.get());
-res_column = ColumnInt64::create(input_rows_count);
-execute_straight(
-input_rows_count, null_map->get_data(),
data_col->get_data(),
-
static_cast*>(res_column->assume_mutable().get())
-->get_data());
-
-} else if constexpr (std::is_same_v) {
-auto data_col = assert_cast*>(argument_column.get());
-res_column = ColumnVector::create(input_rows_count);
-execute_straight, UInt32, UInt32>(
-input_rows_count, null_map->get_data(),
data_col->get_data(),
-
static_cast*>(res_column->assume_mutable().get())
-->get_data());
+ColumnPtr res_column;
+if (is_nullable) {
+auto null_map = ColumnUInt8::create(input_rows_count, 0);
+if constexpr (std::is_same_v ||
+ std::is_same_v) {
+auto data_col = assert_cast*>(argument_column.get());
+res_column = ColumnInt64::create(input_rows_count);
+execute_straight(
+input_rows_count, null_map->get_data(),
data_col->get_data(),
+
static_cast*>(res_column->assume_mutable().get())
+->get_data());
-} else if constexpr (std::is_same_v) {
-auto data_col = assert_cast*>(argument_column.get());
-res_column = ColumnVector::create(input_rows_count);
-execute_straight, UInt32, UInt64>(
-input_rows_count, null_map->get_data(),
data_col->get_data(),
-
static_cast*>(res_column->assume_mutable().get())
-->get_data());
+} else if constexpr (std::is_same_v) {
+auto data_col = assert_cast*>(argument_column.get());
+res_column = ColumnVector::create(input_rows_count);
+execute_straight, UInt32, UInt32>(
+input_rows_count, null_map->get_data(),
data_col->get_data(),
+
static_cast*>(res_column->assume_mutable().get())
+->get_data());
+
+} else if constexpr (std::is_same_v)
{
+auto data_col = assert_cast*>(argument_column.get());
+res_column = ColumnVector::create(input_rows_count);
+execute_straight, UInt32,
UInt64>(
+input_rows_count, null_map->get_data(),
data_col->get_data(),
+
static_cast*>(res_column->assume_mutable().get())
+->get_data());
+}
+block.replace_by_position(
+result, ColumnNullable::create(res_column,
std::move(null_map)));
Review Comment:
warning: passing result of std::move() as a const reference argument; no
move will actually happen [performance-move-const-arg]
```suggestion
result, ColumnNullable::create(res_column,
std::move(null_map)));
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] yiguolei merged pull request #16158: [deps](http)Upgrade springboot version to 2.7.8
yiguolei merged PR #16158: URL: https://github.com/apache/doris/pull/16158 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated: [deps](http)Upgrade springboot version to 2.7.8 (#16158)
This is an automated email from the ASF dual-hosted git repository. yiguolei pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris.git The following commit(s) were added to refs/heads/master by this push: new da28d2faee [deps](http)Upgrade springboot version to 2.7.8 (#16158) da28d2faee is described below commit da28d2faeedc5a68255c3d06a732c7e4f516ec75 Author: jiafeng.zhang AuthorDate: Sat Jan 28 20:13:50 2023 +0800 [deps](http)Upgrade springboot version to 2.7.8 (#16158) * Upgrade springboot version to 2.7.8 * fix --- dist/LICENSE-dist.txt | 46 +++--- fe/fe-core/pom.xml| 10 -- fe/pom.xml| 1 + 3 files changed, 28 insertions(+), 29 deletions(-) diff --git a/dist/LICENSE-dist.txt b/dist/LICENSE-dist.txt index 1ea749cc56..25b6826b9b 100644 --- a/dist/LICENSE-dist.txt +++ b/dist/LICENSE-dist.txt @@ -1075,31 +1075,31 @@ The Apache Software License, Version 2.0 * Spark Project Unsafe: - org.apache.spark:spark-unsafe_2.12:2.4.6 (http://spark.apache.org/) * Spring AOP: -- org.springframework:spring-aop:5.3.22 (https://github.com/spring-projects/spring-framework) +- org.springframework:spring-aop:5.3.25 (https://github.com/spring-projects/spring-framework) * Spring Beans: -- org.springframework:spring-beans:5.3.22 (https://github.com/spring-projects/spring-framework) +- org.springframework:spring-beans:5.3.25 (https://github.com/spring-projects/spring-framework) * Spring Commons Logging Bridge: -- org.springframework:spring-jcl:5.3.22 (https://github.com/spring-projects/spring-framework) +- org.springframework:spring-jcl:5.3.25 (https://github.com/spring-projects/spring-framework) * Spring Context: -- org.springframework:spring-context:5.3.22 (https://github.com/spring-projects/spring-framework) +- org.springframework:spring-context:5.3.25 (https://github.com/spring-projects/spring-framework) * Spring Core: -- org.springframework:spring-core:5.3.22 (https://github.com/spring-projects/spring-framework) +- org.springframework:spring-core:5.3.25 (https://github.com/spring-projects/spring-framework) * Spring Data Core: -- org.springframework.data:spring-data-commons:2.7.2 (https://www.spring.io/spring-data/spring-data-commons) +- org.springframework.data:spring-data-commons:2.7.7 (https://www.spring.io/spring-data/spring-data-commons) * Spring Data LDAP: -- org.springframework.data:spring-data-ldap:2.7.2 (https://github.com/spring-projects/spring-data-ldap) +- org.springframework.data:spring-data-ldap:2.7.7 (https://github.com/spring-projects/spring-data-ldap) * Spring Expression Language (SpEL): -- org.springframework:spring-expression:5.3.22 (https://github.com/spring-projects/spring-framework) +- org.springframework:spring-expression:5.3.25 (https://github.com/spring-projects/spring-framework) * Spring JDBC: -- org.springframework:spring-jdbc:5.3.22 (https://github.com/spring-projects/spring-framework) +- org.springframework:spring-jdbc:5.3.25 (https://github.com/spring-projects/spring-framework) * Spring Object/Relational Mapping: -- org.springframework:spring-orm:5.3.22 (https://github.com/spring-projects/spring-framework) +- org.springframework:spring-orm:5.3.25 (https://github.com/spring-projects/spring-framework) * Spring Transaction: -- org.springframework:spring-tx:5.3.22 (https://github.com/spring-projects/spring-framework) +- org.springframework:spring-tx:5.3.25 (https://github.com/spring-projects/spring-framework) * Spring Web: -- org.springframework:spring-web:5.3.22 (https://github.com/spring-projects/spring-framework) +- org.springframework:spring-web:5.3.25 (https://github.com/spring-projects/spring-framework) * Spring Web MVC: -- org.springframework:spring-webmvc:5.3.22 (https://github.com/spring-projects/spring-framework) +- org.springframework:spring-webmvc:5.3.25 (https://github.com/spring-projects/spring-framework) * Springloaded: - org.springframework:springloaded:1.2.6.RELEASE (https://github.com/spring-projects/spring-loaded) * StAX API: @@ -1246,25 +1246,25 @@ The Apache Software License, Version 2.0 * spark-dpp: - org.apache.doris:spark-dpp:0.15-SNAPSHOT (https://doris.apache.org/spark-dpp/) * spring-boot: -- org.springframework.boot:spring-boot:2.7.3 (https://spring.io/projects/spring-boot) +- org.springframework.boot:spring-boot:2.7.8 (https://spring.io/projects/spring-boot) * spring-boot-autoconfigure: -- org.springframework.boot:spring-boot-autoconfigure:2.7.3 (https://spring.io/projects/spring-boot) +- org.springframework.boot:spring-boot-autoconfigure:2.7.8 (https:
[GitHub] [doris] yiguolei merged pull request #16145: [bugfix](compaction) fix segment writer finalize footer twice
yiguolei merged PR #16145: URL: https://github.com/apache/doris/pull/16145 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated (da28d2faee -> 8ca3aa85e2)
This is an automated email from the ASF dual-hosted git repository. yiguolei pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/doris.git from da28d2faee [deps](http)Upgrade springboot version to 2.7.8 (#16158) add 8ca3aa85e2 [bugfix](compaction) fix segment writer finalize footer twice (#16145) No new revisions were added by this update. Summary of changes: be/src/olap/rowset/segment_v2/segment_writer.cpp | 7 ++- be/src/olap/rowset/segment_v2/segment_writer.h | 1 + 2 files changed, 7 insertions(+), 1 deletion(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] Kikyou1997 opened a new pull request, #16169: [fix](nereids) Delete invalid signature of nvl function
Kikyou1997 opened a new pull request, #16169: URL: https://github.com/apache/doris/pull/16169 1. delete invalid signature of nvl function 2. fix some test cases that failed because of malformed function name # Proposed changes Issue Number: close #xxx ## Problem summary Describe your changes. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [ ] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [ ] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [ ] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on a diff in pull request #16157: [Bug](exec) enable warning on ignoring function return value for vctx
github-actions[bot] commented on code in PR #16157:
URL: https://github.com/apache/doris/pull/16157#discussion_r1089725536
##
be/test/vec/exprs/vexpr_test.cpp:
##
@@ -83,8 +86,11 @@ TEST(TEST_VEXPR, ABSTEST2) {
DescriptorTbl desc_tbl;
Review Comment:
warning: calling a private constructor of class 'doris::DescriptorTbl'
[clang-diagnostic-error]
```cpp
DescriptorTbl desc_tbl;
^
```
**be/src/runtime/descriptors.h:426:** declared private here
```cpp
DescriptorTbl() = default;
^
```
##
be/test/vec/exprs/vexpr_test.cpp:
##
@@ -83,8 +86,11 @@
DescriptorTbl desc_tbl;
desc_tbl._slot_desc_map[0] = tuple_desc->slots()[0];
Review Comment:
warning: '_slot_desc_map' is a private member of 'doris::DescriptorTbl'
[clang-diagnostic-error]
```cpp
desc_tbl._slot_desc_map[0] = tuple_desc->slots()[0];
^
```
**be/src/runtime/descriptors.h:423:** declared private here
```cpp
SlotDescriptorMap _slot_desc_map;
^
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] jackwener merged pull request #16164: [fix](Nereids): fix Ceiling.
jackwener merged PR #16164: URL: https://github.com/apache/doris/pull/16164 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated: [fix](Nereids): fix Ceiling. (#16164)
This is an automated email from the ASF dual-hosted git repository.
jakevin pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 3151d94e9e [fix](Nereids): fix Ceiling. (#16164)
3151d94e9e is described below
commit 3151d94e9e6f63d95bbeeadaf2b94d7b80f0a0e6
Author: jakevin
AuthorDate: Sat Jan 28 20:26:20 2023 +0800
[fix](Nereids): fix Ceiling. (#16164)
---
.../doris/catalog/BuiltinScalarFunctions.java | 4 +-
.../expressions/functions/scalar/Ceiling.java | 69 --
.../expressions/visitor/ScalarFunctionVisitor.java | 5 --
.../nereids_function_p0/scalar_function/A-F.out| 17 ++
.../nereids_function_p0/scalar_function/A-F.groovy | 3 +-
5 files changed, 19 insertions(+), 79 deletions(-)
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/catalog/BuiltinScalarFunctions.java
b/fe/fe-core/src/main/java/org/apache/doris/catalog/BuiltinScalarFunctions.java
index caf6fa48d0..baee16adaa 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/catalog/BuiltinScalarFunctions.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/catalog/BuiltinScalarFunctions.java
@@ -78,7 +78,6 @@ import
org.apache.doris.nereids.trees.expressions.functions.scalar.BitmapXorCoun
import org.apache.doris.nereids.trees.expressions.functions.scalar.Cardinality;
import org.apache.doris.nereids.trees.expressions.functions.scalar.Cbrt;
import org.apache.doris.nereids.trees.expressions.functions.scalar.Ceil;
-import org.apache.doris.nereids.trees.expressions.functions.scalar.Ceiling;
import
org.apache.doris.nereids.trees.expressions.functions.scalar.CharacterLength;
import org.apache.doris.nereids.trees.expressions.functions.scalar.Coalesce;
import org.apache.doris.nereids.trees.expressions.functions.scalar.Concat;
@@ -400,8 +399,7 @@ public class BuiltinScalarFunctions implements
FunctionHelper {
scalar(BitmapXorCount.class, "bitmap_xor_count"),
scalar(Cardinality.class, "cardinality"),
scalar(Cbrt.class, "cbrt"),
-scalar(Ceil.class, "ceil"),
-scalar(Ceiling.class, "ceiling"),
+scalar(Ceil.class, "ceil", "ceiling"),
scalar(CharacterLength.class, "char_length", "character_length"),
scalar(Coalesce.class, "coalesce"),
scalar(Concat.class, "concat"),
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/functions/scalar/Ceiling.java
b/fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/functions/scalar/Ceiling.java
deleted file mode 100644
index 44838a837e..00
---
a/fe/fe-core/src/main/java/org/apache/doris/nereids/trees/expressions/functions/scalar/Ceiling.java
+++ /dev/null
@@ -1,69 +0,0 @@
-// Licensed to the Apache Software Foundation (ASF) under one
-// or more contributor license agreements. See the NOTICE file
-// distributed with this work for additional information
-// regarding copyright ownership. The ASF licenses this file
-// to you under the Apache License, Version 2.0 (the
-// "License"); you may not use this file except in compliance
-// with the License. You may obtain a copy of the License at
-//
-// http://www.apache.org/licenses/LICENSE-2.0
-//
-// Unless required by applicable law or agreed to in writing,
-// software distributed under the License is distributed on an
-// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
-// KIND, either express or implied. See the License for the
-// specific language governing permissions and limitations
-// under the License.
-
-package org.apache.doris.nereids.trees.expressions.functions.scalar;
-
-import org.apache.doris.catalog.FunctionSignature;
-import org.apache.doris.nereids.trees.expressions.Expression;
-import
org.apache.doris.nereids.trees.expressions.functions.ExplicitlyCastableSignature;
-import org.apache.doris.nereids.trees.expressions.functions.PropagateNullable;
-import org.apache.doris.nereids.trees.expressions.shape.UnaryExpression;
-import org.apache.doris.nereids.trees.expressions.visitor.ExpressionVisitor;
-import org.apache.doris.nereids.types.BigIntType;
-import org.apache.doris.nereids.types.DoubleType;
-
-import com.google.common.base.Preconditions;
-import com.google.common.collect.ImmutableList;
-
-import java.util.List;
-
-/**
- * ScalarFunction 'ceiling'. This class is generated by GenerateFunction.
- */
-public class Ceiling extends ScalarFunction
-implements UnaryExpression, ExplicitlyCastableSignature,
PropagateNullable {
-
-public static final List SIGNATURES = ImmutableList.of(
-
FunctionSignature.ret(BigIntType.INSTANCE).args(DoubleType.INSTANCE)
-);
-
-/**
- * constructor with 1 argument.
- */
-public Ceiling(Expression arg) {
-super("ceiling", arg);
-}
-
-/**
- * withChildren.
- */
-@Override
-public Ceiling withChildren
[GitHub] [doris] hello-stephen commented on pull request #16166: [fix](hashjoin) join produce blocks with rows larger than batch size
hello-stephen commented on PR #16166: URL: https://github.com/apache/doris/pull/16166#issuecomment-1407389152 TeamCity pipeline, clickbench performance test result: the sum of best hot time: 35.44 seconds load time: 525 seconds storage size: 17122289679 Bytes https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230128123023_clickbench_pr_85639.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #16167: [Feature-WIP](inverted index) Filter out and remain predicates that do not support apply by inverted index, and add inverted index regr
hello-stephen commented on PR #16167: URL: https://github.com/apache/doris/pull/16167#issuecomment-1407391651 TeamCity pipeline, clickbench performance test result: the sum of best hot time: 34.6 seconds load time: 491 seconds storage size: 17123012999 Bytes https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230128124607_clickbench_pr_85661.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] jackwener opened a new pull request, #16170: [fix](Nereids): fix convert_tz() SIGNATURES.
jackwener opened a new pull request, #16170: URL: https://github.com/apache/doris/pull/16170 # Proposed changes we shouldn't add `DateV2` in `SIGNATURES`, because when meet `DateV2`, it will cast to `DateTimeV2Type` implicitly. ## Problem summary Describe your changes. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [ ] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [ ] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [ ] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #16169: [fix](nereids) Delete invalid signature of nvl function
github-actions[bot] commented on PR #16169: URL: https://github.com/apache/doris/pull/16169#issuecomment-1407394710 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #16169: [fix](nereids) Delete invalid signature of nvl function
github-actions[bot] commented on PR #16169: URL: https://github.com/apache/doris/pull/16169#issuecomment-1407394721 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #16170: [fix](Nereids): fix convert_tz() SIGNATURES.
github-actions[bot] commented on PR #16170: URL: https://github.com/apache/doris/pull/16170#issuecomment-1407397943 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #16170: [fix](Nereids): fix convert_tz() SIGNATURES.
github-actions[bot] commented on PR #16170: URL: https://github.com/apache/doris/pull/16170#issuecomment-1407397954 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] morrySnow merged pull request #16163: [Fix](Nereids)changing test data for more effectively testing for nereids_function_p0.
morrySnow merged PR #16163: URL: https://github.com/apache/doris/pull/16163 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated: [test](Nereids) changing test data for more effectively testing for nereids_function_p0 (#16163)
This is an automated email from the ASF dual-hosted git repository. morrysnow pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris.git The following commit(s) were added to refs/heads/master by this push: new b7379daffa [test](Nereids) changing test data for more effectively testing for nereids_function_p0 (#16163) b7379daffa is described below commit b7379daffa1ffe7477e4e43305904a6166c0e8ad Author: mch_ucchi <41606806+sohardforan...@users.noreply.github.com> AuthorDate: Sat Jan 28 21:23:40 2023 +0800 [test](Nereids) changing test data for more effectively testing for nereids_function_p0 (#16163) --- .../data/nereids_function_p0/fn_test.dat | 26 +- .../nereids_function_p0/scalar_function/A-F.out| 2404 +++ .../nereids_function_p0/scalar_function/G-K.out| 1096 +++ .../nereids_function_p0/scalar_function/L-Q.out| 3150 +--- .../nereids_function_p0/scalar_function/R-W.out| 3108 +-- .../nereids_function_p0/scalar_function/X-Z.out| 1020 +++ .../suites/nereids_function_p0/load.groovy |2 +- .../nereids_function_p0/scalar_function/A-F.groovy |9 +- .../nereids_function_p0/scalar_function/X-Z.groovy | 12 +- 9 files changed, 4970 insertions(+), 5857 deletions(-) diff --git a/regression-test/data/nereids_function_p0/fn_test.dat b/regression-test/data/nereids_function_p0/fn_test.dat index 9c0a8e1b76..6f98c3a87c 100644 --- a/regression-test/data/nereids_function_p0/fn_test.dat +++ b/regression-test/data/nereids_function_p0/fn_test.dat @@ -1,15 +1,13 @@ -0;1;1980;423524523;9223372036854775807;63546353464356436345346345;211.4;4235.432523;25344.543;4324.32523;23352.23524;25344.543;4324.32523;23352.23524;dsgs;Sdgsdfsdg;Nfwkfhwukbvkwbfsejfbwebf;dsgs;Sdgsdfsdg;Nfwkfhwukbvkwbfsejfbwebf;Nfwkfhwukbvkwbfsejfbwebf;1903-12-24;1903-12-24;1451-12-23 13:04:45;1451-12-23 13:04:45;1451-12-14 13:04:45.3436;1451-12-14 13:04:45.34536;1451-12-14 13:04:45.34536;[0];[1];[1980];[423524523];[9223372036854775807];[63546353464356436345346345];[211.4];[1903-12-24, [...] -0;2;1986;2147483647;-9223372036854775807;-54673967938678953673967348967;0.2;534.52352;64564.235;325262.435;235423626.4;64564.235;325262.435;235423626.4;Jehjd;Gsdgwgsg;Fbwekfbwcweubfwkebcwefewfwefbw;Jehdjd;Gsdgwgsg;Fbwekfbwcweubfwkebcwefewfwefbw;Fbwekfbwcweubfwkebcwefewfwefbw;-12-21;-12-21;1411-12-23 13:04:45;1411-12-23 13:04:00;1411-12-14 13:04:45.7667;1411-12-14 13:04:45.76567;1411-12-14 13:04:45.76567;[0];[2];[1986];[2147483647];[-9223372036854775807];[-546739679386789536739673 [...] -0;3;1992;-2147483647;4359080345834638390;86904358903467903579034679304579304;241.154;-35245.52352;52623.25;3543.35;3463634.544;52623.25;3543.35;3463634.544;Fhdr;Gsgwy;Wefbksncweukbfkwbffwegwfcwef;Fhdr;Gsgwy;Wefbksncweukbfkwbffwegwfcwef;Wefbksncweukbfkwbffwegwfcwef;0001-12-21;0001-12-21;1371-12-22 13:04:45;1451-12-23 13:04:45;1451-12-14 13:04:45.3436;1451-12-14 13:04:45.34536;1451-12-14 13:04:45.34536;[0];[3];[1992];[-2147483647];[4359080345834638390];[86904358903467903579034679304579304] [...] -0;4;1998;54534242;3476903769356803;-890235723904023750237409237590232;-1241.45;-5325.325235;0.35;-4324.32523;-45463.53464;0.35;-4324.32523;-45463.53464;Dfhdf;Fsgsdrwgh;Fwefbnckbewvbfwkubcwebfwec;Dfhdf;Fsgsdrwgh;Fwefbnckbewvbfwkubcwebfwec;Fwefbnckbewvbfwkubcwebfwec;2414-01-12;2414-01-12;1331-12-22 13:04:45;1411-12-23 13:04:45;1411-12-14 13:04:45.7657;1411-12-14 13:04:45.76567;1411-12-14 13:04:45.76567;[0];[4];[1998];[54534242];[3476903769356803];[-890235723904023750237409237590232];[-1241 [...] -0;5;32767;234252;-4643347364763456;234325893724892375823742523;124.41;23523.45232;4364.6;-325262.435;23352.23524;4364.6;-325262.435;23352.23524;Heed;Fwnjwbfw;Wehnckwewebvwefwcwe;Heed;Fwnjwbfw;Wehnckwewebvwefwcwe;Wehnckwewebvwefwcwe;1903-12-24;1903-12-24;1291-12-21 13:04:45;1451-12-23 13:04:45;1451-12-14 13:04:45.3436;1451-12-14 13:04:45.34536;1451-12-14 13:04:45.34536;[0];[5];[32767];[234252];[-4643347364763456];[234325893724892375823742523];[124.41];[1903-12-24, 1903-12-24];[1291-12-14 [...] -0;6;-32767;-2352342;34634796439863689;532623456245244235;-41251.1;523.43252;-0.436;-3543.35;235423626.4;-0.436;-3543.35;235423626.4;Hdefd;Ewgweufhwfwf;fwevbwcbkweb;Hdefd;Ewgweufhwfwf;fwevbwcbkweb;fwevbwcbkweb;9599-12-21;9599-12-21;1251-12-21 13:04:45;1411-12-23 13:04:00;1411-12-14 13:04:45.7667;1411-12-14 13:04:45.76567;1411-12-14 13:04:45.76567;[0];[6];[-32767];[-2352342];[34634796439863689];[532623456245244235];[-41251.1];[9599-12-21, 9599-12-21];[1251-12-14 13:04:45];[9599-12-21, 9599 [...] -0;7;255;325265656;-79834967836;3652354346;0.1331;63.64354;346346.346;432534.3252;-346367634.5;346346.346;432534.3252;-346367634.5;gdcg;Fwegwdwegw;Fwefwecwegwehgdrherehjgfd;gdcg;Fwegwdwegw;Fwefwecwegwehgdrherehjgfd;Fwefwecwegwehgdrherehjgfd;1201-12-30;1201-12-30;1211-12-21 13:04:45;1451-12-23 13:04:45;1451-12-14 13:04:45.3436;1451-12-14 13:04:4
[GitHub] [doris] hello-stephen commented on pull request #16159: [datev2](function) make function nullable DEPEND_ON_ARGUMENT
hello-stephen commented on PR #16159: URL: https://github.com/apache/doris/pull/16159#issuecomment-1407398765 TeamCity pipeline, clickbench performance test result: the sum of best hot time: 35.51 seconds load time: 524 seconds storage size: 17122962414 Bytes https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230128132528_clickbench_pr_85693.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] sohardforaname opened a new pull request, #16171: [Fix](Nereids)fix error of X-Z function for nereids
sohardforaname opened a new pull request, #16171: URL: https://github.com/apache/doris/pull/16171 # Proposed changes Issue Number: close #xxx ## Problem summary fix error of X-Z function for nereids ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [x] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [x] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [x] No Need 4. Does it need to update dependencies: - [ ] Yes - [x] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [x] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #16169: [fix](nereids) Delete invalid signature of nvl function
hello-stephen commented on PR #16169: URL: https://github.com/apache/doris/pull/16169#issuecomment-1407406308 TeamCity pipeline, clickbench performance test result: the sum of best hot time: 34.13 seconds load time: 517 seconds storage size: 17122314581 Bytes https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230128140556_clickbench_pr_85706.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #15924: [improvement](agg-function) remove the limit on the max number of agg function parameters
github-actions[bot] commented on PR #15924: URL: https://github.com/apache/doris/pull/15924#issuecomment-1407409582 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #16170: [fix](Nereids): remove `DataV2Type` in ConvertTz, DayCeil and DayFloor SIGNATURES
hello-stephen commented on PR #16170: URL: https://github.com/apache/doris/pull/16170#issuecomment-1407410262 TeamCity pipeline, clickbench performance test result: the sum of best hot time: 35.13 seconds load time: 507 seconds storage size: 17122819584 Bytes https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230128142720_clickbench_pr_85741.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #15924: [improvement](agg-function) Increase the limit maximum number of agg function parameters
github-actions[bot] commented on PR #15924: URL: https://github.com/apache/doris/pull/15924#issuecomment-1407410562 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] nextdreamblue opened a new pull request, #16172: [fix-core](block) clear block row_same_bit when block reuse
nextdreamblue opened a new pull request, #16172: URL: https://github.com/apache/doris/pull/16172 Signed-off-by: nextdreamblue # Proposed changes Issue Number: close #16155 ## Problem summary Describe your changes. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [x] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [x] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [x] No Need 4. Does it need to update dependencies: - [ ] Yes - [x] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [x] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #16172: [fix-core](block) clear block row_same_bit when block reuse
github-actions[bot] commented on PR #16172: URL: https://github.com/apache/doris/pull/16172#issuecomment-1407415617 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #16172: [fix-core](block) clear block row_same_bit when block reuse
github-actions[bot] commented on PR #16172: URL: https://github.com/apache/doris/pull/16172#issuecomment-1407415991 clang-tidy review says "All clean, LGTM! :+1:" -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] eldenmoon opened a new pull request, #16173: [Bug](topn-opt) filter condition for analytic info for two phase read…
eldenmoon opened a new pull request, #16173: URL: https://github.com/apache/doris/pull/16173 … opt two phase read optimization should not be enabled when query has analytic info # Proposed changes Issue Number: close #xxx ## Problem summary Describe your changes. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [ ] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [ ] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [ ] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on a diff in pull request #15718: [fix](schema scanner)change schema_scanner::get_next_row to get_next_block
github-actions[bot] commented on code in PR #15718:
URL: https://github.com/apache/doris/pull/15718#discussion_r1089757755
##
be/src/vec/exec/vschema_scan_node.cpp:
##
@@ -255,74 +257,61 @@ Status VSchemaScanNode::get_next(RuntimeState* state,
vectorized::Block* block,
return Status::InternalError("used before initialize.");
}
RETURN_IF_CANCELLED(state);
-std::vector columns(_slot_num);
bool schema_eos = false;
+block->clear();
+
+std::vector index_map_inv(_src_tuple_desc->slots().size());
+for (int i = 0; i < _slot_num; ++i) {
+auto dest_slot_desc = _dest_tuple_desc->slots()[i];
+
block->insert(ColumnWithTypeAndName(dest_slot_desc->get_empty_mutable_column(),
+
dest_slot_desc->get_data_type_ptr(),
+dest_slot_desc->col_name()));
+
+// Map from index in column of schema table to slots.
+index_map_inv[_index_map[i]] = i;
+}
+
do {
-bool mem_reuse = block->mem_reuse();
-DCHECK(block->rows() == 0);
-
-columns.resize(_slot_num);
-for (int i = 0; i < _slot_num; ++i) {
-if (mem_reuse) {
-columns[i] =
std::move(*block->get_by_position(i).column).mutate();
-} else {
-columns[i] =
_dest_tuple_desc->slots()[i]->get_empty_mutable_column();
-}
+vectorized::Block src_block;
+
+for (int i = 0; i < _src_tuple_desc->slots().size(); ++i) {
+int j = index_map_inv[i];
+auto slot_desc = _dest_tuple_desc->slots()[j];
+
src_block.insert(ColumnWithTypeAndName(slot_desc->get_empty_mutable_column(),
+
slot_desc->get_data_type_ptr(),
+ slot_desc->col_name()));
}
while (true) {
RETURN_IF_CANCELLED(state);
// get all slots from schema table.
-RETURN_IF_ERROR(_schema_scanner->get_next_row(_src_single_tuple,
_tuple_pool.get(),
- &schema_eos));
+RETURN_IF_ERROR(_schema_scanner->get_next_block(&src_block,
&schema_eos));
+
if (schema_eos) {
*eos = true;
break;
}
-// tuple project
-project_tuple();
-for (int i = 0; i < _slot_num; ++i) {
-auto slot_desc = _dest_tuple_desc->slots()[i];
-if (!slot_desc->is_materialized()) {
-continue;
-}
-
-if
(_dest_single_tuple->is_null(slot_desc->null_indicator_offset())) {
-if (slot_desc->is_nullable()) {
-auto* nullable_column =
-
reinterpret_cast(columns[i].get());
-nullable_column->insert_data(nullptr, 0);
-} else {
-return Status::InternalError(
-"nonnull column contains NULL. table={},
column={}", _table_name,
-slot_desc->col_name());
-}
-} else {
-RETURN_IF_ERROR(write_slot_to_vectorized_column(
-
_dest_single_tuple->get_slot(slot_desc->tuple_offset()), slot_desc,
-&columns[i]));
-}
-}
-if (columns[0]->size() == state->batch_size()) {
+if (src_block.rows() >= state->batch_size()) {
break;
}
}
-if (!columns.empty() && !columns[0]->empty()) {
-auto n_columns = 0;
-if (!mem_reuse) {
-for (const auto slot_desc : _dest_tuple_desc->slots()) {
-
block->insert(ColumnWithTypeAndName(std::move(columns[n_columns++]),
-
slot_desc->get_data_type_ptr(),
-
slot_desc->col_name()));
-}
-} else {
-columns.clear();
+
+if (src_block.rows()) {
+// block->check_number_of_rows();
+for (int i = 0; i < _slot_num; ++i) {
+auto dest_slot_desc = _dest_tuple_desc->slots()[i];
+vectorized::MutableColumnPtr column_ptr =
+std::move(*block->get_by_position(i).column).mutate();
Review Comment:
warning: std::move of the const expression has no effect; remove std::move()
[performance-move-const-arg]
```suggestion
*block->get_by_position(i).column.mutate();
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubs
[GitHub] [doris] jackwener opened a new pull request, #16174: [fix](Nereids): fix dceil() dfloor()
jackwener opened a new pull request, #16174: URL: https://github.com/apache/doris/pull/16174 # Proposed changes Issue Number: close #xxx ## Problem summary Describe your changes. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [ ] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [ ] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [ ] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #16172: [fix-core](block) clear block row_same_bit when block reuse
hello-stephen commented on PR #16172: URL: https://github.com/apache/doris/pull/16172#issuecomment-1407425534 TeamCity pipeline, clickbench performance test result: the sum of best hot time: 35.27 seconds load time: 522 seconds storage size: 17120405529 Bytes https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230128154457_clickbench_pr_85767.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] jackwener commented on pull request #16168: [Bug](function) remove datev2 signature of hour_ceil/hour_floor
jackwener commented on PR #16168: URL: https://github.com/apache/doris/pull/16168#issuecomment-1407428342 Maybe need notice: old optimizer `ScalarBuiltins.java` contains the removed type SIGNATURES. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] jackwener commented on pull request #16168: [Bug](function) remove datev2 signature of hour_ceil/hour_floor
jackwener commented on PR #16168: URL: https://github.com/apache/doris/pull/16168#issuecomment-1407428430 #16174 is same with this PR -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] BiteTheDDDDt commented on pull request #16168: [Bug](function) remove datev2 signature of hour_ceil/hour_floor
BiteThet commented on PR #16168: URL: https://github.com/apache/doris/pull/16168#issuecomment-1407430631 > In this case we have wrong signature(the implementation of hour_ceil/hour_floor in `be` is inconsistent with the signature), so I remove it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] BiteTheDDDDt commented on pull request #16170: [fix](Nereids): remove `DataV2Type` in ConvertTz, DayCeil and DayFloor SIGNATURES
BiteThet commented on PR #16170: URL: https://github.com/apache/doris/pull/16170#issuecomment-1407431450 On the one hand, you should make sure that the signature in nereids is consistent with the signature in doris_builtins_functions.py. On the other hand, whether a function like DayCeil should remove the datev2 type signature, I'm not sure. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #16173: [Bug](topn-opt) filter condition for analytic info for two phase read…
hello-stephen commented on PR #16173: URL: https://github.com/apache/doris/pull/16173#issuecomment-1407432708 TeamCity pipeline, clickbench performance test result: the sum of best hot time: 35.68 seconds load time: 494 seconds storage size: 17121999283 Bytes https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230128162516_clickbench_pr_85791.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] jackwener commented on pull request #16168: [Bug](function) remove datev2 signature of hour_ceil/hour_floor
jackwener commented on PR #16168: URL: https://github.com/apache/doris/pull/16168#issuecomment-1407433615 > Maybe need notice: old optimizer `ScalarBuiltins.java` contains the removed type SIGNATURES. signature in it also need to be removed -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #16174: [fix](Nereids): fix dceil() dfloor()
hello-stephen commented on PR #16174: URL: https://github.com/apache/doris/pull/16174#issuecomment-1407433947 TeamCity pipeline, clickbench performance test result: the sum of best hot time: 36.49 seconds load time: 491 seconds storage size: 17123140409 Bytes https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20230128163244_clickbench_pr_85806.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] morningman merged pull request #16160: [docs](docs)update en docs
morningman merged PR #16160: URL: https://github.com/apache/doris/pull/16160 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated: [docs](multi-catalog)update en docs (#16160)
This is an automated email from the ASF dual-hosted git repository.
morningman pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 46ce66cbd8 [docs](multi-catalog)update en docs (#16160)
46ce66cbd8 is described below
commit 46ce66cbd86890aae82ef2eaa97798d67159d2b8
Author: Hu Yanjun <100749531+httpshir...@users.noreply.github.com>
AuthorDate: Sun Jan 29 00:36:31 2023 +0800
[docs](multi-catalog)update en docs (#16160)
---
docs/en/docs/lakehouse/multi-catalog/dlf.md| 78 ++-
docs/en/docs/lakehouse/multi-catalog/hive.md | 147 -
docs/en/docs/lakehouse/multi-catalog/hudi.md | 26 +++-
docs/en/docs/lakehouse/multi-catalog/iceberg.md| 49 ++-
.../docs/lakehouse/multi-catalog/multi-catalog.md | 2 +-
docs/zh-CN/docs/lakehouse/multi-catalog/hive.md| 16 +--
docs/zh-CN/docs/lakehouse/multi-catalog/hudi.md| 2 +-
7 files changed, 304 insertions(+), 16 deletions(-)
diff --git a/docs/en/docs/lakehouse/multi-catalog/dlf.md
b/docs/en/docs/lakehouse/multi-catalog/dlf.md
index 82bdd1f64d..d533ce943e 100644
--- a/docs/en/docs/lakehouse/multi-catalog/dlf.md
+++ b/docs/en/docs/lakehouse/multi-catalog/dlf.md
@@ -1,6 +1,6 @@
---
{
-"title": "Aliyun DLF",
+"title": "Alibaba Cloud DLF",
"language": "en"
}
---
@@ -25,7 +25,79 @@ under the License.
-->
-# Aliyun DLF
+# Alibaba Cloud DLF
+
+Data Lake Formation (DLF) is the unified metadata management service of
Alibaba Cloud. It is compatible with the Hive Metastore protocol.
+
+> [What is DLF](https://www.alibabacloud.com/product/datalake-formation)
+
+Doris can access DLF the same way as it accesses Hive Metastore.
+
+## Connect to DLF
+
+1. Create `hive-site.xml`
+
+ Create the `hive-site.xml` file, and put it in the `fe/conf` directory.
+
+ ```
+
+
+
+
+ hive.metastore.type
+ dlf
+
+
+ dlf.catalog.endpoint
+ dlf-vpc.cn-beijing.aliyuncs.com
+
+
+ dlf.catalog.region
+ cn-beijing
+
+
+ dlf.catalog.proxyMode
+ DLF_ONLY
+
+
+ dlf.catalog.uid
+ 2
+
+
+ dlf.catalog.accessKeyId
+ XXX
+
+
+ dlf.catalog.accessKeySecret
+ X
+
+
+ ```
+
+ * `dlf.catalog.endpoint`: DLF Endpoint. See [Regions and Endpoints of
DLF](https://www.alibabacloud.com/help/en/data-lake-formation/latest/regions-and-endpoints).
+ * `dlf.catalog.region`: DLF Region. See [Regions and Endpoints of
DLF](https://www.alibabacloud.com/help/en/data-lake-formation/latest/regions-and-endpoints).
+ * `dlf.catalog.uid`: Alibaba Cloud account. You can find the "Account ID"
in the upper right corner on the Alibaba Cloud console.
+ * `dlf.catalog.accessKeyId`：AccessKey, which you can create and manage on
the [Alibaba Cloud console](https://ram.console.aliyun.com/manage/ak).
+ * `dlf.catalog.accessKeySecret`：SecretKey, which you can create and manage
on the [Alibaba Cloud console](https://ram.console.aliyun.com/manage/ak).
+
+ Other configuration items are fixed and require no modifications.
+
+2. Restart FE, and create Catalog via the `CREATE CATALOG` statement.
+
+ Doris will read and parse `fe/conf/hive-site.xml`.
+
+ ```sql
+ CREATE CATALOG hive_with_dlf PROPERTIES (
+ "type"="hms",
+ "hive.metastore.uris" = "thrift://127.0.0.1:9083"
+ )
+ ```
+
+`type` should always be `hms`; while `hive.metastore.uris` can be
arbitary since it is not used in real practice, but it should follow the format
of Hive Metastore Thrift URI.
+
+ After the above steps, you can access metadata in DLF the same way as you
access Hive MetaStore.
+
+ Doris supports accessing Hive/Iceberg/Hudi metadata in DLF.
+
-TODO: translate
diff --git a/docs/en/docs/lakehouse/multi-catalog/hive.md
b/docs/en/docs/lakehouse/multi-catalog/hive.md
index fd3bfd8191..18ae073160 100644
--- a/docs/en/docs/lakehouse/multi-catalog/hive.md
+++ b/docs/en/docs/lakehouse/multi-catalog/hive.md
@@ -26,4 +26,149 @@ under the License.
# Hive
-TODO: translate
+Once Doris is connected to Hive Metastore or made compatible with Hive
Metastore metadata service, it can access databases and tables in Hive and
conduct queries.
+
+Besides Hive, many other systems, such as Iceberg and Hudi, use Hive Metastore
to keep their metadata. Thus, Doris can also access these systems via Hive
Catalog.
+
+## Usage
+
+When connnecting to Hive, Doris:
+
+1. Supports Hive version 1/2/3;
+2. Supports both Managed Table and External Table;
+3. Can identify metadata of Hive, Iceberg, and Hudi stored in Hive Metastore;
+4. Supports Hive tables with data stored in JuiceFS, which can be used the
same way as normal Hive tables
