[GitHub] [incubator-doris] EmmyMiao87 edited a comment on issue #7789: [Bug] 0.15.1导出hdfs报错

2022-01-18 Thread GitBox


EmmyMiao87 edited a comment on issue #7789:
URL: 
https://github.com/apache/incubator-doris/issues/7789#issuecomment-1015174532


   1. create table unique_table (k1 int, k2 int) unique key(k1) distributed by 
hash(k1);
   2. EXPORT TABLE unique_table TO "hdfs://xxx:9000/xxx" WITH BROKER "xxx" 
("username"="xxx", "password"="");
   3. ERROR 1054 (42S22): errCode = 2, detailMessage = Unknown column 
'__DORIS_DELETE_SIGN__' in 'table list'


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [incubator-doris] EmmyMiao87 commented on issue #7789: [Bug] 0.15.1导出hdfs报错

2022-01-18 Thread GitBox


EmmyMiao87 commented on issue #7789:
URL: 
https://github.com/apache/incubator-doris/issues/7789#issuecomment-1015174532


   1. create table unique_table (k1 int, k2 int) unique key(k1) distributed by 
hash(k1);
   2. EXPORT TABLE unique_table TO "hdfs://xxx:9000/xxx" WITH BROKER "xxx" 
("username"="xxx", "password"="");


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [incubator-doris] yiguolei commented on issue #7502: Doris Roadmap 2022

2022-01-18 Thread GitBox


yiguolei commented on issue #7502:
URL: 
https://github.com/apache/incubator-doris/issues/7502#issuecomment-1015182922


   Could use vectorized method to optimize load process??


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [incubator-doris] huzk8 closed pull request #6747: feat: use json format to improve spark connector sink compatibility

2022-01-18 Thread GitBox


huzk8 closed pull request #6747:
URL: https://github.com/apache/incubator-doris/pull/6747


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [incubator-doris] i7xh commented on issue #7502: Doris Roadmap 2022

2022-01-18 Thread GitBox


i7xh commented on issue #7502:
URL: 
https://github.com/apache/incubator-doris/issues/7502#issuecomment-1015191906


   Why Doris need push based query engine?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [incubator-doris] i7xh edited a comment on issue #7502: Doris Roadmap 2022

2022-01-18 Thread GitBox


i7xh edited a comment on issue #7502:
URL: 
https://github.com/apache/incubator-doris/issues/7502#issuecomment-1015191906


   Why Doris need push based query execution engine?


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [incubator-doris] zbtzbtzbt opened a new pull request #7790: [Doc] add documents for bitwise functions

2022-01-18 Thread GitBox


zbtzbtzbt opened a new pull request #7790:
URL: https://github.com/apache/incubator-doris/pull/7790


   # Proposed changes
   
   Issue Number: close #xxx
   
   ## Problem Summary:
   
   Describe the overview of changes.
   
   ## Checklist(Required)
   
   1. Does it affect the original behavior: (No)
   2. Has unit tests been added: (No Need)
   3. Has document been added or modified: (No)
   4. Does it need to update dependencies: (No)
   5. Are there any changes that cannot be rolled back: (No)
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [incubator-doris] yiguolei commented on issue #7502: Doris Roadmap 2022

2022-01-18 Thread GitBox


yiguolei commented on issue #7502:
URL: 
https://github.com/apache/incubator-doris/issues/7502#issuecomment-1015197025


   @i7xh 
   Two example:
   1.  Currently doris‘s concurrency control is based on tablet, one tablet 
---> exec fragment, there is only one thread to deal with the data at query 
engine. If use pushed engine, could expand the computing thread num at run time.
   2. In pull engine, if one fragment contains 3 or more node, like  scan--> 
filter--> agg there is only one node is executing, but in  push based engine, 
node executing could be async, for example scan and agg could execute at same 
time.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [incubator-doris] zuochunwei opened a new pull request #7791: (refactor) (olap) refactor SegmentIterator

2022-01-18 Thread GitBox


zuochunwei opened a new pull request #7791:
URL: https://github.com/apache/incubator-doris/pull/7791


   # Proposed changes
   1. BitmapRangeIterator refactor, avoid new/delete, reduce check condition
   2. modify evaluate_vec, avoid set and check new_flags out of 
block_column_predicate->evaluate_vec()
   3. replace _block_rowids.reserve with _block_rowids.resize, access data with 
subscript is illegal after reverving
   4. optimize insert_default_value_res_column and insert_res_column by 
operating PODArray data directly
   5. using empty() interface instead of 'size() > 0', because empty() is 
fittable, sometimes faster, such as for list 
   6. some other small changes
   
   Issue Number: close #xxx
   
   ## Problem Summary:
   
   Describe the overview of changes.
   
   ## Checklist(Required)
   
   1. Does it affect the original behavior: (Yes/No/I Don't know)
   2. Has unit tests been added: (Yes/No/No Need)
   3. Has document been added or modified: (Yes/No/No Need)
   4. Does it need to update dependencies: (Yes/No)
   5. Are there any changes that cannot be rolled back: (Yes/No)
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [incubator-doris] BiteTheDDDDt opened a new issue #7792: [Feature][Vectorized] support function topn && remove some unused code

2022-01-18 Thread GitBox


BiteThet opened a new issue #7792:
URL: https://github.com/apache/incubator-doris/issues/7792


   ### Search before asking
   
   - [X] I had searched in the 
[issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and 
found no similar issues.
   
   
   ### Description
   
   1. support function topn
   2. remove unused header
   3. remove `get_header_file_path()` and `get_address_of_add_function()`
   
   ### Use case
   
   ```sql
   MySQL [tpch]> set enable_vectorized_engine = false;
   Query OK, 0 rows affected (0.001 sec)
   
   MySQL [tpch]> select topn(l_linestatus,5,10) from lineitem;
   +-+
   | topn(`l_linestatus`, 5, 10) |
   +-+
   | {"F":29998258,"O":29987794} |
   +-+
   1 row in set (5.264 sec)
   
   MySQL [tpch]> set enable_vectorized_engine = true;
   Query OK, 0 rows affected (0.001 sec)
   
   MySQL [tpch]> select topn(l_linestatus,5,10) from lineitem;
   +-+
   | topn(`l_linestatus`, 5, 10) |
   +-+
   | {"F":29998258,"O":29987794} |
   +-+
   1 row in set (2.075 sec)
   
   MySQL [tpch]> set enable_vectorized_engine = false;
   Query OK, 0 rows affected (0.001 sec)
   
   MySQL [tpch]> select topn(l_partkey,3) from lineitem;
   +-+
   | topn(`l_partkey`, 3)|
   +-+
   | {"304540":58,"1090021":58,"1457334":57} |
   +-+
   1 row in set (34.988 sec)
   
   MySQL [tpch]> set enable_vectorized_engine = true;
   Query OK, 0 rows affected (0.001 sec)
   
   MySQL [tpch]> select topn(l_partkey,3) from lineitem;
   ++
   | topn(`l_partkey`, 3)   |
   ++
   | {"304540":58,"1090021":58,"782803":57} |
   ++
   1 row in set (18.057 sec)
   ```
   
   ### Related issues
   
   _No response_
   
   ### Are you willing to submit PR?
   
   - [X] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of 
Conduct](https://www.apache.org/foundation/policies/conduct)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [incubator-doris] BiteTheDDDDt opened a new pull request #7793: [Feature][Vectorized] support function topn && remove some unused code

2022-01-18 Thread GitBox


BiteThet opened a new pull request #7793:
URL: https://github.com/apache/incubator-doris/pull/7793


   # Proposed changes
   
   Issue Number: close #7792
   
   ## Problem Summary:
   
   Describe the overview of changes.
   
   ## Checklist(Required)
   
   1. Does it affect the original behavior: (Yes/No/I Don't know)
   2. Has unit tests been added: (Yes/No/No Need)
   3. Has document been added or modified: (Yes/No/No Need)
   4. Does it need to update dependencies: (Yes/No)
   5. Are there any changes that cannot be rolled back: (Yes/No)
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [incubator-doris] zenoyang opened a new issue #7794: [Enhancement](Vectorized) optimize SegmentIterator predication evaluate

2022-01-18 Thread GitBox


zenoyang opened a new issue #7794:
URL: https://github.com/apache/incubator-doris/issues/7794


   ### Search before asking
   
   - [X] I had searched in the 
[issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and 
found no similar issues.
   
   
   ### Description
   
   - optimize SegmentIterator predication evaluate
   - rafactor code
   
   ### Solution
   
   no
   
   ### Are you willing to submit PR?
   
   - [X] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of 
Conduct](https://www.apache.org/foundation/policies/conduct)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [incubator-doris] zenoyang opened a new pull request #7795: [Improvement](Vectorized) optimize SegmentIterator predication evaluate

2022-01-18 Thread GitBox


zenoyang opened a new pull request #7795:
URL: https://github.com/apache/incubator-doris/pull/7795


   # Proposed changes
   
   Issue Number: #7794 
   
   1. Using SIMD to speed up rowid selection
   2. Use `make_unique` instead of reset
   3. `block->clear_column_data()` removes the for loop and reduces calls
   4. Add const to function parameters
   5. using `empty()` interface instead of `size() > 0`
   
   
   ## Problem Summary:
   
   Describe the overview of changes.
   
   ## Checklist(Required)
   
   1. Does it affect the original behavior: (Yes/No/I Don't know)
   2. Has unit tests been added: (Yes/No/No Need)
   3. Has document been added or modified: (Yes/No/No Need)
   4. Does it need to update dependencies: (Yes/No)
   5. Are there any changes that cannot be rolled back: (Yes/No)
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [incubator-doris] wangbo commented on a change in pull request #7795: [Improvement](Vectorized) optimize SegmentIterator predication evaluate

2022-01-18 Thread GitBox


wangbo commented on a change in pull request #7795:
URL: https://github.com/apache/incubator-doris/pull/7795#discussion_r786766718



##
File path: be/src/olap/rowset/segment_v2/segment_iterator.cpp
##
@@ -671,7 +673,7 @@ void SegmentIterator::_vec_init_lazy_materialization() {
 
 } else if (is_predicate_column_exists && !is_non_predicate_column_exists) {
 _first_read_column_ids.assign(pred_column_ids.cbegin(), 
pred_column_ids.cend());
-} else if (!is_predicate_column_exists && is_non_predicate_column_exists) {
+} else if (!is_predicate_column_exists) {

Review comment:
   why remove ```is_non_predicate_column_exists```




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [incubator-doris] wangbo commented on a change in pull request #7795: [Improvement](Vectorized) optimize SegmentIterator predication evaluate

2022-01-18 Thread GitBox


wangbo commented on a change in pull request #7795:
URL: https://github.com/apache/incubator-doris/pull/7795#discussion_r786808199



##
File path: be/src/olap/rowset/segment_v2/segment_iterator.cpp
##
@@ -807,10 +809,31 @@ void 
SegmentIterator::_evaluate_vectorization_predicate(uint16_t* sel_rowid_idx,
 bool ret_flags[selected_size];
 memset(ret_flags, 1, selected_size);
 _pre_eval_block_predicate->evaluate_vec(_current_return_columns, 
selected_size, ret_flags);
-
-for (uint32_t i = 0; i < selected_size; ++i) {
-if (ret_flags[i]) {
-sel_rowid_idx[new_size++] = i;
+
+uint32_t sel_pos = 0;
+const uint32_t sel_end = sel_pos + selected_size;
+static constexpr size_t SIMD_BYTES = 32;
+const uint32_t sel_end_simd = sel_pos + selected_size / SIMD_BYTES * 
SIMD_BYTES;
+
+while (sel_pos < sel_end_simd) {
+auto mask = vectorized::bytes32_mask_to_bits32_mask(
+reinterpret_cast(ret_flags + sel_pos));
+if (mask == 0x) {
+memset(sel_rowid_idx + new_size, true, SIMD_BYTES);
+new_size += SIMD_BYTES;

Review comment:
   why ```memset(true)``` here?
   ```sel_rowid_idx``` should be filled rowid here.
   ```memset(true)```  means you store 257 to ```sel_rowid_idx```.




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [incubator-doris] wangbo opened a new pull request #7796: [Vectorized][Enhancement] SIMD fill sel_rowid_idx

2022-01-18 Thread GitBox


wangbo opened a new pull request #7796:
URL: https://github.com/apache/incubator-doris/pull/7796


   # Proposed changes
   
   A very simple fix.
   
   ## Problem Summary:
   
   Describe the overview of changes.
   
   ## Checklist(Required)
   
   1. Does it affect the original behavior: (Yes/No/I Don't know)
   2. Has unit tests been added: (Yes/No/No Need)
   3. Has document been added or modified: (Yes/No/No Need)
   4. Does it need to update dependencies: (Yes/No)
   5. Are there any changes that cannot be rolled back: (Yes/No)
   
   ## Further comments
   
   If this is a relatively large or complex change, kick off the discussion at 
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you 
chose the solution you did and what alternatives you considered, etc...
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[incubator-doris-website] branch master updated: update cron-deploy-website.yml

2022-01-18 Thread morningman
This is an automated email from the ASF dual-hosted git repository.

morningman pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-doris-website.git


The following commit(s) were added to refs/heads/master by this push:
 new 25fc693  update cron-deploy-website.yml
25fc693 is described below

commit 25fc693dcc757d20de8e5a25d0697e5f848854f7
Author: morningman 
AuthorDate: Tue Jan 18 23:33:38 2022 +0800

update cron-deploy-website.yml
---
 .github/workflows/cron-deploy-website.yml | 1 -
 1 file changed, 1 deletion(-)

diff --git a/.github/workflows/cron-deploy-website.yml 
b/.github/workflows/cron-deploy-website.yml
index 35e80e6..b2bed40 100644
--- a/.github/workflows/cron-deploy-website.yml
+++ b/.github/workflows/cron-deploy-website.yml
@@ -35,7 +35,6 @@ jobs:
 export DORIS_COMMIT=`git rev-parse HEAD`
 
 - name: Deploy Master
-  if: ${{ github.event.inputs.branch == 'master' }}
   run: |
 git fetch
 git checkout -b asf-site remotes/origin/asf-site

-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [incubator-doris] JNSimba commented on pull request #7457: [Feature][flink-connector] support flink delete option

2022-01-18 Thread GitBox


JNSimba commented on pull request #7457:
URL: https://github.com/apache/incubator-doris/pull/7457#issuecomment-1015552412


   only when it is judged to be a unique model, perform the delete operation.


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [incubator-doris] zenoyang commented on a change in pull request #7795: [Improvement](Vectorized) optimize SegmentIterator predication evaluate

2022-01-18 Thread GitBox


zenoyang commented on a change in pull request #7795:
URL: https://github.com/apache/incubator-doris/pull/7795#discussion_r787336118



##
File path: be/src/olap/rowset/segment_v2/segment_iterator.cpp
##
@@ -671,7 +673,7 @@ void SegmentIterator::_vec_init_lazy_materialization() {
 
 } else if (is_predicate_column_exists && !is_non_predicate_column_exists) {
 _first_read_column_ids.assign(pred_column_ids.cbegin(), 
pred_column_ids.cend());
-} else if (!is_predicate_column_exists && is_non_predicate_column_exists) {
+} else if (!is_predicate_column_exists) {

Review comment:
   Because the `is_non_predicate_column_exists` judgment here is redundant, 
I will roll back it, so it looks clearer




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [incubator-doris] zenoyang commented on a change in pull request #7795: [Improvement](Vectorized) optimize SegmentIterator predication evaluate

2022-01-18 Thread GitBox


zenoyang commented on a change in pull request #7795:
URL: https://github.com/apache/incubator-doris/pull/7795#discussion_r787336407



##
File path: be/src/olap/rowset/segment_v2/segment_iterator.cpp
##
@@ -807,10 +809,31 @@ void 
SegmentIterator::_evaluate_vectorization_predicate(uint16_t* sel_rowid_idx,
 bool ret_flags[selected_size];
 memset(ret_flags, 1, selected_size);
 _pre_eval_block_predicate->evaluate_vec(_current_return_columns, 
selected_size, ret_flags);
-
-for (uint32_t i = 0; i < selected_size; ++i) {
-if (ret_flags[i]) {
-sel_rowid_idx[new_size++] = i;
+
+uint32_t sel_pos = 0;
+const uint32_t sel_end = sel_pos + selected_size;
+static constexpr size_t SIMD_BYTES = 32;
+const uint32_t sel_end_simd = sel_pos + selected_size / SIMD_BYTES * 
SIMD_BYTES;
+
+while (sel_pos < sel_end_simd) {
+auto mask = vectorized::bytes32_mask_to_bits32_mask(
+reinterpret_cast(ret_flags + sel_pos));
+if (mask == 0x) {
+memset(sel_rowid_idx + new_size, true, SIMD_BYTES);
+new_size += SIMD_BYTES;

Review comment:
   It is indeed a bug, I have fixed it and am testing it




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [incubator-doris] morningman merged pull request #7773: [fix][chore](thrift) Fix warning when generate cpp code by thrift IDL file and use strict mode

2022-01-18 Thread GitBox


morningman merged pull request #7773:
URL: https://github.com/apache/incubator-doris/pull/7773


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [incubator-doris] morningman closed issue #7765: [Bug] if the query based on a column that with index, the result does not meet the expectation

2022-01-18 Thread GitBox


morningman closed issue #7765:
URL: https://github.com/apache/incubator-doris/issues/7765


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [incubator-doris] morningman merged pull request #7788: [fix](bitmap-index) Fix bug that bitmap index may return wrong result.

2022-01-18 Thread GitBox


morningman merged pull request #7788:
URL: https://github.com/apache/incubator-doris/pull/7788


   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[GitHub] [incubator-doris] wangbo commented on a change in pull request #7795: [Improvement](Vectorized) optimize SegmentIterator predication evaluate

2022-01-18 Thread GitBox


wangbo commented on a change in pull request #7795:
URL: https://github.com/apache/incubator-doris/pull/7795#discussion_r787345637



##
File path: be/src/olap/rowset/segment_v2/segment_iterator.cpp
##
@@ -671,7 +673,7 @@ void SegmentIterator::_vec_init_lazy_materialization() {
 
 } else if (is_predicate_column_exists && !is_non_predicate_column_exists) {
 _first_read_column_ids.assign(pred_column_ids.cbegin(), 
pred_column_ids.cend());
-} else if (!is_predicate_column_exists && is_non_predicate_column_exists) {
+} else if (!is_predicate_column_exists) {

Review comment:
   I think maybe ```is_non_predicate_column_exists``` can be removed,  and 
then add some comment to make it clear




-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org



-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[incubator-doris] branch master updated (5fc0a9f -> aacbc96)

2022-01-18 Thread morningman
This is an automated email from the ASF dual-hosted git repository.

morningman pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-doris.git.


from 5fc0a9f  [improvement](Load) Cancel the load job ASAP when encounter 
unqualified data (#6319)
 add aacbc96  [fix][chore](thrift) Fix warning when generate cpp code by 
thrift IDL file and use strict mode (#7773)

No new revisions were added by this update.

Summary of changes:
 gensrc/thrift/FrontendService.thrift   | 38 +++---
 gensrc/thrift/Makefile |  2 +-
 gensrc/thrift/PaloExternalDataSourceService.thrift |  2 +-
 gensrc/thrift/PlanNodes.thrift |  4 +--
 4 files changed, 23 insertions(+), 23 deletions(-)

-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org



[incubator-doris] branch master updated (aacbc96 -> 7574d39)

2022-01-18 Thread morningman
This is an automated email from the ASF dual-hosted git repository.

morningman pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-doris.git.


from aacbc96  [fix][chore](thrift) Fix warning when generate cpp code by 
thrift IDL file and use strict mode (#7773)
 add 7574d39  [fix](bitmap-index) Fix bug that bitmap index may return 
wrong result. (#7788)

No new revisions were added by this update.

Summary of changes:
 be/src/olap/rowset/segment_v2/index_page.cpp|  2 +-
 be/src/olap/rowset/segment_v2/index_page.h  |  2 ++
 be/src/olap/rowset/segment_v2/indexed_column_reader.cpp | 12 +++-
 be/src/olap/rowset/segment_v2/ordinal_page_index.h  |  3 +++
 4 files changed, 17 insertions(+), 2 deletions(-)

-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org