[GitHub] [incubator-doris] chaoyli merged pull request #1296: Check deserialize result when save meta

2019-06-13 Thread GitBox
chaoyli merged pull request #1296: Check deserialize result when save meta
URL: https://github.com/apache/incubator-doris/pull/1296
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org
For additional commands, e-mail: dev-h...@doris.apache.org



[GitHub] [incubator-doris] chaoyli opened a new pull request #1297: Add copy function to support storage migration task

2019-06-13 Thread GitBox
chaoyli opened a new pull request #1297: Add copy function to support storage 
migration task
URL: https://github.com/apache/incubator-doris/pull/1297
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org
For additional commands, e-mail: dev-h...@doris.apache.org



[GitHub] [incubator-doris] yiguolei commented on a change in pull request #1297: Add copy function to support storage migration task

2019-06-13 Thread GitBox
yiguolei commented on a change in pull request #1297: Add copy function to 
support storage migration task
URL: https://github.com/apache/incubator-doris/pull/1297#discussion_r293239307
 
 

 ##
 File path: be/src/olap/rowset/alpha_rowset.cpp
 ##
 @@ -219,6 +219,20 @@ OLAPStatus AlphaRowset::make_snapshot(const std::string& 
snapshot_path,
 return OLAP_SUCCESS;
 }
 
+OLAPStatus AlphaRowset::copy_files_to_path(const std::string& dest_path,
+   std::vector* 
success_files) {
 
 Review comment:
   should add copy_files_to_path to parent class


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org
For additional commands, e-mail: dev-h...@doris.apache.org



[GitHub] [incubator-doris] yiguolei commented on a change in pull request #1297: Add copy function to support storage migration task

2019-06-13 Thread GitBox
yiguolei commented on a change in pull request #1297: Add copy function to 
support storage migration task
URL: https://github.com/apache/incubator-doris/pull/1297#discussion_r293239820
 
 

 ##
 File path: be/src/olap/rowset/segment_group.cpp
 ##
 @@ -723,6 +723,39 @@ OLAPStatus SegmentGroup::make_snapshot(const std::string& 
snapshot_path,
 return OLAP_SUCCESS;
 }
 
+OLAPStatus SegmentGroup::copy_files_to_path(const std::string& dest_path,
+std::vector* 
success_files) {
+if (_empty) {
+return OLAP_SUCCESS;
+}
+for (int segment_id = 0; segment_id < _num_segments; segment_id++) {
+std::string dest_data_file = construct_data_file_path(dest_path, 
segment_id);
+if (!check_dir_existed(dest_data_file)) {
 
 Review comment:
   If exist file, then should check its length  or remove the exist file?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org
For additional commands, e-mail: dev-h...@doris.apache.org



[GitHub] [incubator-doris] chaoyli commented on a change in pull request #1297: Add copy function to support storage migration task

2019-06-13 Thread GitBox
chaoyli commented on a change in pull request #1297: Add copy function to 
support storage migration task
URL: https://github.com/apache/incubator-doris/pull/1297#discussion_r293240865
 
 

 ##
 File path: be/src/olap/rowset/alpha_rowset.cpp
 ##
 @@ -219,6 +219,20 @@ OLAPStatus AlphaRowset::make_snapshot(const std::string& 
snapshot_path,
 return OLAP_SUCCESS;
 }
 
+OLAPStatus AlphaRowset::copy_files_to_path(const std::string& dest_path,
+   std::vector* 
success_files) {
 
 Review comment:
   Already added.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org
For additional commands, e-mail: dev-h...@doris.apache.org



[GitHub] [incubator-doris] yiguolei opened a new pull request #1298: Add migration lock when migration

2019-06-13 Thread GitBox
yiguolei opened a new pull request #1298: Add migration lock when migration
URL: https://github.com/apache/incubator-doris/pull/1298
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org
For additional commands, e-mail: dev-h...@doris.apache.org



[GitHub] [incubator-doris] imay commented on a change in pull request #1297: Add copy function to support storage migration task

2019-06-13 Thread GitBox
imay commented on a change in pull request #1297: Add copy function to support 
storage migration task
URL: https://github.com/apache/incubator-doris/pull/1297#discussion_r293260962
 
 

 ##
 File path: be/src/olap/task/engine_storage_migration_task.cpp
 ##
 @@ -219,12 +219,16 @@ OLAPStatus 
EngineStorageMigrationTask::_copy_index_and_data_files(
 const string& schema_hash_path,
 const TabletSharedPtr& ref_tablet,
 std::vector& consistent_rowsets) {
-// TODO(lcy). copy function should be implemented
+std::vector success_files;
+OLAPStatus status = OLAP_SUCCESS;
 for (auto& rs : consistent_rowsets) {
-std::vector success_files;
-RETURN_NOT_OK(rs->make_snapshot(schema_hash_path, &success_files));
+status = rs->copy_files_to_path(schema_hash_path, &success_files);
+if (status != OLAP_SUCCESS) {
+while (OLAP_SUCCESS != remove_all_dir(schema_hash_path));
 
 Review comment:
   Don't dead loop here


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org
For additional commands, e-mail: dev-h...@doris.apache.org



[GitHub] [incubator-doris] chaoyli commented on a change in pull request #1297: Add copy function to support storage migration task

2019-06-13 Thread GitBox
chaoyli commented on a change in pull request #1297: Add copy function to 
support storage migration task
URL: https://github.com/apache/incubator-doris/pull/1297#discussion_r293262297
 
 

 ##
 File path: be/src/olap/rowset/segment_group.cpp
 ##
 @@ -723,6 +723,39 @@ OLAPStatus SegmentGroup::make_snapshot(const std::string& 
snapshot_path,
 return OLAP_SUCCESS;
 }
 
+OLAPStatus SegmentGroup::copy_files_to_path(const std::string& dest_path,
+std::vector* 
success_files) {
+if (_empty) {
+return OLAP_SUCCESS;
+}
+for (int segment_id = 0; segment_id < _num_segments; segment_id++) {
+std::string dest_data_file = construct_data_file_path(dest_path, 
segment_id);
+if (!check_dir_existed(dest_data_file)) {
 
 Review comment:
   I check dir exist or not firstly.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org
For additional commands, e-mail: dev-h...@doris.apache.org



[GitHub] [incubator-doris] chaoyli commented on a change in pull request #1297: Add copy function to support storage migration task

2019-06-13 Thread GitBox
chaoyli commented on a change in pull request #1297: Add copy function to 
support storage migration task
URL: https://github.com/apache/incubator-doris/pull/1297#discussion_r293262506
 
 

 ##
 File path: be/src/olap/task/engine_storage_migration_task.cpp
 ##
 @@ -219,12 +219,16 @@ OLAPStatus 
EngineStorageMigrationTask::_copy_index_and_data_files(
 const string& schema_hash_path,
 const TabletSharedPtr& ref_tablet,
 std::vector& consistent_rowsets) {
-// TODO(lcy). copy function should be implemented
+std::vector success_files;
+OLAPStatus status = OLAP_SUCCESS;
 for (auto& rs : consistent_rowsets) {
-std::vector success_files;
-RETURN_NOT_OK(rs->make_snapshot(schema_hash_path, &success_files));
+status = rs->copy_files_to_path(schema_hash_path, &success_files);
+if (status != OLAP_SUCCESS) {
+while (OLAP_SUCCESS != remove_all_dir(schema_hash_path));
 
 Review comment:
   I will fatal in this place.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org
For additional commands, e-mail: dev-h...@doris.apache.org



[GitHub] [incubator-doris] kangpinghuang commented on a change in pull request #1297: Add copy function to support storage migration task

2019-06-13 Thread GitBox
kangpinghuang commented on a change in pull request #1297: Add copy function to 
support storage migration task
URL: https://github.com/apache/incubator-doris/pull/1297#discussion_r293270677
 
 

 ##
 File path: be/src/olap/rowset/alpha_rowset.h
 ##
 @@ -85,7 +85,9 @@ class AlphaRowset : public Rowset {
 int64_t ref_count() const override;
 
 OLAPStatus make_snapshot(const std::string& snapshot_path,
- std::vector* success_files) override;
+ std::vector* success_links) override;
+OLAPStatus copy_files_to_path(const std::string& dest_path,
+  std::vector* success_files);
 
 Review comment:
   ```suggestion
 std::vector* success_files) 
override;
   ```


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org
For additional commands, e-mail: dev-h...@doris.apache.org



[GitHub] [incubator-doris] morningman merged pull request #1286: Add thirdparty lib

2019-06-13 Thread GitBox
morningman merged pull request #1286: Add thirdparty lib
URL: https://github.com/apache/incubator-doris/pull/1286
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org
For additional commands, e-mail: dev-h...@doris.apache.org



[GitHub] [incubator-doris] morningman opened a new pull request #1299: Handle the situation when there is no enough backends for tablet repair

2019-06-13 Thread GitBox
morningman opened a new pull request #1299: Handle the situation when there is 
no enough backends for tablet repair
URL: https://github.com/apache/incubator-doris/pull/1299
 
 
   In cast there are only 3 backends and replication num is 3. If one replica 
of a
   tablet is bad, there is no 4th backend for tablet repair. So we need to 
delete
   a bad replica first to make room for new replica.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org
For additional commands, e-mail: dev-h...@doris.apache.org



[GitHub] [incubator-doris] chaoyli merged pull request #1297: Add copy function to support storage migration task

2019-06-13 Thread GitBox
chaoyli merged pull request #1297: Add copy function to support storage 
migration task
URL: https://github.com/apache/incubator-doris/pull/1297
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org
For additional commands, e-mail: dev-h...@doris.apache.org



[GitHub] [incubator-doris] chaoyli opened a new pull request #1300: Remove olap_table.h and olap_tablet.cpp

2019-06-13 Thread GitBox
chaoyli opened a new pull request #1300: Remove olap_table.h and olap_tablet.cpp
URL: https://github.com/apache/incubator-doris/pull/1300
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org
For additional commands, e-mail: dev-h...@doris.apache.org



[GitHub] [incubator-doris] chaoyli merged pull request #1300: Remove olap_table.h and olap_tablet.cpp

2019-06-13 Thread GitBox
chaoyli merged pull request #1300: Remove olap_table.h and olap_tablet.cpp
URL: https://github.com/apache/incubator-doris/pull/1300
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org
For additional commands, e-mail: dev-h...@doris.apache.org



[GitHub] [incubator-doris] chaoyli merged pull request #1298: Add migration lock when migration

2019-06-13 Thread GitBox
chaoyli merged pull request #1298: Add migration lock when migration
URL: https://github.com/apache/incubator-doris/pull/1298
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org
For additional commands, e-mail: dev-h...@doris.apache.org



[GitHub] [incubator-doris] yiguolei opened a new pull request #1301: Remove data files using olap_header

2019-06-13 Thread GitBox
yiguolei opened a new pull request #1301: Remove data files using olap_header
URL: https://github.com/apache/incubator-doris/pull/1301
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org
For additional commands, e-mail: dev-h...@doris.apache.org



[GitHub] [incubator-doris] chaoyli merged pull request #1301: Remove data files using olap_header

2019-06-13 Thread GitBox
chaoyli merged pull request #1301: Remove data files using olap_header
URL: https://github.com/apache/incubator-doris/pull/1301
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org
For additional commands, e-mail: dev-h...@doris.apache.org



[GitHub] [incubator-doris] chaoyli opened a new pull request #1302: Place _init_seek_columns() in right place

2019-06-13 Thread GitBox
chaoyli opened a new pull request #1302: Place _init_seek_columns() in right 
place
URL: https://github.com/apache/incubator-doris/pull/1302
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org
For additional commands, e-mail: dev-h...@doris.apache.org



[GitHub] [incubator-doris] imay merged pull request #1302: Place _init_seek_columns() in right place

2019-06-13 Thread GitBox
imay merged pull request #1302: Place _init_seek_columns() in right place
URL: https://github.com/apache/incubator-doris/pull/1302
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org
For additional commands, e-mail: dev-h...@doris.apache.org



[GitHub] [incubator-doris] EmmyMiao87 opened a new pull request #1303: Fix the error of duplicated label

2019-06-13 Thread GitBox
EmmyMiao87 opened a new pull request #1303: Fix the error of duplicated label
URL: https://github.com/apache/incubator-doris/pull/1303
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org
For additional commands, e-mail: dev-h...@doris.apache.org



微信号-morningman-cmy无法检索

2019-06-13 Thread fanyu...@eqxiu.com

您好,目前我们公司正在搭建Doris并使用,官网上留的微信号morningman-cmy无法检索到,请问贵方的技术讨论群如何加入?
期待您的回复~


范云丽   Emma
北京中网易企秀科技有限公司
Beijing KNET Eqxiu Technology LTD.
手机:(8610)-15510696906
网址:http://www.eqxiu.com/
使命:助力企业高效营销
价值观:正直/求实/进取/协同/分享/创新
总部地址:北京市海淀区东北旺西路软件园二期华胜天成科研大楼


Re: 微信号-morningman-cmy无法检索

2019-06-13 Thread Zhao Chun
在Apache的社区,是鼓励通过邮件进行交流的。有什么问题可以在邮件组进行沟通。
你可以加我的微信ApacheZhaoc,我会把你加入到群里面

Thanks,
ZHAO Chun



fanyu...@eqxiu.com  于2019年6月13日周四 下午10:40写道:

>
> 您好,目前我们公司正在搭建Doris并使用,官网上留的微信号morningman-cmy无法检索到,请问贵方的技术讨论群如何加入?
> 期待您的回复~
>
>
> 范云丽   Emma
> 北京中网易企秀科技有限公司
> Beijing KNET Eqxiu Technology LTD.
> 手机:(8610)-15510696906
> 网址:http://www.eqxiu.com/
> 使命:助力企业高效营销
> 价值观:正直/求实/进取/协同/分享/创新
> 总部地址:北京市海淀区东北旺西路软件园二期华胜天成科研大楼
>


[GitHub] [incubator-doris] morningman closed pull request #1299: Handle the situation when there is no enough backends for tablet repair

2019-06-13 Thread GitBox
morningman closed pull request #1299: Handle the situation when there is no 
enough backends for tablet repair
URL: https://github.com/apache/incubator-doris/pull/1299
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org
For additional commands, e-mail: dev-h...@doris.apache.org



[GitHub] [incubator-doris] morningman opened a new pull request #1299: Handle the situation when there is no enough backends for tablet repair

2019-06-13 Thread GitBox
morningman opened a new pull request #1299: Handle the situation when there is 
no enough backends for tablet repair
URL: https://github.com/apache/incubator-doris/pull/1299
 
 
   In cast there are only 3 backends and replication num is 3. If one replica 
of a
   tablet is bad, there is no 4th backend for tablet repair. So we need to 
delete
   a bad replica first to make room for new replica.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org
For additional commands, e-mail: dev-h...@doris.apache.org



[GitHub] [incubator-doris] kangpinghuang opened a new pull request #1304: Add bitshuffle page

2019-06-13 Thread GitBox
kangpinghuang opened a new pull request #1304: Add bitshuffle page
URL: https://github.com/apache/incubator-doris/pull/1304
 
 
   1. bitshuffle page
   2. add CMakeLists.txt for segment_v2
   3. unit tests


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org
For additional commands, e-mail: dev-h...@doris.apache.org



[GitHub] [incubator-doris] kangkaisen commented on a change in pull request #1289: Modify colocation creation logic

2019-06-13 Thread GitBox
kangkaisen commented on a change in pull request #1289: Modify colocation 
creation logic
URL: https://github.com/apache/incubator-doris/pull/1289#discussion_r293649761
 
 

 ##
 File path: fe/src/main/java/org/apache/doris/catalog/ColocateGroupSchema.java
 ##
 @@ -0,0 +1,144 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+package org.apache.doris.catalog;
+
+import org.apache.doris.catalog.ColocateTableIndex.GroupId;
+import org.apache.doris.common.DdlException;
+import org.apache.doris.common.ErrorCode;
+import org.apache.doris.common.ErrorReport;
+import org.apache.doris.common.io.Writable;
+
+import com.google.common.collect.Lists;
+
+import java.io.DataInput;
+import java.io.DataOutput;
+import java.io.IOException;
+import java.util.List;
+
+/*
+ * Author: Chenmingyu
+ * Date: Jun 10, 2019
+ */
+
+/*
+ * This class saves the schema of a colocation group
+ */
+public class ColocateGroupSchema implements Writable {
+private GroupId groupId;
+private List distributionCols = Lists.newArrayList();
 
 Review comment:
   I think we really need info is column type?  SerDe the whole column maybe 
unnecessary,we should keep metadata  as small as  possible.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org
For additional commands, e-mail: dev-h...@doris.apache.org



[GitHub] [incubator-doris] kangkaisen commented on a change in pull request #1289: Modify colocation creation logic

2019-06-13 Thread GitBox
kangkaisen commented on a change in pull request #1289: Modify colocation 
creation logic
URL: https://github.com/apache/incubator-doris/pull/1289#discussion_r293651660
 
 

 ##
 File path: fe/src/main/java/org/apache/doris/catalog/ColocateTableIndex.java
 ##
 @@ -33,35 +42,89 @@
 import java.util.HashSet;
 import java.util.List;
 import java.util.Map;
+import java.util.Optional;
 import java.util.Set;
-import java.util.concurrent.CopyOnWriteArraySet;
 import java.util.concurrent.locks.ReentrantReadWriteLock;
+import java.util.stream.Collectors;
 
 /**
  * maintain the colocate table related indexes and meta
  */
 public class ColocateTableIndex implements Writable {
-private transient ReentrantReadWriteLock lock;
+private static final Logger LOG = 
LogManager.getLogger(ColocateTableIndex.class);
+
+public static class GroupId implements Writable {
+public Long dbId;
+public Long grpId;
+
+private GroupId() {
+}
+
+public GroupId(long dbId, long grpId) {
+this.dbId = dbId;
+this.grpId = grpId;
+}
+
+public static GroupId read(DataInput in) throws IOException {
+GroupId groupId = new GroupId();
+groupId.readFields(in);
+return groupId;
+}
+
+@Override
+public void write(DataOutput out) throws IOException {
+out.writeLong(dbId);
+out.writeLong(grpId);
+}
+
+@Override
+public void readFields(DataInput in) throws IOException {
+dbId = in.readLong();
+grpId = in.readLong();
+}
 
+@Override
+public boolean equals(Object obj) {
+if (!(obj instanceof GroupId)) {
+return false;
+}
+GroupId other = (GroupId) obj;
+return dbId.equals(other.dbId) && grpId.equals(other.grpId);
+}
+
+@Override
+public int hashCode() {
+int result = 17;
+result = 31 * result + dbId.hashCode();
+result = 31 * result + grpId.hashCode();
+return result;
+}
+
+@Override
+public String toString() {
+return dbId + "." + grpId;
+}
+}
+
+// group_name -> group_id
+private Map groupName2Id = Maps.newHashMap();
+// group_id -> group_name
+private Map groupId2Name = Maps.newHashMap();
 
 Review comment:
   Do we really need `Map groupId2Name`?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org
For additional commands, e-mail: dev-h...@doris.apache.org



[GitHub] [incubator-doris] kangpinghuang opened a new issue #1305: Add new file format for storage segment

2019-06-13 Thread GitBox
kangpinghuang opened a new issue #1305: Add new file format for storage segment
URL: https://github.com/apache/incubator-doris/issues/1305
 
 
   **Is your feature request related to a problem? Please describe.**
   Now the segment format in BE storage is orc-like format. There are some 
problems:
   1. the file header will be modified after we flush all data, It does not 
apply to cloud environment because the files in distribute file system(eg: 
hdfs), s3 and so do not support random write.
   2. random seek. When read the stream, you first read the StreamHead(8 bytes) 
first, than read the stream data. I think this mechinism is not good.
   3. there are no block cache.
   4. string is stored in plain
   5. it is hard to add secondary index
   6. the data is store in static row number block
   
   So, I would like to add a new format segment for BE to solve the problems 
mentioned above.
   
   To goals to achive include:
   1. write file meta to the footer of the segment file
   2. to support block cache
   3. to support secondary indexes, eg: bitmap index
   4. to support dict encodeing string storage
   5. construct a block in configured size
   6. to support extend the encoding and compression easily.
   
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org
For additional commands, e-mail: dev-h...@doris.apache.org



[GitHub] [incubator-doris] kangkaisen commented on a change in pull request #1289: Modify colocation creation logic

2019-06-13 Thread GitBox
kangkaisen commented on a change in pull request #1289: Modify colocation 
creation logic
URL: https://github.com/apache/incubator-doris/pull/1289#discussion_r293652937
 
 

 ##
 File path: fe/src/main/java/org/apache/doris/http/meta/ColocateMetaService.java
 ##
 @@ -54,8 +58,11 @@
 
 private static ColocateTableIndex colocateIndex = 
Catalog.getCurrentColocateIndex();
 
-private static long checkAndGetGroupId(BaseRequest request) throws 
DdlException {
-long groupId = 
Long.valueOf(request.getSingleParameter(GROUP_ID).trim());
+private static GroupId checkAndGetGroupId(BaseRequest request) throws 
DdlException {
+long grpId = Long.valueOf(request.getSingleParameter(GROUP_ID).trim());
+long dbId = Long.valueOf(request.getSingleParameter(DB_ID).trim());
+GroupId groupId = new GroupId(dbId, grpId);
+
 
 Review comment:
   `palo/fe/src/main/java/org/apache/doris/http/Http API Reference` this file 
seems need to update?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org
For additional commands, e-mail: dev-h...@doris.apache.org



[GitHub] [incubator-doris] kangkaisen commented on a change in pull request #1289: Modify colocation creation logic

2019-06-13 Thread GitBox
kangkaisen commented on a change in pull request #1289: Modify colocation 
creation logic
URL: https://github.com/apache/incubator-doris/pull/1289#discussion_r293653425
 
 

 ##
 File path: fe/src/main/java/org/apache/doris/persist/TablePropertyInfo.java
 ##
 @@ -33,45 +37,45 @@
 private long dbId;
 
 Review comment:
   the `dbId` could be removed?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org
For additional commands, e-mail: dev-h...@doris.apache.org



[GitHub] [incubator-doris] kangpinghuang opened a new issue #1306: Add new file format design

2019-06-13 Thread GitBox
kangpinghuang opened a new issue #1306: Add new file format design
URL: https://github.com/apache/incubator-doris/issues/1306
 
 
   To solve the problem of ISSUE #1305 , I propose a design of new file format.
   Please refer to PR #1267 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org
For additional commands, e-mail: dev-h...@doris.apache.org



[GitHub] [incubator-doris] kangpinghuang closed issue #1306: Add new file format design

2019-06-13 Thread GitBox
kangpinghuang closed issue #1306: Add new file format design
URL: https://github.com/apache/incubator-doris/issues/1306
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org
For additional commands, e-mail: dev-h...@doris.apache.org



[GitHub] [incubator-doris] kangpinghuang opened a new issue #1307: Add new file format meta design

2019-06-13 Thread GitBox
kangpinghuang opened a new issue #1307: Add new file format meta design
URL: https://github.com/apache/incubator-doris/issues/1307
 
 
   Parent Issue: #1305 
   
   I propose a new file meta design in protobuf, pls refer to #1268 


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org
For additional commands, e-mail: dev-h...@doris.apache.org



[GitHub] [incubator-doris] kangpinghuang closed issue #1307: Add new file format meta design

2019-06-13 Thread GitBox
kangpinghuang closed issue #1307: Add new file format meta design
URL: https://github.com/apache/incubator-doris/issues/1307
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org
For additional commands, e-mail: dev-h...@doris.apache.org



[GitHub] [incubator-doris] kangpinghuang opened a new issue #1308: Add some common and options for new file format segment

2019-06-13 Thread GitBox
kangpinghuang opened a new issue #1308: Add some common and options for new 
file format segment
URL: https://github.com/apache/incubator-doris/issues/1308
 
 
   Parent Issue #1305 
   
   pls refer to PR #1269 


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org
For additional commands, e-mail: dev-h...@doris.apache.org



[GitHub] [incubator-doris] kangkaisen commented on a change in pull request #1289: Modify colocation creation logic

2019-06-13 Thread GitBox
kangkaisen commented on a change in pull request #1289: Modify colocation 
creation logic
URL: https://github.com/apache/incubator-doris/pull/1289#discussion_r293654151
 
 

 ##
 File path: fe/src/test/java/org/apache/doris/catalog/ColocateTableTest.java
 ##
 @@ -477,62 +377,4 @@ public void testDistributionColumnsType() throws 
Exception {
 
 catalog.createTable(childStmt);
 }
-
-@Test
-public void testParentTableNotExist() throws Exception {
-testParentTableNotExist("t8");
-}
-
-@Test
-public void testParentTableNotExistCaseSensitive() throws Exception {
-testParentTableNotExist(tableName1.toUpperCase());
-}
-
-private void testParentTableNotExist(String tableName) throws Exception {
-Map properties = new HashMap();
-properties.put(PropertyAnalyzer.PROPERTIES_COLOCATE_WITH, tableName);
-
-int bucketNum = 1;
-CreateTableStmt parentStmt = new CreateTableStmt(false, false, 
dbTableName1, columnDefs, "olap",
-new KeysDesc(KeysType.AGG_KEYS, columnNames), null,
-new HashDistributionDesc(bucketNum, 
Lists.newArrayList("key1")), properties, null);
-parentStmt.analyze(analyzer);
-
-expectedEx.expect(DdlException.class);
-expectedEx.expectMessage(String.format("Colocate table '%s' no exist", 
tableName));
-
-catalog.createTable(parentStmt);
-}
-
-@Test
-public void testParentTableType() throws Exception {
 
 Review comment:
   Why remove `testParentTableType` method?  I think we still only support 
OlapTable.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org
For additional commands, e-mail: dev-h...@doris.apache.org



[GitHub] [incubator-doris] kangpinghuang closed issue #1308: Add some common and options for new file format segment

2019-06-13 Thread GitBox
kangpinghuang closed issue #1308: Add some common and options for new file 
format segment
URL: https://github.com/apache/incubator-doris/issues/1308
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org
For additional commands, e-mail: dev-h...@doris.apache.org



[GitHub] [incubator-doris] kangpinghuang closed issue #1309: Add page api for new file format segment

2019-06-13 Thread GitBox
kangpinghuang closed issue #1309: Add page api for new file format segment
URL: https://github.com/apache/incubator-doris/issues/1309
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org
For additional commands, e-mail: dev-h...@doris.apache.org



[GitHub] [incubator-doris] kangpinghuang opened a new issue #1309: Add page api for new file format segment

2019-06-13 Thread GitBox
kangpinghuang opened a new issue #1309: Add page api for new file format segment
URL: https://github.com/apache/incubator-doris/issues/1309
 
 
   Parent Issue #1305 
   include:
   1. page builder api
   2. page decoder api
   pls refer to PR #1270 


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org
For additional commands, e-mail: dev-h...@doris.apache.org



[GitHub] [incubator-doris] kangpinghuang opened a new issue #1310: Add some utils for new file format

2019-06-13 Thread GitBox
kangpinghuang opened a new issue #1310: Add some utils for new file format
URL: https://github.com/apache/incubator-doris/issues/1310
 
 
   parent Issue #1305 
   Add some utils for new file format, including:
   1. faststring
   2. cpu
   
   pls refer to PR #1281


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org
For additional commands, e-mail: dev-h...@doris.apache.org



[GitHub] [incubator-doris] kangpinghuang closed issue #1310: Add some utils for new file format

2019-06-13 Thread GitBox
kangpinghuang closed issue #1310: Add some utils for new file format
URL: https://github.com/apache/incubator-doris/issues/1310
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org
For additional commands, e-mail: dev-h...@doris.apache.org



[GitHub] [incubator-doris] kangkaisen commented on a change in pull request #1289: Modify colocation creation logic

2019-06-13 Thread GitBox
kangkaisen commented on a change in pull request #1289: Modify colocation 
creation logic
URL: https://github.com/apache/incubator-doris/pull/1289#discussion_r293655591
 
 

 ##
 File path: fe/src/main/java/org/apache/doris/catalog/Catalog.java
 ##
 @@ -4109,6 +4093,13 @@ private void createTablets(String clusterName, 
MaterializedIndex index, ReplicaS
 db.writeUnlock();
 }
 }
+
+// firstColocateTable is true, means this may be the first table 
of colocation group,
+// or this is just a normal table.
+boolean firstColocateTable = backendsPerBucketSeq == null || 
backendsPerBucketSeq.isEmpty();
 
 Review comment:
   the variate name `firstColocateTable ` would better change: 
shouldChooseBackendsRandom?  chooseBackendsFromColocateMeta?  or  
isColocateChildTable ?


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org
For additional commands, e-mail: dev-h...@doris.apache.org



[GitHub] [incubator-doris] kangpinghuang opened a new issue #1311: Add some thirdparty libs for new encoding method

2019-06-13 Thread GitBox
kangpinghuang opened a new issue #1311: Add some thirdparty libs for new 
encoding method
URL: https://github.com/apache/incubator-doris/issues/1311
 
 
   Parent Issue #1305 
   We want to support bitshuffle encoding and bitmap index. So we need to add 
some libs:
   1. add bitshuffle lib
   2. add CRoaring lib


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org
For additional commands, e-mail: dev-h...@doris.apache.org



[GitHub] [incubator-doris] kangpinghuang commented on issue #1311: Add some thirdparty libs for new encoding method

2019-06-13 Thread GitBox
kangpinghuang commented on issue #1311: Add some thirdparty libs for new 
encoding method
URL: 
https://github.com/apache/incubator-doris/issues/1311#issuecomment-501963412
 
 
   pls refer to PR #1286


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org
For additional commands, e-mail: dev-h...@doris.apache.org



[GitHub] [incubator-doris] kangpinghuang closed issue #1311: Add some thirdparty libs for new encoding method

2019-06-13 Thread GitBox
kangpinghuang closed issue #1311: Add some thirdparty libs for new encoding 
method
URL: https://github.com/apache/incubator-doris/issues/1311
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org
For additional commands, e-mail: dev-h...@doris.apache.org



[GitHub] [incubator-doris] morningman opened a new pull request #1312: Fix bug that FE web frontend can not get static resource files

2019-06-13 Thread GitBox
morningman opened a new pull request #1312: Fix bug that FE web frontend can 
not get static resource files
URL: https://github.com/apache/incubator-doris/pull/1312
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org
For additional commands, e-mail: dev-h...@doris.apache.org



[GitHub] [incubator-doris] morningman commented on a change in pull request #1292: Support Grouping Sets, Rollup and Cube to extend group by statement

2019-06-13 Thread GitBox
morningman commented on a change in pull request #1292: Support Grouping Sets, 
Rollup and Cube to extend group by statement
URL: https://github.com/apache/incubator-doris/pull/1292#discussion_r292808242
 
 

 ##
 File path: be/src/exec/repeat_node.cpp
 ##
 @@ -0,0 +1,214 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements.  See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership.  The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License.  You may obtain a copy of the License at
+//
+//   http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied.  See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#include "exec/repeat_node.h"
+
+#include "exprs/expr.h"
+#include "runtime/raw_value.h"
+#include "runtime/row_batch.h"
+#include "runtime/runtime_state.h"
+#include "util/runtime_profile.h"
+
+namespace doris {
+
+RepeatNode::RepeatNode(ObjectPool* pool, const TPlanNode& tnode,
+ const DescriptorTbl& descs)
+: ExecNode(pool, tnode, descs),
+_slot_id_set_list(tnode.repeat_node.slot_id_set_list),
+_repeat_id_list(tnode.repeat_node.repeat_id_list),
+_output_tuple_id(tnode.repeat_node.output_tuple_id),
+_tuple_desc(nullptr),
+_child_row_batch(nullptr),
+_child_eos(false),
+_repeat_id_idx(0),
+_runtime_state(nullptr) {
+}
+
+RepeatNode::~RepeatNode() {
+}
+
+Status RepeatNode::prepare(RuntimeState* state) {
+SCOPED_TIMER(_runtime_profile->total_time_counter());
+RETURN_IF_ERROR(ExecNode::prepare(state));
+
+_runtime_state = state;
+_tuple_desc = state->desc_tbl().get_tuple_descriptor(_output_tuple_id);
+if (_tuple_desc == NULL) {
+return Status("Failed to get tuple descriptor.");
+}
+
+return Status::OK;
+}
+
+Status RepeatNode::open(RuntimeState* state) {
+SCOPED_TIMER(_runtime_profile->total_time_counter());
+RETURN_IF_ERROR(ExecNode::open(state));
+RETURN_IF_CANCELLED(state);
+RETURN_IF_ERROR(child(0)->open(state));
+return Status::OK;
+}
+
+Status RepeatNode::get_repeated_batch(
+RowBatch* child_row_batch, int repeat_id_idx, RowBatch* row_batch) 
{
+DCHECK(repeat_id_idx >= 0);
+DCHECK(repeat_id_idx <= (int)_repeat_id_list.size());
+DCHECK(child_row_batch != nullptr);
+DCHECK_EQ(row_batch->num_rows(), 0);
+
+// Fill all slots according to child
+MemPool* tuple_pool = row_batch->tuple_data_pool();
+const vector& src_tuple_descs = 
child_row_batch->row_desc().tuple_descriptors();
+const vector& dst_tuple_descs = 
row_batch->row_desc().tuple_descriptors();
+vector dst_tuples(src_tuple_descs.size(), nullptr);
+for (int i = 0; i < child_row_batch->num_rows(); ++i) {
+int row_idx = row_batch->add_row();
+TupleRow* dst_row = row_batch->get_row(row_idx);
+TupleRow* src_row = child_row_batch->get_row(i);
+
+vector::const_iterator src_it = 
src_tuple_descs.begin();
+vector::const_iterator dst_it = 
dst_tuple_descs.begin();
+for (int j = 0; src_it != src_tuple_descs.end() && dst_it != 
dst_tuple_descs.end(); 
+++src_it, ++dst_it, ++j) {
+Tuple* src_tuple = src_row->get_tuple(j);
+if (src_tuple == NULL) {
+continue;
+}
+
+if (dst_tuples[j] == nullptr) {
+int size = row_batch->capacity() * (*dst_it)->byte_size();
+void* tuple_buffer = tuple_pool->allocate(size);
+if (tuple_buffer == nullptr) {
+return Status("Allocate memory for row batch failed.");
+}
+dst_tuples[j] = reinterpret_cast(tuple_buffer);
+} else {
+char* new_tuple = reinterpret_cast(dst_tuples[j]);
+new_tuple += (*dst_it)->byte_size();
+dst_tuples[j] = reinterpret_cast(new_tuple);
+}
+dst_row->set_tuple(j, dst_tuples[j]);
+memset(dst_tuples[j], 0, (*dst_it)->num_null_bytes());
+
+for (int k = 0; k < (*src_it)->slots().size(); k++) {
+SlotDescriptor* src_slot_desc = (*src_it)->slots()[k];
+SlotDescriptor* dst_slot_desc = (*dst_it)->slots()[k];
+DCHECK_EQ(src_slot_desc->type().type, 
dst_slot_desc->type().type);
+DCHECK_EQ(src_slot_desc->col_name(), 
dst_slot_desc->col_name());
+
+if (_slot_id_set_list[0].find(src_slot_desc->id()) != 
_slot_id_set_list[0].end()) {
+std::set r

[GitHub] [incubator-doris] imay closed pull request #1303: Fix the error of duplicated label

2019-06-13 Thread GitBox
imay closed pull request #1303: Fix the error of duplicated label
URL: https://github.com/apache/incubator-doris/pull/1303
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org
For additional commands, e-mail: dev-h...@doris.apache.org



[GitHub] [incubator-doris] EmmyMiao87 opened a new pull request #1303: Fix the error of duplicated label

2019-06-13 Thread GitBox
EmmyMiao87 opened a new pull request #1303: Fix the error of duplicated label
URL: https://github.com/apache/incubator-doris/pull/1303
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org
For additional commands, e-mail: dev-h...@doris.apache.org



[GitHub] [incubator-doris] yiguolei opened a new pull request #1313: Use remove_all_dir to remote non-empty dir

2019-06-13 Thread GitBox
yiguolei opened a new pull request #1313: Use remove_all_dir to remote 
non-empty dir
URL: https://github.com/apache/incubator-doris/pull/1313
 
 
   1. Use remove_all_dir to remote non-empty dir
   2. Gc should not hold write lock too much time
   3. skip tablet dir if dir already exists when create tablet


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org
For additional commands, e-mail: dev-h...@doris.apache.org



[GitHub] [incubator-doris] morningman closed pull request #1299: Handle the situation when there is no enough backends for tablet repair

2019-06-13 Thread GitBox
morningman closed pull request #1299: Handle the situation when there is no 
enough backends for tablet repair
URL: https://github.com/apache/incubator-doris/pull/1299
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org
For additional commands, e-mail: dev-h...@doris.apache.org



[GitHub] [incubator-doris] imay merged pull request #1303: Fix the error of duplicated label

2019-06-13 Thread GitBox
imay merged pull request #1303: Fix the error of duplicated label
URL: https://github.com/apache/incubator-doris/pull/1303
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org
For additional commands, e-mail: dev-h...@doris.apache.org



[GitHub] [incubator-doris] morningman opened a new pull request #1299: Handle the situation when there is no enough backends for tablet repair

2019-06-13 Thread GitBox
morningman opened a new pull request #1299: Handle the situation when there is 
no enough backends for tablet repair
URL: https://github.com/apache/incubator-doris/pull/1299
 
 
   In cast there are only 3 backends and replication num is 3. If one replica 
of a
   tablet is bad, there is no 4th backend for tablet repair. So we need to 
delete
   a bad replica first to make room for new replica.


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org
For additional commands, e-mail: dev-h...@doris.apache.org



[GitHub] [incubator-doris] chaoyli merged pull request #1313: Use remove_all_dir to remote non-empty dir

2019-06-13 Thread GitBox
chaoyli merged pull request #1313: Use remove_all_dir to remote non-empty dir
URL: https://github.com/apache/incubator-doris/pull/1313
 
 
   


This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
 
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org


With regards,
Apache Git Services

-
To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org
For additional commands, e-mail: dev-h...@doris.apache.org