Re: [VOTE] Release Apache Doris 0.9.0-incubating-rc01
Hi Makoto, We got it, thank you. We are going to recheck and improve docker building scripts. Best Regards, Reed On 2019/1/12 上午1:12， "Makoto Yui" wrote: >With PARALLEL=1, the build fails as follows > >/usr/bin/../bin/g++ -DHAVE_CONFIG_H -I. -I../.. >-I../../lib/cpp/src/thrift -I./src >-I/tmp/doris/apache-doris-0.9.0.rc01-incubating-src/thirdparty/installed/i >nclude >-Wall -Wextra -pedantic -g -O2 -std=c++11 -MT >src/generate/thrift-t_as3_generator.o -MD -MP -MF >src/generate/.deps/thrift-t_as3_generator.Tpo -c -o >src/generate/thrift-t_as3_generator.o `test -f >'src/generate/t_as3_generator.cc' || echo >'./'`src/generate/t_as3_generator.cc > >src/generate/t_as3_generator.cc: In member function 'void >t_as3_generator::generate_as3_struct(t_struct*, bool)': > >src/generate/t_as3_generator.cc:663:1: internal compiler error: >Segmentation >fault > > } > > ^ > >Please submit a full bug report, > >with preprocessed source if appropriate. > >See for instructions. > >Makefile:1066: recipe for target 'src/generate/thrift-t_as3_generator.o' >failed > >make[3]: *** [src/generate/thrift-t_as3_generator.o] Error 1 > >make[3]: Leaving directory >'/tmp/doris/apache-doris-0.9.0.rc01-incubating-src/thirdparty/src/thrift-0 >.9.3/compiler/cpp' > >Makefile:588: recipe for target 'all' failed > >make[2]: *** [all] Error 2 > >make[2]: Leaving directory >'/tmp/doris/apache-doris-0.9.0.rc01-incubating-src/thirdparty/src/thrift-0 >.9.3/compiler/cpp' > >Makefile:609: recipe for target 'all-recursive' failed > >make[1]: *** [all-recursive] Error 1 > >make[1]: Leaving directory >'/tmp/doris/apache-doris-0.9.0.rc01-incubating-src/thirdparty/src/thrift-0 >.9.3' > >Makefile:530: recipe for target 'all' failed > >make: *** [all] Error 2 > >I'm not sure it's host memory issue. Host has enough memory left. > >BTW, the project page[1] should have disclaimer [2] of undergoing >incubation message. >[1] https://doris.incubator.apache.org/ >[2] https://incubator.apache.org/guides/branding.html#disclaimers > >Makoto > >2019年1月11日(金) 17:07 Li,De(BDG) : > >> Hi Makoto, >> >> Thank you for your check. >> >> >> Why not cancel rc1 and create rc2 fixing license header issue >>that >> Willem >> >> pointed? >> >> It seems it's better to be fixed. >> >> Actually, we have fixed these issues in branch-0.9.0 (#471, #473) after >> Willem pointed. >> >> I’m afraid it will take everyone’s more time to check repeatedly and so >> I’m going to fix them in next version. >> >> >> It also seems that no one from IPMC succeeded to build >>distribution >> from src >> >> Indeed, the building is not good as we expect, but I noticed the main >> cause is that it can’t compile in OSX and it needs GCC 5.3.1+. >> But it seems Dave Meikle has succeeded to build in Ubuntu 18.04 and we >> have tested pass in CentOS and Ubuntu. >> We will continue to check and complete the building, also docker >> environment as you mentioned. >> >> Best Regards, >> Reed >> >> >> On 2019/1/11 上午2:23， "Makoto Yui" wrote: >> >> >Reed, >> > >> >Why not cancel rc1 and create rc2 fixing license header issue that >>Willem >> >pointed? >> >It seems it's better to be fixed. >> > >> >It also seems that no one from IPMC succeeded to build distribution >>from >> >src (no sure for Luke though). >> > >> >I got the following build error (which might be gcc-5 version issue): >> > >> >[ 57%] Building CXX object >> >>>projects/compiler-rt/lib/msan/CMakeFiles/clang_rt.msan-x86_64.dir/msan_i >>>nt >> >erceptors.cc.o >> > >> >>>/tmp/doris/apache-doris-0.9.0.rc01-incubating-src/thirdparty/src/llvm-3. >>>4. >> >2.src/projects/compiler-rt/lib/msan/msan_interceptors.cc: >> >In function 'void __msan::InitializeInterceptors()': >> > >> >>>/tmp/doris/apache-doris-0.9.0.rc01-incubating-src/thirdparty/src/llvm-3. >>>4. >> >2.src/projects/compiler-rt/lib/msan/msan_interceptors.cc:1573:1: >> >internal >> >compiler error: Segmentation fault >> > >> > } >> > >> > ^ >> > >> >Please submit a full bug report, >> > >> >with preprocessed source if appropriate. >> > >> >See for instructions. >> > >> >projects/compiler-rt/lib/msan/CMakeFiles/clang_rt.msan-x86_64.dir/ >> build.ma >> >ke:134: >> >recipe for target >> >>>'projects/compiler-rt/lib/msan/CMakeFiles/clang_rt.msan-x86_64.dir/msan_ >>>in >> >terceptors.cc.o' >> >failed >> > >> >I used ubuntu xenial on docker on OSX. >> >Software versions meets requirements (except Maven version) as seen in >> > >> >docker run ubuntu:xenial -it >> > >> > >> >$ apt-get install wget openjdk-8-jdk maven gcc-5 bzip2 python cmake zip >> >xz-utils patch byacc flex automake libtool g++ >> > >> > >> >root@d9e5b7017e7b:/tmp/doris/apache-doris-0.9.0.rc01-incubating-src# >>gcc >> >--version | head -1 >> > >> >gcc (Ubuntu 5.4.0-6ubuntu1~16.04.11) 5.4.0 20160609 >> > >> > >> >root@d9e5b7017e7b:/tmp/doris/apache-doris-0.9.0.rc01-incubating-src# >>java >> >-version >> > >> >openjdk version "1.8.0_191" >> > >> >OpenJDK Runtime Environment (build >> >1.8.0_191-8u191-b12-0ubuntu0.16
[CANCELLED][VOTE] Release Apache Doris 0.9.0-incubating-rc01
Hi -, I'm cancelling the vote due to license and building issues after discussed in the Doris community. For details see: https://lists.apache.org/thread.html/6c8177522029d5bac98d0b300dbe16ffc22e3d0 6351ffc88501b2100@%3Cgeneral.incubator.apache.org%3E The Doris community will fix these issues and send out rc02 for another vote. Best Regards, Reed
[GitHub] kangpinghuang commented on a change in pull request #529: Remove segment group from delta writer
kangpinghuang commented on a change in pull request #529: Remove segment group
from delta writer
URL: https://github.com/apache/incubator-doris/pull/529#discussion_r247392934
##
File path: be/src/olap/delta_writer.cpp
##
@@ -143,20 +148,7 @@ OLAPStatus DeltaWriter::write(Tuple* tuple) {
_mem_table->insert(tuple);
if (_mem_table->memory_usage() >= config::write_buffer_size) {
-RETURN_NOT_OK(_mem_table->flush(_writer));
-
-++_segment_group_id;
-//_cur_segment_group = new SegmentGroup(_tablet.get(), false,
_segment_group_id, 0, true,
-// _req.partition_id, _req.transaction_id);
-_cur_segment_group = nullptr;
-DCHECK(_cur_segment_group != nullptr) << "failed to malloc
SegmentGroup";
-_cur_segment_group->acquire();
-_cur_segment_group->set_load_id(_req.load_id);
-_segment_group_vec.push_back(_cur_segment_group);
-
-SAFE_DELETE(_writer);
-_writer = ColumnDataWriter::create(_cur_segment_group, true,
_tablet->compress_kind(), _tablet->bloom_filter_fpp());
-DCHECK(_writer != nullptr) << "memory error occur when creating
writer";
+RETURN_NOT_OK(_mem_table->flush(_rowset_builder.get()));
Review comment:
OK, I will modify it
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org
For additional commands, e-mail: dev-h...@doris.apache.org
[GitHub] kangpinghuang commented on a change in pull request #529: Remove segment group from delta writer
kangpinghuang commented on a change in pull request #529: Remove segment group
from delta writer
URL: https://github.com/apache/incubator-doris/pull/529#discussion_r247392985
##
File path: be/src/olap/delta_writer.cpp
##
@@ -172,15 +164,18 @@ OLAPStatus
DeltaWriter::close(google::protobuf::RepeatedPtrField* t
return st;
}
}
-RETURN_NOT_OK(_mem_table->close(_writer));
+RETURN_NOT_OK(_mem_table->close(_rowset_builder.get()));
OLAPStatus res = OLAP_SUCCESS;
//add pending data to tablet
-RETURN_NOT_OK(_tablet->add_pending_version(_req.partition_id,
_req.transaction_id, nullptr));
-for (SegmentGroup* segment_group : _segment_group_vec) {
-RETURN_NOT_OK(_tablet->add_pending_segment_group(segment_group));
-RETURN_NOT_OK(segment_group->load());
-}
+RETURN_NOT_OK(_tablet->add_pending_version(_req.partition_id, _req.txn_id,
nullptr));
+// use rowset meta manager to save meta
+_cur_rowset = _rowset_builder->build();
+RowsetMetaManager::save(
+_tablet->data_dir()->get_meta(),
+_cur_rowset->rowset_id(),
+_cur_rowset->rowset_meta().get());
Review comment:
OK, I will modify it
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org
For additional commands, e-mail: dev-h...@doris.apache.org
[GitHub] kangpinghuang commented on a change in pull request #529: Remove segment group from delta writer
kangpinghuang commented on a change in pull request #529: Remove segment group
from delta writer
URL: https://github.com/apache/incubator-doris/pull/529#discussion_r247392970
##
File path: be/src/olap/delta_writer.cpp
##
@@ -172,15 +164,18 @@ OLAPStatus
DeltaWriter::close(google::protobuf::RepeatedPtrField* t
return st;
}
}
-RETURN_NOT_OK(_mem_table->close(_writer));
+RETURN_NOT_OK(_mem_table->close(_rowset_builder.get()));
OLAPStatus res = OLAP_SUCCESS;
//add pending data to tablet
-RETURN_NOT_OK(_tablet->add_pending_version(_req.partition_id,
_req.transaction_id, nullptr));
-for (SegmentGroup* segment_group : _segment_group_vec) {
-RETURN_NOT_OK(_tablet->add_pending_segment_group(segment_group));
-RETURN_NOT_OK(segment_group->load());
-}
+RETURN_NOT_OK(_tablet->add_pending_version(_req.partition_id, _req.txn_id,
nullptr));
Review comment:
OK, I think this should be removed
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
With regards,
Apache Git Services
-
To unsubscribe, e-mail: dev-unsubscr...@doris.apache.org
For additional commands, e-mail: dev-h...@doris.apache.org
