[GitHub] [incubator-doris] HappenLee commented on a change in pull request #6993: [Alter] Support alter table engine type from MySQL to ODBC
HappenLee commented on a change in pull request #6993: URL: https://github.com/apache/incubator-doris/pull/6993#discussion_r744216445 ## File path: docs/en/sql-reference/sql-statements/Data Definition/ALTER TABLE.md ## @@ -395,6 +401,10 @@ under the License. 20. Modify column comment ALTER TABLE example_db.my_table MODIFY COLUMN k1 COMMENT "k1", MODIFY COLUMN k2 COMMENT "k2"; + +21. Modify Engine Type Review comment: The doc should keep as old. 'Engine Type' -> ‘engine type’ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #6993: [Alter] Support alter table engine type from MySQL to ODBC
github-actions[bot] commented on pull request #6993: URL: https://github.com/apache/incubator-doris/pull/6993#issuecomment-962568378 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] morningman commented on pull request #7024: fix core problem: use object replace pointer
morningman commented on pull request #7024: URL: https://github.com/apache/incubator-doris/pull/7024#issuecomment-962578999 > > Conflict > > done Unit test compile failed. You need to run `run-be-ut.sh` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] morningman merged pull request #6577: Custom config handler #6508
morningman merged pull request #6577: URL: https://github.com/apache/incubator-doris/pull/6577 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] morningman closed issue #6508: [Feature] Custom config handler, support callback when set mutable config at FE and BE
morningman closed issue #6508: URL: https://github.com/apache/incubator-doris/issues/6508 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[incubator-doris] branch master updated: [Config] Support custom config handler (#6577)
This is an automated email from the ASF dual-hosted git repository.
morningman pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-doris.git
The following commit(s) were added to refs/heads/master by this push:
new 3dd5570 [Config] Support custom config handler (#6577)
3dd5570 is described below
commit 3dd55701ba455d25f8ff0c1b339fc12595527aa8
Author: ccoffline <45881148+ccoffl...@users.noreply.github.com>
AuthorDate: Sun Nov 7 17:39:24 2021 +0800
[Config] Support custom config handler (#6577)
Support custom config handler callback and types.
---
.../java/org/apache/doris/common/ConfigBase.java | 204 +
.../apache/doris/http/rest/SetConfigAction.java| 50 ++---
.../apache/doris/httpv2/rest/SetConfigAction.java | 68 ++-
.../doris/httpv2/rest/manager/NodeAction.java | 33 ++--
.../java/org/apache/doris/qe/ShowExecutor.java | 24 +--
.../doris/analysis/AdminSetConfigStmtTest.java | 2 +-
6 files changed, 137 insertions(+), 244 deletions(-)
diff --git a/fe/fe-core/src/main/java/org/apache/doris/common/ConfigBase.java
b/fe/fe-core/src/main/java/org/apache/doris/common/ConfigBase.java
index b7298aa..1287c19 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/common/ConfigBase.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/common/ConfigBase.java
@@ -17,6 +17,7 @@
package org.apache.doris.common;
+import org.apache.doris.catalog.Catalog;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
@@ -39,21 +40,36 @@ import java.util.Map;
import java.util.Properties;
import java.util.regex.Matcher;
import java.util.regex.Pattern;
+import java.util.stream.Collectors;
+import java.util.stream.Stream;
public class ConfigBase {
private static final Logger LOG = LogManager.getLogger(ConfigBase.class);
@Retention(RetentionPolicy.RUNTIME)
-public static @interface ConfField {
+public @interface ConfField {
String value() default "";
boolean mutable() default false;
boolean masterOnly() default false;
String comment() default "";
+Class callback() default
DefaultConfHandler.class;
+}
+
+public interface ConfHandler {
+void handle(Field field, String confVal) throws Exception;
+}
+
+static class DefaultConfHandler implements ConfHandler {
+@Override
+public void handle(Field field, String confVal) throws Exception{
+setConfigField(field, confVal);
+}
}
private static String confFile;
private static String customConfFile;
public static Class confClass;
+public static Map confFields;
private static String ldapConfFile;
private static String ldapCustomConfFile;
@@ -66,6 +82,15 @@ public class ConfigBase {
if (!isLdapConfig) {
confClass = this.getClass();
confFile = configFile;
+confFields = Maps.newHashMap();
+for (Field field : confClass.getFields()) {
+ConfField confField = field.getAnnotation(ConfField.class);
+if (confField == null) {
+continue;
+}
+confFields.put(confField.value().equals("") ? field.getName()
: confField.value(), field);
+}
+
initConf(confFile);
} else {
ldapConfClass = this.getClass();
@@ -90,42 +115,48 @@ public class ConfigBase {
replacedByEnv(props);
setFields(props, isLdapConfig);
}
-
-public static HashMap dump() throws Exception {
-HashMap map = new HashMap();
-Field[] fields = confClass.getFields();
+
+public static HashMap dump() {
+HashMap map = new HashMap<>();
+Field[] fields = confClass.getFields();
for (Field f : fields) {
-if (f.getAnnotation(ConfField.class) == null) {
-continue;
+ConfField anno = f.getAnnotation(ConfField.class);
+if (anno != null) {
+map.put(anno.value().isEmpty() ? f.getName() : anno.value(),
getConfValue(f));
}
-if (f.getType().isArray()) {
-switch (f.getType().getSimpleName()) {
+}
+return map;
+}
+
+public static String getConfValue(Field field) {
+try {
+if (field.getType().isArray()) {
+switch (field.getType().getSimpleName()) {
+case "boolean[]":
+return Arrays.toString((boolean[]) field.get(null));
+case "char[]":
+return Arrays.toString((char[]) field.get(null));
+case "byte[]":
+return Arrays.toString((byte[]) field.get(null));
case "short[]":
-map.put(f.getName(), Arrays.toString((short[])
f.get(null)));
-
[GitHub] [incubator-doris] morningman merged pull request #6600: [Feature] Extend logger interface, support structured log output #6529
morningman merged pull request #6600: URL: https://github.com/apache/incubator-doris/pull/6600 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] morningman closed issue #6529: [Proposal] Extend logging framework to support structured log output and custom log format
morningman closed issue #6529: URL: https://github.com/apache/incubator-doris/issues/6529 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[incubator-doris] branch master updated: [Feature] Extend logger interface, support structured log output (#6600)
This is an automated email from the ASF dual-hosted git repository.
morningman pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-doris.git
The following commit(s) were added to refs/heads/master by this push:
new ca8268f [Feature] Extend logger interface, support structured log
output (#6600)
ca8268f is described below
commit ca8268f1c9669aeca510ea705e146d3a4a44a983
Author: ccoffline <45881148+ccoffl...@users.noreply.github.com>
AuthorDate: Sun Nov 7 17:39:53 2021 +0800
[Feature] Extend logger interface, support structured log output (#6600)
Support structured logging.
---
be/src/common/logconfig.cpp| 17
be/src/runtime/plan_fragment_executor.cpp | 25 +++---
be/src/util/logging.h | 90 ++
.../doris/common/logger/DefaultDorisLogger.java| 55 +
.../apache/doris/common/logger/LoggerProvider.java | 51
.../org/apache/doris/common/logger/TagKey.java | 27 +++
.../apache/doris/common/logger/TaggableLogger.java | 64 +++
.../doris/common/logger/TaggedLogFormat.java | 26 +++
.../java/org/apache/doris/common/logger/Tags.java | 24 ++
9 files changed, 366 insertions(+), 13 deletions(-)
diff --git a/be/src/common/logconfig.cpp b/be/src/common/logconfig.cpp
index e3463f1..49684ac 100644
--- a/be/src/common/logconfig.cpp
+++ b/be/src/common/logconfig.cpp
@@ -159,4 +159,21 @@ std::string FormatTimestampForLog(MicrosecondsInt64
micros_since_epoch) {
tm_time.tm_hour, tm_time.tm_min, tm_time.tm_sec,
usecs);
}
+/// Custom your log format here
+void TaggableLogger::flush() {
+_stream << _message;
+Tags *head = _tags;
+Tags *next;
+while (head) {
+next = head->next;
+_stream << "|" << head->key << "=" << head->value;
+delete head;
+head = next;
+}
+}
+
+/// Modify these tag names to suit your log format and collector.
+const std::string TaggableLogger::QUERY_ID = "query_id";
+const std::string TaggableLogger::INSTANCE_ID = "instance_id";
+
} // namespace doris
diff --git a/be/src/runtime/plan_fragment_executor.cpp
b/be/src/runtime/plan_fragment_executor.cpp
index d0f2cb2..670248e 100644
--- a/be/src/runtime/plan_fragment_executor.cpp
+++ b/be/src/runtime/plan_fragment_executor.cpp
@@ -21,7 +21,6 @@
#include
-#include "common/logging.h"
#include "common/object_pool.h"
#include "exec/data_sink.h"
#include "exec/exchange_node.h"
@@ -40,7 +39,7 @@
#include "util/mem_info.h"
#include "util/parse_util.h"
#include "util/pretty_printer.h"
-#include "util/uid_util.h"
+#include "util/logging.h"
namespace doris {
@@ -70,9 +69,9 @@ Status PlanFragmentExecutor::prepare(const
TExecPlanFragmentParams& request,
const TPlanFragmentExecParams& params = request.params;
_query_id = params.query_id;
-LOG(INFO) << "Prepare(): query_id=" << print_id(_query_id)
- << " fragment_instance_id=" <<
print_id(params.fragment_instance_id)
- << " backend_num=" << request.backend_num;
+TAG(LOG(INFO)).log("PlanFragmentExecutor::prepare")
+ .query_id(_query_id).instance_id(params.fragment_instance_id)
+ .tag("backend_num", std::to_string(request.backend_num));
// VLOG_CRITICAL << "request:\n" <<
apache::thrift::ThriftDebugString(request);
const TQueryGlobals& query_globals =
@@ -223,10 +222,10 @@ Status PlanFragmentExecutor::prepare(const
TExecPlanFragmentParams& request,
}
Status PlanFragmentExecutor::open() {
-LOG(INFO) << "Open(): fragment_instance_id="
- << print_id(_runtime_state->fragment_instance_id())
- << ", Using query memory limit: "
- <<
PrettyPrinter::print(_runtime_state->fragment_mem_tracker()->limit(),
TUnit::BYTES);
+int64_t mem_limit = _runtime_state->fragment_mem_tracker()->limit();
+TAG(LOG(INFO)).log("PlanFragmentExecutor::open, using query memory limit:
" + PrettyPrinter::print(mem_limit, TUnit::BYTES))
+
.query_id(_query_id).instance_id(_runtime_state->fragment_instance_id())
+ .tag("mem_limit", std::to_string(mem_limit));
// we need to start the profile-reporting thread before calling Open(),
since it
// may block
@@ -445,8 +444,8 @@ Status PlanFragmentExecutor::get_next(RowBatch** batch) {
update_status(status);
if (_done) {
-LOG(INFO) << "Finished executing fragment query_id=" <<
print_id(_query_id)
- << " instance_id=" <<
print_id(_runtime_state->fragment_instance_id());
+TAG(LOG(INFO)).log("PlanFragmentExecutor::get_next finished")
+
.query_id(_query_id).instance_id(_runtime_state->fragment_instance_id());
// Query is done, return the thread token
stop_report_thread();
send_report(true);
@@ -504,8 +503,8 @@ void PlanFragmen
[GitHub] [incubator-doris] morningman merged pull request #6936: Fix some return logic error in init BE encoding_map
morningman merged pull request #6936: URL: https://github.com/apache/incubator-doris/pull/6936 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[incubator-doris] branch master updated: [Bug] Fix some return logic error in init BE encoding_map (#6936)
This is an automated email from the ASF dual-hosted git repository.
morningman pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-doris.git
The following commit(s) were added to refs/heads/master by this push:
new 9b1a801 [Bug] Fix some return logic error in init BE encoding_map
(#6936)
9b1a801 is described below
commit 9b1a80114eb977e348bd176480b0eb5d245e95fd
Author: Zhikai Zuo
AuthorDate: Sun Nov 7 17:40:18 2021 +0800
[Bug] Fix some return logic error in init BE encoding_map (#6936)
Checking _encoding_map in the original code to return in advance will cause
some encoding methods cannot be pushed to default_encoding_type_map_ or
value_seek_encoding_map_ in EncodingInfoResolver constructor.
E.g:
EncodingInfoResolver::EncodingInfoResolver() {
_add_map();
_add_map();
...
}
The second line code is invilid.
---
be/src/olap/rowset/segment_v2/encoding_info.cpp | 11 +--
1 file changed, 5 insertions(+), 6 deletions(-)
diff --git a/be/src/olap/rowset/segment_v2/encoding_info.cpp
b/be/src/olap/rowset/segment_v2/encoding_info.cpp
index e4ce43c..bcad01f 100644
--- a/be/src/olap/rowset/segment_v2/encoding_info.cpp
+++ b/be/src/olap/rowset/segment_v2/encoding_info.cpp
@@ -189,12 +189,6 @@ private:
// Not thread-safe
template
void _add_map() {
-auto key = std::make_pair(type, encoding_type);
-auto it = _encoding_map.find(key);
-if (it != _encoding_map.end()) {
-return;
-}
-
EncodingTraits traits;
std::unique_ptr encoding(new EncodingInfo(traits));
if (_default_encoding_type_map.find(type) ==
std::end(_default_encoding_type_map)) {
@@ -204,6 +198,11 @@ private:
_value_seek_encoding_map.find(type) ==
_value_seek_encoding_map.end()) {
_value_seek_encoding_map[type] = encoding_type;
}
+auto key = std::make_pair(type, encoding_type);
+auto it = _encoding_map.find(key);
+if (it != _encoding_map.end()) {
+return;
+}
_encoding_map.emplace(key, encoding.release());
}
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[incubator-doris] branch master updated: [Refactor] Refactor part of RuntimeFilter's code (#6998)
This is an automated email from the ASF dual-hosted git repository.
morningman pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-doris.git
The following commit(s) were added to refs/heads/master by this push:
new 29ca776 [Refactor] Refactor part of RuntimeFilter's code (#6998)
29ca776 is described below
commit 29ca77622fe492ec767221b7d87b6dfcfff1aa8c
Author: Pxl <952130...@qq.com>
AuthorDate: Sun Nov 7 17:40:45 2021 +0800
[Refactor] Refactor part of RuntimeFilter's code (#6998)
#6997
---
be/src/exec/hash_join_node.cpp | 5 +-
be/src/exec/hash_join_node.h | 1 +
be/src/exprs/bloomfilter_predicate.cpp | 22 --
be/src/exprs/hybrid_set.cpp| 19 +++--
be/src/exprs/hybrid_set.h | 16 ++--
be/src/exprs/runtime_filter.cpp| 108 +-
be/src/exprs/runtime_filter.h | 46 +--
be/src/exprs/runtime_filter_slots.h| 137 +
be/src/runtime/primitive_type.h| 4 +
9 files changed, 197 insertions(+), 161 deletions(-)
diff --git a/be/src/exec/hash_join_node.cpp b/be/src/exec/hash_join_node.cpp
index 3fc0dfe..b1b9356 100644
--- a/be/src/exec/hash_join_node.cpp
+++ b/be/src/exec/hash_join_node.cpp
@@ -242,8 +242,7 @@ Status HashJoinNode::open(RuntimeState* state) {
_runtime_filter_descs);
RETURN_IF_ERROR(thread_status.get_future().get());
-RETURN_IF_ERROR(runtime_filter_slots.init(state, _pool,
expr_mem_tracker().get(),
- _hash_tbl->size()));
+RETURN_IF_ERROR(runtime_filter_slots.init(state, _hash_tbl->size()));
{
SCOPED_TIMER(_push_compute_timer);
auto func = [&](TupleRow* row) { runtime_filter_slots.insert(row);
};
@@ -252,7 +251,7 @@ Status HashJoinNode::open(RuntimeState* state) {
COUNTER_UPDATE(_build_timer, _push_compute_timer->value());
{
SCOPED_TIMER(_push_down_timer);
-runtime_filter_slots.publish(this);
+runtime_filter_slots.publish();
}
Status open_status = child(0)->open(state);
RETURN_IF_ERROR(open_status);
diff --git a/be/src/exec/hash_join_node.h b/be/src/exec/hash_join_node.h
index 2379f77..f097a2c 100644
--- a/be/src/exec/hash_join_node.h
+++ b/be/src/exec/hash_join_node.h
@@ -25,6 +25,7 @@
#include "exec/exec_node.h"
#include "exec/hash_table.h"
+#include "exprs/runtime_filter_slots.h"
#include "gen_cpp/PlanNodes_types.h"
namespace doris {
diff --git a/be/src/exprs/bloomfilter_predicate.cpp
b/be/src/exprs/bloomfilter_predicate.cpp
index 44906fd..4bc7584 100644
--- a/be/src/exprs/bloomfilter_predicate.cpp
+++ b/be/src/exprs/bloomfilter_predicate.cpp
@@ -40,31 +40,37 @@ IBloomFilterFuncBase*
IBloomFilterFuncBase::create_bloom_filter(MemTracker* trac
return new BloomFilterFunc(tracker);
case TYPE_BIGINT:
return new BloomFilterFunc(tracker);
+case TYPE_LARGEINT:
+return new BloomFilterFunc(tracker);
+
case TYPE_FLOAT:
return new BloomFilterFunc(tracker);
case TYPE_DOUBLE:
return new BloomFilterFunc(tracker);
+
+case TYPE_DECIMALV2:
+return new BloomFilterFunc(tracker);
+
+case TYPE_TIME:
+return new BloomFilterFunc(tracker);
case TYPE_DATE:
return new BloomFilterFunc(tracker);
case TYPE_DATETIME:
return new BloomFilterFunc(tracker);
-case TYPE_DECIMALV2:
-return new BloomFilterFunc(tracker);
-case TYPE_LARGEINT:
-return new BloomFilterFunc(tracker);
+
case TYPE_CHAR:
return new BloomFilterFunc(tracker);
case TYPE_VARCHAR:
return new BloomFilterFunc(tracker);
case TYPE_STRING:
-return new BloomFilterFunc(tracker);
+return new BloomFilterFunc(tracker);
+
default:
-return nullptr;
+DCHECK(false) << "Invalid type.";
}
return nullptr;
}
-
BloomFilterPredicate::BloomFilterPredicate(const TExprNode& node)
: Predicate(node),
_is_prepare(false),
@@ -74,7 +80,7 @@ BloomFilterPredicate::BloomFilterPredicate(const TExprNode&
node)
BloomFilterPredicate::~BloomFilterPredicate() {
VLOG_NOTICE << "bloom filter rows:" << _filtered_rows << ",scan_rows:" <<
_scan_rows
- << ",rate:" << (double)_filtered_rows / _scan_rows;
+<< ",rate:" << (double)_filtered_rows / _scan_rows;
}
BloomFilterPredicate::BloomFilterPredicate(const BloomFilterPredicate& other)
diff --git a/be/src/exprs/hybrid_set.cpp b/be/src/exprs/hybrid_set.cpp
index 4ebdabc..1b3fd37 100644
--- a/be/src/exprs/hybrid_set.cpp
+++ b/be/src/exprs/hybrid_set.cpp
@@ -36,21 +36,22 @@ HybridSetBase* HybridSetBase::create_set(PrimitiveType
type) {
case TYPE_BIGINT:
return new (std::nothrow) HybridSet();
+case TYPE_LARGE
[GitHub] [incubator-doris] morningman closed issue #6997: [Feature] Refactor part of RuntimeFilter's code
morningman closed issue #6997: URL: https://github.com/apache/incubator-doris/issues/6997 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] morningman merged pull request #6998: [Refactor] Refactor part of RuntimeFilter's code
morningman merged pull request #6998: URL: https://github.com/apache/incubator-doris/pull/6998 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] morningman opened a new pull request #7029: [Compile] Fix FE compile problem
morningman opened a new pull request #7029: URL: https://github.com/apache/incubator-doris/pull/7029 ## Proposed changes Introduced from #6600 ## Types of changes What types of changes does your code introduce to Doris? _Put an `x` in the boxes that apply_ - [ ] Bugfix (non-breaking change which fixes an issue) - [ ] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) - [ ] Documentation Update (if none of the other choices apply) - [ ] Code refactor (Modify the code structure, format the code, etc...) - [ ] Optimization. Including functional usability improvements and performance improvements. - [ ] Dependency. Such as changes related to third-party components. - [ ] Other. ## Checklist _Put an `x` in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code._ - [ ] I have created an issue on (Fix #ISSUE) and described the bug/feature there in detail - [ ] Compiling and unit tests pass locally with my changes - [ ] I have added tests that prove my fix is effective or that my feature works - [ ] If these changes need document changes, I have updated the document - [ ] Any dependent changes have been merged ## Further comments If this is a relatively large or complex change, kick off the discussion at d...@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] xy720 opened a new pull request #7030: [Feature] Print peak memory use of all backend after query in audit log
xy720 opened a new pull request #7030: URL: https://github.com/apache/incubator-doris/pull/7030 ## Proposed changes  ## Types of changes What types of changes does your code introduce to Doris? _Put an `x` in the boxes that apply_ - [x] New feature (non-breaking change which adds functionality) ## Checklist _Put an `x` in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code._ - [x] I have created an issue on (Fix #6945 ) and described the bug/feature there in detail - [x] Compiling and unit tests pass locally with my changes ## Further comments If this is a relatively large or complex change, kick off the discussion at d...@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] xinyiZzz opened a new issue #7031: [Feature] Support cancel outdated query after FE restart
xinyiZzz opened a new issue #7031: URL: https://github.com/apache/incubator-doris/issues/7031 ### Search before asking - [X] I had searched in the [issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and found no similar issues. ### Description After the FE restarts, the query submitted when the FE was started last time will continue to run and eventually fail. This can lead to wasted resources and in some cases failure to recover from failures in time. ### Use case For example, a large query causes doris to be stuck, and it can only be recovered by restarting all BE. which is usually very slow in a large cluster. ### Related issues _No response_ ### Are you willing to submit PR? - [X] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] xinyiZzz opened a new pull request #7032: [Heartbeat] Support cancel outdated query after FE restart
xinyiZzz opened a new pull request #7032: URL: https://github.com/apache/incubator-doris/pull/7032 ## Proposed changes Support cancel outdated query after FE restart, through three steps: 1. Add startTime to Frontend Info and pass it in the heartbeat between Frontend. 2. Add the startTime of all Frontends to the heartbeat between Master and Backend. 3. Backend identifies whether the query is out of date according to the startTime of Frontend. Recognizable obsolete select query: Master restart, other Frontend stop or restart. Recognizable obsolete load query: Master restart. In the future, it is expected to support Backend to cancel outdated queries after failing to receive Master's heartbeat for a long time. ## Types of changes What types of changes does your code introduce to Doris? _Put an `x` in the boxes that apply_ - [ ] Bugfix (non-breaking change which fixes an issue) - [x] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) - [ ] Documentation Update (if none of the other choices apply) - [ ] Code refactor (Modify the code structure, format the code, etc...) - [ ] Optimization. Including functional usability improvements and performance improvements. - [ ] Dependency. Such as changes related to third-party components. - [ ] Other. ## Checklist _Put an `x` in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code._ - [x] I have created an issue on (Fix #ISSUE) and described the bug/feature there in detail - [x] Compiling and unit tests pass locally with my changes - [x] I have added tests that prove my fix is effective or that my feature works - [ ] If these changes need document changes, I have updated the document - [ ] Any dependent changes have been merged ## Further comments If this is a relatively large or complex change, kick off the discussion at d...@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #7029: [Compile] Fix FE compile problem
github-actions[bot] commented on pull request #7029: URL: https://github.com/apache/incubator-doris/pull/7029#issuecomment-962757713 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] yangzhg merged pull request #7029: [Compile] Fix FE compile problem
yangzhg merged pull request #7029: URL: https://github.com/apache/incubator-doris/pull/7029 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[incubator-doris] branch master updated: [Compile] Fix FE compile problem (#7029)
This is an automated email from the ASF dual-hosted git repository.
yangzhg pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-doris.git
The following commit(s) were added to refs/heads/master by this push:
new 9c12060 [Compile] Fix FE compile problem (#7029)
9c12060 is described below
commit 9c12060db3616051359d9ac500a124cecf3c358e
Author: Mingyu Chen
AuthorDate: Mon Nov 8 10:35:49 2021 +0800
[Compile] Fix FE compile problem (#7029)
Co-authored-by: morningman
---
.../src/main/java/org/apache/doris/common/logger/TaggableLogger.java | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/common/logger/TaggableLogger.java
b/fe/fe-core/src/main/java/org/apache/doris/common/logger/TaggableLogger.java
index 43d069a..fa98e78 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/common/logger/TaggableLogger.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/common/logger/TaggableLogger.java
@@ -18,7 +18,7 @@
package org.apache.doris.common.logger;
import org.apache.doris.common.util.DebugUtil;
-import org.apache.doris.proto.PUniqueId;
+import org.apache.doris.proto.Types;
import org.apache.doris.thrift.TUniqueId;
import org.apache.logging.log4j.Logger;
@@ -57,7 +57,7 @@ public interface TaggableLogger extends Logger {
return tag(TagKey.QUERY_ID, DebugUtil.printId(queryId));
}
-default TaggableLogger queryId(final PUniqueId queryId) {
+default TaggableLogger queryId(final Types.PUniqueId queryId) {
return tag(TagKey.QUERY_ID, DebugUtil.printId(queryId));
}
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] morningman commented on a change in pull request #7022: [Bug] Change DateTimeValue Memmory Layout To Old
morningman commented on a change in pull request #7022:
URL: https://github.com/apache/incubator-doris/pull/7022#discussion_r743644425
##
File path: be/src/runtime/datetime_value.h
##
@@ -628,25 +628,25 @@ class DateTimeValue {
// 1 bits for neg. 3 bits for type. 12bit for second
Review comment:
We should add a NOTICE here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] weajun opened a new pull request #7036: [Manager] Adapt to manger product prototype
weajun opened a new pull request #7036: URL: https://github.com/apache/incubator-doris/pull/7036 ## Proposed changes 1. Support recording task status when installing a cluster 2. Integrated user permission verification, etc. 3. Use JDBC to add BE nodes 4. Increase the installation progress query, log query and other interfaces 4. Modify the interface document 5. Optimize the agent log code ## Types of changes What types of changes does your code introduce to Doris? _Put an `x` in the boxes that apply_ - [ ] Bugfix (non-breaking change which fixes an issue) - [ ] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) - [ ] Documentation Update (if none of the other choices apply) - [ ] Code refactor (Modify the code structure, format the code, etc...) - [ ] Optimization. Including functional usability improvements and performance improvements. - [ ] Dependency. Such as changes related to third-party components. - [ ] Other. ## Checklist _Put an `x` in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code._ - [ ] I have created an issue on (Fix #ISSUE) and described the bug/feature there in detail - [ ] Compiling and unit tests pass locally with my changes - [ ] I have added tests that prove my fix is effective or that my feature works - [ ] If these changes need document changes, I have updated the document - [ ] Any dependent changes have been merged ## Further comments If this is a relatively large or complex change, kick off the discussion at d...@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] zcdJason opened a new issue #7037: [Bug]
zcdJason opened a new issue #7037: URL: https://github.com/apache/incubator-doris/issues/7037 ### Search before asking - [X] I had searched in the [issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and found no similar issues. ### Version 0.14.0 ### What's Wrong? when I query set the parameter "set query_timeout = 3000;" there will be an immediate error as follows: org.jkiss.dbeaver.model.sql.DBSQLException: SQL 错误 [1064] [42000]: errCode = 2, detailMessage = query timeout at org.jkiss.dbeaver.model.impl.jdbc.exec.JDBCStatementImpl.executeStatement(JDBCStatementImpl.java:133) at org.jkiss.dbeaver.ui.editors.sql.execute.SQLQueryJob.executeStatement(SQLQueryJob.java:513) at org.jkiss.dbeaver.ui.editors.sql.execute.SQLQueryJob.lambda$0(SQLQueryJob.java:444) at org.jkiss.dbeaver.model.exec.DBExecUtils.tryExecuteRecover(DBExecUtils.java:171) at org.jkiss.dbeaver.ui.editors.sql.execute.SQLQueryJob.executeSingleQuery(SQLQueryJob.java:431) at org.jkiss.dbeaver.ui.editors.sql.execute.SQLQueryJob.extractData(SQLQueryJob.java:816) at org.jkiss.dbeaver.ui.editors.sql.SQLEditor$QueryResultsContainer.readData(SQLEditor.java:3468) at org.jkiss.dbeaver.ui.controls.resultset.ResultSetJobDataRead.lambda$0(ResultSetJobDataRead.java:118) at org.jkiss.dbeaver.model.exec.DBExecUtils.tryExecuteRecover(DBExecUtils.java:171) at org.jkiss.dbeaver.ui.controls.resultset.ResultSetJobDataRead.run(ResultSetJobDataRead.java:116) at org.jkiss.dbeaver.ui.controls.resultset.ResultSetViewer$ResultSetDataPumpJob.run(ResultSetViewer.java:4796) at org.jkiss.dbeaver.model.runtime.AbstractJob.run(AbstractJob.java:105) at org.eclipse.core.internal.jobs.Worker.run(Worker.java:63) Caused by: java.sql.SQLSyntaxErrorException: errCode = 2, detailMessage = query timeout at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:120) at com.mysql.cj.jdbc.exceptions.SQLError.createSQLException(SQLError.java:97) at com.mysql.cj.jdbc.exceptions.SQLExceptionsMapping.translateException(SQLExceptionsMapping.java:122) at com.mysql.cj.jdbc.StatementImpl.executeInternal(StatementImpl.java:764) at com.mysql.cj.jdbc.StatementImpl.execute(StatementImpl.java:648) at org.jkiss.dbeaver.model.impl.jdbc.exec.JDBCStatementImpl.execute(JDBCStatementImpl.java:330) at org.jkiss.dbeaver.model.impl.jdbc.exec.JDBCStatementImpl.executeStatement(JDBCStatementImpl.java:130) ... 12 more ### What You Expected? return the result. ### How to Reproduce? _No response_ ### Anything Else? the Class "ResultReceiver.java" constructor parameter 'timeoutMs' should use Long type, use int will be overflow。 ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] HappenLee commented on a change in pull request #6515: [Memory] BitShufflePageDecoder use memory allocated by ChunkAllocator instead of Faststring
HappenLee commented on a change in pull request #6515:
URL: https://github.com/apache/incubator-doris/pull/6515#discussion_r744386965
##
File path: be/src/olap/rowset/segment_v2/bitshuffle_page.h
##
@@ -385,7 +394,7 @@ class BitShufflePageDecoder : public PageDecoder {
int _size_of_element;
size_t _cur_index;
-faststring _decoded;
+Chunk chunk;
Review comment:
'_chunk` not ‘chunk’
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] tinkerrrr commented on issue #6806: Release note 0.15.0 (WIP)
tinke commented on issue #6806: URL: https://github.com/apache/incubator-doris/issues/6806#issuecomment-962788714 > > 请问大概什么时候 Release？比较期待这个功能：Spark-Doris-Connector 支持数据写入Doris！😁 > > You can download the trunk code and compile the `spark-doris-connector` in `extensions/` dir to try it now. @morningman I tried this but got error `/bin/sh: thrift: command not found`, even though I've built the whole project include be, fe, and ui. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #7022: [Bug] Change DateTimeValue Memmory Layout To Old
github-actions[bot] commented on pull request #7022: URL: https://github.com/apache/incubator-doris/pull/7022#issuecomment-962810166 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] tianhui5 commented on pull request #7025: Add tablets number limit to affend wrong usage
tianhui5 commented on pull request #7025: URL: https://github.com/apache/incubator-doris/pull/7025#issuecomment-962845142 > I think we can use replicaQuotaSize in Database to limit. You are right, it can reach out same result. But the threshold value can only be changed in db level, I'll change the value from constant to config, so that we can change it in cluster level. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] code81192 opened a new issue #7040: [Bug] Compiler error
code81192 opened a new issue #7040: URL: https://github.com/apache/incubator-doris/issues/7040 ### Search before asking - [X] I had searched in the [issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and found no similar issues. ### Version Docker image version: 1.4.1 Source code version: 0.14.0 ### What's Wrong? [ERROR] COMPILATION ERROR : [INFO] - [ERROR] /root/incubator-doris/fe/fe-core/src/main/java/org/apache/doris/common/logger/TaggableLogger.java:[21,30] cannot find symbol symbol: class PUniqueId location: package org.apache.doris.proto [ERROR] /root/incubator-doris/fe/fe-core/src/main/java/org/apache/doris/common/logger/TaggableLogger.java:[60,42] cannot find symbol symbol: class PUniqueId location: interface org.apache.doris.common.logger.TaggableLogger [INFO] 2 errors ### What You Expected? '' ### How to Reproduce? Execute the compile command after cloning the source code in the container ### Anything Else? _No response_ ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] morningman commented on issue #7040: [Bug] Compiler error
morningman commented on issue #7040: URL: https://github.com/apache/incubator-doris/issues/7040#issuecomment-962854003 Fixed: #7029 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] morningman closed issue #7040: [Bug] Compiler error
morningman closed issue #7040: URL: https://github.com/apache/incubator-doris/issues/7040 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] yangzhg opened a new pull request #7041: [enhancement] Increase compatibility with mysql
yangzhg opened a new pull request #7041: URL: https://github.com/apache/incubator-doris/pull/7041 ## Proposed changes Increase compatibility with mysql 1. Added two system tables files and partitions 2. Improved the return logic of mysql error code to make the error code more compatible with mysql 3. Added lock/unlock tables statement and show columns statement for compatibility with mysql dump 4. Compatible with mysqldump tool ## Types of changes What types of changes does your code introduce to Doris? _Put an `x` in the boxes that apply_ - [ ] Bugfix (non-breaking change which fixes an issue) - [ ] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) - [ ] Documentation Update (if none of the other choices apply) - [ ] Code refactor (Modify the code structure, format the code, etc...) - [x] Optimization. Including functional usability improvements and performance improvements. - [ ] Dependency. Such as changes related to third-party components. - [ ] Other. ## Checklist _Put an `x` in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code._ - [ ] I have created an issue on (Fix #ISSUE) and described the bug/feature there in detail - [ ] Compiling and unit tests pass locally with my changes - [ ] I have added tests that prove my fix is effective or that my feature works - [ ] If these changes need document changes, I have updated the document - [ ] Any dependent changes have been merged ## Further comments If this is a relatively large or complex change, kick off the discussion at d...@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #6971: hll optimize: trace memory usage, new explicit data when really need
github-actions[bot] commented on pull request #6971: URL: https://github.com/apache/incubator-doris/pull/6971#issuecomment-962862614 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] xinyiZzz commented on a change in pull request #6515: [Memory] BitShufflePageDecoder use memory allocated by ChunkAllocator instead of Faststring
xinyiZzz commented on a change in pull request #6515:
URL: https://github.com/apache/incubator-doris/pull/6515#discussion_r77405
##
File path: be/src/olap/rowset/segment_v2/bitshuffle_page.h
##
@@ -385,7 +394,7 @@ class BitShufflePageDecoder : public PageDecoder {
int _size_of_element;
size_t _cur_index;
-faststring _decoded;
+Chunk chunk;
Review comment:
done
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] HappenLee commented on a change in pull request #6971: hll optimize: trace memory usage, new explicit data when really need
HappenLee commented on a change in pull request #6971:
URL: https://github.com/apache/incubator-doris/pull/6971#discussion_r744450364
##
File path: be/src/olap/hll.h
##
@@ -33,6 +33,7 @@ class Slice;
const static int HLL_COLUMN_PRECISION = 14;
const static int HLL_ZERO_COUNT_BITS = (64 - HLL_COLUMN_PRECISION);
const static int HLL_EXPLICIT_INT64_NUM = 160;
+const static int HLL_EXPLICIT_INT64_NUM_DOUBLE = HLL_EXPLICIT_INT64_NUM*2;
Review comment:
HLL_EXPLICIT_INT64_NUM * 2
##
File path: be/src/olap/hll.h
##
@@ -154,7 +174,7 @@ class HyperLogLog {
HllDataType _type = HLL_DATA_EMPTY;
uint32_t _explicit_data_num = 0;
-uint64_t _explicit_data[HLL_EXPLICIT_INT64_NUM * 2];
+uint64_t* _explicit_data = nullptr;
Review comment:
directly use the raw ptr is dangerous and complex. Rethink shoule we
replace is use std::unique_ptr
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] HappenLee commented on a change in pull request #6971: hll optimize: trace memory usage, new explicit data when really need
HappenLee commented on a change in pull request #6971:
URL: https://github.com/apache/incubator-doris/pull/6971#discussion_r744450847
##
File path: be/src/olap/hll.h
##
@@ -154,7 +174,7 @@ class HyperLogLog {
HllDataType _type = HLL_DATA_EMPTY;
uint32_t _explicit_data_num = 0;
-uint64_t _explicit_data[HLL_EXPLICIT_INT64_NUM * 2];
+uint64_t* _explicit_data = nullptr;
Review comment:
directly use the raw ptr is dangerous and complex. Rethink should we
replace it by std::unique_ptr
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] HappenLee commented on pull request #6515: [Memory] BitShufflePageDecoder use memory allocated by ChunkAllocator instead of Faststring
HappenLee commented on pull request #6515: URL: https://github.com/apache/incubator-doris/pull/6515#issuecomment-962876079 LGTM -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #6515: [Memory] BitShufflePageDecoder use memory allocated by ChunkAllocator instead of Faststring
github-actions[bot] commented on pull request #6515: URL: https://github.com/apache/incubator-doris/pull/6515#issuecomment-962876479 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] BiteTheDDDDt opened a new issue #7042: [Bug] core dump at TAG log
BiteThet opened a new issue #7042: URL: https://github.com/apache/incubator-doris/issues/7042 ### Search before asking - [X] I had searched in the [issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and found no similar issues. ### Version master ### What's Wrong? core dump at TAG log ### What You Expected? fix ### How to Reproduce? _No response_ ### Anything Else? _No response_ ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] BiteTheDDDDt opened a new pull request #7043: [Bug] fix Tags empty reference core dump
BiteThet opened a new pull request #7043: URL: https://github.com/apache/incubator-doris/pull/7043 ## Proposed changes key may have been destructed when key reference is called. ## Types of changes What types of changes does your code introduce to Doris? _Put an `x` in the boxes that apply_ - [x] Bugfix (non-breaking change which fixes an issue) - [ ] New feature (non-breaking change which adds functionality) - [ ] Breaking change (fix or feature that would cause existing functionality to not work as expected) - [ ] Documentation Update (if none of the other choices apply) - [ ] Code refactor (Modify the code structure, format the code, etc...) - [ ] Optimization. Including functional usability improvements and performance improvements. - [ ] Dependency. Such as changes related to third-party components. - [ ] Other. ## Checklist _Put an `x` in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code._ - [x] I have created an issue on (Fix #7042) and described the bug/feature there in detail - [] Compiling and unit tests pass locally with my changes - [ ] I have added tests that prove my fix is effective or that my feature works - [ ] If these changes need document changes, I have updated the document - [ ] Any dependent changes have been merged ## Further comments If this is a relatively large or complex change, kick off the discussion at d...@doris.apache.org by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #7043: [Bug] fix Tags empty reference core dump
github-actions[bot] commented on pull request #7043: URL: https://github.com/apache/incubator-doris/pull/7043#issuecomment-962879295 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
