[GitHub] [incubator-doris] morningman merged pull request #8136: [community](doc) Refactor the release and verify doc
morningman merged pull request #8136: URL: https://github.com/apache/incubator-doris/pull/8136 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[incubator-doris] branch master updated (02531cc -> 13d217b)
This is an automated email from the ASF dual-hosted git repository.
morningman pushed a change to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-doris.git.
from 02531cc [doc]fix start/stop command (#8150)
add 13d217b [community](doc) Refactor the release and verify doc (#8136)
No new revisions were added by this update.
Summary of changes:
.../public/images/staging-repositories.png | Bin 0 -> 249300 bytes
docs/.vuepress/public/images/staging_repo.png | Bin 392700 -> 0 bytes
docs/.vuepress/sidebar/en.js | 30 +-
docs/.vuepress/sidebar/zh-CN.js| 30 +-
.../commit-format-specification.md | 2 +-
.../{ => how-to-contribute}/committer-guide.md | 0
.../{ => how-to-contribute}/how-to-contribute.md | 8 +-
.../{ => how-to-contribute}/pull-request.md| 0
docs/en/community/how-to-deploy-to-maven.md| 185 -
.../release-and-verify/release-complete.md | 198 ++
.../release-and-verify/release-doris-connectors.md | 275
.../release-and-verify/release-doris-core.md | 324 +
.../release-and-verify/release-prepare.md | 342 ++
.../community/release-and-verify/release-verify.md | 101 +++
docs/en/community/release-process.md | 750 -
docs/en/community/{members.md => team.md} | 8 +-
docs/en/community/verify-apache-release.md | 92 ---
docs/en/extending-doris/flink-doris-connector.md | 23 +-
.../commit-format-specification.md | 2 +-
.../{ => how-to-contribute}/committer-guide.md | 0
.../{ => how-to-contribute}/how-to-contribute.md | 10 +-
.../{ => how-to-contribute}/pull-request.md| 0
docs/zh-CN/community/how-to-deploy-to-maven.md | 185 -
.../release-and-verify/release-complete.md | 194 ++
.../release-and-verify/release-doris-connectors.md | 279
.../release-and-verify/release-doris-core.md | 323 +
.../release-and-verify/release-prepare.md | 343 ++
.../release-verify.md} | 30 +-
docs/zh-CN/community/release-process.md| 731
docs/zh-CN/community/{members.md => team.md} | 7 +-
.../zh-CN/extending-doris/flink-doris-connector.md | 22 +-
31 files changed, 2472 insertions(+), 2022 deletions(-)
create mode 100644 docs/.vuepress/public/images/staging-repositories.png
delete mode 100644 docs/.vuepress/public/images/staging_repo.png
rename docs/en/community/{ =>
how-to-contribute}/commit-format-specification.md (99%)
rename docs/en/community/{ => how-to-contribute}/committer-guide.md (100%)
rename docs/en/community/{ => how-to-contribute}/how-to-contribute.md (88%)
rename docs/en/community/{ => how-to-contribute}/pull-request.md (100%)
delete mode 100644 docs/en/community/how-to-deploy-to-maven.md
create mode 100644 docs/en/community/release-and-verify/release-complete.md
create mode 100644
docs/en/community/release-and-verify/release-doris-connectors.md
create mode 100644 docs/en/community/release-and-verify/release-doris-core.md
create mode 100644 docs/en/community/release-and-verify/release-prepare.md
create mode 100644 docs/en/community/release-and-verify/release-verify.md
delete mode 100644 docs/en/community/release-process.md
rename docs/en/community/{members.md => team.md} (97%)
delete mode 100644 docs/en/community/verify-apache-release.md
rename docs/zh-CN/community/{ =>
how-to-contribute}/commit-format-specification.md (99%)
rename docs/zh-CN/community/{ => how-to-contribute}/committer-guide.md (100%)
rename docs/zh-CN/community/{ => how-to-contribute}/how-to-contribute.md (86%)
rename docs/zh-CN/community/{ => how-to-contribute}/pull-request.md (100%)
delete mode 100644 docs/zh-CN/community/how-to-deploy-to-maven.md
create mode 100644 docs/zh-CN/community/release-and-verify/release-complete.md
create mode 100644
docs/zh-CN/community/release-and-verify/release-doris-connectors.md
create mode 100644
docs/zh-CN/community/release-and-verify/release-doris-core.md
create mode 100644 docs/zh-CN/community/release-and-verify/release-prepare.md
rename docs/zh-CN/community/{verify-apache-release.md =>
release-and-verify/release-verify.md} (71%)
delete mode 100644 docs/zh-CN/community/release-process.md
rename docs/zh-CN/community/{members.md => team.md} (97%)
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] yiguolei commented on a change in pull request #8041: [Feature](create_table) Support create table with random distribution to avoid data skew
yiguolei commented on a change in pull request #8041:
URL: https://github.com/apache/incubator-doris/pull/8041#discussion_r810593964
##
File path: be/src/exec/tablet_sink.cpp
##
@@ -683,7 +683,13 @@ Status OlapTableSink::init(const TDataSink& t_sink) {
if (table_sink.__isset.send_batch_parallelism &&
table_sink.send_batch_parallelism > 1) {
_send_batch_parallelism = table_sink.send_batch_parallelism;
}
-
+if (table_sink.partition.distributed_columns.empty()) {
+if (table_sink.__isset.single_tablet_load_per_sink &&
table_sink.single_tablet_load_per_sink) {
Review comment:
Please add a comment here, the meanings of these two params.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] yiguolei commented on a change in pull request #8041: [Feature](create_table) Support create table with random distribution to avoid data skew
yiguolei commented on a change in pull request #8041:
URL: https://github.com/apache/incubator-doris/pull/8041#discussion_r810595114
##
File path: be/src/exec/tablet_sink.h
##
@@ -460,6 +462,11 @@ class OlapTableSink : public DataSink {
// TODO(cmy): this should be removed after we switch to rpc attachment by
default.
bool _transfer_data_by_brpc_attachment = false;
+
+enum FindTabletMode {
+FIND_TABLET_EVERY_ROW, FIND_TABLET_EVERY_BATCH, FIND_TABLET_EVERY_SINK
Review comment:
Add comment and explain here
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] yiguolei commented on a change in pull request #8041: [Feature](create_table) Support create table with random distribution to avoid data skew
yiguolei commented on a change in pull request #8041:
URL: https://github.com/apache/incubator-doris/pull/8041#discussion_r810595342
##
File path: be/src/vec/sink/vtablet_sink.cpp
##
@@ -117,7 +120,16 @@ Status VOlapTableSink::send(RuntimeState* state,
vectorized::Block* input_block)
continue;
}
_partition_ids.emplace(partition->id);
-uint32_t tablet_index = dist_hash % partition->num_buckets;
+if (findTabletMode != FindTabletMode::FIND_TABLET_EVERY_ROW) {
Review comment:
How to deal with find tablet every sink?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] caiconghui commented on a change in pull request #8041: [Feature](create_table) Support create table with random distribution to avoid data skew
caiconghui commented on a change in pull request #8041:
URL: https://github.com/apache/incubator-doris/pull/8041#discussion_r810596382
##
File path: be/src/vec/sink/vtablet_sink.cpp
##
@@ -117,7 +120,16 @@ Status VOlapTableSink::send(RuntimeState* state,
vectorized::Block* input_block)
continue;
}
_partition_ids.emplace(partition->id);
-uint32_t tablet_index = dist_hash % partition->num_buckets;
+if (findTabletMode != FindTabletMode::FIND_TABLET_EVERY_ROW) {
Review comment:
if we find first tablet for one partition, we don't clear the
partition_to_tablet_map before processing every row batch
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] hf200012 opened a new pull request #8154: [Doc]Correct parameter name
hf200012 opened a new pull request #8154: URL: https://github.com/apache/incubator-doris/pull/8154 Correct parameter name # Proposed changes Issue Number: close #xxx ## Problem Summary: Describe the overview of changes. ## Checklist(Required) 1. Does it affect the original behavior: (No) 2. Has unit tests been added: (No) 3. Has document been added or modified: (No) 4. Does it need to update dependencies: (No) 5. Are there any changes that cannot be rolled back: (No) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] HappenLee closed issue #8140: [Vectorized][Bug] Query with null const may cause coredump in vec query engine
HappenLee closed issue #8140: URL: https://github.com/apache/incubator-doris/issues/8140 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] zbtzbtzbt commented on a change in pull request #8138: [Vectorized][Feature] support some bitmap functions
zbtzbtzbt commented on a change in pull request #8138:
URL: https://github.com/apache/incubator-doris/pull/8138#discussion_r810614593
##
File path: be/src/vec/functions/function_bitmap_variadic.cpp
##
@@ -0,0 +1,229 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#include "vec/columns/column_complex.h"
+#include "vec/data_types/data_type.h"
+#include "vec/data_types/data_type_bitmap.h"
+#include "vec/functions/function_const.h"
+#include "vec/functions/function_string.h"
+#include "vec/functions/function_totype.h"
+#include "vec/functions/simple_function_factory.h"
+
+namespace doris::vectorized {
+
+struct BitmapAnd {
+static constexpr auto name = "bitmap_and";
+using ResultDataType = DataTypeBitMap;
+
+static Status vector_vector(ColumnPtr argument_columns[], size_t col_size,
+size_t input_rows_count,
std::vector& res) {
+
+res = assert_cast(argument_columns[0].get())->get_data();
+
+for (size_t col = 1; col < col_size; ++col) {
+auto& col_data =
+assert_cast(argument_columns[col].get())->get_data();
+for (size_t row = 0; row < input_rows_count; ++row) {
+res[row] &= col_data[row];
+}
+}
+return Status::OK();
+}
+};
+
+struct BitmapOr {
+static constexpr auto name = "bitmap_or";
+using ResultDataType = DataTypeBitMap;
+
+static Status vector_vector(ColumnPtr argument_columns[], size_t col_size,
+size_t input_rows_count,
std::vector& res) {
+
+res = assert_cast(argument_columns[0].get())->get_data();
+for (size_t col = 1; col < col_size; ++col) {
+auto& col_data =
+assert_cast(argument_columns[col].get())->get_data();
+for (size_t row = 0; row < input_rows_count; ++row) {
+res[row] |= col_data[row];
+}
+}
+return Status::OK();
+}
+};
+
+struct BitmapXor {
+using ResultDataType = DataTypeBitMap;
+static constexpr auto name = "bitmap_xor";
+
+static Status vector_vector(ColumnPtr argument_columns[], size_t col_size,
+size_t input_rows_count,
std::vector& res) {
+
+res = assert_cast(argument_columns[0].get())->get_data();
+for (size_t col = 1; col < col_size; ++col) {
+auto& col_data =
+assert_cast(argument_columns[col].get())->get_data();
+for (size_t row = 0; row < input_rows_count; ++row) {
+res[row] ^= col_data[row];
+}
+}
+return Status::OK();
+}
+};
+
+struct BitmapOrCount {
+static constexpr auto name = "bitmap_or_count";
+using ResultDataType = DataTypeInt64;
+using TData = std::vector;
+using ResTData = typename ColumnVector::Container;
+
+static Status vector_vector(ColumnPtr argument_columns[], size_t col_size,
+size_t input_rows_count, ResTData& res) {
+
+TData vals(input_rows_count);
+for (size_t col = 0; col < col_size; ++col) {
+auto& col_data =
+assert_cast(argument_columns[col].get())->get_data();
+for (size_t row = 0; row < input_rows_count; ++row) {
+vals[row] |= col_data[row];
+}
+}
+
+for (size_t row = 0; row < input_rows_count; ++row) {
+res[row] = vals[row].cardinality();
+}
+return Status::OK();
+}
+};
+
+struct BitmapAndCount {
+static constexpr auto name = "bitmap_and_count";
+using ResultDataType = DataTypeInt64;
+using TData = std::vector;
+using ResTData = typename ColumnVector::Container;
+
+static Status vector_vector(ColumnPtr argument_columns[], size_t col_size,
+size_t input_rows_count, ResTData& res) {
+
+TData vals(input_rows_count);
+vals = assert_cast(argument_columns[0].get())->get_data();
+
+for (size_t col = 1; col < col_size; ++col) {
+auto& col_data =
+assert_cast(argument_columns[col].get())->get_data();
+for (size_t row = 0; row <
[GitHub] [incubator-doris] HappenLee merged pull request #8139: [Bug][vec] Fix bug of nullable const value convert to argument cause coredump
HappenLee merged pull request #8139: URL: https://github.com/apache/incubator-doris/pull/8139 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[incubator-doris] branch master updated: [Bug][vec] Fix bug of nullable const value convert to argument cause coredump (#8139)
This is an automated email from the ASF dual-hosted git repository.
lihaopeng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-doris.git
The following commit(s) were added to refs/heads/master by this push:
new 56adc7f [Bug][vec] Fix bug of nullable const value convert to
argument cause coredump (#8139)
56adc7f is described below
commit 56adc7f56bb450fe9b766355cfb473223a2fba8f
Author: HappenLee
AuthorDate: Sun Feb 20 20:05:23 2022 +0800
[Bug][vec] Fix bug of nullable const value convert to argument cause
coredump (#8139)
Co-authored-by: lihaopeng
---
be/src/vec/functions/function_conv.cpp | 5 -
be/src/vec/functions/function_convert_tz.h | 5 -
be/src/vec/functions/function_string.h | 22 ++
be/src/vec/functions/function_totype.h | 10 --
4 files changed, 34 insertions(+), 8 deletions(-)
diff --git a/be/src/vec/functions/function_conv.cpp
b/be/src/vec/functions/function_conv.cpp
index ae88f68..1385603 100644
--- a/be/src/vec/functions/function_conv.cpp
+++ b/be/src/vec/functions/function_conv.cpp
@@ -54,9 +54,12 @@ public:
for (int i = 0; i < 3; ++i) {
argument_columns[i] =
block.get_by_position(arguments[i]).column->convert_to_full_column_if_const();
if (auto* nullable =
check_and_get_column(*argument_columns[i])) {
-argument_columns[i] = nullable->get_nested_column_ptr();
+// Danger: Here must dispose the null map data first! Because
+// argument_columns[i]=nullable->get_nested_column_ptr(); will
release the mem
+// of column nullable mem of null map
VectorizedUtils::update_null_map(result_null_map_column->get_data(),
nullable->get_null_map_data());
+argument_columns[i] = nullable->get_nested_column_ptr();
}
}
diff --git a/be/src/vec/functions/function_convert_tz.h
b/be/src/vec/functions/function_convert_tz.h
index 739279d..729 100644
--- a/be/src/vec/functions/function_convert_tz.h
+++ b/be/src/vec/functions/function_convert_tz.h
@@ -54,9 +54,12 @@ public:
argument_columns[i] =
block.get_by_position(arguments[i]).column->convert_to_full_column_if_const();
if (auto* nullable =
check_and_get_column(*argument_columns[i])) {
-argument_columns[i] = nullable->get_nested_column_ptr();
+// Danger: Here must dispose the null map data first! Because
+// argument_columns[i]=nullable->get_nested_column_ptr(); will
release the mem
+// of column nullable mem of null map
VectorizedUtils::update_null_map(result_null_map_column->get_data(),
nullable->get_null_map_data());
+argument_columns[i] = nullable->get_nested_column_ptr();
}
}
diff --git a/be/src/vec/functions/function_string.h
b/be/src/vec/functions/function_string.h
index 934053e..c14e639 100644
--- a/be/src/vec/functions/function_string.h
+++ b/be/src/vec/functions/function_string.h
@@ -127,9 +127,12 @@ struct SubstringUtil {
argument_columns[i] =
block.get_by_position(arguments[i]).column->convert_to_full_column_if_const();
if (auto* nullable =
check_and_get_column(*argument_columns[i])) {
-argument_columns[i] = nullable->get_nested_column_ptr();
+// Danger: Here must dispose the null map data first! Because
+// argument_columns[i]=nullable->get_nested_column_ptr(); will
release the mem
+// of column nullable mem of null map
VectorizedUtils::update_null_map(null_map->get_data(),
nullable->get_null_map_data());
+argument_columns[i] = nullable->get_nested_column_ptr();
}
}
@@ -521,13 +524,18 @@ public:
std::vector null_list(argument_size);
ColumnPtr argument_columns[argument_size];
+ColumnPtr argument_null_columns[argument_size];
for (size_t i = 0; i < argument_size; ++i) {
argument_columns[i] =
block.get_by_position(arguments[i]).column->convert_to_full_column_if_const();
if (auto* nullable = check_and_get_column(*argument_columns[i])) {
-argument_columns[i] = nullable->get_nested_column_ptr();
+// Danger: Here must dispose the null map data first! Because
+// argument_columns[i]=nullable->get_nested_column_ptr(); will
release the mem
+// of column nullable mem of null map
null_list[i] = &nullable->get_null_map_data();
+argument_null_columns[i] = nullable->get_null_map_column_ptr();
+argument
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #8154: [Doc]Correct parameter name
github-actions[bot] commented on pull request #8154: URL: https://github.com/apache/incubator-doris/pull/8154#issuecomment-1046226821 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] yiguolei commented on pull request #8041: [Feature](create_table) Support create table with random distribution to avoid data skew
yiguolei commented on pull request #8041: URL: https://github.com/apache/incubator-doris/pull/8041#issuecomment-1046230442 LGTM -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] hf200012 merged pull request #7984: [Docs] add rpc function document
hf200012 merged pull request #7984: URL: https://github.com/apache/incubator-doris/pull/7984 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[incubator-doris] branch master updated: [Docs] add rpc function document (#7984)
This is an automated email from the ASF dual-hosted git repository.
jiafengzheng pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-doris.git
The following commit(s) were added to refs/heads/master by this push:
new b27122d [Docs] add rpc function document (#7984)
b27122d is described below
commit b27122df8bedd25e3d3b8d5622b74e0d70093eeb
Author: Stalary <452024...@qq.com>
AuthorDate: Sun Feb 20 23:26:52 2022 +0800
[Docs] add rpc function document (#7984)
* add rpc function document
---
docs/.vuepress/sidebar/en.js | 3 +-
docs/.vuepress/sidebar/zh-CN.js| 3 +-
...function.md => native-user-defined-function.md} | 4 +-
.../udf/remote-user-defined-function.md| 106 ++
...function.md => native-user-defined-function.md} | 2 +-
.../udf/remote-user-defined-function.md| 108 ++
samples/doris-demo/pom.xml | 1 +
samples/doris-demo/udf-demo/pom.xml| 113 +++
.../java/org/apache/doris/udf/Application.java | 36
.../java/org/apache/doris/udf/FunctionGrpc.java| 102 ++
.../udf-demo/src/main/proto/function_service.proto | 63 ++
.../doris-demo/udf-demo/src/main/proto/types.proto | 217 +
12 files changed, 753 insertions(+), 5 deletions(-)
diff --git a/docs/.vuepress/sidebar/en.js b/docs/.vuepress/sidebar/en.js
index c8c1f48..5f314bc 100644
--- a/docs/.vuepress/sidebar/en.js
+++ b/docs/.vuepress/sidebar/en.js
@@ -255,7 +255,8 @@ module.exports = [
directoryPath: "udf/",
children: [
"contribute-udf",
- "user-defined-function",
+ "native-user-defined-function",
+ "remote-user-defined-function",
],
},
],
diff --git a/docs/.vuepress/sidebar/zh-CN.js b/docs/.vuepress/sidebar/zh-CN.js
index 0a57884..4c63d4c 100644
--- a/docs/.vuepress/sidebar/zh-CN.js
+++ b/docs/.vuepress/sidebar/zh-CN.js
@@ -256,7 +256,8 @@ module.exports = [
directoryPath: "udf/",
children: [
"contribute-udf",
- "user-defined-function",
+ "native-user-defined-function",
+ "remote-user-defined-function",
],
},
],
diff --git a/docs/en/extending-doris/udf/user-defined-function.md
b/docs/en/extending-doris/udf/native-user-defined-function.md
similarity index 99%
rename from docs/en/extending-doris/udf/user-defined-function.md
rename to docs/en/extending-doris/udf/native-user-defined-function.md
index f1ebcce..32311bf 100644
--- a/docs/en/extending-doris/udf/user-defined-function.md
+++ b/docs/en/extending-doris/udf/native-user-defined-function.md
@@ -1,6 +1,6 @@
---
{
-"title": "User Defined Function",
+"title": "Native User Defined Function",
"language": "en"
}
---
@@ -24,7 +24,7 @@ specific language governing permissions and limitations
under the License.
-->
-# User Define Function
+# Native User Defined Function
UDF is mainly suitable for scenarios where the analytical capabilities that
users need do not possess. Users can implement custom functions according to
their own needs, and register with Doris through the UDF framework to expand
Doris' capabilities and solve user analysis needs.
There are two types of analysis requirements that UDF can meet: UDF and UDAF.
UDF in this article refers to both.
diff --git a/docs/en/extending-doris/udf/remote-user-defined-function.md
b/docs/en/extending-doris/udf/remote-user-defined-function.md
new file mode 100644
index 000..2a59f58
--- /dev/null
+++ b/docs/en/extending-doris/udf/remote-user-defined-function.md
@@ -0,0 +1,106 @@
+---
+{
+"title": "Remote User Defined Function Service",
+"language": "en"
+}
+---
+
+
+
+# User Defined Function Rpc
+
+Remote UDF Service The Remote UDF Service can be accessed through RPC to
implement the execution of user-defined functions. Compared with Native UDF
implementations, Remote UDF Service has the following advantages and
limitations:
+1. The advantage
+ * Cross-language: UDF services can be written in all languages supported by
Protobuf.
+ * Security: UDF execution failure or crash only affects the UDF Service and
does not cause the Doris process to crash.
+ * Flexibility: Any other Service or library class can be invoked within a
UDF Service to meet a wider variety of business requirements.
+
+2. Restrictions on use
+ * Performance: Compared to Native UDFs, UDF services incur extra network
overhead and thus have much lower performance than Native UDFs. At the same
time, the implementation of the UDF Service also affects the execution
efficiency of the function. Users need to deal with problems such as high
concurrency and thread safety by themselves.
+ * Single line mode and batch mode: Doris's original query execution
framework based on row memory would execute one UDF RPC call for each row of
data, so the exe
[GitHub] [incubator-doris] HappenLee commented on a change in pull request #8138: [Vectorized][Feature] support some bitmap functions
HappenLee commented on a change in pull request #8138:
URL: https://github.com/apache/incubator-doris/pull/8138#discussion_r810653688
##
File path: be/src/vec/functions/function_bitmap_variadic.cpp
##
@@ -0,0 +1,229 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#include "vec/columns/column_complex.h"
+#include "vec/data_types/data_type.h"
+#include "vec/data_types/data_type_bitmap.h"
+#include "vec/functions/function_const.h"
+#include "vec/functions/function_string.h"
+#include "vec/functions/function_totype.h"
+#include "vec/functions/simple_function_factory.h"
+
+namespace doris::vectorized {
+
+struct BitmapAnd {
+static constexpr auto name = "bitmap_and";
+using ResultDataType = DataTypeBitMap;
+
+static Status vector_vector(ColumnPtr argument_columns[], size_t col_size,
+size_t input_rows_count,
std::vector& res) {
+
+res = assert_cast(argument_columns[0].get())->get_data();
+
+for (size_t col = 1; col < col_size; ++col) {
+auto& col_data =
+assert_cast(argument_columns[col].get())->get_data();
+for (size_t row = 0; row < input_rows_count; ++row) {
+res[row] &= col_data[row];
+}
+}
+return Status::OK();
+}
+};
+
+struct BitmapOr {
+static constexpr auto name = "bitmap_or";
+using ResultDataType = DataTypeBitMap;
+
+static Status vector_vector(ColumnPtr argument_columns[], size_t col_size,
+size_t input_rows_count,
std::vector& res) {
+
+res = assert_cast(argument_columns[0].get())->get_data();
+for (size_t col = 1; col < col_size; ++col) {
+auto& col_data =
+assert_cast(argument_columns[col].get())->get_data();
+for (size_t row = 0; row < input_rows_count; ++row) {
+res[row] |= col_data[row];
+}
+}
+return Status::OK();
+}
+};
+
+struct BitmapXor {
+using ResultDataType = DataTypeBitMap;
+static constexpr auto name = "bitmap_xor";
+
+static Status vector_vector(ColumnPtr argument_columns[], size_t col_size,
+size_t input_rows_count,
std::vector& res) {
+
+res = assert_cast(argument_columns[0].get())->get_data();
+for (size_t col = 1; col < col_size; ++col) {
+auto& col_data =
+assert_cast(argument_columns[col].get())->get_data();
+for (size_t row = 0; row < input_rows_count; ++row) {
+res[row] ^= col_data[row];
+}
+}
+return Status::OK();
+}
+};
+
+struct BitmapOrCount {
+static constexpr auto name = "bitmap_or_count";
+using ResultDataType = DataTypeInt64;
+using TData = std::vector;
+using ResTData = typename ColumnVector::Container;
+
+static Status vector_vector(ColumnPtr argument_columns[], size_t col_size,
+size_t input_rows_count, ResTData& res) {
+
+TData vals(input_rows_count);
+for (size_t col = 0; col < col_size; ++col) {
+auto& col_data =
+assert_cast(argument_columns[col].get())->get_data();
+for (size_t row = 0; row < input_rows_count; ++row) {
+vals[row] |= col_data[row];
+}
+}
+
+for (size_t row = 0; row < input_rows_count; ++row) {
+res[row] = vals[row].cardinality();
+}
+return Status::OK();
+}
+};
+
+struct BitmapAndCount {
+static constexpr auto name = "bitmap_and_count";
+using ResultDataType = DataTypeInt64;
+using TData = std::vector;
+using ResTData = typename ColumnVector::Container;
+
+static Status vector_vector(ColumnPtr argument_columns[], size_t col_size,
+size_t input_rows_count, ResTData& res) {
+
+TData vals(input_rows_count);
+vals = assert_cast(argument_columns[0].get())->get_data();
+
+for (size_t col = 1; col < col_size; ++col) {
+auto& col_data =
+assert_cast(argument_columns[col].get())->get_data();
+for (size_t row = 0; row <
[GitHub] [incubator-doris] HappenLee commented on a change in pull request #8119: [Vectorized][HashJoin] Opt HashJoin Performance
HappenLee commented on a change in pull request #8119:
URL: https://github.com/apache/incubator-doris/pull/8119#discussion_r810656345
##
File path: be/src/vec/exec/vset_operation_node.h
##
@@ -81,6 +81,7 @@ class VSetOperationNode : public ExecNode {
//record insert column id during probe
std::vector _probe_column_inserted_id;
+Block _build_block;
Review comment:
do the same opt in vset_operation_node, only one block cause mem problem?
##
File path: be/src/vec/exec/join/vhash_join_node.h
##
@@ -203,6 +203,7 @@ class HashJoinNode : public ::doris::ExecNode {
HashTableVariants _hash_table_variants;
AcquireList _acquire_list;
+std::vector _build_block;
Review comment:
_build_blocks
##
File path: be/src/vec/exec/join/vhash_join_node.cpp
##
@@ -184,19 +179,21 @@ struct ProcessHashTableProbe {
KeyGetter key_getter(_probe_raw_ptrs, _join_node->_probe_key_sz,
nullptr);
-std::vector items_counts(_probe_rows);
+uint32_t items_counts[_probe_rows];
auto& mcol = mutable_block.mutable_columns();
int current_offset = 0;
+//build_block_index size may be largger than _batch_size, 100 is
safety.
+int build_block_row_num[_batch_size + 100];
Review comment:
it's dangerous to have a pre safety set for noknown data, may should use
vector.
##
File path: be/src/vec/exec/join/join_op.h
##
@@ -26,15 +26,16 @@ namespace doris::vectorized {
struct RowRef {
using SizeT = uint32_t; /// Do not use size_t cause of memory economy
-const Block* block = nullptr;
+// const Block* block = nullptr;
Review comment:
directly delete the code
##
File path: be/src/vec/exec/join/vhash_join_node.cpp
##
@@ -184,19 +179,21 @@ struct ProcessHashTableProbe {
KeyGetter key_getter(_probe_raw_ptrs, _join_node->_probe_key_sz,
nullptr);
-std::vector items_counts(_probe_rows);
+uint32_t items_counts[_probe_rows];
Review comment:
use vector to init the uint32_t to 0.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] westgatepark closed issue #8149: [Bug] 当开启向量化时函数或语法报错
westgatepark closed issue #8149: URL: https://github.com/apache/incubator-doris/issues/8149 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] westgatepark opened a new issue #8155: [Bug] s3 BROKER LOAD 不识别s3前缀
westgatepark opened a new issue #8155:
URL: https://github.com/apache/incubator-doris/issues/8155
### Search before asking
- [X] I had searched in the
[issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and
found no similar issues.
### Version
0.15.1-rc09
### What's Wrong?
使用broker load 装载s3数据时，报错：ERROR 1064 (HY000): errCode = 2, detailMessage =
Invalid export path. please use valid 'HDFS://', 'AFS://' or 'BOS://' path.
模版参考的show broker load 中的：
LOAD LABEL example_db.label11
(
DATA INFILE("s3a://my_bucket/input/file")
INTO TABLE `my_table`
FORMAT AS "parquet"
(k1, k2, k3)
)
WITH BROKER my_s3a_broker
(
"fs.s3a.access.key" = "xx",
"fs.s3a.secret.key" = "",
"fs.s3a.endpoint" = "s3.amazonaws.com"
);
可以正常通过s3sdk装载数据:
LOAD LABEL load_label
(
...
)
WITH S3
(
..
);
但是这样就没法指定节点去执行装载任务，因为be做了资源隔离,而s3 sdk装载任务不受隔离限制。
### What You Expected?
期望当开启资源隔离时，根据 用户来分配执行 s3 sdk load 所占用的资源。
或者使s3 broker load 正常运行，这样可以根据broker name 来映射对应的资源组。
### How to Reproduce?
_No response_
### Anything Else?
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of
Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] yiguolei commented on issue #8155: [Bug] s3 BROKER LOAD 不识别s3前缀
yiguolei commented on issue #8155: URL: https://github.com/apache/incubator-doris/issues/8155#issuecomment-1046385157 ”BE 做了资源隔离“ 具体是怎么做的啊？ 你是隔离了broker吗？ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] westgatepark commented on issue #8155: [Bug] s3 BROKER LOAD 不识别s3前缀
westgatepark commented on issue #8155:
URL:
https://github.com/apache/incubator-doris/issues/8155#issuecomment-1046388686
be:
alter system modify backend "***:9050" set ("tag.location" = "group_batch");
...
...
alter system modify backend "***:9050" set ("tag.location" = "group_stream");
...
...
user:
set property for '***' 'resource_tags.location' = 'group_batch';
set property for '***' 'resource_tags.location' = 'group_stream';
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] yangzhg merged pull request #8153: Some doc improvements and typo fix
yangzhg merged pull request #8153: URL: https://github.com/apache/incubator-doris/pull/8153 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[incubator-doris] branch master updated (b27122d -> 826738d)
This is an automated email from the ASF dual-hosted git repository. yangzhg pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-doris.git. from b27122d [Docs] add rpc function document (#7984) add 826738d [docs]Some doc improvements and typo fix (#8153) No new revisions were added by this update. Summary of changes: be/src/exprs/agg_fn_evaluator.h | 2 +- be/src/runtime/row_batch.cpp| 2 +- docs/en/installing/compilation-with-ldb-toolchain.md| 6 +++--- docs/en/installing/compilation.md | 2 ++ docs/zh-CN/installing/compilation-with-ldb-toolchain.md | 6 +++--- docs/zh-CN/installing/compilation.md| 2 +- 6 files changed, 11 insertions(+), 9 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] yangzhg merged pull request #8126: Update data-model-rollup.md
yangzhg merged pull request #8126: URL: https://github.com/apache/incubator-doris/pull/8126 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[incubator-doris] branch master updated (826738d -> 16bf60e)
This is an automated email from the ASF dual-hosted git repository. yangzhg pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-doris.git. from 826738d [docs]Some doc improvements and typo fix (#8153) add 16bf60e [docs] Update data-model-rollup.md (#8126) No new revisions were added by this update. Summary of changes: docs/en/getting-started/data-model-rollup.md| 4 ++-- docs/zh-CN/getting-started/data-model-rollup.md | 2 +- 2 files changed, 3 insertions(+), 3 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] yangzhg merged pull request #8144: [refactor] add some log when close parquet file
yangzhg merged pull request #8144: URL: https://github.com/apache/incubator-doris/pull/8144 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[incubator-doris] branch master updated (16bf60e -> 409aefd)
This is an automated email from the ASF dual-hosted git repository. yangzhg pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-doris.git. from 16bf60e [docs] Update data-model-rollup.md (#8126) add 409aefd [refactor] add some log when close parquet file (#8144) No new revisions were added by this update. Summary of changes: be/src/exec/csv_scan_node.cpp | 1 - be/src/exec/parquet_reader.cpp | 4 ++-- be/src/exec/parquet_writer.cpp | 4 ++-- be/src/olap/rowset/segment_v2/segment_iterator.cpp | 4 ++-- be/src/runtime/exec_env_init.cpp | 17 + 5 files changed, 15 insertions(+), 15 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] yangzhg merged pull request #8071: Correct the wrong words in the prompt information
yangzhg merged pull request #8071: URL: https://github.com/apache/incubator-doris/pull/8071 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[incubator-doris] branch doris-manager updated: [refactor] Correct the wrong words in the prompt information (#8071)
This is an automated email from the ASF dual-hosted git repository.
yangzhg pushed a commit to branch doris-manager
in repository https://gitbox.apache.org/repos/asf/incubator-doris.git
The following commit(s) were added to refs/heads/doris-manager by this push:
new 0d61d43 [refactor] Correct the wrong words in the prompt information
(#8071)
0d61d43 is described below
commit 0d61d4305975debd76eaa014fcbc249d10c6bb68
Author: liteng2430 <91309604+liteng2...@users.noreply.github.com>
AuthorDate: Mon Feb 21 09:37:44 2022 +0800
[refactor] Correct the wrong words in the prompt information (#8071)
---
.../main/java/org/apache/doris/manager/agent/api/CommandController.java | 2 +-
.../main/java/org/apache/doris/manager/agent/api/CommonControllder.java | 2 +-
.../java/org/apache/doris/manager/agent/command/CommandFactory.java | 2 +-
.../src/main/java/org/apache/doris/stack/agent/AgentHeatbeatRunner.java | 2 +-
.../java/org/apache/doris/stack/component/TaskInstanceComponent.java| 2 +-
5 files changed, 5 insertions(+), 5 deletions(-)
diff --git
a/manager/dm-agent/src/main/java/org/apache/doris/manager/agent/api/CommandController.java
b/manager/dm-agent/src/main/java/org/apache/doris/manager/agent/api/CommandController.java
index 840138b..c314539 100644
---
a/manager/dm-agent/src/main/java/org/apache/doris/manager/agent/api/CommandController.java
+++
b/manager/dm-agent/src/main/java/org/apache/doris/manager/agent/api/CommandController.java
@@ -40,7 +40,7 @@ public class CommandController {
@PostMapping("/execute")
public RResult execute(@RequestBody CommandRequest commandRequest) {
if
(Objects.isNull(CommandType.findByName(commandRequest.getCommandType( {
-return RResult.error("Unkonwn command");
+return RResult.error("unknown command");
}
Task task = CommandFactory.get(commandRequest).setup().execute();
CommandResult commandResult = new CommandResult(task.getTaskResult());
diff --git
a/manager/dm-agent/src/main/java/org/apache/doris/manager/agent/api/CommonControllder.java
b/manager/dm-agent/src/main/java/org/apache/doris/manager/agent/api/CommonControllder.java
index e22797e..b2c8351 100644
---
a/manager/dm-agent/src/main/java/org/apache/doris/manager/agent/api/CommonControllder.java
+++
b/manager/dm-agent/src/main/java/org/apache/doris/manager/agent/api/CommonControllder.java
@@ -79,7 +79,7 @@ public class CommonControllder {
BrokerService brokerService = new
BrokerService(register.getInstallDir());
ServiceContext.register(brokerService);
} else {
-return RResult.error("unkown service role");
+return RResult.error("unknown service role");
}
return RResult.success();
diff --git
a/manager/dm-agent/src/main/java/org/apache/doris/manager/agent/command/CommandFactory.java
b/manager/dm-agent/src/main/java/org/apache/doris/manager/agent/command/CommandFactory.java
index 5423a1f..cc7d2b4 100644
---
a/manager/dm-agent/src/main/java/org/apache/doris/manager/agent/command/CommandFactory.java
+++
b/manager/dm-agent/src/main/java/org/apache/doris/manager/agent/command/CommandFactory.java
@@ -101,7 +101,7 @@ public class CommandFactory {
}
return new
WriteBrokerConfCommand(writeBrokerConfCommandRequestBody);
default:
-throw new AgentException("unkown CommandType");
+throw new AgentException("unknown CommandType");
}
}
diff --git
a/manager/dm-server/src/main/java/org/apache/doris/stack/agent/AgentHeatbeatRunner.java
b/manager/dm-server/src/main/java/org/apache/doris/stack/agent/AgentHeatbeatRunner.java
index f934a33..ed3224a 100644
---
a/manager/dm-server/src/main/java/org/apache/doris/stack/agent/AgentHeatbeatRunner.java
+++
b/manager/dm-server/src/main/java/org/apache/doris/stack/agent/AgentHeatbeatRunner.java
@@ -76,7 +76,7 @@ public class AgentHeatbeatRunner implements ApplicationRunner
{
agent.setStatus(AgentStatus.STOP);
agentRepository.save(agent);
agentCache.putAgent(agent);
-log.warn("agent {} is unhealthly", agent.getHost());
+log.warn("agent {} is unhealthy", agent.getHost());
}
}
}
diff --git
a/manager/dm-server/src/main/java/org/apache/doris/stack/component/TaskInstanceComponent.java
b/manager/dm-server/src/main/java/org/apache/doris/stack/component/TaskInstanceComponent.java
index d976613..b9c8f8b 100644
---
a/manager/dm-server/src/main/java/org/apache/doris/stack/component/TaskInstanceComponent.java
+++
b/manager/dm-server/src/main/java/org/apache/doris/stack/component/TaskInstanceComponent.java
@@ -119,7 +119,7 @@ public class TaskInstanceComponent {
}
for (TaskInstanceEntity task : taskInstanceEntities) {
if (Flag.NO.equals(task.getFinish())) {
-log.error("ta
[GitHub] [incubator-doris] yangzhg closed issue #8151: [Bug] explode_json_array_string return unexpected return
yangzhg closed issue #8151: URL: https://github.com/apache/incubator-doris/issues/8151 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] yangzhg merged pull request #8152: [fix](lateral-view) Fix bug that explode_json_array_string return unstable result
yangzhg merged pull request #8152: URL: https://github.com/apache/incubator-doris/pull/8152 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[incubator-doris] branch master updated: [fix](lateral-view) Fix bug that explode_json_array_string return unstable result (#8152)
This is an automated email from the ASF dual-hosted git repository.
yangzhg pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/incubator-doris.git
The following commit(s) were added to refs/heads/master by this push:
new 16020cb [fix](lateral-view) Fix bug that explode_json_array_string
return unstable result (#8152)
16020cb is described below
commit 16020cbdf9d1ec685514380b63eb59a8897477da
Author: Mingyu Chen
AuthorDate: Mon Feb 21 09:38:36 2022 +0800
[fix](lateral-view) Fix bug that explode_json_array_string return unstable
result (#8152)
Co-authored-by: morningman
---
be/src/exprs/table_function/explode_json_array.cpp | 17 +++--
be/src/exprs/table_function/explode_json_array.h | 1 -
thirdparty/download-thirdparty.sh | 8 +---
3 files changed, 16 insertions(+), 10 deletions(-)
diff --git a/be/src/exprs/table_function/explode_json_array.cpp
b/be/src/exprs/table_function/explode_json_array.cpp
index e82c9ec..43fb1f3 100644
--- a/be/src/exprs/table_function/explode_json_array.cpp
+++ b/be/src/exprs/table_function/explode_json_array.cpp
@@ -68,8 +68,10 @@ int ParsedData::set_output(ExplodeJsonArrayType type,
rapidjson::Document& docum
switch (v.GetType()) {
case rapidjson::Type::kStringType:
_backup_string.emplace_back(v.GetString(),
v.GetStringLength());
-_data_string.emplace_back(_backup_string.back());
_string_nulls.push_back(false);
+// do not set _data_string here.
+// Because the address of the string stored in
`_backup_string` may
+// change each time `emplace_back()` is called.
break;
case rapidjson::Type::kNumberType:
if (v.IsUint()) {
@@ -84,8 +86,10 @@ int ParsedData::set_output(ExplodeJsonArrayType type,
rapidjson::Document& docum
wbytes = sprintf(tmp_buf, "%f", v.GetDouble());
}
_backup_string.emplace_back(tmp_buf, wbytes);
-_data_string.emplace_back(_backup_string.back());
_string_nulls.push_back(false);
+// do not set _data_string here.
+// Because the address of the string stored in
`_backup_string` may
+// change each time `emplace_back()` is called.
break;
case rapidjson::Type::kFalseType:
_data_string.emplace_back(true_value);
@@ -106,6 +110,11 @@ int ParsedData::set_output(ExplodeJsonArrayType type,
rapidjson::Document& docum
}
++i;
}
+// Must set _data_string at the end, so that we can
+// save the real addr of string in `_backup_string` to
`_data_string`.
+for (auto& str : _backup_string) {
+_data_string.emplace_back(str);
+}
break;
}
default:
@@ -139,21 +148,17 @@ Status ExplodeJsonArrayTableFunction::process(TupleRow*
tuple_row) {
StringVal text =
_expr_context->root()->get_child(0)->get_string_val(_expr_context, tuple_row);
if (text.is_null || text.len == 0) {
-// _set_null_output();
_is_current_empty = true;
} else {
rapidjson::Document document;
document.Parse((char*) text.ptr, text.len);
if (UNLIKELY(document.HasParseError()) || !document.IsArray() ||
document.GetArray().Size() == 0) {
-// _set_null_output();
_is_current_empty = true;
} else {
_cur_size = _parsed_data.set_output(_type, document);
_cur_offset = 0;
-// _eos = _cur_size == 0;
}
}
-// _is_current_empty = _eos;
return Status::OK();
}
diff --git a/be/src/exprs/table_function/explode_json_array.h
b/be/src/exprs/table_function/explode_json_array.h
index ae5c657..8ecbd2d 100644
--- a/be/src/exprs/table_function/explode_json_array.h
+++ b/be/src/exprs/table_function/explode_json_array.h
@@ -91,7 +91,6 @@ struct ParsedData {
*output = _data[offset];
break;
case ExplodeJsonArrayType::STRING:
-// LOG(INFO) << "cmy get_value offset: " << offset << ", is
null: " << _string_nulls[offset] << ", data: " << (_string_nulls[offset] ?
"null2" : _backup_string[offset]);
*output = _string_nulls[offset] ? nullptr :
&_data_string[offset];
break;
default:
diff --git a/thirdparty/download-thirdparty.sh
b/thirdparty/download-thirdparty.sh
index c6650bd..768e92d 100755
--- a/thirdparty/download-thirdparty.sh
+++ b/thirdparty/download-thirdparty.sh
@@ -72,6 +72,8 @@ md5sum_func() {
return 0
}
+# return
[GitHub] [incubator-doris] westgatepark commented on issue #8155: [Bug] s3 BROKER LOAD 不识别s3前缀
westgatepark commented on issue #8155: URL: https://github.com/apache/incubator-doris/issues/8155#issuecomment-1046389796 我把一些be节点卸载了broker组件,这些节点依然参与了s3load，所以目前没法对装载任务占用资源隔离. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #8154: [Doc]Correct parameter name
github-actions[bot] commented on pull request #8154: URL: https://github.com/apache/incubator-doris/pull/8154#issuecomment-1046390033 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[incubator-doris] branch master updated (16020cb -> c3348b8)
This is an automated email from the ASF dual-hosted git repository. yangzhg pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/incubator-doris.git. from 16020cb [fix](lateral-view) Fix bug that explode_json_array_string return unstable result (#8152) add c3348b8 [docs] fix config enable_force_drop_redundant_replica name not correct in docs (#8154) No new revisions were added by this update. Summary of changes: docs/en/administrator-guide/config/fe_config.md| 2 +- docs/zh-CN/administrator-guide/config/fe_config.md | 4 +++- 2 files changed, 4 insertions(+), 2 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] yangzhg merged pull request #8154: [Doc]Correct parameter name
yangzhg merged pull request #8154: URL: https://github.com/apache/incubator-doris/pull/8154 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #8110: (refactor) (runtime) tidy up the plan_fragment_executor codes
github-actions[bot] commented on pull request #8110: URL: https://github.com/apache/incubator-doris/pull/8110#issuecomment-1046391246 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] zhangstar333 commented on a change in pull request #8138: [Vectorized][Feature] support some bitmap functions
zhangstar333 commented on a change in pull request #8138:
URL: https://github.com/apache/incubator-doris/pull/8138#discussion_r810741885
##
File path: be/src/vec/functions/function_bitmap_variadic.cpp
##
@@ -0,0 +1,229 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#include "vec/columns/column_complex.h"
+#include "vec/data_types/data_type.h"
+#include "vec/data_types/data_type_bitmap.h"
+#include "vec/functions/function_const.h"
+#include "vec/functions/function_string.h"
+#include "vec/functions/function_totype.h"
+#include "vec/functions/simple_function_factory.h"
+
+namespace doris::vectorized {
+
+struct BitmapAnd {
+static constexpr auto name = "bitmap_and";
+using ResultDataType = DataTypeBitMap;
+
+static Status vector_vector(ColumnPtr argument_columns[], size_t col_size,
+size_t input_rows_count,
std::vector& res) {
+
+res = assert_cast(argument_columns[0].get())->get_data();
+
+for (size_t col = 1; col < col_size; ++col) {
+auto& col_data =
+assert_cast(argument_columns[col].get())->get_data();
+for (size_t row = 0; row < input_rows_count; ++row) {
+res[row] &= col_data[row];
+}
+}
+return Status::OK();
+}
+};
+
+struct BitmapOr {
+static constexpr auto name = "bitmap_or";
+using ResultDataType = DataTypeBitMap;
+
+static Status vector_vector(ColumnPtr argument_columns[], size_t col_size,
+size_t input_rows_count,
std::vector& res) {
+
+res = assert_cast(argument_columns[0].get())->get_data();
+for (size_t col = 1; col < col_size; ++col) {
+auto& col_data =
+assert_cast(argument_columns[col].get())->get_data();
+for (size_t row = 0; row < input_rows_count; ++row) {
+res[row] |= col_data[row];
+}
+}
+return Status::OK();
+}
+};
+
+struct BitmapXor {
+using ResultDataType = DataTypeBitMap;
+static constexpr auto name = "bitmap_xor";
+
+static Status vector_vector(ColumnPtr argument_columns[], size_t col_size,
+size_t input_rows_count,
std::vector& res) {
+
+res = assert_cast(argument_columns[0].get())->get_data();
+for (size_t col = 1; col < col_size; ++col) {
+auto& col_data =
+assert_cast(argument_columns[col].get())->get_data();
+for (size_t row = 0; row < input_rows_count; ++row) {
+res[row] ^= col_data[row];
+}
+}
+return Status::OK();
+}
+};
+
+struct BitmapOrCount {
+static constexpr auto name = "bitmap_or_count";
+using ResultDataType = DataTypeInt64;
+using TData = std::vector;
+using ResTData = typename ColumnVector::Container;
+
+static Status vector_vector(ColumnPtr argument_columns[], size_t col_size,
+size_t input_rows_count, ResTData& res) {
+
+TData vals(input_rows_count);
+for (size_t col = 0; col < col_size; ++col) {
+auto& col_data =
+assert_cast(argument_columns[col].get())->get_data();
+for (size_t row = 0; row < input_rows_count; ++row) {
+vals[row] |= col_data[row];
+}
+}
+
+for (size_t row = 0; row < input_rows_count; ++row) {
+res[row] = vals[row].cardinality();
+}
+return Status::OK();
+}
+};
+
+struct BitmapAndCount {
+static constexpr auto name = "bitmap_and_count";
+using ResultDataType = DataTypeInt64;
+using TData = std::vector;
+using ResTData = typename ColumnVector::Container;
+
+static Status vector_vector(ColumnPtr argument_columns[], size_t col_size,
+size_t input_rows_count, ResTData& res) {
+
+TData vals(input_rows_count);
+vals = assert_cast(argument_columns[0].get())->get_data();
+
+for (size_t col = 1; col < col_size; ++col) {
+auto& col_data =
+assert_cast(argument_columns[col].get())->get_data();
+for (size_t row = 0; row
[GitHub] [incubator-doris] caiconghui commented on issue #8155: [Bug] s3 BROKER LOAD 不识别s3前缀
caiconghui commented on issue #8155: URL: https://github.com/apache/incubator-doris/issues/8155#issuecomment-1046417835 just simply separated by broker name? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] westgatepark commented on issue #8155: [Bug] s3 BROKER LOAD 不识别s3前缀
westgatepark commented on issue #8155: URL: https://github.com/apache/incubator-doris/issues/8155#issuecomment-1046421242 您是说指定broker name吗： LOAD LABEL ... WITH BROKER ... 这样会报错识别不了s3前缀，所以我只能s3 sdk load： ... WITH S3 ... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #8117: (refactor)(common) optimize Status implemention: no dynamic new
github-actions[bot] commented on pull request #8117: URL: https://github.com/apache/incubator-doris/pull/8117#issuecomment-1046439380 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] weizuo93 opened a new pull request #8156: [Transactions][Stmt] Support view transactions info for specified status by `SHOW TRANSACTION` stmt
weizuo93 opened a new pull request #8156: URL: https://github.com/apache/incubator-doris/pull/8156 # Proposed changes Support view all transactions info for specified status by `SHOW TRANSACTION` stmt. ``` SHOW TRANSACTION WHERE STATUS = 'prepare/precommitted/committed/visible/aborted'; ``` ## Checklist(Required) 1. Does it affect the original behavior: No 2. Has unit tests been added: No Need 3. Has document been added or modified: Yes 4. Does it need to update dependencies: No 5. Are there any changes that cannot be rolled back: No -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] BiteTheDDDDt opened a new issue #8157: [Feature][Vectorized] support function `json_array()`/`json_object`/`json_quote`
BiteThet opened a new issue #8157:
URL: https://github.com/apache/incubator-doris/issues/8157
### Search before asking
- [X] I had searched in the
[issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and
found no similar issues.
### Description
support function `json_array()`/`json_object`/`json_quote`
### Use case
```sql
MySQL [test]> select json_array(1,2);
+--+
| json_array(1, 2) |
+--+
| [1,2]|
+--+
MySQL [test]> select json_object('abc',2);
+---+
| json_object('abc', 2) |
+---+
| {"abc":2} |
+---+
MySQL [test]> select json_quote(1);
+---+
| json_quote(1) |
+---+
| "1" |
+---+
```
### Related issues
_No response_
### Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of
Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] BiteTheDDDDt opened a new pull request #8158: [Feature][Vectorized] support function `json_array()`/`json_object()`/`json_quote()`
BiteThet opened a new pull request #8158: URL: https://github.com/apache/incubator-doris/pull/8158 @HappenLee # Proposed changes Issue Number: close #8157 ## Problem Summary: Describe the overview of changes. ## Checklist(Required) 1. Does it affect the original behavior: (Yes/No/I Don't know) 2. Has unit tests been added: (Yes/No/No Need) 3. Has document been added or modified: (Yes/No/No Need) 4. Does it need to update dependencies: (Yes/No) 5. Are there any changes that cannot be rolled back: (Yes/No) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] HappenLee commented on a change in pull request #8138: [Vectorized][Feature] support some bitmap functions
HappenLee commented on a change in pull request #8138:
URL: https://github.com/apache/incubator-doris/pull/8138#discussion_r810787724
##
File path: be/src/vec/functions/function_bitmap_variadic.cpp
##
@@ -0,0 +1,227 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#include "vec/columns/column_complex.h"
+#include "vec/data_types/data_type.h"
+#include "vec/data_types/data_type_bitmap.h"
+#include "vec/functions/function_const.h"
+#include "vec/functions/function_string.h"
+#include "vec/functions/function_totype.h"
+#include "vec/functions/simple_function_factory.h"
+
+namespace doris::vectorized {
+
+struct BitmapAnd {
Review comment:
use template or macro to simple the code
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] xingtanzjr opened a new pull request #8159: Fix some typos in both comment and variable naming
xingtanzjr opened a new pull request #8159: URL: https://github.com/apache/incubator-doris/pull/8159 ## Problem Summary: Fix some typos in both comment and variable naming. ## Checklist(Required) 1. Does it affect the original behavior: (No) 2. Has unit tests been added: (No Need) 3. Has document been added or modified: (No Need) 4. Does it need to update dependencies: (No) 5. Are there any changes that cannot be rolled back: (Yes) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] github-actions[bot] commented on pull request #8158: [Feature][Vectorized] support function `json_array()`/`json_object()`/`json_quote()`
github-actions[bot] commented on pull request #8158: URL: https://github.com/apache/incubator-doris/pull/8158#issuecomment-1046501365 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] boboyunz opened a new issue #8160: [Bug] Failed to get scan range, no queryable replica found in tablet: 44605
boboyunz opened a new issue #8160:
URL: https://github.com/apache/incubator-doris/issues/8160
### Search before asking
- [X] I had searched in the
[issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and
found no similar issues.
### Version
0.14.00
### What's Wrong?
[Err] 1064 - errCode = 2, detailMessage = Failed to get scan range, no
queryable replica found in tablet: 44605
查询数据表报错
**show tablet 44605**
default_cluster:QHYGJcargis P202202 cargis 13726 39843 39735
39844 trueSHOW PROC '/dbs/13726/39843/partitions/39735/39844/44605';
**SHOW PROC '/dbs/13726/39843/partitions/39735/39844/44605';**
4460610002 130361 -1 130569 3970544862712453573 130362
-1 1083286175 191451111882920 NORMAL false 206
-6750335797368435219
http://63.1.21.103:8140/api/meta/header/44605/1083286175
http://63.1.21.103:8140/api/compaction/show?tablet_id=44605&schema_hash=1083286175
**http://63.1.21.103:8140/api/meta/header/44605/1083286175**
{
"table_id": 39843,
"partition_id": 39735,
"tablet_id": 44605,
"schema_hash": 1083286175,
"shard_id": 760,
"creation_time": 1626325743,
"cumulative_layer_point": -1,
"tablet_state": "PB_RUNNING",
"schema": {
"keys_type": "UNIQUE_KEYS",
"column": [
{
"unique_id": 0,
"name": "cph",
"type": "VARCHAR",
"is_key": true,
"aggregation": "NONE",
"is_nullable": false,
"length": 22,
"index_length": 20,
"visible": true
},
{
"unique_id": 1,
"name": "cpys",
"type": "VARCHAR",
"is_key": true,
"aggregation": "NONE",
"is_nullable": false,
"length": 12,
"index_length": 20,
"visible": true
},
..
**http://63.1.21.103:8140/api/compaction/show?tablet_id=44605&schema_hash=1083286175**
{
"cumulative policy type": "SIZE_BASED",
"cumulative point": -1,
"last cumulative failure time": "1970-01-01 08:00:00.000",
"last base failure time": "1970-01-01 08:00:00.000",
"last cumulative success time": "1970-01-01 08:00:00.000",
"last base success time": "1970-01-01 08:00:00.000",
"rowsets": [
"[0-1] 0 DATA NONOVERLAPPING
02002ca85f425ab768bad300dd04bb39a57ab9ad 0",
"[2-130361] 1 DATA NONOVERLAPPING
0209994253df273cff02ba9995b1071360a6 16.09 MB",
"[130366-130366] 1 DATA NONOVERLAPPING
020d994253df273cff02ba9995b1071360a6 10.74 KB",
### What You Expected?
希望协助解决问题
### How to Reproduce?
_No response_
### Anything Else?
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of
Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] DarvenDuan opened a new pull request #8161: [typo] (docs) update backup and restore docs
DarvenDuan opened a new pull request #8161: URL: https://github.com/apache/incubator-doris/pull/8161 # Proposed changes ## Problem Summary: The 'backup' and 'restore' commands already support the 'EXCLUDE' clause, but viewing the help documentation with 'help backup' and 'help restore' commands has no information about it. ## Checklist(Required) 1. Does it affect the original behavior: (No) 2. Has unit tests been added: (No) 3. Has document been added or modified: (Yes) 4. Does it need to update dependencies: (No) 5. Are there any changes that cannot be rolled back: (No) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] zuochunwei opened a new pull request #8162: [improvement] (storage) Dict decoder improvement
zuochunwei opened a new pull request #8162: URL: https://github.com/apache/incubator-doris/pull/8162 # Proposed changes replace PR#7897, resolve confilit Issue Number: close #xxx ## Problem Summary: Describe the overview of changes. ## Checklist(Required) 1. Does it affect the original behavior: (Yes/No/I Don't know) 2. Has unit tests been added: (Yes/No/No Need) 3. Has document been added or modified: (Yes/No/No Need) 4. Does it need to update dependencies: (Yes/No) 5. Are there any changes that cannot be rolled back: (Yes/No) ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [incubator-doris] zuochunwei commented on pull request #7897: [improvement] (storage) Dict decoder optimization
zuochunwei commented on pull request #7897: URL: https://github.com/apache/incubator-doris/pull/7897#issuecomment-1046569364 too much conflicts, so i new a PR instead, this PR will be deleted later -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
