[GitHub] [doris] awakeljw opened a new issue, #13638: [Bug] Some Variables in be/CMakeList.txt is not add "${}"
awakeljw opened a new issue, #13638:
URL: https://github.com/apache/doris/issues/13638
### Search before asking
- [X] I had searched in the
[issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and
found no similar issues.
### Version
the master version
### What's Wrong?
In file be/CMakeLists.txt, there is a lot of variables which is not added
"${}", if the variables is empty. Maybe it wil report error.
### What You Expected?
fix the potential error.
https://stackoverflow.com/questions/39707772/cmake-error-if-given-arguments-followed-by-parenthesis-not-equals-and-si
### How to Reproduce?
_No response_
### Anything Else?
_No response_
### Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of
Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] awakeljw commented on pull request #13637: fix cmake error
awakeljw commented on PR #13637: URL: https://github.com/apache/doris/pull/13637#issuecomment-1290077258 https://github.com/apache/doris/issues/13638 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] zenyanggongmian1995 opened a new issue, #13639: [Bug] Select后的字段解析有问题。
zenyanggongmian1995 opened a new issue, #13639:
URL: https://github.com/apache/doris/issues/13639
### Search before asking
- [X] I had searched in the
[issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and
found no similar issues.
### Version
1.1.2.rc02
### What's Wrong?
#
错误就在第二行，可以看见我month_id之后没有逗号，但是该SQL可以正确执行，而且结果集中没有month字段，这个位置解析应该有问题。当我加上逗号之后，结果集中会出现month_id字段
SELECT
month_id
standard_app_id,
split_part(key_value1, '$', 1) AS avg_uv_name,
split_part(key_value1, '$', 2) AS avg_uv_value
from (
SELECT *
FROM (
SELECT
month_id,
standard_app_id,
concat_ws('$', 'avg_uv01', avg_uv01) as con_avg_uv01,
concat_ws('$', 'avg_uv02', avg_uv02) as con_avg_uv02,
concat_ws('$', 'avg_uv03', avg_uv03) as con_avg_uv03
FROM (
select
substring(day_id,1,6) month_id,
standard_app_id,
avg(CASEwhen(day_id-concat(substring(day_id,1,6),'00') ) =
1 THEN uv ELSEnullEND ) avg_uv01,
avg(CASEwhen(day_id-concat(substring(day_id,1,6),'00') ) <=
2 THEN uv ELSEnullEND ) avg_uv02,
avg(CASEwhen(day_id-concat(substring(day_id,1,6),'00') ) <=
3 THEN uv ELSEnullEND ) avg_uv03
from
(select * from ids.ids_app_rank_product_a4_d where day_id
>=DATE_SUB(CURRENT_DATE(),INTERVAL 132 DAY) and day_id
<=DATE_SUB(CURRENT_DATE(),INTERVAL 2 DAY) and uv!=0
union all
select * from ids.ids_app_game_rank_product_a4_d where day_id
>=DATE_SUB(CURRENT_DATE(),INTERVAL 132 DAY) and day_id
<=DATE_SUB(CURRENT_DATE(),INTERVAL 2 DAY) and uv!=0
) b
group by 1,2
) bb
) bbview lateral VIEW explode_split ( CONCAT_WS( ',', con_avg_uv01,
con_avg_uv02,con_avg_uv03), ',' ) tmp1 AS key_value1)xxx
where month_id=202207
### What You Expected?
解决涉及到laterview的查询时，Select字段解析的问题。
此前：#13431 ，使用lateral view之后，内层表别名解析有问题。
更前：我发现 lateral view之后的过滤条件无法生效，现在已经修复。
### How to Reproduce?
_No response_
### Anything Else?
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of
Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] Wilson-BT opened a new issue, #13640: [Bug] [Bug] Sql Exec Failed Raise IndexOutOfBoundsException
Wilson-BT opened a new issue, #13640: URL: https://github.com/apache/doris/issues/13640 ### Search before asking - [X] I had searched in the [issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and found no similar issues. ### Version 1.1.3 ### What's Wrong? when i exec sql like this，IndexOutOfBoundsException will be raised. we don`t know why. ``` with b as ( select store_no,order_unit_no,item_no,max(item_name) as item_naa from retail_gms_ods.inventory group by store_no,order_unit_no,item_no order by store_no,order_unit_no,item_no limit 100 ) SELECT a.store_no, a.order_unit_no, a.item_code, a.item_name, a.item_no, a.color_no, a.sku_no, a.size_kind, a.balance_qty FROM retail_gms_ods.inventory a INNER JOIN b ON a.store_no = b.store_no AND a.order_unit_no = b.order_unit_no AND a.item_no = b.item_no ``` but we can explain it success. ### What You Expected? Sql exec can be more robust. ### How to Reproduce? _No response_ ### Anything Else? _No response_ ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] Kikyou1997 commented on issue #13628: statistics persistence
Kikyou1997 commented on issue #13628: URL: https://github.com/apache/doris/issues/13628#issuecomment-1290089018 # 统计信息设计文档 - [统计信息设计文档](#统计信息设计文档) - [持久化](#持久化) - [表更新](#表更新) - [统计信息粒度](#统计信息粒度) - [缓存](#缓存) - [收集](#收集) - [SQL](#sql) - [自动收集](#自动收集) - [TODO](#todo) ## 持久化 目前主流的方案是将统计信息放在单独的表中，Postgres，MySQL，StarRocks等皆如是。我门也将遵循这个设计。 Doris兼容了MySQL5.1, 在`information_schema`库中保有`statistics`表，但其字段与我们的需求有较大的gap，且Doris不对该表进行维护，该表永远为空。另外Doris对该库做了很多特殊的处理，我们不希望这些逻辑对统计信息的代码构成干扰。 因此，将使用一个独立的库表用以保存统计数据，这样维护起来以较为方便也方便用户和开发者对已有的数据做进一步的聚合查询。 库名：`internal_statistics` 表类型：`OLAP` 表副本数： 1，`fe.conf`增加配置项`statistics_replica_num` 主要有如下几张表： 表名：`column_statistics` bucket col：`id` unique key: `id` 表字段及类型： |字段名|类型|Nullable|Comments| |--|--|---|--| |id|VARCHAR(4096)|false|由`catalog_id` `db_id` `tbl_id` `col_id` `part_id`拼接而成，分隔符`-`| |catalog_id|VARCHAR(1024)|| |db_id|VARCHAR(1024)|n| |tbl_id|VARCHAR(1024)|n| |col_id|VARCHAR(1024)|n|名为id实则name| |part_id|VARCHAR(1024)||为空表示该row为表级别数据| |row_count|BIGINT|| |ndv|BIGINT|| |null_count|BIGINT| |min|VARCHAR(ScalarType.MAX_VARCHAR_LENGTH)|| |max|VARCHAR(ScalarType.MAX_VARCHAR_LENGTH)|| |data_size|BIGINT|| |update_time|DATETIME||数据插入时间| 注意，在这里，对于列上的最大最小值我们用字符串记录其原始值，在从表中load时转换为double类型。 ### 表更新 此外，由于存在列名被修改的情况，我们需要在相关的alter语句执行时，对`column_statistics`中的记录执行相应的modify操作。但这里如果在列的alter语句执行时，这个操作一并阻塞执行的话，那么存在潜在的性能问题，如果是异步的操作，则存在一致性问题。如果更新不及时，还存在拿到错误的统计信息的可能，比如A列被重名为B， C列被重命名为A的情况。 结论： 这种情况提交一个作业异步的删除旧的统计信息，由于不太可能对表schema的高频且持续不断的修改，并且统计信息本身的收集本身就必然存在滞后性，一定时间窗口内的部分的不一致是可以接受的，而通过周期性的收集任务我们可以保证表中列的统计信息和实际的列最终一致。 ### 统计信息粒度 就前者而言，当SQL中存在下推到OlapScan的等值条件`slot=constant`，且左侧为distributed column时，Doris会根据hash值对tablet进行裁剪，统计每个tablet的信息有助于我们在这种条件下更准确的估计scan出的rowCount，但如果将列的统计信息的收集拆分到`TABLET`的级别的话，如何合理的估计distinct是个问题， 考虑到一些用户会设置bucket数量为一个非常大值，如果再乘以partition数量，那么每张表上将需要执行非常多的任务去收集统计信息。这样 1. 一方面会对其他逻辑构成较大影响 2. 另一方面也意味着`column_statistics`表的过分膨胀。 3. FE任务调度的压力也更大 4. 基于bucket粒度的统计信息 聚合估计出的ndv偏差更大 class：`StatisticsStorageManager` method： * saveStatistics：在StatisticJob的状态被更新为true时，调用 * removeExpiredStatistics：周期性任务。由于统计信息本身不会占用太多的资源，对一致性要求也不高，我们不必追求及时删除。这里有一个周期性的任务检查statistics表中记录的库表列和目前元数据中保有的之间的diff，然后起一个SQL把statistics中多余的记录删掉即可。 * createStatisticsTableIfNotExists：FE启动时检查 ## 缓存 我们引入一个缓存层，这样的做的目的是 1. 在统计信息收集和统计信息查询之间引入一个中间层，这样之后对前者进行重构时，不会对其他部分产生影响，同时屏蔽了收集过程中的各类状态，使得代码结构更为清晰易懂 2. 另外，目前除非用户显式的执行删除操作，否则已经收集到的统计信息会永远驻留在FE内存中，在拥有持久化能力之后，可以将长期不用的统计信息淘汰掉减少FE内存占用 各个操作基本设计如下： * 载入：按需从statistic表中查询，如果没有相应的统计数据，返回一个默认的值 * 更新：每次执行`ANALYZE`语句，或`ANALYZE JOB`周期性执行时 * 删除： 1. 过期（超过指定时间未被access, ，可通过`information_schema_stats_expiry`配置），或相应的db/table/column被删除时删除对应的cache。 2. cache条目达到上限,可通过参数`statistics_max_cache_size配置` * 刷新时间：一天，从statistics表重新load数据 * 淘汰策略：LRU * 多FE同步：Doris常部署多个FE提供query服务，这里我们不通过额外的通信来保证各个FE间cache的统计信息的一致性，通过定时的刷新机制来保证cache中的数据不会是严重过时的 class: `StatisticsCache` method: * getStatistics(dbid, tblid, colid) * tryToLoadStatitistics(dbid, tblid, colid) * eraseEntry(bid, tblid, colid) Doris中大量使用了Google Guava库，我们将基于该库提供的`LoadingCache`结构实现缓存能力，[Guava LoadingCache 相关文档](https://github.com/google/guava/wiki/CachesExplained) ## 收集 目前的实现中，将每个列的统计信息的收集拆分为了多个单独的SQL去执行，同时为了避免统计信息收集占用过多资源，如果队列中的Job数量超过20个，用户执行`ANALYZE`语句时会抛出异常。这么做的问题有： 1. 用户体验较差，用户执行ANALYZE语句不应因为这样粗糙的推测可能负载较高而直接报错 2. 代码较为复杂，不便维护和扩展。比如，如果由于每个ColumnStat的统计项都有一个单独的SQL去收集，那么，我们要么在每次一项执行完后，执行一次insert，要么引入同步机制，等到所有的统计项收集完毕后再执行insert进行持久化。此外，为StatisticJob定义了，`FINISHED` `FAILED`等状态，如果Job中的任意一个task失败了就认为Job执行失败了，但是该Job中由其他task完成收集的统计项依然能够被查询到。 重构计划： 1. 删除`TABLET`这一级的收集操作 2. 移除任务拆分逻辑，改为仅基于partition做收集 3. 统一为使用 SQL查询各项统计信息 4. 通过SQL粒度的会话变量限制SQL执行开销 5. 不再对聚合表和明细表进行区分，统一用SQL收集行数 6. 修改`TableStat` `ColumnStat`相关字段为`final` 7. 在`fe.conf`中添加变量`statistics_max_parallism`控制同时可执行的SQL数量，默认为10。 8. 统一命名：单个对象的统计信息使用`XXXStatistic `如`ColumnSatistic`，抽象表示整个统计信息框架，或聚合了多个对象的统计信息类的，用`Statistics` ### SQL partition统计信息SQL定义如下： ```sql private static final String ANALYZE_PARTITION_SQL_TEMPLATE = "INSERT INTO " + StatisticConstants.STATISTIC_DB_NAME + StatisticConstants.STATISTIC_TBL_NAME + " SELECT " + "CONCAT($catalogId, $dbId, $colId, $partId) AS id, " + "COUNT(1) AS row_count, " + "NDV($columnName) AS ndv, " + "SUM(CASE $colName IS NULL THEN 1 ELSE 0) AS null_count, " + "MAX($colName) AS max, " + "MIN($colName) AS min, " // SUM(LENGTH($colName)) as data_size + "$dataSizeFunction AS data_size, " + "MAX(LENGTH($colName)
[GitHub] [doris] BiteTheDDDDt merged pull request #13612: [feature-array](array-type) Add array function array_enumerate
BiteThet merged PR #13612: URL: https://github.com/apache/doris/pull/13612 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated: [feature-array](array-type) Add array function array_enumerate (#13612)
This is an automated email from the ASF dual-hosted git repository.
panxiaolei pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 235c105554 [feature-array](array-type) Add array function
array_enumerate (#13612)
235c105554 is described below
commit 235c105554d240a09cef3d21d1f3db3572c11198
Author: lihangyu <15605149...@163.com>
AuthorDate: Tue Oct 25 15:12:11 2022 +0800
[feature-array](array-type) Add array function array_enumerate (#13612)
Add array function array_enumerate
---
be/src/vec/CMakeLists.txt | 1 +
.../functions/array/function_array_enumerate.cpp | 98 ++
.../functions/array/function_array_register.cpp| 2 +
.../array-functions/array_enumerate.md | 60 +
.../array-functions/array_enumerate.md | 59 +
gensrc/script/doris_builtins_functions.py | 17
.../array_functions/test_array_functions.out | 36
.../array_functions/test_array_functions.groovy| 4 +
8 files changed, 277 insertions(+)
diff --git a/be/src/vec/CMakeLists.txt b/be/src/vec/CMakeLists.txt
index 845a1c0d75..d3ae2849de 100644
--- a/be/src/vec/CMakeLists.txt
+++ b/be/src/vec/CMakeLists.txt
@@ -152,6 +152,7 @@ set(VEC_FILES
functions/array/function_array_intersect.cpp
functions/array/function_array_slice.cpp
functions/array/function_array_difference.cpp
+ functions/array/function_array_enumerate.cpp
exprs/table_function/vexplode_json_array.cpp
functions/math.cpp
functions/function_bitmap.cpp
diff --git a/be/src/vec/functions/array/function_array_enumerate.cpp
b/be/src/vec/functions/array/function_array_enumerate.cpp
new file mode 100644
index 00..8f63e46ab1
--- /dev/null
+++ b/be/src/vec/functions/array/function_array_enumerate.cpp
@@ -0,0 +1,98 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+
+namespace doris::vectorized {
+
+class FunctionArrayEnumerate : public IFunction {
+public:
+static constexpr auto name = "array_enumerate";
+static FunctionPtr create() { return
std::make_shared(); }
+String get_name() const override { return name; }
+bool use_default_implementation_for_nulls() const override { return false;
}
+size_t get_number_of_arguments() const override { return 1; }
+DataTypePtr get_return_type_impl(const DataTypes& arguments) const
override {
+const DataTypeArray* array_type =
+
check_and_get_data_type(remove_nullable(arguments[0]).get());
+if (!array_type) {
+LOG(FATAL) << "First argument for function " + get_name() +
+ " must be an array but it has type " +
arguments[0]->get_name() +
+ ".";
+}
+
+auto nested_type = assert_cast(*array_type).get_nested_type();
+bool is_nested_nullable = nested_type->is_nullable();
+bool is_nullable = arguments[0]->is_nullable();
+auto return_nested_type = std::make_shared();
+DataTypePtr return_type = std::make_shared(
+is_nested_nullable ? make_nullable(return_nested_type) :
return_nested_type);
+if (is_nullable) {
+return_type = make_nullable(return_type);
+}
+return return_type;
+}
+Status execute_impl(FunctionContext* context, Block& block, const
ColumnNumbers& arguments,
+size_t result, size_t input_rows_count) override {
+auto left_column =
+
block.get_by_position(arguments[0]).column->convert_to_full_column_if_const();
+const ColumnArray* array =
+
check_and_get_column(remove_nullable(left_column->get_ptr()));
+if (!array) {
+return Status::RuntimeError(
+fmt::format("Illegal column {}, of first argument of
function {}",
+left_column->get_name(), get_name()));
+}
+const ColumnArray::Offsets64& offsets = array->get_offsets();
+
[GitHub] [doris] eldenmoon opened a new pull request, #13641: [feature-array](array-type) Add array function array_popback
eldenmoon opened a new pull request, #13641:
URL: https://github.com/apache/doris/pull/13641
Remove the last element from array.
```
mysql> select array_popback(['test', NULL, 'value']);
+-+
| array_popback(ARRAY('test', NULL, 'value')) |
+-+
| [test, NULL]|
+-+
```
# Proposed changes
Issue Number: close #xxx
## Problem summary
Describe your changes.
## Checklist(Required)
1. Does it affect the original behavior:
- [ ] Yes
- [ ] No
- [ ] I don't know
2. Has unit tests been added:
- [ ] Yes
- [ ] No
- [ ] No Need
3. Has document been added or modified:
- [ ] Yes
- [ ] No
- [ ] No Need
4. Does it need to update dependencies:
- [ ] Yes
- [ ] No
5. Are there any changes that cannot be rolled back:
- [ ] Yes (If Yes, please explain WHY)
- [ ] No
## Further comments
If this is a relatively large or complex change, kick off the discussion at
[d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you
chose the solution you did and what alternatives you considered, etc...
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] wangshuo128 commented on a diff in pull request #13601: [Feature](Nereids) Support join hint.
wangshuo128 commented on code in PR #13601:
URL: https://github.com/apache/doris/pull/13601#discussion_r1004084822
##
fe/fe-core/src/main/java/org/apache/doris/nereids/properties/RequestPropertyDeriver.java:
##
@@ -128,25 +129,45 @@ public Void
visitPhysicalLocalQuickSort(PhysicalLocalQuickSort s
@Override
public Void visitPhysicalHashJoin(PhysicalHashJoin hashJoin, PlanContext context) {
-// for broadcast join
-if (JoinUtils.couldBroadcast(hashJoin)) {
-addToRequestPropertyToChildren(PhysicalProperties.ANY,
PhysicalProperties.REPLICATED);
-}
+JoinHint hint = hashJoin.getHint();
Review Comment:
Please help to check the change here, thanks~ @morrySnow
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] luozenglin opened a new issue, #13642: [Bug] cross join statistics exception
luozenglin opened a new issue, #13642: URL: https://github.com/apache/doris/issues/13642 ### Search before asking - [X] I had searched in the [issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and found no similar issues. ### Version master ### What's Wrong? ``` mysql> select t1.k1 from test t1 cross join test t2 where t1.k1 = t2.k1 + 1 group by t1.k1; ERROR 1105 (HY000): errCode = 2, detailMessage = Unexpected exception: joinOp is not supported ``` ### What You Expected? ``` mysql> select t1.k1 from test t1 cross join test t2 where t1.k1 = t2.k1 + 1 group by t1.k1; +--+ | k1 | +--+ |3 | |2 | +--+ ``` ### How to Reproduce? _No response_ ### Anything Else? _No response_ ### Are you willing to submit PR? - [X] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] morningman opened a new pull request, #13643: [improvement](test) add sync for test_agg_keys_schema_change_datev2
morningman opened a new pull request, #13643: URL: https://github.com/apache/doris/pull/13643 # Proposed changes Issue Number: close #xxx ## Problem summary To avoid some potential meta sync problem when running regression test on multi-node cluster ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [ ] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [ ] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [ ] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen commented on a diff in pull request #13607: [fix](thirdparty) g++-11: error: unrecognized command-line option '-m…
hello-stephen commented on code in PR #13607:
URL: https://github.com/apache/doris/pull/13607#discussion_r1004109610
##
thirdparty/build-thirdparty.sh:
##
@@ -1345,8 +1345,13 @@ build_hdfs3() {
cd "${BUILD_DIR}"
rm -rf ./*
+if [[ "$(uname -m)" == "aarch64" ]]; then
+SSE_OPTION='-DENABLE_SSE=OFF'
+else
+SSE_OPTION='-DENABLE_SSE=ON'
+fi
Review Comment:
I found this when I try to compile on the 'TencentOS Server 3.1 for ARM64
(TK4)' system, it does interrupt the compilation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] adonis0147 commented on a diff in pull request #13607: [fix](thirdparty) g++-11: error: unrecognized command-line option '-m…
adonis0147 commented on code in PR #13607:
URL: https://github.com/apache/doris/pull/13607#discussion_r1004111444
##
thirdparty/build-thirdparty.sh:
##
@@ -1345,8 +1345,13 @@ build_hdfs3() {
cd "${BUILD_DIR}"
rm -rf ./*
+if [[ "$(uname -m)" == "aarch64" ]]; then
+SSE_OPTION='-DENABLE_SSE=OFF'
+else
+SSE_OPTION='-DENABLE_SSE=ON'
+fi
Review Comment:
> I found this when I try to compile on the 'TencentOS Server 3.1 for ARM64
(TK4)' system, it does interrupt the compilation.
Ok.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] carlvinhust2012 commented on pull request #13627: [fix](array-type) support CTAS for ARRAY column from collect_list and collect_set
carlvinhust2012 commented on PR #13627: URL: https://github.com/apache/doris/pull/13627#issuecomment-1290124917 LGTM -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #13600: [fix](planner) inlineView alias error
github-actions[bot] commented on PR #13600: URL: https://github.com/apache/doris/pull/13600#issuecomment-1290126561 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #13600: [fix](planner) inlineView alias error
github-actions[bot] commented on PR #13600: URL: https://github.com/apache/doris/pull/13600#issuecomment-1290126605 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] Gabriel39 opened a new pull request, #13644: [Chore](regression) Fix wrong result for decimal
Gabriel39 opened a new pull request, #13644: URL: https://github.com/apache/doris/pull/13644 # Proposed changes Issue Number: close #xxx ## Problem summary Describe your changes. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [ ] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [ ] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [ ] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] luozenglin opened a new pull request, #13645: [fix](statistics) fix cross join statistics exception
luozenglin opened a new pull request, #13645: URL: https://github.com/apache/doris/pull/13645 # Proposed changes Issue Number: close #13642 ## Problem summary Describe your changes. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [x] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [x] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [x] No Need 4. Does it need to update dependencies: - [ ] Yes - [x] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [x] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] zhang91466 commented on issue #13503: [Bug] 没有任何SYNC同步JOB，但是日志一直报CanalSyncDataReceiver Connection refused
zhang91466 commented on issue #13503: URL: https://github.com/apache/doris/issues/13503#issuecomment-1290138925 > Hello！看样子可能是配置有问题，可以加我微信：lyy0222 ，我们一起看下哈！ Hi 我添加了此微信号，但是被告知加错人了，麻烦指条明路，谢谢 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris-website] branch master updated: link 404
This is an automated email from the ASF dual-hosted git repository. jiafengzheng pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris-website.git The following commit(s) were added to refs/heads/master by this push: new 4b205bca302 link 404 4b205bca302 is described below commit 4b205bca302e524e8d412cc101997e048a1f5de4 Author: jiafeng.zhang AuthorDate: Tue Oct 25 15:52:17 2022 +0800 link 404 --- docs/admin-manual/config/be-config.md | 2 +- .../maint-monitor/metadata-operation.md| 2 +- docs/admin-manual/maint-monitor/multi-tenant.md| 238 - .../maint-monitor/tablet-repair-and-balance.md | 2 +- docs/advanced/alter-table/replace-table.md | 2 +- docs/advanced/alter-table/schema-change.md | 2 +- docs/data-operate/export/outfile.md| 2 +- .../import/import-scenes/external-storage-load.md | 2 +- .../data-operate/import/import-scenes/jdbc-load.md | 2 +- .../import/import-scenes/kafka-load.md | 2 +- docs/data-table/basic-usage.md | 2 +- docs/data-table/hit-the-rollup.md | 2 +- docs/ecosystem/doris-manager/space-list.md | 2 +- docs/ecosystem/logstash.md | 2 +- docs/ecosystem/udf/contribute-udf.md | 2 +- docs/faq/install-faq.md| 8 +- .../Alter/ALTER-TABLE-PARTITION.md | 2 +- .../Alter/ALTER-TABLE-REPLACE.md | 2 +- .../Alter/ALTER-TABLE-ROLLUP.md| 2 +- .../Drop/DROP-DATABASE.md | 2 +- .../Data-Definition-Statements/Drop/DROP-TABLE.md | 2 +- .../Load/BROKER-LOAD.md| 18 +- .../Load/CREATE-SYNC-JOB.md| 2 +- .../Load/STREAM-LOAD.md| 10 +- .../SHOW-ALTER-TABLE-MATERIALIZED-VIEW.md | 2 +- .../cluster-management/elastic-expansion.md| 2 +- .../current/admin-manual/config/be-config.md | 2 +- .../maint-monitor/metadata-operation.md| 2 +- .../maint-monitor/tablet-repair-and-balance.md | 2 +- .../current/admin-manual/multi-tenant.md | 232 .../current/advanced/alter-table/replace-table.md | 2 +- .../current/advanced/alter-table/schema-change.md | 2 +- .../import/import-scenes/external-storage-load.md | 2 +- .../data-operate/import/import-scenes/jdbc-load.md | 2 +- .../current/data-table/basic-usage.md | 2 +- .../current/data-table/hit-the-rollup.md | 2 +- .../current/ecosystem/doris-manager/space-list.md | 2 +- .../current/ecosystem/logstash.md | 2 +- .../current/ecosystem/udf/contribute-udf.md| 2 +- .../Alter/ALTER-TABLE-PARTITION.md | 2 +- .../Alter/ALTER-TABLE-REPLACE.md | 4 +- .../Alter/ALTER-TABLE-ROLLUP.md| 2 +- .../Drop/DROP-DATABASE.md | 2 +- .../Data-Definition-Statements/Drop/DROP-TABLE.md | 2 +- .../Load/BROKER-LOAD.md| 22 +- .../Load/CREATE-SYNC-JOB.md| 2 +- .../SHOW-ALTER-TABLE-MATERIALIZED-VIEW.md | 2 +- .../alter-table/alter-table-bitmap-index.md| 20 +- .../alter-table/alter-table-replace-table.md | 4 +- .../alter-table/alter-table-temp-partition.md | 2 +- .../administrator-guide/config/fe_config.md| 2 +- .../load-data/broker-load-manual.md| 2 +- .../load-data/routine-load-manual.md | 2 +- .../administrator-guide/operation/disk-capacity.md | 4 +- .../version-0.15/administrator-guide/outfile.md| 2 +- .../administrator-guide/resource-management.md | 2 +- .../best-practices/star-schema-benchmark.md| 8 +- .../version-0.15/extending-doris/logstash.md | 4 +- .../alter-table/alter-table-bitmap-index.md| 18 +- .../alter-table/alter-table-replace-table.md | 4 +- .../alter-table/alter-table-temp-partition.md | 2 +- .../administrator-guide/config/fe_config.md| 2 +- .../load-data/broker-load-manual.md| 2 +- .../load-data/routine-load-manual.md | 2 +- .../administrator-guide/operation/disk-capacity.md | 4 +- .../version-0.15/administrator-guide/outfile.md| 2 +- .../administrator-guide/resource-management.md | 2 +- .../best-practices/star-schema-benchmark.md| 4 +- 68 files changed, 118 insertions(+), 588 deletions(-) diff --git a/docs/admin-manual/config/be-config.md b/docs/admin-manual/config/be-config.md index 66282b1acfb..e27118a83ed 100644 --- a/docs/admin-manual/config/be-config.md +++ b/docs/admin-manual/config/be-config.md @@ -450,7 +450,7 @@ Cgroups assigned to doris ### `doris_max_scan_key_num` * Type: int
[GitHub] [doris] yiguolei merged pull request #13619: [fix](plugin) Fix dirty uninstallation of plugin for 1.1-lts
yiguolei merged PR #13619: URL: https://github.com/apache/doris/pull/13619 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch branch-1.1-lts updated: [fix](plugin) Fix dirty uninstallation of plugin for 1.1-lts (#13619)
This is an automated email from the ASF dual-hosted git repository.
yiguolei pushed a commit to branch branch-1.1-lts
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-1.1-lts by this push:
new 0c83509e98 [fix](plugin) Fix dirty uninstallation of plugin for
1.1-lts (#13619)
0c83509e98 is described below
commit 0c83509e98c2967546307769bf680649d5cf05bb
Author: wxy
AuthorDate: Tue Oct 25 16:06:43 2022 +0800
[fix](plugin) Fix dirty uninstallation of plugin for 1.1-lts (#13619)
* [fix](plugin) bugfix for dirty uninstallation of dynamic plugin (#13540)
* [fix](plugin) bugfix for dirty uninstallation of dynamic plugin (#13540)
Co-authored-by: wangxian...@360shuke.com
---
.../java/org/apache/doris/plugin/DynamicPluginLoader.java | 13 +++--
1 file changed, 7 insertions(+), 6 deletions(-)
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/plugin/DynamicPluginLoader.java
b/fe/fe-core/src/main/java/org/apache/doris/plugin/DynamicPluginLoader.java
index cfb2c25d77..2ce00fd760 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/plugin/DynamicPluginLoader.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/plugin/DynamicPluginLoader.java
@@ -111,14 +111,15 @@ public class DynamicPluginLoader extends PluginLoader {
movePlugin();
-plugin = dynamicLoadPlugin(true);
+try {
+plugin = dynamicLoadPlugin(true);
+
+pluginInstallValid();
-pluginInstallValid();
+pluginContext.setPluginPath(installPath.toString());
-pluginContext.setPluginPath(installPath.toString());
-try {
plugin.init(pluginInfo, pluginContext);
-} catch (Error e) {
+} catch (Throwable e) {
throw new UserException(e.getMessage());
}
}
@@ -152,7 +153,7 @@ public class DynamicPluginLoader extends PluginLoader {
/**
* reload plugin if plugin has already been installed, else will
re-install.
* Notice that this method will create a new instance of plugin.
- *
+ *
* @throws PluginException
*/
public void reload() throws IOException, UserException {
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] yiguolei commented on pull request #13543: [fix](plugin) bugfix for dirty uninstallation of dynamic plugin (#13540)
yiguolei commented on PR #13543: URL: https://github.com/apache/doris/pull/13543#issuecomment-1290156145 https://github.com/apache/doris/pull/13619 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] ByteYue opened a new pull request, #13646: [enhancement](metric)Add metric for exec_state prepare function
ByteYue opened a new pull request, #13646: URL: https://github.com/apache/doris/pull/13646 # Proposed changes Issue Number: close #xxx ## Problem summary Previously, we found out that sometimes the rpc service might timeout due to the extremely time consuming operation during exec_state prepare, thus this pr adds one metric to monitor the prepare time cost of each single fragment. After compiling with this commit, BE would be able to generate data compatible with grafana, and you can watch the data using dashboard like below.  As for the total time consumption of all the fragments prepare, you can refer to the brpc port to watch the rpc metric for more detailed message. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [x] No - [ ] I don't know 2. Has unit tests been added: - [x] Yes - [ ] No - [ ] No Need 3. Has document been added or modified: - [x] Yes - [ ] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [x] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [x] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] BiteTheDDDDt merged pull request #13594: [chore](be version) Check BE version by script
BiteThet merged PR #13594: URL: https://github.com/apache/doris/pull/13594 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated: [chore](be version) Check BE version by script (#13594)
This is an automated email from the ASF dual-hosted git repository.
panxiaolei pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 78278f5943 [chore](be version) Check BE version by script (#13594)
78278f5943 is described below
commit 78278f594330acc311ac5e3d973f244e854c0ffd
Author: Gabriel
AuthorDate: Tue Oct 25 16:20:38 2022 +0800
[chore](be version) Check BE version by script (#13594)
Check BE version by script
---
bin/check_be_version.sh | 93 +
build.sh| 1 +
2 files changed, 94 insertions(+)
diff --git a/bin/check_be_version.sh b/bin/check_be_version.sh
new file mode 100755
index 00..b44d0c8f64
--- /dev/null
+++ b/bin/check_be_version.sh
@@ -0,0 +1,93 @@
+#!/usr/bin/env bash
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+
+set -eo pipefail
+
+curdir="$(cd "$(dirname "${BASH_SOURCE[0]}")" &>/dev/null && pwd)"
+
+if [[ "$(uname -s)" == 'Darwin' ]] && command -v brew &>/dev/null; then
+PATH="$(brew --prefix)/opt/gnu-getopt/bin:${PATH}"
+export PATH
+fi
+
+DORIS_HOME="$(
+cd "${curdir}/.."
+pwd
+)"
+export DORIS_HOME
+
+jdk_version() {
+local java_cmd="${1}"
+local result
+local IFS=$'\n'
+
+if [[ -z "${java_cmd}" ]]; then
+result=no_java
+return 1
+else
+local version
+# remove \r for Cygwin
+version="$("${java_cmd}" -Xms32M -Xmx32M -version 2>&1 | tr '\r' '\n'
| grep version | awk '{print $3}')"
+version="${version//\"/}"
+if [[ "${version}" =~ ^1\. ]]; then
+result="$(echo "${version}" | awk -F '.' '{print $2}')"
+else
+result="$(echo "${version}" | awk -F '.' '{print $1}')"
+fi
+fi
+echo "${result}"
+return 0
+}
+
+setup_java_env() {
+local java_version
+
+if [[ -z "${JAVA_HOME}" ]]; then
+return 1
+fi
+
+local jvm_arch='amd64'
+if [[ "$(uname -m)" == 'aarch64' ]]; then
+jvm_arch='aarch64'
+fi
+java_version="$(
+set -e
+jdk_version "${JAVA_HOME}/bin/java"
+)"
+if [[ "${java_version}" -gt 8 ]]; then
+export
LD_LIBRARY_PATH="${JAVA_HOME}/lib/server:${JAVA_HOME}/lib:${LD_LIBRARY_PATH}"
+# JAVA_HOME is jdk
+elif [[ -d "${JAVA_HOME}/jre" ]]; then
+export
LD_LIBRARY_PATH="${JAVA_HOME}/jre/lib/${jvm_arch}/server:${JAVA_HOME}/jre/lib/${jvm_arch}:${LD_LIBRARY_PATH}"
+# JAVA_HOME is jre
+else
+export
LD_LIBRARY_PATH="${JAVA_HOME}/lib/${jvm_arch}/server:${JAVA_HOME}/lib/${jvm_arch}:${LD_LIBRARY_PATH}"
+fi
+}
+
+# prepare jvm if needed
+setup_java_env || true
+
+if [[ -e "${DORIS_HOME}/bin/palo_env.sh" ]]; then
+# shellcheck disable=1091
+source "${DORIS_HOME}/bin/palo_env.sh"
+fi
+
+chmod 755 "${DORIS_HOME}/lib/doris_be"
+
+"${DORIS_HOME}"/lib/doris_be --version
diff --git a/build.sh b/build.sh
index 8ea1ced091..650c030248 100755
--- a/build.sh
+++ b/build.sh
@@ -496,6 +496,7 @@ if [[ "${BUILD_BE}" -eq 1 ]]; then
"${DORIS_OUTPUT}/udf/include"
cp -r -p "${DORIS_HOME}/be/output/bin"/* "${DORIS_OUTPUT}/be/bin"/
+cp -r -p "${DORIS_HOME}/bin/check_be_version.sh" "${DORIS_OUTPUT}/be/bin"/
cp -r -p "${DORIS_HOME}/be/output/conf"/* "${DORIS_OUTPUT}/be/conf"/
# Fix Killed: 9 error on MacOS (arm64).
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] morrySnow opened a new pull request, #13647: [enhancement](chore) Enhance build script compatibility
morrySnow opened a new pull request, #13647: URL: https://github.com/apache/doris/pull/13647 # Proposed changes 1. build fe-common when build java-udf 2. add jvm environment parameter '-Dos.arch=x86_64' when build fe on MacOS ## Problem summary Describe your changes. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [ ] No - [ ] I don't know 3. Has unit tests been added: - [ ] Yes - [ ] No - [ ] No Need 4. Has document been added or modified: - [ ] Yes - [ ] No - [ ] No Need 5. Does it need to update dependencies: - [ ] Yes - [ ] No 6. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [ ] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] adonis0147 commented on a diff in pull request #13647: [enhancement](chore) Enhance build script compatibility
adonis0147 commented on code in PR #13647:
URL: https://github.com/apache/doris/pull/13647#discussion_r1004163133
##
build.sh:
##
@@ -451,7 +452,11 @@ if [[ "${FE_MODULES}" != '' ]]; then
if [[ "${CLEAN}" -eq 1 ]]; then
clean_fe
fi
-"${MVN_CMD}" package -pl ${FE_MODULES:+${FE_MODULES}} -DskipTests
+MVN_PARAMS="-DskipTests"
+if [[ "$(uname -s)" = 'Darwin' ]]; then
+ MVN_PARAMS="${MVN_PARAMS} -Dos.arch=x86_64"
+fi
Review Comment:
It seems that we don't need this option `-Dos.arch=x86_64` anymore. See
#13571.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[doris-website] branch master updated: SHOW-ALTER-TABLE-MATERIALIZED-VIEW
This is an automated email from the ASF dual-hosted git repository. jiafengzheng pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris-website.git The following commit(s) were added to refs/heads/master by this push: new 888cdfe6d4a SHOW-ALTER-TABLE-MATERIALIZED-VIEW 888cdfe6d4a is described below commit 888cdfe6d4aaf489669100a4857b4f46ed6e83c9 Author: jiafeng.zhang AuthorDate: Tue Oct 25 16:32:43 2022 +0800 SHOW-ALTER-TABLE-MATERIALIZED-VIEW --- sidebars.json | 1 + 1 file changed, 1 insertion(+) diff --git a/sidebars.json b/sidebars.json index 08f7f0777ea..0c58b003d39 100644 --- a/sidebars.json +++ b/sidebars.json @@ -720,6 +720,7 @@ "type": "category", "label": "Show", "items": [ + "sql-manual/sql-reference/Show-Statements/SHOW-ALTER-TABLE-MATERIALIZED-VIEW", "sql-manual/sql-reference/Show-Statements/SHOW-ALTER", "sql-manual/sql-reference/Show-Statements/SHOW-BACKENDS", "sql-manual/sql-reference/Show-Statements/SHOW-BACKUP", - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] morrySnow closed pull request #13647: [enhancement](chore) Enhance build script compatibility
morrySnow closed pull request #13647: [enhancement](chore) Enhance build script compatibility URL: https://github.com/apache/doris/pull/13647 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] morrySnow commented on pull request #13647: [enhancement](chore) Enhance build script compatibility
morrySnow commented on PR #13647: URL: https://github.com/apache/doris/pull/13647#issuecomment-1290192482 already fixed by #13571 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] dataroaring merged pull request #13578: [fix](regression) add 'if not exists' to 'create table' to support parallel test (#13576)
dataroaring merged PR #13578: URL: https://github.com/apache/doris/pull/13578 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] dataroaring closed issue #13576: [Bug] regression test failed for duplicated table creation
dataroaring closed issue #13576: [Bug] regression test failed for duplicated table creation URL: https://github.com/apache/doris/issues/13576 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated (78278f5943 -> b85c78ee00)
This is an automated email from the ASF dual-hosted git repository. dataroaring pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/doris.git from 78278f5943 [chore](be version) Check BE version by script (#13594) add b85c78ee00 [fix](regression) add 'if not exists' to 'create table' to support parallel test (#13576) (#13578) No new revisions were added by this update. Summary of changes: regression-test/common/table/bitmap_basic_agg.sql | 2 +- regression-test/common/table/hll_basic_agg.sql | 2 +- regression-test/common/table/test_basic_agg.sql| 2 +- .../bitmap_functions/test_bitmap_intersect.groovy | 2 +- .../bloom_filter_p0/test_bloom_filter.groovy | 4 ++-- .../test_bloom_filter_is_not_null.groovy | 2 +- .../test_create_table_with_bloom_filter.groovy | 2 +- .../create_table_use_partition_policy.groovy | 8 +++ .../use_policy/create_table_use_policy.groovy | 2 +- .../use_policy/use_default_storage_policy.groovy | 4 ++-- .../compaction/test_compaction_agg_keys.groovy | 2 +- .../compaction/test_compaction_dup_keys.groovy | 2 +- .../compaction/test_compaction_uniq_keys.groovy| 2 +- ...st_constant_push_down_through_outer_join.groovy | 4 ++-- .../test_crossjoin_inlineview_slot.groovy | 10 .../test_lateral_view_in_subquery.groovy | 4 ++-- .../test_outer_join_with_subquery.groovy | 4 ++-- .../correctness/test_pushdown_pred_to_view.groovy | 2 +- .../correctness/test_union_with_subquery.groovy| 2 +- .../correctness/test_view_varchar_length.groovy| 2 +- .../suites/correctness_p0/test_bitmap_count.groovy | 2 +- .../correctness_p0/test_bitmap_intersect.groovy| 2 +- .../correctness_p0/test_bitmap_serialize.groovy| 2 +- .../test_bucket_join_with_colocate_table.groovy| 4 ++-- .../correctness_p0/test_current_timestamp.groovy | 2 +- .../test_dict_range_predicate.groovy | 2 +- .../correctness_p0/test_first_value_window.groovy | 2 +- .../correctness_p0/test_group_by_constant.groovy | 2 +- .../test_join_should_not_reorder.groovy| 6 ++--- .../test_join_with_projection.groovy | 4 ++-- .../correctness_p0/test_lag_lead_window.groovy | 2 +- .../correctness_p0/test_last_value_window.groovy | 2 +- .../test_like_predicate_with_nullable.groovy | 2 +- .../correctness_p0/test_min_max_window.groovy | 2 +- .../correctness_p0/test_no_grouping_window.groovy | 2 +- .../correctness_p0/test_null_predicate.groovy | 2 +- .../correctness_p0/test_outer_join_sort.groovy | 6 ++--- .../test_outer_join_with_cross_join.groovy | 8 +++ .../test_outer_join_with_empty_node.groovy | 4 ++-- .../test_outer_join_with_grouping.groovy | 4 ++-- .../test_outer_join_with_order_by.groovy | 6 ++--- .../test_outer_join_with_window_function.groovy| 8 +++ ...t_outer_join_with_window_function_datev2.groovy | 8 +++ .../correctness_p0/test_pushdown_constant.groovy | 2 +- .../test_select_stddev_variance_window.groovy | 2 +- .../correctness_p0/test_select_variance_agg.groovy | 2 +- .../suites/correctness_p0/test_sort.groovy | 2 +- .../correctness_p0/test_string_pad_function.groovy | 2 +- .../correctness_p0/test_subquery_with_agg.groovy | 2 +- .../csv_header_p0/test_csv_with_header.groovy | 2 +- .../storage/test_dup_tab_basic_int.groovy | 2 +- .../storage/test_dup_tab_basic_int_nullable.groovy | 2 +- .../storage/test_dup_tab_basic_varchar.groovy | 2 +- .../test_dup_tab_basic_varchar_nullable.groovy | 2 +- .../duplicate/storage/test_dup_tab_char.groovy | 2 +- .../storage/test_dup_tab_char_nullable.groovy | 2 +- .../duplicate/storage/test_dup_tab_date.groovy | 2 +- .../storage/test_dup_tab_date_nullable.groovy | 2 +- .../duplicate/storage/test_dup_tab_datetime.groovy | 2 +- .../storage/test_dup_tab_datetime_nullable.groovy | 2 +- .../duplicate/storage/test_dup_tab_decimal.groovy | 2 +- .../storage/test_dup_tab_decimal_nullable.groovy | 2 +- .../storage/test_dup_tab_mixed_type.groovy | 2 +- .../test_dup_tab_mixed_type_nullable.groovy| 2 +- .../datatype_p0/bitmap/test_bitmap_int.groovy | 4 ++-- .../suites/datatype_p0/hll/test_hll_int.groovy | 2 +- .../datatype_p0/string/test_string_basic.groovy| 8 +++ regression-test/suites/ddl_p0/test_ctas.groovy | 4 ++-- regression-test/suites/ddl_p0/test_ctl.groovy | 2 +- .../delete_p0/test_array_column_delete.groovy | 2 +- .../suites/delete_p0/test_delete.groovy| 4 ++-- .../suites/delete_p0/test_delete_sign_mow.sql | 2 +- .../suites/delete_p0/test_delete_sign_unique.sql | 2 +- .../delete_p0/test_multi_columns_delete.groovy | 2 +- .../delete_p0/test_segment_iterator_delete.groovy | 6
[GitHub] [doris] morningman commented on a diff in pull request #13611: [test](multi-catalog)Regression test for external hive parquet table.
morningman commented on code in PR #13611:
URL: https://github.com/apache/doris/pull/13611#discussion_r1004175554
##
regression-test/suites/external_catalog_p0/hive/test_hive_parquet.groovy:
##
@@ -0,0 +1,185 @@
+// Licensed to the Apache Software Foundation (ASF) under one
+// or more contributor license agreements. See the NOTICE file
+// distributed with this work for additional information
+// regarding copyright ownership. The ASF licenses this file
+// to you under the Apache License, Version 2.0 (the
+// "License"); you may not use this file except in compliance
+// with the License. You may obtain a copy of the License at
+//
+// http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing,
+// software distributed under the License is distributed on an
+// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+// KIND, either express or implied. See the License for the
+// specific language governing permissions and limitations
+// under the License.
+
+suite("test_hive_parquet", "p0") {
+def q01 = {
+qt_q01 """
+select * from partition_table order by l_orderkey;
+"""
+}
+
+def q02 = {
+qt_q02 """
+select count(*) from partition_table;
+"""
+}
+
+def q03 = {
+qt_q03 """
+select count(city) from partition_table;
+"""
+}
+
+def q04 = {
+qt_q04 """
+select count(nation) from partition_table;
+"""
+}
+
+def q05 = {
+qt_q05 """
+select distinct city from partition_table order by city;
+"""
+}
+
+def q06 = {
+qt_q06 """
+select distinct nation from partition_table order by nation;
+"""
+}
+
+def q07 = {
+qt_q07 """
+select city from partition_table order by city limit 10;
+"""
+}
+
+def q08 = {
+qt_q08 """
+select nation from partition_table order by nation limit 10;
+"""
+}
+
+def q09 = {
+qt_q09 """
+select city, nation from partition_table order by nation, city limit
10;
+"""
+}
+
+def q10 = {
+qt_q10 """
+select nation, city, count(*) from partition_table group by nation,
city order by nation, city;
+"""
+}
+
+def q11 = {
+qt_q11 """
+select city, count(*) from partition_table group by city order by city;
+"""
+}
+
+def q12 = {
+qt_q12 """
+select nation, count(*) from partition_table group by nation order by
nation;
+"""
+}
+
+def q13 = {
+qt_q13 """
+select city, l_orderkey, nation from partition_table order by
l_orderkey limit 10;
+"""
+}
+
+def q14 = {
+qt_q14 """
+select l_orderkey from partition_table order by l_orderkey limit 10;
+"""
+}
+
+def q15 = {
+qt_q15 """
+select count(l_orderkey) from partition_table;
+"""
+}
+
+def q16 = {
+qt_q16 """
+select count(*) from partition_table where city='beijing';
+"""
+}
+
+def q17 = {
+qt_q17 """
+select count(*) from partition_table where nation='cn';
+"""
+}
+
+
+def set_be_config = { flag ->
+String[][] backends = sql """ show backends; """
+assertTrue(backends.size() > 0)
+for (String[] backend in backends) {
+StringBuilder setConfigCommand = new StringBuilder();
+setConfigCommand.append("curl -X POST http://";)
+setConfigCommand.append(backend[2])
+setConfigCommand.append(":")
+setConfigCommand.append(backend[5])
+setConfigCommand.append("/api/update_config?")
+String command1 = setConfigCommand.toString() +
"enable_new_load_scan_node=$flag"
+logger.info(command1)
+String command2 = setConfigCommand.toString() +
"enable_new_file_scanner=$flag"
+logger.info(command2)
+def process1 = command1.execute()
+int code = process1.waitFor()
+assertEquals(code, 0)
+def process2 = command2.execute()
+code = process1.waitFor()
+assertEquals(code, 0)
+}
+}
+
+String enabled = context.config.otherConfigs.get("enableHiveTest")
+if (enabled != null && enabled.equalsIgnoreCase("true")) {
+try {
+String hms_port = context.config.otherConfigs.get("hms_port")
+sql """admin set frontend config ("enable_multi_catalog" =
"true")"""
+sql """admin set frontend config ("enable_new_load_scan_node" =
"true");"""
+set_be_config.call('true')
+sql """drop catalog if exists hive"""
+sql """
+create catalog hive properties (
+"type"="hms",
+'hive.metastore.uris' = 'thrift://127.0.0.1:${hms_port}'
+);
+"""
+sql """switch hive"""
+
[GitHub] [doris] wangshuo128 commented on a diff in pull request #13416: [Feature](Nereids) Support materialized index selection.
wangshuo128 commented on code in PR #13416:
URL: https://github.com/apache/doris/pull/13416#discussion_r1003970016
##
fe/fe-core/src/main/java/org/apache/doris/nereids/trees/plans/logical/LogicalOlapScan.java:
##
@@ -164,30 +164,31 @@ public long getSelectedIndexId() {
return selectedIndexId;
}
-public boolean isRollupSelected() {
-return rollupSelected;
+public boolean isIndexSelected() {
+return indexSelected;
}
public PreAggStatus getPreAggStatus() {
return preAggStatus;
}
/**
- * Should apply {@link SelectRollupWithAggregate} or not.
+ * Should apply {@link SelectMaterializedIndexWithAggregate} or not.
*/
-public boolean shouldSelectRollup() {
+public boolean shouldSelectIndex() {
Review Comment:
This was to do two things:
1. judge `indexSelected` is true or false
2. make sure the keys type of the OLAP table is `DUP_KEYS`, `UNIQUE_KEYS`,
or `AGG_KEYS`, but it should not be `PRIMARY_KEYS`. Even if it seems
`PRIMARY_KEYS` is not used, it's safe to ensure the scope to apply materialized
view selection rules is right.
I would keep the getter method of the status of `indexSelected` in
`LogicalOlapScan` and move the condition of applying materialized view
selection to the rule itself.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] yiguolei merged pull request #13613: [fix](sort)order by constant expr bug
yiguolei merged PR #13613: URL: https://github.com/apache/doris/pull/13613 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated (b85c78ee00 -> e103531e69)
This is an automated email from the ASF dual-hosted git repository. yiguolei pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/doris.git from b85c78ee00 [fix](regression) add 'if not exists' to 'create table' to support parallel test (#13576) (#13578) add e103531e69 [fix](sort)order by constant expr bug (#13613) No new revisions were added by this update. Summary of changes: .../src/main/java/org/apache/doris/analysis/SelectStmt.java | 8 .../suites/correctness_p0/test_group_by_constant.groovy | 9 + 2 files changed, 17 insertions(+) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] zhangyongxianggithub commented on issue #9839: [Bug]Report No Doris FE is avaliable，actually the fe and be is running
zhangyongxianggithub commented on issue #9839:
URL: https://github.com/apache/doris/issues/9839#issuecomment-1290200774
I had the same problem and solved this bug，fellow is my solution，the reason ：
doris fe server version is too low and your jar version is new。
the problem file is org.apache.doris.flink.rest.RestService，fellow:
```java
public static String randomBackend(DorisOptions options,
DorisReadOptions readOptions, Logger logger) throws DorisException, IOException
{
List backends = getBackendsV2(options,
readOptions, logger);
logger.trace("Parse beNodes '{}'.", backends);
if (backends == null || backends.isEmpty()) {
logger.error(ILLEGAL_ARGUMENT_MESSAGE, "beNodes", backends);
throw new IllegalArgumentException("beNodes",
String.valueOf(backends));
}
Collections.shuffle(backends);
BackendV2.BackendRowV2 backend = backends.get(0);
return backend.getIp() + ":" + backend.getHttpPort();
}
```
there is two getBackends[V2] methods, one is getBackends() for old version
doris fe, another is getBackendsV2() for newer version doris fe. the
diffierences between them is url. method getBackends() will visist
/rest/v1/system?path=//backends, method getBackendsV2() will visit
/api/backends?is_alive=true, if your doris fe version is too low, your call to
getBackendsV2() will get 404.so you can write a
org.apache.doris.flink.rest.RestService or upgrade doris version
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch branch-1.1-lts updated: [fix](sort)order by constant expr bug (#13614)
This is an automated email from the ASF dual-hosted git repository.
yiguolei pushed a commit to branch branch-1.1-lts
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/branch-1.1-lts by this push:
new 7e4b4dd79f [fix](sort)order by constant expr bug (#13614)
7e4b4dd79f is described below
commit 7e4b4dd79f1bb4deeda80c10152473606becf9e6
Author: starocean999 <40539150+starocean...@users.noreply.github.com>
AuthorDate: Tue Oct 25 16:44:42 2022 +0800
[fix](sort)order by constant expr bug (#13614)
---
.../src/main/java/org/apache/doris/analysis/SelectStmt.java | 8
1 file changed, 8 insertions(+)
diff --git a/fe/fe-core/src/main/java/org/apache/doris/analysis/SelectStmt.java
b/fe/fe-core/src/main/java/org/apache/doris/analysis/SelectStmt.java
index 4e98276954..2fe58db9ec 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/analysis/SelectStmt.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/analysis/SelectStmt.java
@@ -1409,7 +1409,15 @@ public class SelectStmt extends QueryStmt {
}
if (orderByElements != null) {
for (OrderByElement orderByElem : orderByElements) {
+// we must make sure the expr is analyzed before rewrite
+try {
+orderByElem.getExpr().analyze(analyzer);
+} catch (AnalysisException ex) {
+//ignore any exception
+}
orderByElem.setExpr(rewriter.rewrite(orderByElem.getExpr(),
analyzer));
+// after rewrite, need reset the analyze status for later
re-analyze
+orderByElem.getExpr().reset();
}
}
}
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] yiguolei merged pull request #13614: [fix](sort)order by constant expr bug
yiguolei merged PR #13614: URL: https://github.com/apache/doris/pull/13614 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated: [fix](Nereids) add exchange node check between local and global agg in plan translator (#12913)
This is an automated email from the ASF dual-hosted git repository.
morrysnow pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new f209b7ab6e [fix](Nereids) add exchange node check between local and
global agg in plan translator (#12913)
f209b7ab6e is described below
commit f209b7ab6e22ba85a9e12b6affdfa08a6bff5a70
Author: yinzhijian <373141...@qq.com>
AuthorDate: Tue Oct 25 16:55:29 2022 +0800
[fix](Nereids) add exchange node check between local and global agg in plan
translator (#12913)
### table schema
CREATE TABLE `t1` (
`k1` int(11) NULL,
`v1` int(11) NULL
) ENGINE=OLAP
DUPLICATE KEY(`k1`, `v1`)
COMMENT 'OLAP'
DISTRIBUTED BY HASH(`k1`)
BUCKETS 3
PROPERTIES('replication_num'='1')
### query
select k1,count(distinct v1+1) from t1 group by k1;
### error
java.lang.ClassCastException: org.apache.doris.planner.OlapScanNode cannot
be cast to org.apache.doris.planner.AggregationNode
---
.../doris/nereids/glue/translator/PhysicalPlanTranslator.java | 8 +++-
regression-test/data/nereids_syntax_p0/function.out | 5 +
regression-test/suites/nereids_syntax_p0/function.groovy | 4
3 files changed, 16 insertions(+), 1 deletion(-)
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/nereids/glue/translator/PhysicalPlanTranslator.java
b/fe/fe-core/src/main/java/org/apache/doris/nereids/glue/translator/PhysicalPlanTranslator.java
index 1b973daf47..8fc0ba7f92 100644
---
a/fe/fe-core/src/main/java/org/apache/doris/nereids/glue/translator/PhysicalPlanTranslator.java
+++
b/fe/fe-core/src/main/java/org/apache/doris/nereids/glue/translator/PhysicalPlanTranslator.java
@@ -201,7 +201,13 @@ public class PhysicalPlanTranslator extends
DefaultPlanVisitor
[GitHub] [doris] morrySnow merged pull request #12913: [fix](Nereids) Add exchange node check between local and global agg
morrySnow merged PR #12913: URL: https://github.com/apache/doris/pull/12913 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] nextdreamblue opened a new issue, #13648: [Bug] broker load with s3a error
nextdreamblue opened a new issue, #13648:
URL: https://github.com/apache/doris/issues/13648
### Search before asking
- [X] I had searched in the
[issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and
found no similar issues.
### Version
master
### What's Wrong?
MySQL [test]> LOAD LABEL label15
-> (
-> DATA INFILE("s3a:///a.csv")
-> INTO TABLE `lineitem`
-> COLUMNS TERMINATED BY ","
-> )
-> WITH BROKER Broker_Doris
-> (
-> "fs.s3a.access.key" = "xxx",
-> "fs.s3a.secret.key" = "xxx",
-> "fs.s3a.endpoint" = "xxx"
-> );
ERROR 1105 (HY000): errCode = 2, detailMessage = Invalid export path. please
use valid 'HDFS://', 'AFS://' , 'BOS://', or 'ofs://' path.
这个路径校验导致不能使用s3a协议进行对象存储的导入
### What You Expected?
应该可以正确提交broker load任务
### How to Reproduce?
_No response_
### Anything Else?
_No response_
### Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of
Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] zhouaoe opened a new issue, #13649: [Enhancement] broker update hadoop to higher version because hadoop-huaweicloud which is used by doris-broker now only provide 2.8.3
zhouaoe opened a new issue, #13649:
URL: https://github.com/apache/doris/issues/13649
### Search before asking
- [X] I had searched in the
[issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and
found no similar issues.
### Description
华为 Huaweicloud.jar 只提供2.8.3版本导致broker的hadoop版本无法升级。
Background information：Doris broker support object storage API，like s3
,obs,kfs. I am integrating aliyun-oss into supportion list of broker.
Aliyun-oss is integreated since hadoop-2.9.1（Aliyun-oss is supported naturall
by hadoop-2.9.x, hadoop-2.10.x ,hadoop-3.0.x ,hadoop-3.1.x
,hadoop-3.2.x,hadoop-3.3.x ）, doris-broker use hadoop-2.8.3 now，so i think it
would be better that update hadoop to 2.9.1 version。
Problem: I find a problem that doris-broker can't support hadoop-2.9.x
,3.0.x，3.2.x，3.3.x，because hadoop-huaweicloud which is used by doris only
provide 2.8.3 and 3.1.1 。 I think this problem is a obstructive factor for
doris-borker。
### Solution
Huaweicloud should release all version from 2.9.x to 3.3.x 。
Before it's done，its version should be 2.8.3 but
not ${hadoop.version}
### Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of
Conduct](https://www.apache.org/foundation/policies/conduct)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] nextdreamblue opened a new pull request, #13650: [fix](broker) fix bug when broker load with s3a
nextdreamblue opened a new pull request, #13650: URL: https://github.com/apache/doris/pull/13650 Signed-off-by: nextdreamblue # Proposed changes Issue Number: close #13648 ## Problem summary Describe your changes. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [x] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [x] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [x] No Need 4. Does it need to update dependencies: - [ ] Yes - [x] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [x] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] Gabriel39 merged pull request #13593: [opt](planer) remove unless cast of avg function
Gabriel39 merged PR #13593: URL: https://github.com/apache/doris/pull/13593 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated (f209b7ab6e -> 1741a20689)
This is an automated email from the ASF dual-hosted git repository. gabriellee pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/doris.git from f209b7ab6e [fix](Nereids) add exchange node check between local and global agg in plan translator (#12913) add 1741a20689 [opt](planer) remove unless cast of avg function (#13593) No new revisions were added by this update. Summary of changes: .../java/org/apache/doris/catalog/FunctionSet.java | 81 +- 1 file changed, 19 insertions(+), 62 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] Gabriel39 merged pull request #13587: [Del](vec) Support in predicate in delete condition of or and
Gabriel39 merged PR #13587: URL: https://github.com/apache/doris/pull/13587 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated (1741a20689 -> 2c70b17a47)
This is an automated email from the ASF dual-hosted git repository. gabriellee pushed a change to branch master in repository https://gitbox.apache.org/repos/asf/doris.git from 1741a20689 [opt](planer) remove unless cast of avg function (#13593) add 2c70b17a47 [Del](vec) Support in predicate in delete condition of or and (#13587) No new revisions were added by this update. Summary of changes: be/src/olap/column_predicate.h | 1 + be/src/olap/in_list_predicate.h | 109 ++-- 2 files changed, 107 insertions(+), 3 deletions(-) - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] yixiutt opened a new pull request, #13651: [improvement](publish) add publish version cost
yixiutt opened a new pull request, #13651: URL: https://github.com/apache/doris/pull/13651 # Proposed changes Issue Number: close #xxx ## Problem summary Describe your changes. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [ ] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [ ] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [ ] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] Gabriel39 merged pull request #13588: [Chore](build) add a environment variable DISABLE_JAVA_UDF
Gabriel39 merged PR #13588: URL: https://github.com/apache/doris/pull/13588 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated: [Chore](build) add a environment variable DISABLE_JAVA_UDF (#13588)
This is an automated email from the ASF dual-hosted git repository.
gabriellee pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new bd884d3298 [Chore](build) add a environment variable DISABLE_JAVA_UDF
(#13588)
bd884d3298 is described below
commit bd884d3298836dd8a08722b1c3f3b3499d266266
Author: Pxl
AuthorDate: Tue Oct 25 17:46:56 2022 +0800
[Chore](build) add a environment variable DISABLE_JAVA_UDF (#13588)
---
build.sh | 25 +++--
1 file changed, 11 insertions(+), 14 deletions(-)
diff --git a/build.sh b/build.sh
index 650c030248..3f1af56b52 100755
--- a/build.sh
+++ b/build.sh
@@ -46,14 +46,13 @@ Usage: $0
--auditbuild audit loader. Default ON.
--spark-dppbuild Spark DPP application. Default ON.
--hive-udf build Hive UDF library for Spark Load. Default ON.
- --java-udf build Java UDF library. Default ON.
--cleanclean and build target
-j build Backend parallel
Environment variables:
USE_AVX2If the CPU does not support AVX2 instruction set,
please set USE_AVX2=0. Default is ON.
STRIP_DEBUG_INFOIf set STRIP_DEBUG_INFO=ON, the debug information in
the compiled binaries will be stored separately in the 'be/lib/debug_info'
directory. Default is OFF.
-
+DISABLE_JAVA_UDFIf set DISABLE_JAVA_UDF=ON, we will do not build
binary with java-udf. Default is OFF.
Eg.
$0 build all
$0 --be build Backend
@@ -62,7 +61,7 @@ Usage: $0
$0 --fe --be --cleanclean and build Frontend, Spark
Dpp application and Backend
$0 --spark-dpp build Spark DPP application alone
$0 --broker build Broker
-$0 --be --fe --java-udf build Backend, Frontend, Spark Dpp
application and Java UDF library
+$0 --be --febuild Backend, Frontend, Spark Dpp
application and Java UDF library
USE_AVX2=0 $0 --be build Backend and not using AVX2
instruction.
USE_AVX2=0 STRIP_DEBUG_INFO=ON $0 build all and not using AVX2
instruction, and strip the debug info for Backend
@@ -113,7 +112,6 @@ if ! OPTS="$(getopt \
-l 'audit' \
-l 'meta-tool' \
-l 'spark-dpp' \
--l 'java-udf' \
-l 'hive-udf' \
-l 'clean' \
-l 'help' \
@@ -131,7 +129,7 @@ BUILD_BROKER=0
BUILD_AUDIT=0
BUILD_META_TOOL='OFF'
BUILD_SPARK_DPP=0
-BUILD_JAVA_UDF=0
+BUILD_JAVA_UDF=1
BUILD_HIVE_UDF=0
CLEAN=0
HELP=0
@@ -145,7 +143,6 @@ if [[ "$#" == 1 ]]; then
BUILD_AUDIT=1
BUILD_META_TOOL='OFF'
BUILD_SPARK_DPP=1
-BUILD_JAVA_UDF=1
BUILD_HIVE_UDF=1
CLEAN=0
else
@@ -154,13 +151,11 @@ else
--fe)
BUILD_FE=1
BUILD_SPARK_DPP=1
-BUILD_JAVA_UDF=1
BUILD_HIVE_UDF=1
shift
;;
--be)
BUILD_BE=1
-BUILD_JAVA_UDF=1
shift
;;
--broker)
@@ -179,12 +174,6 @@ else
BUILD_SPARK_DPP=1
shift
;;
---java-udf)
-BUILD_JAVA_UDF=1
-BUILD_FE=1
-BUILD_SPARK_DPP=1
-shift
-;;
--hive-udf)
BUILD_HIVE_UDF=1
shift
@@ -296,6 +285,10 @@ if [[ -z "${USE_DWARF}" ]]; then
USE_DWARF='OFF'
fi
+if [[ -z "${DISABLE_JAVA_UDF}" ]]; then
+DISABLE_JAVA_UDF='OFF'
+fi
+
if [[ -z "${RECORD_COMPILER_SWITCHES}" ]]; then
RECORD_COMPILER_SWITCHES='OFF'
fi
@@ -304,6 +297,10 @@ if [[ "${BUILD_JAVA_UDF}" -eq 1 && "$(uname -s)" ==
'Darwin' ]]; then
BUILD_JAVA_UDF=0
fi
+if [[ "${DISABLE_JAVA_UDF}" == "ON" ]]; then
+BUILD_JAVA_UDF=0
+fi
+
echo "Get params:
BUILD_FE-- ${BUILD_FE}
BUILD_BE-- ${BUILD_BE}
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] dataroaring opened a new pull request, #13652: (jsonreader) release memory of both value and parse allocator
dataroaring opened a new pull request, #13652: URL: https://github.com/apache/doris/pull/13652 # Proposed changes Issue Number: close #xxx ## Problem summary Describe your changes. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [ ] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [ ] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [ ] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris-website] branch master updated: fix
This is an automated email from the ASF dual-hosted git repository. jiafengzheng pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris-website.git The following commit(s) were added to refs/heads/master by this push: new 8e47a9294cf fix 8e47a9294cf is described below commit 8e47a9294cf95871020eb025dc5b5efe7eb7c6d0 Author: jiafeng.zhang AuthorDate: Tue Oct 25 18:05:56 2022 +0800 fix --- blog/Annoucing.md | 2 +- .../cluster-management/elastic-expansion.md| 4 ++-- docs/advanced/alter-table/replace-table.md | 4 ++-- .../import/import-scenes/external-storage-load.md | 2 +- docs/data-operate/import/import-scenes/jdbc-load.md| 4 ++-- docs/data-table/basic-usage.md | 14 +++--- docs/data-table/hit-the-rollup.md | 4 ++-- .../Alter/ALTER-TABLE-REPLACE.md | 2 +- .../Alter/ALTER-TABLE-ROLLUP.md| 4 ++-- .../Data-Definition-Statements/Drop/DROP-DATABASE.md | 2 +- .../Data-Manipulation-Statements/Load/BROKER-LOAD.md | 4 ++-- .../SHOW-ALTER-TABLE-MATERIALIZED-VIEW.md | 2 +- i18n/zh-CN/docusaurus-plugin-content-blog/Annoucing.md | 2 +- .../cluster-management/elastic-expansion.md| 4 ++-- .../current/advanced/alter-table/replace-table.md | 2 +- .../import/import-scenes/external-storage-load.md | 8 .../data-operate/import/import-scenes/jdbc-load.md | 4 ++-- .../current/data-table/basic-usage.md | 18 +- .../current/data-table/hit-the-rollup.md | 2 +- .../Alter/ALTER-TABLE-REPLACE.md | 2 +- .../Alter/ALTER-TABLE-ROLLUP.md| 4 ++-- .../Data-Definition-Statements/Drop/DROP-DATABASE.md | 2 +- .../SHOW-ALTER-TABLE-MATERIALIZED-VIEW.md | 2 +- .../best-practices/star-schema-benchmark.md| 4 ++-- .../extending-doris/udf/user-defined-function.md | 2 +- .../best-practices/star-schema-benchmark.md| 4 ++-- .../extending-doris/udf/user-defined-function.md | 2 +- 27 files changed, 55 insertions(+), 55 deletions(-) diff --git a/blog/Annoucing.md b/blog/Annoucing.md index 77bef3a0853..7ce634eb78d 100644 --- a/blog/Annoucing.md +++ b/blog/Annoucing.md @@ -52,4 +52,4 @@ Once again, we sincerely thank all contributors who participated in the construc **See How to subscribe:** -[https://doris.apache.org/zh-CN/community/subscribe-mail-list](https://doris.apache.org/zh-CN/community/subscribe-mail-list) +[https://doris.apache.org/community/subscribe-mail-list](https://doris.apache.org/community/subscribe-mail-list/) diff --git a/docs/admin-manual/cluster-management/elastic-expansion.md b/docs/admin-manual/cluster-management/elastic-expansion.md index 17cb9111553..ea821f69bbb 100644 --- a/docs/admin-manual/cluster-management/elastic-expansion.md +++ b/docs/admin-manual/cluster-management/elastic-expansion.md @@ -106,7 +106,7 @@ You can also view the BE node through the front-end page connection: ``http://fe All of the above methods require Doris's root user rights. -The expansion and scaling process of BE nodes does not affect the current system operation and the tasks being performed, and does not affect the performance of the current system. Data balancing is done automatically. Depending on the amount of data available in the cluster, the cluster will be restored to load balancing in a few hours to a day. For cluster load, see the [Tablet Load Balancing Document](../maint-monitor/tablet-repair-and-balance.md). +The expansion and scaling process of BE nodes does not affect the current system operation and the tasks being performed, and does not affect the performance of the current system. Data balancing is done automatically. Depending on the amount of data available in the cluster, the cluster will be restored to load balancing in a few hours to a day. For cluster load, see the [Tablet Load Balancing Document](../maint-monitor/tablet-repair-and-balance). ### Add BE nodes @@ -140,7 +140,7 @@ DECOMMISSION clause: > ```CANCEL ALTER SYSTEM DECOMMISSION BACKEND > "be_host:be_heartbeat_service_port";``` > The order was cancelled. When cancelled, the data on the BE will > maintain the current amount of data remaining. Follow-up Doris re-load > balancing -**For expansion and scaling of BE nodes in multi-tenant deployment environments, please refer to the [Multi-tenant Design Document](../multi-tenant.md).** +**For expansion and scaling of BE nodes in multi-tenant deployment environments, please refer to the [Multi-tenant Design Document](../multi-tenant).** ## Broker Expansion and Shrinkage diff --git a/docs/advanced/alter-table/replace-table.md b/docs/advanced/alter-table/replace-table.md index 9aa755710f0..13ad8d5cc4d 100644 --- a
[GitHub] [doris] github-actions[bot] commented on pull request #13607: [fix](thirdparty) g++-11: error: unrecognized command-line option '-m…
github-actions[bot] commented on PR #13607: URL: https://github.com/apache/doris/pull/13607#issuecomment-1290333606 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #13607: [fix](thirdparty) g++-11: error: unrecognized command-line option '-m…
github-actions[bot] commented on PR #13607: URL: https://github.com/apache/doris/pull/13607#issuecomment-1290333673 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris-website] branch master updated: fix
This is an automated email from the ASF dual-hosted git repository. jiafengzheng pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris-website.git The following commit(s) were added to refs/heads/master by this push: new 4f58e9d9f3e fix 4f58e9d9f3e is described below commit 4f58e9d9f3ec4009a57d7d6fae2a91113b535968 Author: jiafeng.zhang AuthorDate: Tue Oct 25 18:33:47 2022 +0800 fix --- docs/admin-manual/cluster-management/elastic-expansion.md | 2 +- docs/admin-manual/config/be-config.md | 2 +- docs/advanced/alter-table/schema-change.md| 2 +- docs/data-operate/import/import-scenes/external-storage-load.md | 2 +- docs/data-operate/import/import-scenes/jdbc-load.md | 4 ++-- docs/data-table/basic-usage.md| 4 ++-- .../Data-Definition-Statements/Alter/ALTER-TABLE-PARTITION.md | 2 +- .../Data-Definition-Statements/Alter/ALTER-TABLE-ROLLUP.md| 4 ++-- .../current/admin-manual/cluster-management/elastic-expansion.md | 2 +- .../current/admin-manual/config/be-config.md | 2 +- .../Data-Definition-Statements/Alter/ALTER-TABLE-PARTITION.md | 2 +- .../Data-Definition-Statements/Alter/ALTER-TABLE-ROLLUP.md| 4 ++-- 12 files changed, 16 insertions(+), 16 deletions(-) diff --git a/docs/admin-manual/cluster-management/elastic-expansion.md b/docs/admin-manual/cluster-management/elastic-expansion.md index ea821f69bbb..e8fd4525298 100644 --- a/docs/admin-manual/cluster-management/elastic-expansion.md +++ b/docs/admin-manual/cluster-management/elastic-expansion.md @@ -106,7 +106,7 @@ You can also view the BE node through the front-end page connection: ``http://fe All of the above methods require Doris's root user rights. -The expansion and scaling process of BE nodes does not affect the current system operation and the tasks being performed, and does not affect the performance of the current system. Data balancing is done automatically. Depending on the amount of data available in the cluster, the cluster will be restored to load balancing in a few hours to a day. For cluster load, see the [Tablet Load Balancing Document](../maint-monitor/tablet-repair-and-balance). +The expansion and scaling process of BE nodes does not affect the current system operation and the tasks being performed, and does not affect the performance of the current system. Data balancing is done automatically. Depending on the amount of data available in the cluster, the cluster will be restored to load balancing in a few hours to a day. For cluster load, see the [Tablet Load Balancing Document](../../maint-monitor/tablet-repair-and-balance). ### Add BE nodes diff --git a/docs/admin-manual/config/be-config.md b/docs/admin-manual/config/be-config.md index e27118a83ed..90e9d044208 100644 --- a/docs/admin-manual/config/be-config.md +++ b/docs/admin-manual/config/be-config.md @@ -450,7 +450,7 @@ Cgroups assigned to doris ### `doris_max_scan_key_num` * Type: int -* Description: Used to limit the maximum number of scan keys that a scan node can split in a query request. When a conditional query request reaches the scan node, the scan node will try to split the conditions related to the key column in the query condition into multiple scan key ranges. After that, these scan key ranges will be assigned to multiple scanner threads for data scanning. A larger value usually means that more scanner threads can be used to increase the parallelism of the s [...] +* Description: Used to limit the maximum number of scan keys that a scan node can split in a query request. When a conditional query request reaches the scan node, the scan node will try to split the conditions related to the key column in the query condition into multiple scan key ranges. After that, these scan key ranges will be assigned to multiple scanner threads for data scanning. A larger value usually means that more scanner threads can be used to increase the parallelism of the s [...] * Default value: 1024 When the concurrency cannot be improved in high concurrency scenarios, try to reduce this value and observe the impact. diff --git a/docs/advanced/alter-table/schema-change.md b/docs/advanced/alter-table/schema-change.md index 3cf43598ae7..23f9fa77084 100644 --- a/docs/advanced/alter-table/schema-change.md +++ b/docs/advanced/alter-table/schema-change.md @@ -282,5 +282,5 @@ SHOW ALTER TABLE COLUMN\G; ## More Help -For more detailed syntax and best practices used by Schema Change, see [ALTER TABLE COLUMN](../../../sql-manual/sql-reference/Data-Definition-Statements/Alter/ALTER-TABLE-COLUMN.md ) command manual, you can also enter `HELP ALTER TABLE COLUMN` in the MySql client command line for more help information. +For more detailed syntax and best practices used by Schema Change, see [AL
[GitHub] [doris] jacktengg opened a new issue, #13653: [Enhancement]
jacktengg opened a new issue, #13653: URL: https://github.com/apache/doris/issues/13653 ### Search before asking - [X] I had searched in the [issues](https://github.com/apache/incubator-doris/issues?q=is%3Aissue) and found no similar issues. ### Description In vhash join node, if there are a lot of unique build side rows, hash table resize will alloc big consecutive memory and alloc will easily fail: ``` #0 0x7fabaebd0207 in raise () from /lib64/libc.so.6 #1 0x7fabaebd18f8 in abort () from /lib64/libc.so.6 #2 0x55ef7322b1ec in __gnu_cxx::__verbose_terminate_handler() [clone .cold] () #3 0x55ef7758dfa6 in __cxxabiv1::__terminate(void (*)()) () #4 0x55ef7758e011 in std::terminate() () #5 0x55ef7758e165 in __cxa_throw () #6 0x55ef7307a95b in doris::vectorized::throwFromErrno (s=..., code=code@entry=54, e=e@entry=12) at /mnt/disk1/dafeng/build/ldb/include/c++/11/optional:361 #7 0x55ef74209d49 in Allocator::alloc_no_track () at /mnt/disk1/dafeng/build/ldb/include/c++/11/bits/char_traits.h:371 #8 0x55ef74209f24 in alloc (alignment=0, size=2147483648, this=) at /mnt/disk1/dafeng/binary/1.1.2-lts/doris/be/src/vec/common/allocator.h:103 #9 Allocator::realloc () at /mnt/disk1/dafeng/binary/1.1.2-lts/doris/be/src/vec/common/allocator.h:171 #10 0x55ef7420c98a in HashTable, HashTableNoState>, HashCRC32, HashTableGrower<10ul>, Allocator >::resize (this=0x55f196b423b0, for_num_elems=26787725, for_buf_size=) at /mnt/disk1/dafeng/binary/1.1.2-lts/doris/be/src/vec/common/hash_table/hash_table.h:932 #11 0x55ef743086a8 in expanse_for_add_elem (num_elem=, this=0x55f196b423b0) at /mnt/disk1/dafeng/binary/1.1.2-lts/doris/be/src/vec/common/hash_table/hash_table.h:250 #12 doris::vectorized::ProcessHashTableBuild, true, false>::operator() (this=this@entry=0x7faa28a60d00, hash_table_ctx=..., null_map=0x7faa28a61600, has_runtime_filter=has_runtime_filter@entry=false) at /mnt/disk1/dafeng/binary/1.1.2-lts/doris/be/src/vec/exec/join/vhash_join_node.cpp:73 #13 0x55ef7429512a in _ZZN5doris10vectorized12HashJoinNode20_process_build_blockEPNS_12RuntimeStateERNS0_5BlockEhENKUlOT_E0_clIRNS0_27PrimaryTypeHashTableContextImDaS7_ () at /mnt/disk1/dafeng/binary/1.1.2-lts/doris/be/src/vec/exec/join/vhash_join_node.cpp:43 #14 0x55ef742a058e in __do_visit, doris::vectorized::HashJoinNode::_process_build_block(doris::RuntimeState*, doris::vectorized::Block&, uint8_t)::, std::variant, doris::vectorized::PrimaryTypeHashTableContext, doris::vectorized::PrimaryTypeHashTableContext, doris::vectorized::PrimaryTypeHashTableContext, doris::vectorized::PrimaryTypeHashTableContext, doris::vectorized::PrimaryTypeHashTableContext, doris::vectorized::FixedKeyHashTableContext, doris::vectorized::FixedKeyHashTableContext, doris::vectorized::FixedKeyHashTableContext, doris::vectorized::FixedKeyHashTableContext, doris::vectorized::FixedKeyHashTableContext, doris::vectorized::FixedKeyHashTableContext >&> ( __visitor=) at /mnt/disk1/dafeng/build/ldb/include/c++/11/variant:1708 #15 visit, std::variant, doris::vectorized::PrimaryTypeHashTableContext, doris::vectorized::PrimaryTypeHashTableContext, doris::vectorized::PrimaryTypeHashTableContext, doris::vectorized::PrimaryTypeHashTableContext, doris::vectorized::PrimaryTypeHashTableContext, doris::vectorized::FixedKeyHashTableContext, doris::vectorized::FixedKeyHashTableContext, doris::vectorized::FixedKeyHashTableContext, doris::vectorized::FixedKeyHashTableContext, doris::vectorized::FixedKeyHashTableContext, doris::vectorized::FixedKeyHashTableContext >&> (__visitor=) at /mnt/disk1/dafeng/build/ldb/include/c++/11/variant:1764 #16 doris::vectorized::HashJoinNode::_process_build_block(doris::RuntimeState*, doris::vectorized::Block&, unsigned char) () ``` We can consider using two level hash table in vhash join node, to reduce the possibility of alloc fail when doing hash table resize. ### Solution _No response_ ### Are you willing to submit PR? - [X] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] jacktengg opened a new pull request, #13654: [improvement](hashjoin) support two-level hash table in hash join
jacktengg opened a new pull request, #13654: URL: https://github.com/apache/doris/pull/13654 # Proposed changes Issue Number: close #13653 ## Problem summary Describe your changes. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [ ] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [ ] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [ ] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] morningman commented on a diff in pull request #13646: [enhancement](metric)Add metric for exec_state prepare function
morningman commented on code in PR #13646:
URL: https://github.com/apache/doris/pull/13646#discussion_r1004315541
##
docs/zh-CN/docs/admin-manual/maint-monitor/monitor-metrics/metrics.md:
##
@@ -210,6 +210,7 @@ curl http://be_host:webserver_port/metrics?type=json
|| `{status="skip",type="report_all_tablets"}` | Num | xxx 类型任务被跳过执行的次数的累计值 | |
|`doris_be_fragment_endpoint_count`| | Num| 同 | FIXME: 同
`doris_be_data_stream_receiver_count` 数目。并且向量化引擎缺失 |
|`doris_be_fragment_request_duration_us`| | 微秒| 所有 fragment intance 的执行时间累计 |
通过斜率观测 instance 的执行耗时 | P0 |
+|`doris_be_fragment_prepare_duration_ns`| | 纳秒| 单个 fragment intance
的prepare时间累计 | 通过斜率观测 instance 的prepare耗时 | P0 |
Review Comment:
How about use same unit as `doris_be_fragment_request_duration_us`?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] starocean999 opened a new pull request, #13655: [fix](joinoutput all value from no-null side of outer join
starocean999 opened a new pull request, #13655: URL: https://github.com/apache/doris/pull/13655 # Proposed changes Issue Number: close #xxx ## Problem summary 1. move _convert_block_to_null right after _do_evaluate function 2. output all values from no-null side of outer join ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [ ] No - [ ] I don't know 3. Has unit tests been added: - [ ] Yes - [ ] No - [ ] No Need 4. Has document been added or modified: - [ ] Yes - [ ] No - [ ] No Need 5. Does it need to update dependencies: - [ ] Yes - [ ] No 6. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [ ] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] morrySnow closed issue #12303: [Feature] Support rollup and materialized view in nereids planner.
morrySnow closed issue #12303: [Feature] Support rollup and materialized view in nereids planner. URL: https://github.com/apache/doris/issues/12303 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] morrySnow merged pull request #13416: [Feature](Nereids) Support materialized index selection.
morrySnow merged PR #13416: URL: https://github.com/apache/doris/pull/13416 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated: [feature](Nereids) support materialized index selection (#13416)
This is an automated email from the ASF dual-hosted git repository. morrysnow pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris.git The following commit(s) were added to refs/heads/master by this push: new d6c3470c8d [feature](Nereids) support materialized index selection (#13416) d6c3470c8d is described below commit d6c3470c8db1cc4e91a00165ce23cc17a3f93dcd Author: Shuo Wang AuthorDate: Tue Oct 25 19:25:58 2022 +0800 [feature](Nereids) support materialized index selection (#13416) This PR unified the selection of rollup index and materialized view index into uniform logic, which is called selecting materialized index. Main steps: ### Find candidate indexes 1. When aggregate is present, it's handled in `SelectMaterializedIndexWithAggregate`. The base index and indexes that could use pre-aggregation should be used. The pre-aggregation status is determined by aggregation function, grouping expression, and pushdown predicates. 2. When aggregate is not on top of scan node, it's handled in `SelectMaterializedIndexWithoutAggregate`. The base index and indexes that have all the key columns could be used. ### Filter and order the candidate indexes 1. filter indexes that contain all the required output scan columns. 2. filter indexes that could match prefix index most. 3. order the result index by row count, column count, and index id. --- .../glue/translator/PhysicalPlanTranslator.java| 12 +- .../jobs/batch/NereidsRewriteJobExecutor.java |8 +- .../org/apache/doris/nereids/rules/RuleType.java | 16 +- .../mv/AbstractSelectMaterializedIndexRule.java| 273 ++ ...a => SelectMaterializedIndexWithAggregate.java} | 323 ++ .../SelectMaterializedIndexWithoutAggregate.java | 147 +++ .../rules/mv/SelectRollupWithoutAggregate.java | 60 -- .../trees/plans/logical/LogicalOlapScan.java | 34 +- .../org/apache/doris/planner/OlapScanNode.java |5 + .../rules/mv/BaseMaterializedIndexSelectTest.java | 51 + .../doris/nereids/rules/mv/SelectMvIndexTest.java | 1035 ...tRollupTest.java => SelectRollupIndexTest.java} | 191 ++-- .../apache/doris/utframe/TestWithFeService.java| 53 +- 13 files changed, 1798 insertions(+), 410 deletions(-) diff --git a/fe/fe-core/src/main/java/org/apache/doris/nereids/glue/translator/PhysicalPlanTranslator.java b/fe/fe-core/src/main/java/org/apache/doris/nereids/glue/translator/PhysicalPlanTranslator.java index 8fc0ba7f92..292656b518 100644 --- a/fe/fe-core/src/main/java/org/apache/doris/nereids/glue/translator/PhysicalPlanTranslator.java +++ b/fe/fe-core/src/main/java/org/apache/doris/nereids/glue/translator/PhysicalPlanTranslator.java @@ -33,7 +33,6 @@ import org.apache.doris.analysis.TupleId; import org.apache.doris.catalog.OlapTable; import org.apache.doris.catalog.Table; import org.apache.doris.common.Pair; -import org.apache.doris.nereids.exceptions.AnalysisException; import org.apache.doris.nereids.properties.DistributionSpecHash; import org.apache.doris.nereids.properties.DistributionSpecHash.ShuffleType; import org.apache.doris.nereids.properties.OrderKey; @@ -320,22 +319,13 @@ public class PhysicalPlanTranslator extends DefaultPlanVisitorhttp://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, +// software distributed under the License is distributed on an +// "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +// KIND, either express or implied. See the License for the +// specific language governing permissions and limitations +// under the License. + +package org.apache.doris.nereids.rules.mv; + +import org.apache.doris.catalog.Column; +import org.apache.doris.catalog.MaterializedIndex; +import org.apache.doris.catalog.OlapTable; +import org.apache.doris.nereids.trees.expressions.ComparisonPredicate; +import org.apache.doris.nereids.trees.expressions.EqualTo; +import org.apache.doris.nereids.trees.expressions.ExprId; +import org.apache.doris.nereids.trees.expressions.Expression; +import org.apache.doris.nereids.trees.expressions.InPredicate; +import org.apache.doris.nereids.trees.expressions.NamedExpression; +import org.apache.doris.nereids.trees.expressions.NullSafeEqual; +import org.apache.doris.nereids.trees.expressions.Slot; +import org.apache.doris.nereids.trees.expressions.literal.Literal; +import org.apache.doris.nereids.trees.expressions.visitor.ExpressionVisitor; +import org.apache.doris.nereids.trees.plans.logical.LogicalOlapScan; +import org.apache.doris.nereids.util.ExpressionUtils; + +import com.google.common.collect.ImmutableSet; +import com.google.common.collect.Lists; + +import java.util.Comparator; +import java.util.List; +import java.util.Map; +import java.util.Optional; +import java.util.Set; +import java.util.TreeMap; +import java.util.function.Function; +import java.util.stream.Collectors; +i
[doris-website] branch master updated: fix
This is an automated email from the ASF dual-hosted git repository. jiafengzheng pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris-website.git The following commit(s) were added to refs/heads/master by this push: new ae4125aa154 fix ae4125aa154 is described below commit ae4125aa1545ea7e94b9a104fa1e29f8d452fb77 Author: jiafeng.zhang AuthorDate: Tue Oct 25 19:27:57 2022 +0800 fix --- docs/admin-manual/cluster-management/elastic-expansion.md | 2 +- docs/admin-manual/cluster-management/upgrade.md | 2 +- docs/admin-manual/http-actions/fe/table-schema-action.md | 2 +- docs/admin-manual/maint-monitor/disk-capacity.md | 2 +- docs/advanced/alter-table/schema-change.md| 4 ++-- docs/data-operate/import/import-scenes/external-storage-load.md | 8 docs/data-operate/import/import-scenes/jdbc-load.md | 4 ++-- docs/data-table/basic-usage.md| 6 +++--- docs/ecosystem/external-table/hive-bitmap-udf.md | 4 ++-- .../Data-Definition-Statements/Alter/ALTER-TABLE-PARTITION.md | 2 +- .../Data-Manipulation-Statements/Load/STREAM-LOAD.md | 8 .../current/admin-manual/cluster-management/elastic-expansion.md | 2 +- .../current/admin-manual/cluster-management/upgrade.md| 2 +- .../current/admin-manual/http-actions/fe/table-schema-action.md | 2 +- .../data-operate/import/import-scenes/external-storage-load.md| 8 .../current/ecosystem/external-table/hive-bitmap-udf.md | 4 ++-- .../administrator-guide/operation/metadata-operation.md | 4 ++-- .../administrator-guide/alter-table/alter-table-rollup.md | 4 ++-- .../administrator-guide/operation/metadata-operation.md | 4 ++-- versioned_docs/version-0.15/extending-doris/logstash.md | 2 +- .../version-0.15/extending-doris/udf/user-defined-function.md | 2 +- 21 files changed, 39 insertions(+), 39 deletions(-) diff --git a/docs/admin-manual/cluster-management/elastic-expansion.md b/docs/admin-manual/cluster-management/elastic-expansion.md index e8fd4525298..b10b7f57c68 100644 --- a/docs/admin-manual/cluster-management/elastic-expansion.md +++ b/docs/admin-manual/cluster-management/elastic-expansion.md @@ -140,7 +140,7 @@ DECOMMISSION clause: > ```CANCEL ALTER SYSTEM DECOMMISSION BACKEND > "be_host:be_heartbeat_service_port";``` > The order was cancelled. When cancelled, the data on the BE will > maintain the current amount of data remaining. Follow-up Doris re-load > balancing -**For expansion and scaling of BE nodes in multi-tenant deployment environments, please refer to the [Multi-tenant Design Document](../multi-tenant).** +**For expansion and scaling of BE nodes in multi-tenant deployment environments, please refer to the [Multi-tenant Design Document](../../multi-tenant).** ## Broker Expansion and Shrinkage diff --git a/docs/admin-manual/cluster-management/upgrade.md b/docs/admin-manual/cluster-management/upgrade.md index a04a69a5394..99b7cbea5dd 100644 --- a/docs/admin-manual/cluster-management/upgrade.md +++ b/docs/admin-manual/cluster-management/upgrade.md @@ -113,7 +113,7 @@ Therefore, it is recommended to upgrade some nodes and observe the business oper **Illegal rollback operation may cause data loss and damage.** ## Documentation -1. [Doris metadata design document](../../../community/design/metadata-design) +1. [Doris metadata design document](/community/design/metadata-design) 2. [Metadata Operations and Maintenance](../../admin-manual/maint-monitor/metadata-operation.md) 3. [Data replica management](../../admin-manual/maint-monitor/tablet-repair-and-balance.md) 4. [Installation Deployment Document](../../install/install-deploy.md) diff --git a/docs/admin-manual/http-actions/fe/table-schema-action.md b/docs/admin-manual/http-actions/fe/table-schema-action.md index c2f7cc7..122b7ed06d9 100644 --- a/docs/admin-manual/http-actions/fe/table-schema-action.md +++ b/docs/admin-manual/http-actions/fe/table-schema-action.md @@ -97,7 +97,7 @@ None "count": 0 } ``` -Note: The difference is that the `http` method returns more `aggregation_type` fields than the `http v2` method. The `http v2` is enabled by setting `enable_http_server_v2`. For detailed parameter descriptions, see [fe parameter settings](../admin-manual/config/fe-config) +Note: The difference is that the `http` method returns more `aggregation_type` fields than the `http v2` method. The `http v2` is enabled by setting `enable_http_server_v2`. For detailed parameter descriptions, see [fe parameter settings](../../../admin-manual/config/fe-config) ## Examples diff --git a/docs/admin-manual/maint-monitor/disk-capacity.md b/docs/admin-manual/maint-monitor/disk-capacity.md index 59412f846b2..1f377faa897 100644 --- a/docs/admin-ma
[GitHub] [doris] HappenLee opened a new pull request, #13656: [opt](tpch) opt the q9 of tpch to change instance to 8
HappenLee opened a new pull request, #13656: URL: https://github.com/apache/doris/pull/13656 # Proposed changes After change instance num, Q9 3.7s -> 3.0s ## Problem summary Describe your changes. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [ ] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [ ] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [ ] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] Gabriel39 opened a new pull request, #13657: [Bug](jdbc) Fix memory leak for JDBC datasource
Gabriel39 opened a new pull request, #13657: URL: https://github.com/apache/doris/pull/13657 # Proposed changes Issue Number: close #xxx ## Problem summary Describe your changes. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [ ] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [ ] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [ ] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris-website] branch master updated: fix
This is an automated email from the ASF dual-hosted git repository. jiafengzheng pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris-website.git The following commit(s) were added to refs/heads/master by this push: new 401d4f87f6d fix 401d4f87f6d is described below commit 401d4f87f6d06f051bc89b57c0c26538f693108d Author: jiafeng.zhang AuthorDate: Tue Oct 25 20:11:52 2022 +0800 fix --- docs/admin-manual/http-actions/fe/table-schema-action.md| 2 +- docs/admin-manual/maint-monitor/disk-capacity.md| 2 +- docs/advanced/alter-table/schema-change.md | 2 +- .../sql-reference/Data-Manipulation-Statements/Load/STREAM-LOAD.md | 2 +- .../current/admin-manual/http-actions/fe/table-schema-action.md | 2 +- .../sql-reference/Show-Statements/SHOW-ALTER-TABLE-MATERIALIZED-VIEW.md | 2 +- .../version-0.15/best-practices/star-schema-benchmark.md| 2 +- versioned_docs/version-0.15/best-practices/star-schema-benchmark.md | 2 +- 8 files changed, 8 insertions(+), 8 deletions(-) diff --git a/docs/admin-manual/http-actions/fe/table-schema-action.md b/docs/admin-manual/http-actions/fe/table-schema-action.md index 122b7ed06d9..3af0088b31f 100644 --- a/docs/admin-manual/http-actions/fe/table-schema-action.md +++ b/docs/admin-manual/http-actions/fe/table-schema-action.md @@ -97,7 +97,7 @@ None "count": 0 } ``` -Note: The difference is that the `http` method returns more `aggregation_type` fields than the `http v2` method. The `http v2` is enabled by setting `enable_http_server_v2`. For detailed parameter descriptions, see [fe parameter settings](../../../admin-manual/config/fe-config) +Note: The difference is that the `http` method returns more `aggregation_type` fields than the `http v2` method. The `http v2` is enabled by setting `enable_http_server_v2`. For detailed parameter descriptions, see [fe parameter settings](../../../../admin-manual/config/fe-config) ## Examples diff --git a/docs/admin-manual/maint-monitor/disk-capacity.md b/docs/admin-manual/maint-monitor/disk-capacity.md index 1f377faa897..e84ca8c1f9a 100644 --- a/docs/admin-manual/maint-monitor/disk-capacity.md +++ b/docs/admin-manual/maint-monitor/disk-capacity.md @@ -162,6 +162,6 @@ When the disk capacity is higher than High Watermark or even Flood Stage, many o ```rm -rf data/0/12345/``` -* Delete tablet metadata (refer to [Tablet metadata management tool](../../tablet-meta-tool)） +* Delete tablet metadata (refer to [Tablet metadata management tool](../tablet-meta-tool)） ```./lib/meta_tool --operation=delete_header --root_path=/path/to/root_path --tablet_id=12345 --schema_hash= 35278``` diff --git a/docs/advanced/alter-table/schema-change.md b/docs/advanced/alter-table/schema-change.md index bf0fa481c95..809015c4dd0 100644 --- a/docs/advanced/alter-table/schema-change.md +++ b/docs/advanced/alter-table/schema-change.md @@ -68,7 +68,7 @@ The basic process of executing a Schema Change is to generate a copy of the inde Before starting the conversion of historical data, Doris will obtain a latest transaction ID. And wait for all import transactions before this Transaction ID to complete. This Transaction ID becomes a watershed. This means that Doris guarantees that all import tasks after the watershed will generate data for both the original Index and the new Index. In this way, when the historical data conversion is completed, the data in the new Index can be guaranteed to be complete. ## Create Job -The specific syntax for creating a Schema Change can be found in the help [ALTER TABLE COLUMN](../../sql-manual/sql-reference/Data-Definition-Statements/Alter/ALTER-TABLE-COLUMN) for the description of the Schema Change section . +The specific syntax for creating a Schema Change can be found in the help [ALTER TABLE COLUMN](../../../sql-manual/sql-reference/Data-Definition-Statements/Alter/ALTER-TABLE-COLUMN) for the description of the Schema Change section . The creation of Schema Change is an asynchronous process. After the job is submitted successfully, the user needs to view the job progress through the `SHOW ALTER TABLE COLUMN` command. ## View Job diff --git a/docs/sql-manual/sql-reference/Data-Manipulation-Statements/Load/STREAM-LOAD.md b/docs/sql-manual/sql-reference/Data-Manipulation-Statements/Load/STREAM-LOAD.md index 210c4706c6f..897be9f24da 100644 --- a/docs/sql-manual/sql-reference/Data-Manipulation-Statements/Load/STREAM-LOAD.md +++ b/docs/sql-manual/sql-reference/Data-Manipulation-Statements/Load/STREAM-LOAD.md @@ -432,7 +432,7 @@ curl --location-trusted -u root -H "columns: k1,k2,source_sequence,v1,v2" -H "fu 7. Strict Mode - The `strict_mode` attribute is used to set whether the import task runs in strict mode. The format affects the results of column mapping, transformation, and filtering. For a de
[GitHub] [doris] github-actions[bot] commented on pull request #13647: [enhancement](chore) build fe-common when build java-udf
github-actions[bot] commented on PR #13647:
URL: https://github.com/apache/doris/pull/13647#issuecomment-1290501846
`sh-checker report`
To get the full details, please check in the
[job]("https://github.com/apache/doris/actions/runs/3319291572";) output.
shellcheck errors
```
'shellcheck ' returned error 1 finding the following syntactical issues:
--
In build.sh line 459:
"${MVN_CMD}" package -pl ${FE_MODULES:+${FE_MODULES}} ${MVN_PARAMS}
^---^
SC2086 (info): Double quote to prevent globbing and word splitting.
Did you mean:
"${MVN_CMD}" package -pl ${FE_MODULES:+${FE_MODULES}} "${MVN_PARAMS}"
For more information:
https://www.shellcheck.net/wiki/SC2086 -- Double quote to prevent globbing
...
--
You can address the above issues in one of three ways:
1. Manually correct the issue in the offending shell script;
2. Disable specific issues by adding the comment:
# shellcheck disable=
above the line that contains the issue, where is the error code;
3. Add '-e ' to the SHELLCHECK_OPTS setting in your .yml action file.
```
shfmt errors
```
'shfmt ' returned error 1 finding the following formatting issues:
--
--- build.sh.orig
+++ build.sh
@@ -454,7 +454,7 @@
fi
MVN_PARAMS="-DskipTests"
if [[ "$(uname -s)" = 'Darwin' ]]; then
- MVN_PARAMS="${MVN_PARAMS} -Dos.arch=x86_64"
+MVN_PARAMS="${MVN_PARAMS} -Dos.arch=x86_64"
fi
"${MVN_CMD}" package -pl ${FE_MODULES:+${FE_MODULES}} ${MVN_PARAMS}
cd "${DORIS_HOME}"
--
You can reformat the above files to meet shfmt's requirements by typing:
shfmt -w filename
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] zhangstar333 commented on a diff in pull request #13657: [Bug](jdbc) Fix memory leak for JDBC datasource
zhangstar333 commented on code in PR #13657:
URL: https://github.com/apache/doris/pull/13657#discussion_r1004446614
##
be/src/vec/exec/vjdbc_connector.cpp:
##
@@ -41,22 +42,26 @@ const char* JDBC_EXECUTOR_TRANSACTION_SIGNATURE = "()V";
JdbcConnector::JdbcConnector(const JdbcConnectorParam& param)
: TableConnector(param.tuple_desc, param.query_string),
_conn_param(param) {}
-JdbcConnector::~JdbcConnector() {
+Status JdbcConnector::close() {
if (!_is_open) {
-return;
+return Status::OK();
}
if (_is_in_transaction) {
-abort_trans();
+RETURN_IF_ERROR(abort_trans());
}
JNIEnv* env;
-Status status;
-RETURN_IF_STATUS_ERROR(status, JniUtil::GetJNIEnv(&env));
+RETURN_IF_ERROR(JniUtil::GetJNIEnv(&env));
+env->DeleteGlobalRef(_executor_clazz);
+env->DeleteGlobalRef(_executor_list_clazz);
+env->DeleteGlobalRef(_executor_object_clazz);
+env->DeleteGlobalRef(_executor_string_clazz);
Review Comment:
could add those variables: jclass _executor_##RETURN_TYPE##_clazz;
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] dataalive commented on issue #13584: [Bug] Be crash at mem_tracker_task_pool.cpp
dataalive commented on issue #13584: URL: https://github.com/apache/doris/issues/13584#issuecomment-1290506239 what(): Resource temporarily unavailable is there any more error msg? or try `ulimit -a` to check fd limit ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] dataalive commented on issue #13640: [Bug] [Bug] Sql Exec Failed Raise IndexOutOfBoundsException
dataalive commented on issue #13640: URL: https://github.com/apache/doris/issues/13640#issuecomment-1290513501 this bug has been fixed by https://github.com/apache/doris/pull/13300 and pull the latest code of branch 1.1-lts to fix -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #13647: [enhancement](chore) build fe-common when build java-udf
github-actions[bot] commented on PR #13647:
URL: https://github.com/apache/doris/pull/13647#issuecomment-1290519787
`sh-checker report`
To get the full details, please check in the
[job]("https://github.com/apache/doris/actions/runs/3319388203";) output.
shellcheck errors
```
'shellcheck ' returned error 1 finding the following syntactical issues:
--
In build.sh line 459:
"${MVN_CMD}" package -pl ${FE_MODULES:+${FE_MODULES}} ${MVN_PARAMS}
^---^
SC2086 (info): Double quote to prevent globbing and word splitting.
Did you mean:
"${MVN_CMD}" package -pl ${FE_MODULES:+${FE_MODULES}} "${MVN_PARAMS}"
For more information:
https://www.shellcheck.net/wiki/SC2086 -- Double quote to prevent globbing
...
--
You can address the above issues in one of three ways:
1. Manually correct the issue in the offending shell script;
2. Disable specific issues by adding the comment:
# shellcheck disable=
above the line that contains the issue, where is the error code;
3. Add '-e ' to the SHELLCHECK_OPTS setting in your .yml action file.
```
shfmt errors
```
'shfmt ' returned error 1 finding the following formatting issues:
--
--- build.sh.orig
+++ build.sh
@@ -454,7 +454,7 @@
fi
MVN_PARAMS="-DskipTests"
if [[ "$(uname -s)" = 'Darwin' ]]; then
- MVN_PARAMS="${MVN_PARAMS} -Dos.arch=x86_64"
+MVN_PARAMS="${MVN_PARAMS} -Dos.arch=x86_64"
fi
"${MVN_CMD}" package -pl ${FE_MODULES:+${FE_MODULES}} ${MVN_PARAMS}
cd "${DORIS_HOME}"
--
You can reformat the above files to meet shfmt's requirements by typing:
shfmt -w filename
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] team-317 commented on issue #7582: [Roadmap] Enhanced system observability
team-317 commented on issue #7582: URL: https://github.com/apache/doris/issues/7582#issuecomment-1290523589 I am interested in this task, can you assign it to me? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] yuanyuan8983 commented on issue #13503: [Bug] 没有任何SYNC同步JOB，但是日志一直报CanalSyncDataReceiver Connection refused
yuanyuan8983 commented on issue #13503: URL: https://github.com/apache/doris/issues/13503#issuecomment-1290523679 我邮件回复你了，加我就行哈 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] Wilson-BT closed issue #13640: [Bug] [Bug] Sql Exec Failed Raise IndexOutOfBoundsException
Wilson-BT closed issue #13640: [Bug] [Bug] Sql Exec Failed Raise IndexOutOfBoundsException URL: https://github.com/apache/doris/issues/13640 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] Wilson-BT closed issue #13584: [Bug] Be crash at mem_tracker_task_pool.cpp
Wilson-BT closed issue #13584: [Bug] Be crash at mem_tracker_task_pool.cpp URL: https://github.com/apache/doris/issues/13584 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] siriume closed issue #13610: [Bug] doris be core dump
siriume closed issue #13610: [Bug] doris be core dump URL: https://github.com/apache/doris/issues/13610 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] zhangstar333 commented on a diff in pull request #13657: [Bug](jdbc) Fix memory leak for JDBC datasource
zhangstar333 commented on code in PR #13657:
URL: https://github.com/apache/doris/pull/13657#discussion_r1004482088
##
be/src/vec/exec/vjdbc_connector.cpp:
##
@@ -121,7 +128,7 @@ Status JdbcConnector::query() {
RETURN_IF_ERROR(JniUtil::GetJNIEnv(&env));
jstring query_sql = env->NewStringUTF(_sql_str.c_str());
jint colunm_count = env->CallNonvirtualIntMethod(_executor_obj,
_executor_clazz,
- _executor_query_id,
query_sql);
+ _executor_read_id,
query_sql);
Review Comment:
I see register read function is `int read() `?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] hello-stephen commented on pull request #13645: [fix](statistics) fix cross join statistics exception
hello-stephen commented on PR #13645: URL: https://github.com/apache/doris/pull/13645#issuecomment-1290550920 TeamCity pipeline, clickbench performance test result: the sum of best hot time: 37.46 seconds load time: 560 seconds storage size: 17154887219 Bytes https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20221025132734_clickbench_pr_33949.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #13647: [enhancement](chore) build fe-common when build java-udf
github-actions[bot] commented on PR #13647: URL: https://github.com/apache/doris/pull/13647#issuecomment-1290563393 PR approved by at least one committer and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] github-actions[bot] commented on pull request #13647: [enhancement](chore) build fe-common when build java-udf
github-actions[bot] commented on PR #13647: URL: https://github.com/apache/doris/pull/13647#issuecomment-1290563480 PR approved by anyone and no changes requested. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] adonis0147 merged pull request #13607: [fix](thirdparty) g++-11: error: unrecognized command-line option '-m…
adonis0147 merged PR #13607: URL: https://github.com/apache/doris/pull/13607 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated: [fix](thirdparty) g++-11: error: unrecognized command-line option '-m… (#13607)
This is an automated email from the ASF dual-hosted git repository.
adonisling pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 4db18c139c [fix](thirdparty) g++-11: error: unrecognized command-line
option '-m… (#13607)
4db18c139c is described below
commit 4db18c139c668fdb6fee22fbfb0f24c7097f2442
Author: Dongyang Li
AuthorDate: Tue Oct 25 21:35:31 2022 +0800
[fix](thirdparty) g++-11: error: unrecognized command-line option '-m…
(#13607)
When compiling third-party hdfs on aarch64 machine,
got error: g++-11: error: unrecognized command-line option '-msse4.2'
---
thirdparty/build-thirdparty.sh | 7 ++-
1 file changed, 6 insertions(+), 1 deletion(-)
diff --git a/thirdparty/build-thirdparty.sh b/thirdparty/build-thirdparty.sh
index c7906546c5..e404757a2b 100755
--- a/thirdparty/build-thirdparty.sh
+++ b/thirdparty/build-thirdparty.sh
@@ -1345,8 +1345,13 @@ build_hdfs3() {
cd "${BUILD_DIR}"
rm -rf ./*
+if [[ "$(uname -m)" == "x86_64" ]]; then
+SSE_OPTION='-DENABLE_SSE=ON'
+else
+SSE_OPTION='-DENABLE_SSE=OFF'
+fi
cmake -DCMAKE_BUILD_TYPE=RelWithDebInfo
-DCMAKE_INSTALL_PREFIX="${TP_INSTALL_DIR}" \
--DBUILD_STATIC_LIBS=ON -DBUILD_SHARED_LIBS=OFF -DBUILD_TEST=OFF \
+-DBUILD_STATIC_LIBS=ON -DBUILD_SHARED_LIBS=OFF -DBUILD_TEST=OFF
"${SSE_OPTION}" \
-DProtobuf_PROTOC_EXECUTABLE="${TP_INSTALL_DIR}/bin/protoc" \
-DProtobuf_INCLUDE_DIR="${TP_INSTALL_DIR}/include" \
-DProtobuf_LIBRARIES="${TP_INSTALL_DIR}/lib/libprotoc.a" \
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] morningman closed issue #13482: [Bug] java.util.ConcurrentModificationException: null of PolicyMgr
morningman closed issue #13482: [Bug] java.util.ConcurrentModificationException: null of PolicyMgr URL: https://github.com/apache/doris/issues/13482 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] morningman merged pull request #13497: [Bug](policy) Add readlock for show policy
morningman merged PR #13497: URL: https://github.com/apache/doris/pull/13497 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated: [fix](policy) Add readlock for show policy (#13497)
This is an automated email from the ASF dual-hosted git repository.
morningman pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new cb39671a73 [fix](policy) Add readlock for show policy (#13497)
cb39671a73 is described below
commit cb39671a73364e07721c3c95789494370ced058a
Author: Stalary
AuthorDate: Tue Oct 25 21:42:40 2022 +0800
[fix](policy) Add readlock for show policy (#13497)
Add readlock for show policy resolve ConcurrentModificationException
---
.../org/apache/doris/analysis/AlterPolicyStmt.java | 6 +-
.../java/org/apache/doris/catalog/S3Resource.java | 2 +-
.../java/org/apache/doris/policy/PolicyMgr.java| 120 ++---
.../apache/doris/service/FrontendServiceImpl.java | 2 +-
4 files changed, 88 insertions(+), 42 deletions(-)
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/analysis/AlterPolicyStmt.java
b/fe/fe-core/src/main/java/org/apache/doris/analysis/AlterPolicyStmt.java
index ae3fb503e4..b058846ba0 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/analysis/AlterPolicyStmt.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/analysis/AlterPolicyStmt.java
@@ -37,6 +37,9 @@ import java.util.List;
import java.util.Map;
import java.util.Optional;
+/**
+ * Alter policy
+ **/
@Data
public class AlterPolicyStmt extends DdlStmt {
private final String policyName;
@@ -65,7 +68,8 @@ public class AlterPolicyStmt extends DdlStmt {
}
// check resource existence
-List policiesByType =
Env.getCurrentEnv().getPolicyMgr().getPoliciesByType(PolicyTypeEnum.STORAGE);
+List policiesByType = Env.getCurrentEnv().getPolicyMgr()
+.getCopiedPoliciesByType(PolicyTypeEnum.STORAGE);
Optional hasPolicy = policiesByType.stream()
.filter(policy ->
policy.getPolicyName().equals(this.policyName)).findAny();
StoragePolicy storagePolicy = (StoragePolicy) hasPolicy.orElseThrow(
diff --git a/fe/fe-core/src/main/java/org/apache/doris/catalog/S3Resource.java
b/fe/fe-core/src/main/java/org/apache/doris/catalog/S3Resource.java
index e3178bafd9..aa303cfaaa 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/catalog/S3Resource.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/catalog/S3Resource.java
@@ -209,7 +209,7 @@ public class S3Resource extends Resource {
this.usedByPolicySet.forEach(
policy -> {
List policiesByType =
Env.getCurrentEnv().getPolicyMgr()
-.getPoliciesByType(PolicyTypeEnum.STORAGE);
+
.getCopiedPoliciesByType(PolicyTypeEnum.STORAGE);
Optional findPolicy = policiesByType.stream()
.filter(p -> p.getType() ==
PolicyTypeEnum.STORAGE && policy.equals(p.getPolicyName()))
.findAny();
diff --git a/fe/fe-core/src/main/java/org/apache/doris/policy/PolicyMgr.java
b/fe/fe-core/src/main/java/org/apache/doris/policy/PolicyMgr.java
index e360b98cc4..0e096a8b30 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/policy/PolicyMgr.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/policy/PolicyMgr.java
@@ -34,6 +34,7 @@ import org.apache.doris.qe.ShowResultSet;
import com.google.common.base.Joiner;
import com.google.common.base.Strings;
+import com.google.common.collect.ImmutableList;
import com.google.common.collect.Lists;
import com.google.common.collect.Maps;
import com.google.common.collect.Sets;
@@ -89,6 +90,9 @@ public class PolicyMgr implements Writable {
lock.readLock().unlock();
}
+/**
+ * Create default storage policy used by master.
+ **/
public void createDefaultStoragePolicy() {
writeLock();
try {
@@ -164,8 +168,13 @@ public class PolicyMgr implements Writable {
* @return exist or not
*/
public boolean existPolicy(Policy checkedPolicy) {
-List policies = getPoliciesByType(checkedPolicy.getType());
-return policies.stream().anyMatch(policy ->
policy.matchPolicy(checkedPolicy));
+readLock();
+try {
+List policies = getPoliciesByType(checkedPolicy.getType());
+return policies.stream().anyMatch(policy ->
policy.matchPolicy(checkedPolicy));
+} finally {
+readUnlock();
+}
}
/**
@@ -175,8 +184,13 @@ public class PolicyMgr implements Writable {
* @return exist or not
*/
private boolean existPolicy(DropPolicyLog checkedDropPolicy) {
-List policies = getPoliciesByType(checkedDropPolicy.getType());
-return policies.stream().anyMatch(policy ->
policy.matchPolicy(checkedDropPolicy));
+readLock();
+try {
+List policies =
getPoliciesByType(checkedDropPolicy.getType());
+return policies.stream().
[GitHub] [doris] morningman merged pull request #13231: [feature-wip](CN Node)Support compute node
morningman merged PR #13231: URL: https://github.com/apache/doris/pull/13231 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated: [feature-wip](CN Node)Support compute node (#13231)
This is an automated email from the ASF dual-hosted git repository. morningman pushed a commit to branch master in repository https://gitbox.apache.org/repos/asf/doris.git The following commit(s) were added to refs/heads/master by this push: new 17ba40f947 [feature-wip](CN Node)Support compute node (#13231) 17ba40f947 is described below commit 17ba40f947ee505677107fe776de75ff8555fcf8 Author: huangzhaowei AuthorDate: Tue Oct 25 21:44:33 2022 +0800 [feature-wip](CN Node)Support compute node (#13231) Introduce the node role to doris, and the table creation and tablet scheduler will control the storage only assign to the BE nodes. --- be/src/agent/heartbeat_server.cpp | 1 + be/src/common/config.h | 4 ++ .../sql-reference/Show-Statements/SHOW-BACKENDS.md | 2 + .../sql-reference/Show-Statements/SHOW-BACKENDS.md | 2 + .../java/org/apache/doris/catalog/OlapTable.java | 4 +- .../main/java/org/apache/doris/catalog/Tablet.java | 2 +- .../apache/doris/clone/ClusterLoadStatistic.java | 4 ++ .../clone/ColocateTableCheckerAndBalancer.java | 2 + .../org/apache/doris/clone/TabletScheduler.java| 5 +- .../main/java/org/apache/doris/common/Config.java | 8 +++ .../apache/doris/common/proc/BackendsProcDir.java | 5 +- .../doris/common/proc/ClusterLoadStatByTag.java| 2 +- .../org/apache/doris/planner/BrokerScanNode.java | 10 +--- .../org/apache/doris/planner/OlapScanNode.java | 3 ++ .../doris/planner/external/BackendPolicy.java | 13 ++--- .../main/java/org/apache/doris/resource/Tag.java | 11 +++- .../main/java/org/apache/doris/system/Backend.java | 25 + .../org/apache/doris/system/BackendHbResponse.java | 20 +++ .../org/apache/doris/system/BeSelectionPolicy.java | 60 - .../java/org/apache/doris/system/HeartbeatMgr.java | 7 ++- .../org/apache/doris/system/SystemInfoService.java | 12 +++-- .../doris/clone/ClusterLoadStatisticsTest.java | 18 +++ .../clone/ColocateTableCheckerAndBalancerTest.java | 53 +- .../apache/doris/system/SystemInfoServiceTest.java | 63 ++ .../doris/utframe/DemoMultiBackendsTest.java | 8 +-- gensrc/thrift/HeartbeatService.thrift | 1 + 26 files changed, 308 insertions(+), 37 deletions(-) diff --git a/be/src/agent/heartbeat_server.cpp b/be/src/agent/heartbeat_server.cpp index 492fe10379..f1895dc561 100644 --- a/be/src/agent/heartbeat_server.cpp +++ b/be/src/agent/heartbeat_server.cpp @@ -80,6 +80,7 @@ void HeartbeatServer::heartbeat(THeartbeatResult& heartbeat_result, heartbeat_result.backend_info.__set_brpc_port(config::brpc_port); heartbeat_result.backend_info.__set_version(get_short_version()); heartbeat_result.backend_info.__set_be_start_time(_be_epoch); +heartbeat_result.backend_info.__set_be_node_role(config::be_node_role); } } diff --git a/be/src/common/config.h b/be/src/common/config.h index 94ff9a6122..b30cc1b005 100644 --- a/be/src/common/config.h +++ b/be/src/common/config.h @@ -857,6 +857,10 @@ CONF_mInt64(nodechannel_pending_queue_max_bytes, "67108864"); // so as to avoid occupying the execution thread for a long time. CONF_mInt32(max_fragment_start_wait_time_seconds, "30"); +// Node role tag for backend. Mix role is the default role, and computation role have no +// any tablet. +CONF_String(be_node_role, "mix"); + // Hide webserver page for safety. // Hide the be config page for webserver. CONF_Bool(hide_webserver_config_page, "false"); diff --git a/docs/en/docs/sql-manual/sql-reference/Show-Statements/SHOW-BACKENDS.md b/docs/en/docs/sql-manual/sql-reference/Show-Statements/SHOW-BACKENDS.md index 72b1850921..456c66598c 100644 --- a/docs/en/docs/sql-manual/sql-reference/Show-Statements/SHOW-BACKENDS.md +++ b/docs/en/docs/sql-manual/sql-reference/Show-Statements/SHOW-BACKENDS.md @@ -52,6 +52,8 @@ illustrate: 10. UsedPct Indicates the percentage of disk used. 11. ErrMsg is used to display the error message when the heartbeat fails. 12. Status is used to display some status information of BE in JSON format, including the time information of the last time BE reported its tablet. + 13. HeartbeatFailureCounter: The current number of heartbeats that have failed consecutively. If the number exceeds the `max_backend_heartbeat_failure_tolerance_count` configuration, the isAlive will be set to false. + 14. NodeRole is used to display the role of Backend node. Now there are two roles: mix and computation. Mix node represent the origin Backend node and computation Node represent the compute only node. ### Example diff --git a/docs/zh-CN/docs/sql-manual/sql-reference/Show-Statements/SHOW-BACKENDS.md b/docs/zh-CN/docs/sql-manual/sql-reference/Show-Statements/SHOW-BACKENDS.md index 277121ee47..9917b50e83 100644 --- a/docs/zh-CN/docs/sql-manual/sql-reference/Show-Statements/SHO
[GitHub] [doris] hello-stephen commented on pull request #13644: [Chore](regression) Fix wrong result for decimal
hello-stephen commented on PR #13644: URL: https://github.com/apache/doris/pull/13644#issuecomment-1290589003 TeamCity pipeline, clickbench performance test result: the sum of best hot time: 37.26 seconds load time: 559 seconds storage size: 17154887178 Bytes https://doris-community-test-1308700295.cos.ap-hongkong.myqcloud.com/tmp/20221025214612_clickbench_pr_33955.html -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] nextdreamblue commented on pull request #13650: [fix](broker) fix bug when broker load with s3a
nextdreamblue commented on PR #13650: URL: https://github.com/apache/doris/pull/13650#issuecomment-1290599449 > Why remove bos and afs? broker代码中实际就没这俩吧 fs_brokers/apache_hdfs_broker/src/main/java/org/apache/doris/broker/hdfs/FileSystemManager.java // supported scheme private static final String HDFS_SCHEME = "hdfs"; private static final String S3A_SCHEME = "s3a"; private static final String KS3_SCHEME = "ks3"; private static final String CHDFS_SCHEME = "ofs"; private static final String OBS_SCHEME = "obs"; -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] sohardforaname opened a new pull request, #13659: [Feature](Nereids)support nereids event for logging the cascades states and transformation.
sohardforaname opened a new pull request, #13659: URL: https://github.com/apache/doris/pull/13659 # Proposed changes Issue Number: close #xxx ## Problem summary Add an event producer, channel, consumer system to support the feature as title and you can turn it on using set enable_nereids_event = true; ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [x] No - [ ] I don't know 2. Has unit tests been added: - [x] Yes - [ ] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [x] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [x] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [x] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] morningman commented on pull request #13650: [fix](broker) fix bug when broker load with s3a
morningman commented on PR #13650: URL: https://github.com/apache/doris/pull/13650#issuecomment-1290631255 > > Why remove bos and afs? > > broker代码中实际就没这俩吧 > > fs_brokers/apache_hdfs_broker/src/main/java/org/apache/doris/broker/hdfs/FileSystemManager.java > > ``` > // supported scheme > private static final String HDFS_SCHEME = "hdfs"; > private static final String S3A_SCHEME = "s3a"; > private static final String KS3_SCHEME = "ks3"; > private static final String CHDFS_SCHEME = "ofs"; > private static final String OBS_SCHEME = "obs"; > ``` > > 实际上只支持这几种 Oh, I see -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[GitHub] [doris] morningman commented on a diff in pull request #13657: [Bug](jdbc) Fix memory leak for JDBC datasource
morningman commented on code in PR #13657:
URL: https://github.com/apache/doris/pull/13657#discussion_r1004558418
##
be/src/util/jni-util.cpp:
##
@@ -39,13 +39,17 @@ void FindOrCreateJavaVM() {
if (rv == 0) {
JNIEnv* env;
JavaVMInitArgs vm_args;
-JavaVMOption options[1];
+JavaVMOption options[2];
char* str = getenv("DORIS_JNI_CLASSPATH_PARAMETER");
options[0].optionString = str;
+// Set max heap size to 4G.
+std::string heap_size("-Xmx4096M");
Review Comment:
This should be a `config`
##
fe/java-udf/pom.xml:
##
@@ -70,6 +70,11 @@ under the License.
${junit.version}
test
+
+com.zaxxer
+HikariCP
+${hikaricp.version}
Review Comment:
Add this to `dist/LICENSE-dist.txt`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch master updated: [improvement](test) add sync for test_agg_keys_schema_change_datev2 (#13643)
This is an automated email from the ASF dual-hosted git repository.
morningman pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 6f18726f01 [improvement](test) add sync for
test_agg_keys_schema_change_datev2 (#13643)
6f18726f01 is described below
commit 6f18726f013971b62ae69e975d355efa2f50e6e4
Author: Mingyu Chen
AuthorDate: Tue Oct 25 22:29:05 2022 +0800
[improvement](test) add sync for test_agg_keys_schema_change_datev2 (#13643)
1. add "sync" to avoid some potential meta sync problem when running
regression test on multi-node cluster
2. Use /tmp dir as dest dir of outfile test, to avoid "No such file or
directory" error.
---
regression-test/suites/export_p0/test_outfile.groovy | 3 ++-
regression-test/suites/export_p0/test_outfile_expr.groovy| 3 ++-
regression-test/suites/export_p0/test_outfile_parquet.groovy | 3 ++-
.../suites/export_p0/test_outfile_separator.groovy | 3 ++-
.../suites/nereids_syntax_p0/agg_with_const.groovy | 1 +
.../datev2/test_agg_keys_schema_change_datev2.groovy | 12
6 files changed, 21 insertions(+), 4 deletions(-)
diff --git a/regression-test/suites/export_p0/test_outfile.groovy
b/regression-test/suites/export_p0/test_outfile.groovy
index 1271ddad37..4fa3350268 100644
--- a/regression-test/suites/export_p0/test_outfile.groovy
+++ b/regression-test/suites/export_p0/test_outfile.groovy
@@ -49,7 +49,8 @@ suite("test_outfile") {
return
}
def tableName = "outfile_test"
-def outFilePath = """${context.file.parent}/test_outfile"""
+def uuid = UUID.randomUUID().toString()
+def outFilePath = """/tmp/test_outfile_${uuid}"""
try {
sql """ DROP TABLE IF EXISTS ${tableName} """
sql """
diff --git a/regression-test/suites/export_p0/test_outfile_expr.groovy
b/regression-test/suites/export_p0/test_outfile_expr.groovy
index 0e3f5adbb9..6649123958 100644
--- a/regression-test/suites/export_p0/test_outfile_expr.groovy
+++ b/regression-test/suites/export_p0/test_outfile_expr.groovy
@@ -49,7 +49,8 @@ suite("test_outfile_expr") {
return
}
def tableName = "outfile_test_expr"
-def outFilePath = """${context.file.parent}/tmp_expr"""
+def uuid = UUID.randomUUID().toString()
+def outFilePath = """/tmp/test_outfile_expr_${uuid}"""
try {
sql """ DROP TABLE IF EXISTS ${tableName} """
sql """
diff --git a/regression-test/suites/export_p0/test_outfile_parquet.groovy
b/regression-test/suites/export_p0/test_outfile_parquet.groovy
index ac710410e9..8b1944d2fb 100644
--- a/regression-test/suites/export_p0/test_outfile_parquet.groovy
+++ b/regression-test/suites/export_p0/test_outfile_parquet.groovy
@@ -53,7 +53,8 @@ suite("test_outfile_parquet") {
}
def tableName = "outfile_parquet_test"
def tableName2 = "outfile_parquet_test2"
-def outFilePath = """${context.file.parent}/test_outfile_parquet"""
+def uuid = UUID.randomUUID().toString()
+def outFilePath = """/tmp/test_outfile_parquet_${uuid}"""
try {
sql """ DROP TABLE IF EXISTS ${tableName} """
sql """
diff --git a/regression-test/suites/export_p0/test_outfile_separator.groovy
b/regression-test/suites/export_p0/test_outfile_separator.groovy
index 07766db680..4d11af6b09 100644
--- a/regression-test/suites/export_p0/test_outfile_separator.groovy
+++ b/regression-test/suites/export_p0/test_outfile_separator.groovy
@@ -50,7 +50,8 @@ suite("test_outfile_separator") {
}
def dbName = context.config.getDbNameByFile(context.file)
def tableName = "outfile_test_separator"
-def outFilePath = """${context.file.parent}/test_outfile_separator"""
+def uuid = UUID.randomUUID().toString()
+def outFilePath = """/tmp/test_outfile_separator_${uuid}"""
try {
sql """ DROP TABLE IF EXISTS ${tableName} """
sql """
diff --git a/regression-test/suites/nereids_syntax_p0/agg_with_const.groovy
b/regression-test/suites/nereids_syntax_p0/agg_with_const.groovy
index c1d48d7766..c5d6abe08f 100644
--- a/regression-test/suites/nereids_syntax_p0/agg_with_const.groovy
+++ b/regression-test/suites/nereids_syntax_p0/agg_with_const.groovy
@@ -38,6 +38,7 @@ suite("agg_with_const") {
sql "SET enable_fallback_to_original_planner=false"
+sql """sync"""
qt_select """
select count(2) + 1, sum(2) + sum(col1) from agg_with_const_tbl
"""
diff --git
a/regression-test/suites/schema_change_p0/datev2/test_agg_keys_schema_change_datev2.groovy
b/regression-test/suites/schema_change_p0/datev2/test_agg_keys_schema_change_datev2.groovy
index 6784c533a0..796793ae5a 100644
---
a/regression-test/suites/schema_change_p0/datev2/test_agg_keys_schema_change_datev2.groovy
+++
b/regression-test/suites/schema_change_p0/datev2/test_agg_keys_schema_change_datev2.groovy
@@ -144,10 +1
[GitHub] [doris] morningman merged pull request #13643: [improvement](test) add sync for test_agg_keys_schema_change_datev2
morningman merged PR #13643: URL: https://github.com/apache/doris/pull/13643 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
[doris] branch branch-1.2-lts updated (c08568e13e -> 6f18726f01)
This is an automated email from the ASF dual-hosted git repository. morningman pushed a change to branch branch-1.2-lts in repository https://gitbox.apache.org/repos/asf/doris.git omit c08568e13e [fix](mem) failure of allocating memory (#13414) add f0dbbe5b46 [Bug](funciton) fix repeat coredump when step is to long (#13408) add 18f2db6064 [feature](nereids) let minValue and maxValue in stats support for Date, CHAR and VARCHAR type (#13311) add cd3450bd9d [Improvement](join) optimize join probing phase (#13357) add 3f964ad5a8 [Regression](javaudf) add regression test for javaudf (#13266) add 125def5102 [enhancement](macOS M1) Support building from source on macOS (M1) (#13195) add 21f233d7e7 [feature-wip](multi-catalog) use apache orc reader to read orc file (#13404) add 6d322f85ac [improvement](compaction) delete num based compaction policy (#13409) add d8e53da764 [feature-wip](statistics) collect statistics by sampling sql-tasks (#13399) add 174054e32d [fix](conf) aggressive_memory_decommit and chunk_reserve_limits can not be changed when running (#13427) add a8fd76fe32 [Fix](docs) fix error description of `LDAP_ADMIN_PASSWORD` in the document (#13405) add 971eb9172f [fix](mem) failure of allocating memory (#13414) add ac037e57f5 [fix](sort)the sort expr's nullability property may not be right (#13328) add 755a946516 [feature](jsonb) jsonb functions (#13366) add 248ca14df7 [fix](test) let each case uses its own table name (#13419) add 8a068c8c92 [function](string_function) add new string function 'not_null_or_empty' (#13418) add c449028a5f [fix](year) fix `year()` results are not as expected (#13426) add 2745a88814 [enhancement](memtracker) Fix brpc causing query mem tracker to be inaccurate #13401 add 1e42598fe6 [memory](podarray) revert not allocate too much memory in podarray change (#13457) add 5423de68dd [refactor](new-scan) remove old file scan node (#13433) add 0b368fbbfa [Bugfix](vec) Fix all create mv using to_bitmap() on negative value columns when enable_vectorized_alter_table is true (#13448) add 29b4d8dcad [typo](docs) fix some problem #13462 add eeb2b0acdb [doc][fix](multi-catalog) Add multi-catalog es doc (#13429) add e65a4a9f9f [Improvement](multi-catalog)Support refresh external catalog. (#13363) add c4b5ba2a4f [Regression](java-udf) Move source code used by Java UDF test case (#13476) add 9ac4cfc9bb [bugfix](array-type) ColumnDate lost is_date_type after cloned (#13420) add 410e36ef5b [enhancement](macOS) Refine the build scripts for macOS (#13473) add 3a2d5db914 [fix](String) fix string type length set to -1 when load stirng data (#13475) add 697fa5f586 [Enhancement](profile) support configure the number of query profile (#13421) add 4fa3b14bf0 [Fix](multi-catalog)Fix NPE caused by GsonUtils created objects. #13489 add 50e2d0fd3e [opt](storage) opt the read by column decimal (#13488) add 3821f8420d [opt](tpch) after change the config to speed up q21 (#13460) add f329d33666 [chore](fix) Fix some spell errors in be's comments. #13452 add bc08854a35 [doc](storage policy) add cold and hot separation docs (#13096) add 60d5e4dfce [improvement](spark-load) support parquet and orc file (#13438) add 4996eafe74 [bugfix](VecDateTimeValue) eat the value of microsecond in function from_date_format_str (#13446) add 8637ac1ca3 [regression](framework)set random parallel_fragment_exec_instance_num… (#13383) add f7c69ade18 [feature-wip](multi-catalog) implement predicate pushdown in native OrcReader (#13453) add b5cd167713 [fix](hashjoin) fix coredump of hash join in ubsan build (#13479) add 2b328eafbb [function](string_function) add new string function 'extract_url_parameter' (#13323) add 3c837a9bdd [regression](load) modify variable definition (#13506) add 1892e8f66e [Enhancement](scanner) support split avg key range (#13166) add 32b1456b28 [feature-wip](array) remove array config and check array nested depth (#13428) add 1e774036f1 [fix](function)fix be coredump when using json_object function (#13443) add 7109cbfe6f [feature-wip](unique-key-merge-on-write) fix that delete the bitmap of stale rowset (#13393) add 4ae777bfc5 [fix](Nereids) NPE caused by GroupExpression has null owner group when choosing best plan (#13252) add 483a46d17c [feature](Nereids) generate ExprId from 0 for each statement (#13382) add 95f437c506 [fix] Fix potential unhandled exception cause data inconsistency (#11029) add d624ff0580 [chore](macOS) Avoid using binutils from Homebrew to build third parties (#13512) add 736d113700 [fix](memtracker) Fix transmit_tracker null pointer because phamp is not thread safe #13528 add d2be5096d6 [Revert](mem) revert the mem config cause perfermace degradation (#13526) add 3dd00df24b [fix](jsonrea
[doris] branch branch-1.2-unstable updated (66ebb075e7 -> 6f18726f01)
This is an automated email from the ASF dual-hosted git repository. morningman pushed a change to branch branch-1.2-unstable in repository https://gitbox.apache.org/repos/asf/doris.git omit 66ebb075e7 [fix](load) clear and notify when an error happens in flushing omit 6b27d29e68 [Bug](predicate) fix core dump on bool type runtime filter (#13417) omit 58c7edf8e7 [Fix](array-type) bugfix for array column with delete condition (#13361) omit f4948a06bf [fix](function)fix be coredump when using json_object function (#13443) omit e8858a09c3 [improvement](heartbeat) Add some relaxation strategies to reduce the failure probability of regression testing (#13568) omit 83b95f7ebb [memory](podarray) revert not allocate too much memory in podarray change (#13457) omit dcc8d57f46 [Revert](mem) revert the mem config cause perfermace degradation (#13526) omit c08568e13e [fix](mem) failure of allocating memory (#13414) add f0dbbe5b46 [Bug](funciton) fix repeat coredump when step is to long (#13408) add 18f2db6064 [feature](nereids) let minValue and maxValue in stats support for Date, CHAR and VARCHAR type (#13311) add cd3450bd9d [Improvement](join) optimize join probing phase (#13357) add 3f964ad5a8 [Regression](javaudf) add regression test for javaudf (#13266) add 125def5102 [enhancement](macOS M1) Support building from source on macOS (M1) (#13195) add 21f233d7e7 [feature-wip](multi-catalog) use apache orc reader to read orc file (#13404) add 6d322f85ac [improvement](compaction) delete num based compaction policy (#13409) add d8e53da764 [feature-wip](statistics) collect statistics by sampling sql-tasks (#13399) add 174054e32d [fix](conf) aggressive_memory_decommit and chunk_reserve_limits can not be changed when running (#13427) add a8fd76fe32 [Fix](docs) fix error description of `LDAP_ADMIN_PASSWORD` in the document (#13405) add 971eb9172f [fix](mem) failure of allocating memory (#13414) add ac037e57f5 [fix](sort)the sort expr's nullability property may not be right (#13328) add 755a946516 [feature](jsonb) jsonb functions (#13366) add 248ca14df7 [fix](test) let each case uses its own table name (#13419) add 8a068c8c92 [function](string_function) add new string function 'not_null_or_empty' (#13418) add c449028a5f [fix](year) fix `year()` results are not as expected (#13426) add 2745a88814 [enhancement](memtracker) Fix brpc causing query mem tracker to be inaccurate #13401 add 1e42598fe6 [memory](podarray) revert not allocate too much memory in podarray change (#13457) add 5423de68dd [refactor](new-scan) remove old file scan node (#13433) add 0b368fbbfa [Bugfix](vec) Fix all create mv using to_bitmap() on negative value columns when enable_vectorized_alter_table is true (#13448) add 29b4d8dcad [typo](docs) fix some problem #13462 add eeb2b0acdb [doc][fix](multi-catalog) Add multi-catalog es doc (#13429) add e65a4a9f9f [Improvement](multi-catalog)Support refresh external catalog. (#13363) add c4b5ba2a4f [Regression](java-udf) Move source code used by Java UDF test case (#13476) add 9ac4cfc9bb [bugfix](array-type) ColumnDate lost is_date_type after cloned (#13420) add 410e36ef5b [enhancement](macOS) Refine the build scripts for macOS (#13473) add 3a2d5db914 [fix](String) fix string type length set to -1 when load stirng data (#13475) add 697fa5f586 [Enhancement](profile) support configure the number of query profile (#13421) add 4fa3b14bf0 [Fix](multi-catalog)Fix NPE caused by GsonUtils created objects. #13489 add 50e2d0fd3e [opt](storage) opt the read by column decimal (#13488) add 3821f8420d [opt](tpch) after change the config to speed up q21 (#13460) add f329d33666 [chore](fix) Fix some spell errors in be's comments. #13452 add bc08854a35 [doc](storage policy) add cold and hot separation docs (#13096) add 60d5e4dfce [improvement](spark-load) support parquet and orc file (#13438) add 4996eafe74 [bugfix](VecDateTimeValue) eat the value of microsecond in function from_date_format_str (#13446) add 8637ac1ca3 [regression](framework)set random parallel_fragment_exec_instance_num… (#13383) add f7c69ade18 [feature-wip](multi-catalog) implement predicate pushdown in native OrcReader (#13453) add b5cd167713 [fix](hashjoin) fix coredump of hash join in ubsan build (#13479) add 2b328eafbb [function](string_function) add new string function 'extract_url_parameter' (#13323) add 3c837a9bdd [regression](load) modify variable definition (#13506) add 1892e8f66e [Enhancement](scanner) support split avg key range (#13166) add 32b1456b28 [feature-wip](array) remove array config and check array nested depth (#13428) add 1e774036f1 [fix](function)fix be coredump when using json_object function (#13443) add 7109cbfe6f [feature-wip](unique-key-merge-on-writ
[GitHub] [doris] adonis0147 opened a new pull request, #13660: [enhancement](java-udf) Support loading libjvm at runtime
adonis0147 opened a new pull request, #13660: URL: https://github.com/apache/doris/pull/13660 # Proposed changes Load libjvm at runtime. ## Problem summary Currently, we build BE with Java UDF support by default. Unlike other dependencies which are linked with BE statically, we link BE and libjvm dynamically. As a result, we should set the environment variable `LD_LIBRARY_PATH` before we start BE up, otherwise we will get an error (error while loading shared libraries). That is really inconvenient. The workaround is that we can load the libjvm at runtime. ## Checklist(Required) 1. Does it affect the original behavior: - [ ] Yes - [ ] No - [ ] I don't know 2. Has unit tests been added: - [ ] Yes - [ ] No - [ ] No Need 3. Has document been added or modified: - [ ] Yes - [ ] No - [ ] No Need 4. Does it need to update dependencies: - [ ] Yes - [ ] No 5. Are there any changes that cannot be rolled back: - [ ] Yes (If Yes, please explain WHY) - [ ] No ## Further comments If this is a relatively large or complex change, kick off the discussion at [d...@doris.apache.org](mailto:d...@doris.apache.org) by explaining why you chose the solution you did and what alternatives you considered, etc... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org For additional commands, e-mail: commits-h...@doris.apache.org
