This is an automated email from the ASF dual-hosted git repository.
dataroaring pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/doris.git
The following commit(s) were added to refs/heads/master by this push:
new 888fda7e8b6 [fix](multi-table-load) fix fe log too much when using
multi table load (#36458)
888fda7e8b6 is described below
commit 888fda7e8b6bb8d250ba28bdfc15602377985476
Author: hui lai <1353307...@qq.com>
AuthorDate: Sat Jun 22 07:52:42 2024 +0800
[fix](multi-table-load) fix fe log too much when using multi table load
(#36458)
## Proposed changes
Log introduce by https://github.com/apache/doris/pull/32716
There is lots of log in FE log when using multi table load:
```
2024-06-18 11:13:05,360 INFO (thrift-server-pool-3645|93075)
[FrontendServiceImpl.streamLoadMultiTablePut():2042] receive stream load multi
table put requ
est result: TStreamLoadMultiTablePutResult(status:TStatus(status_code:OK),
pipeline_params:[TPipelineFragmentParams(protocol_version:V1, query_id:TUniqueI
d(hi:8161224298039997004, lo:3829166682228620956), fragment_id:185,
per_exch_num_senders:{},
desc_tbl:TDescriptorTable(slotDescriptors:[TSlotDescriptor(id
:0, parent:0, slotType:TTypeDesc(types:[TTypeNode(type:SCALAR,
scalar_type:TScalarType(type:DATEV2))], byte_size:-1), columnPos:-1,
byteOffset:-1, nullInd
icatorByte:0, nullIndicatorBit:-1, colName:l_shipdate, slotIdx:0,
isMaterialized:true, col_unique_id:0, is_key:true, need_materialize:true,
is_auto_increm
ent:false, primitive_type:DATEV2), TSlotDescriptor(id:2, parent:0,
slotType:TTypeDesc(types:[TTypeNode(type:SCALAR,
scalar_type:TScalarType(type:BIGINT))]
, byte_size:-1), columnPos:-1, byteOffset:-1, nullIndicatorByte:0,
nullIndicatorBit:-1, colName:l_orderkey, slotIdx:3, isMaterialized:true,
col_unique_id:
1, is_key:true, need_materialize:true, is_auto_increment:false,
primitive_type:BIGINT), TSlotDescriptor(id:4, parent:0,
slotType:TTypeDesc(types:[TTypeNod
e(type:SCALAR, scalar_type:TScalarType(type:BIGINT))], byte_size:-1),
columnPos:-1, byteOffset:-1, nullIndicatorByte:0, nullIndicatorBit:-1,
colName:l_lin
enumber, slotIdx:4, isMaterialized:true, col_unique_id:2, is_key:true,
need_materialize:true, is_auto_increment:false, primitive_type:BIGINT),
TSlotDescri
ptor(id:6, parent:0, slotType:TTypeDesc(types:[TTypeNode(type:SCALAR,
scalar_type:TScalarType(type:BIGINT))], byte_size:-1), columnPos:-1,
byteOffset:-1,
nullIndicatorByte:0, nullIndicatorBit:-1, colName:l_partkey, slotIdx:5,
isMaterialized:true, col_unique_id:3, is_key:true, need_materialize:true,
is_auto_
increment:false, primitive_type:BIGINT), TSlotDescriptor(id:8, parent:0,
slotType:TTypeDesc(types:[TTypeNode(type:SCALAR,
scalar_type:TScalarType(type:BIG
INT))], byte_size:-1), columnPos:-1, byteOffset:-1, nullIndicatorByte:0,
nullIndicatorBit:-1, colName:l_suppkey, slotIdx:6, isMaterialized:true,
col_uniqu
e_id:4, is_key:true, need_materialize:true, is_auto_increment:false,
primitive_type:BIGINT), TSlotDescriptor(id:10, parent:0,
slotType:TTypeDesc(types:[TT
ypeNode(type:SCALAR, scalar_type:TScalarType(type:DECIMAL64, precision:15,
scale:2))], byte_size:-1), columnPos:-1, byteOffset:-1, nullIndicatorByte:0, nu
llIndicatorBit:-1, colName:l_quantity, slotIdx:7, isMaterialized:true,
col_unique_id:5, is_key:false, need_materialize:true, is_auto_increment:false,
prim
itive_type:DECIMAL64), TSlotDescriptor(id:12, parent:0,
slotType:TTypeDesc(types:[TTypeNode(type:SCALAR,
scalar_type:TScalarType(type:DECIMAL64, precision
:15, scale:2))], byte_size:-1), columnPos:-1, byteOffset:-1,
nullIndicatorByte:0, nullIndicatorBit:-1, colName:l_extendedprice, slotIdx:8,
isMaterialized:
true, col_unique_id:6, is_key:false, need_materialize:true,
is_auto_increment:false, primitive_type:DECIMAL64), TSlotDescriptor(id:14,
parent:0, slotType:
TTypeDesc(types:[TTypeNode(type:SCALAR,
scalar_type:TScalarType(type:DECIMAL64, precision:15, scale:2))],
byte_size:-1), columnPos:-1, byteOffset:-1, null
IndicatorByte:0, nullIndicatorBit:-1, colName:l_discount, slotIdx:9,
isMaterialized:true, col_unique_id:7, is_key:false, need_materialize:true,
is_auto_in
crement:false, primitive_type:DECIMAL64), TSlotDescriptor(id:16, parent:0,
slotType:TTypeDesc(types:[TTypeNode(type:SCALAR, scalar_type:TScalarType(type:D
ECIMAL64, precision:15, scale:2))], byte_size:-1), columnPos:-1,
byteOffset:-1, nullIndicatorByte:0, nullIndicatorBit:-1, colName:l_tax,
slotIdx:10, isMat
erialized:true, col_unique_id:8, is_key:false, need_materialize:true,
is_auto_increment:false, primitive_type:DECIMAL64), TSlotDescriptor(id:18,
parent:0,
slotType:TTypeDesc(types:[TTypeNode(type:SCALAR,
scalar_type:TScalarType(type:VARCHAR, len:1))], byte_size:1), columnPos:-1,
byteOffset:-1, nullIndicator
Byte:0, nullIndicatorBit:-1, colName:l_returnflag, slotIdx:11,
isMaterialized:true, col_unique_id:9, is_key:false, need_materialize:true,
is_auto_incremen
t:false, primitive_type:VARCHAR), TSlotDescriptor(id:20, parent:0,
slotType:TTypeDesc(types:[TTypeNode(type:SCALAR,
scalar_type:TScalarType(type:VARCHAR,
len:1))], byte_size:1), columnPos:-1, byteOffset:-1, nullIndicatorByte:0,
nullIndicatorBit:-1, colName:l_linestatus, slotIdx:12, isMaterialized:true, col_
unique_id:10, is_key:false, need_materialize:true, is_auto_increment:false,
primitive_type:VARCHAR), TSlotDescriptor(id:22, parent:0, slotType:TTypeDesc(t
ypes:[TTypeNode(type:SCALAR, scalar_type:TScalarType(type:DATEV2))],
byte_size:-1), columnPos:-1, byteOffset:-1, nullIndicatorByte:0,
nullIndicatorBit:-1,
colName:l_commitdate, slotIdx:1, isMaterialized:true, col_unique_id:11,
is_key:false, need_materialize:true, is_auto_increment:false,
primitive_type:DATE
V2), TSlotDescriptor(id:24, parent:0,
slotType:TTypeDesc(types:[TTypeNode(type:SCALAR,
scalar_type:TScalarType(type:DATEV2))], byte_size:-1), columnPos:-1
, byteOffset:-1, nullIndicatorByte:0, nullIndicatorBit:-1,
colName:l_receiptdate, slotIdx:2, isMaterialized:true, col_unique_id:12,
is_key:false, need_mat
erialize:true, is_auto_increment:false, primitive_type:DATEV2),
TSlotDescriptor(id:26, parent:0,
slotType:TTypeDesc(types:[TTypeNode(type:SCALAR, scalar_t
ype:TScalarType(type:VARCHAR, len:25))], byte_size:25), columnPos:-1,
byteOffset:-1, nullIndicatorByte:0, nullIndicatorBit:-1,
colName:l_shipinstruct, slo
tIdx:13, isMaterialized:true, col_unique_id:13, is_key:false,
need_materialize:true, is_auto_increment:false, primitive_type:VARCHAR),
TSlotDescriptor(id:
28, parent:0, slotType:TTypeDesc(types:[TTypeNode(type:SCALAR,
scalar_type:TScalarType(type:VARCHAR, len:40))], byte_size:40), columnPos:-1,
byteOffset:-1
, nullIndicatorByte:0, nullIndicatorBit:-1, colName:l_shipmode, slotIdx:14,
isMaterialized:true, col_unique_id:14, is_key:false, need_materialize:true,
is_auto_increment:false, primitive_type:VARCHAR), TSlotDescriptor(id:30,
parent:0, slotType:TTypeDesc(types:[TTypeNode(type:SCALAR,
scalar_type:TScalarType(type:VARCHAR, len:44))], byte_size:44), columnPos:-1,
byteOffset:-1, nullIndicatorByte:0, nullIndicatorBit:-1, colName:l_comment,
slotIdx:15, isMaterialized:true, col_unique_id:15, [...]
```
Due to the presence of a large number of such logs, the speed of the log
file rolling up is very fast.
---
.../src/main/java/org/apache/doris/service/FrontendServiceImpl.java | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git
a/fe/fe-core/src/main/java/org/apache/doris/service/FrontendServiceImpl.java
b/fe/fe-core/src/main/java/org/apache/doris/service/FrontendServiceImpl.java
index a3573528bc9..a1848fc6e42 100644
--- a/fe/fe-core/src/main/java/org/apache/doris/service/FrontendServiceImpl.java
+++ b/fe/fe-core/src/main/java/org/apache/doris/service/FrontendServiceImpl.java
@@ -2041,7 +2041,9 @@ public class FrontendServiceImpl implements
FrontendService.Iface {
ConnectContext.remove();
}
result.setPipelineParams(planFragmentParamsList);
- LOG.info("receive stream load multi table put request result: {}",
result);
+ if (LOG.isDebugEnabled()) {
+ LOG.debug("receive stream load multi table put request result:
{}", result);
+ }
return result;
}
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
