zhangbutao commented on code in PR #34775:
URL: https://github.com/apache/doris/pull/34775#discussion_r1598152392
##########
fe/fe-core/src/main/java/org/apache/doris/datasource/iceberg/source/IcebergScanNode.java:
##########
@@ -212,60 +212,73 @@ private List<Split> doGetSplits() throws UserException {
HashSet<String> partitionPathSet = new HashSet<>();
boolean isPartitionedTable = icebergTable.spec().isPartitioned();
+ TPushAggOp aggOp = getPushDownAggNoGroupingOp();
+ boolean canPushCount = aggOp.equals(TPushAggOp.COUNT) &&
getCountFromSnapshot() > 0;
+
CloseableIterable<FileScanTask> fileScanTasks =
TableScanUtil.splitFiles(scan.planFiles(), splitSize);
try (CloseableIterable<CombinedScanTask> combinedScanTasks =
TableScanUtil.planTasks(fileScanTasks, splitSize, 1, 0)) {
- combinedScanTasks.forEach(taskGrp ->
taskGrp.files().forEach(splitTask -> {
- String dataFilePath =
normalizeLocation(splitTask.file().path().toString());
-
- List<String> partitionValues = new ArrayList<>();
- if (isPartitionedTable) {
- StructLike structLike = splitTask.file().partition();
- List<PartitionField> fields = splitTask.spec().fields();
- Types.StructType structType =
icebergTable.schema().asStruct();
-
- // set partitionValue for this IcebergSplit
- for (int i = 0; i < structLike.size(); i++) {
- Object obj = structLike.get(i, Object.class);
- String value = String.valueOf(obj);
- PartitionField partitionField = fields.get(i);
- if (partitionField.transform().isIdentity()) {
- Type type =
structType.fieldType(partitionField.name());
- if (type != null &&
type.typeId().equals(Type.TypeID.DATE)) {
- // iceberg use integer to store date,
- // we need transform it to string
- value = DateTimeUtil.daysToIsoDate((Integer)
obj);
+ for (CombinedScanTask taskGrp : combinedScanTasks) {
Review Comment:
I didn't find a better way to end the loop in `forEach`, so i use `for ` to
replace `forEach`. :(
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
---------------------------------------------------------------------
To unsubscribe, e-mail: commits-unsubscr...@doris.apache.org
For additional commands, e-mail: commits-h...@doris.apache.org
