This is an automated email from the ASF dual-hosted git repository.
Gabriel39 pushed a change to branch refact_reader_branch
in repository https://gitbox.apache.org/repos/asf/doris.git
from d176f97195a [improvement](be) Cast literals for slot literal file
predicates (#63939)
add dc3ce055b92 [refactor](be) Refine new parquet reader layering (#63929)
No new revisions were added by this update.
Summary of changes:
be/src/format/new_parquet/column_reader.cpp | 1548 --------------------
be/src/format/new_parquet/parquet_file_context.cpp | 164 +++
.../new_parquet/parquet_file_context.h} | 34 +-
be/src/format/new_parquet/parquet_reader.cpp | 614 ++------
be/src/format/new_parquet/parquet_reader.h | 21 +-
be/src/format/new_parquet/parquet_scan.cpp | 511 +++++++
be/src/format/new_parquet/parquet_scan.h | 125 ++
be/src/format/new_parquet/parquet_statistics.cpp | 331 ++++-
be/src/format/new_parquet/parquet_statistics.h | 37 +-
.../reader/arrow_leaf_reader_adapter.cpp | 325 ++++
.../new_parquet/reader/arrow_leaf_reader_adapter.h | 69 +
be/src/format/new_parquet/reader/column_reader.cpp | 426 ++++++
.../new_parquet/{ => reader}/column_reader.h | 0
.../new_parquet/reader/list_column_reader.cpp | 182 +++
.../format/new_parquet/reader/list_column_reader.h | 64 +
.../new_parquet/reader/map_column_reader.cpp | 269 ++++
.../format/new_parquet/reader/map_column_reader.h | 64 +
.../new_parquet/reader/nested_column_reader.cpp | 415 ++++++
.../new_parquet/reader/nested_column_reader.h | 789 ++++++++++
.../new_parquet/reader/scalar_column_reader.cpp | 97 ++
.../new_parquet/reader/scalar_column_reader.h | 67 +
.../reader/shape_only_column_reader.cpp | 125 ++
.../new_parquet/reader/shape_only_column_reader.h | 66 +
.../new_parquet/reader/struct_column_reader.cpp | 317 ++++
.../new_parquet/reader/struct_column_reader.h | 70 +
be/src/format/new_parquet/selection_vector.h | 28 +
be/src/format/reader/column_mapper.cpp | 119 +-
be/src/format/reader/column_mapper.h | 8 +-
be/src/format/reader/file_reader.h | 7 +-
be/src/format/reader/table_reader.cpp | 5 +
be/src/format/reader/table_reader.h | 198 ++-
be/src/format/table/iceberg_reader_v2.cpp | 2 +-
.../new_parquet/parquet_column_reader_test.cpp | 1322 ++++++++++++++++-
be/test/format/new_parquet/parquet_reader_test.cpp | 418 +++++-
be/test/format/reader/table_reader_test.cpp | 82 ++
...s-arrow-parquet-complex-types-implementation.md | 559 -------
docs/doris-arrow-parquet-reader-implementation.md | 349 -----
docs/doris-new-parquet-dictionary-pushdown.md | 359 -----
docs/doris-new-parquet-reader-internal-layering.md | 569 +++++++
39 files changed, 7297 insertions(+), 3458 deletions(-)
delete mode 100644 be/src/format/new_parquet/column_reader.cpp
create mode 100644 be/src/format/new_parquet/parquet_file_context.cpp
copy be/src/{service/brpc_service.h =>
format/new_parquet/parquet_file_context.h} (59%)
create mode 100644 be/src/format/new_parquet/parquet_scan.cpp
create mode 100644 be/src/format/new_parquet/parquet_scan.h
create mode 100644
be/src/format/new_parquet/reader/arrow_leaf_reader_adapter.cpp
create mode 100644 be/src/format/new_parquet/reader/arrow_leaf_reader_adapter.h
create mode 100644 be/src/format/new_parquet/reader/column_reader.cpp
rename be/src/format/new_parquet/{ => reader}/column_reader.h (100%)
create mode 100644 be/src/format/new_parquet/reader/list_column_reader.cpp
create mode 100644 be/src/format/new_parquet/reader/list_column_reader.h
create mode 100644 be/src/format/new_parquet/reader/map_column_reader.cpp
create mode 100644 be/src/format/new_parquet/reader/map_column_reader.h
create mode 100644 be/src/format/new_parquet/reader/nested_column_reader.cpp
create mode 100644 be/src/format/new_parquet/reader/nested_column_reader.h
create mode 100644 be/src/format/new_parquet/reader/scalar_column_reader.cpp
create mode 100644 be/src/format/new_parquet/reader/scalar_column_reader.h
create mode 100644
be/src/format/new_parquet/reader/shape_only_column_reader.cpp
create mode 100644 be/src/format/new_parquet/reader/shape_only_column_reader.h
create mode 100644 be/src/format/new_parquet/reader/struct_column_reader.cpp
create mode 100644 be/src/format/new_parquet/reader/struct_column_reader.h
delete mode 100644 docs/doris-arrow-parquet-complex-types-implementation.md
delete mode 100644 docs/doris-arrow-parquet-reader-implementation.md
delete mode 100644 docs/doris-new-parquet-dictionary-pushdown.md
create mode 100644 docs/doris-new-parquet-reader-internal-layering.md
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
