github-actions[bot] commented on code in PR #69313:
URL: https://github.com/apache/doris/pull/69313#discussion_r4625271261


##########
be/src/storage/compaction/compaction.cpp:
##########
@@ -1817,6 +1819,25 @@ void 
Compaction::construct_index_compaction_columns(RowsetWriterContext& ctx) {
             continue;
         }
 
+        // A norms-bearing index without term frequencies cannot reconstruct 
its exact
+        // token count after rows are removed or split across destination 
segments.
+        // Byte norms are lossy (for example, length 41 decodes as 40), so 
rebuild from
+        // the raw column. CLucene compaction is selected per column: a 
sibling index
+        // must not re-enable native merging for this column later in this 
loop.
+        const auto column_indexes = 
_cur_tablet_schema->inverted_indexs(col_unique_id);
+        const bool needs_norms_rebuild =
+                std::ranges::any_of(column_indexes, [](const TabletIndex* 
column_index) {
+                    return 
segment_v2::inverted_index::InvertedIndexAnalyzer::should_analyzer(
+                                   column_index->properties()) &&
+                           should_write_index_norms(*column_index) &&
+                           get_parser_phrase_support_string_from_properties(
+                                   column_index->properties()) !=
+                                   INVERTED_INDEX_PARSER_PHRASE_SUPPORT_YES;
+                });
+        if (needs_norms_rebuild) {

Review Comment:
   [P2] Update the existing compaction fixtures for this exclusion. 
`IndexCompactionTest::_build_tablet()` creates column 2 with `parser=unicode` 
and no `support_phrase`/`norms`, so the defaults make this branch exclude it. 
However, `tes_write_index_normally` still expects native columns `{1, 2}` 
(lines 828-830), and `test_col_unique_ids_empty` and several other fixture 
tests explicitly require column 2. Those assertions now fail; 
`_run_normal_wiki_test` has the same assumption for phrase-disabled indexes. 
Update the fixtures intended to test native merging (for example, enable phrase 
support) and adjust/add expectations for the raw-rebuild cases.



-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to