xylaaaaa commented on issue #69884:
URL: https://github.com/apache/doris/issues/69884#issuecomment-6942964335

   **R7 reproduced with Spark-written Delta tables (2026-11-17).** Spark 4.0.1 
with Delta 4.4.1 accepts a single partition column named `p,q` in both name and 
ID mapping modes. In the two-row fixture, ID 41 belongs to west and ID 42 to 
east.
   
   With the Spark writer default, the values are present in both the Delta log 
and Parquet, and Doris returns both rows correctly. With 
`delta.writePartitionColumnsToParquet=false`, the same SQL and rows leave the 
partition values only in the log. Spark still returns west/east. Doris returns 
NULL for both rows and an empty result for the west filter. EXPLAIN selects the 
very file whose log entry says west and whose Parquet row has ID 41. Both cases 
use the supported [Delta writer 
option](https://github.com/delta-io/delta/blob/v4.4.1/spark/src/main/scala/org/apache/spark/sql/delta/DeltaConfig.scala#L1036-L1045).
   
   Filesystem and open-source Unity Catalog, each with synchronous and 
streaming scheduling, gave the same results. Of 72 Doris queries, the 16 
incorrect results were the projection and filter in the two mapped tables with 
a comma and no partition column in Parquet. Plain partition names and two 
distinct partition columns passed.
   
   The FE serializes the schema partition names and scan partition keys as 
comma-separated strings. Both consumers interpret the single name as two names, 
so the reader treats the projected column as an ordinary data column. The Delta 
split already carries the full name and the correct value. R7 remains open.
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to