924060929 commented on code in PR #67687:
URL: https://github.com/apache/doris/pull/67687#discussion_r3985530197
##########
fe/pom.xml:
##########
@@ -360,7 +360,7 @@ under the License.
<!-- ATTN: avro version must be consistent with Iceberg version -->
<!-- Please modify iceberg.version and avro.version together,
you can find avro version info in iceberg mvn repository -->
- <iceberg.version>1.10.1</iceberg.version>
+ <iceberg.version>1.11.0</iceberg.version>
Review Comment:
Follow-up on the current head `dab9c2ef7ad0aa720f9a52b1fa1d6451b6545232`:
the Schema-only bridge does not close this compatibility issue for the
supported **BE-first, FE-second** upgrade order.
During that upgrade window, an old FE using Iceberg 1.10.1 sends a
Java-serialized system-table `FileScanTask` to a new BE using Iceberg 1.11.0. I
checked the actual two jars: `AllManifestsTable$ManifestListReadTask`, used by
Doris `$all_manifests`, changed its generated `serialVersionUID` from
`-5290087248369820236` to `-686031333743179684`. Its retained field also
changed from `String manifestListLocation` to `ManifestListFile manifestList`.
The new BE input stream only substitutes the nested `Schema` descriptor, so it
still rejects the outer 1.10.1 task class before `asDataTask().rows()`.
The committed 1.10.1 fixture is a `StaticDataTask`, whose own UID is
unchanged across these versions, and the `$snapshots` test uses a single 1.11.0
classpath. Neither exercises the production `$all_manifests` task shape. A
new-FE backend fence also cannot protect this BE-first window because the task
is produced by the old FE.
**Required in this PR:** cover the actual supported direction—an Iceberg
1.10.1 `$all_manifests` task produced by the old FE and fully consumed through
the new 1.11.0 BE scanner path, including `asDataTask().rows()`—and make that
path compatible, or avoid upgrading the BE-side Iceberg reader until such a
bridge exists. Bidirectional compatibility and a complete wire-protocol
redesign are not required for this PR.
**Maintainer follow-up only:** replace Java native serialization with a
Doris-owned versioned representation so future Iceberg upgrades do not depend
on third-party implementation-class layouts.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
