[
https://issues.apache.org/jira/browse/IGNITE-29185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18132225#comment-18132225
]
Ignite TC Bot commented on IGNITE-29185:
----------------------------------------
Persistence: Fixed node startup after upgrade when reading checkpoint map
snapshots written by Ignite 2.17 or 2.18.
The Release Note field has been filled with the wording above. Please review it
and edit it if needed.
> cpMapSnapshot.bin written by Ignite 2.17/2.18 can't be read by current
> version (generated IDTO serializer changed UUID format)
> ------------------------------------------------------------------------------------------------------------------------------
>
> Key: IGNITE-29185
> URL: https://issues.apache.org/jira/browse/IGNITE-29185
> Project: Ignite
> Issue Type: Bug
> Affects Versions: 2.19
> Reporter: Nikolay Izhikov
> Assignee: Nikolay Izhikov
> Priority: Major
> Labels: ise
> Fix For: 2.19
>
> Attachments: IGNITE-29185-reproducer.patch
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> h3. Problem
> {{cpMapSnapshot.bin}} (earliest checkpoint map snapshot, see
> {{CheckpointMarkersStorage#EARLIEST_CP_SNAPSHOT_FILE}}) written by Ignite
> 2.17 / 2.18 can't be deserialized by current master (future 2.19).
> The file is an {{EarliestCheckpointMapSnapshot}} marshalled with
> {{JdkMarshaller}}. In 2.17 and 2.18 the payload is written by hand:
> {code:java}
> U.writeMap(out, data); // keys (UUID) and values via
> out.writeObject(...)
> U.writeCollection(out, checkpointIds); // UUID elements via
> out.writeObject(...)
> {code}
> Since IGNITE-27709 the class is serialized by generated code, and since
> IGNITE-27775 the generator emits typed code for collections. The generated
> {{EarliestCheckpointMapSnapshotSerializer}} now writes/reads the outer map
> keys and the {{checkpointIds}} elements with {{U.writeUuid / U.readUuid}}
> (null flag + two longs) instead of {{writeObject / readObject}}
> (Java-serialized {{java.util.UUID}}). The two formats are not
> interchangeable, so the file produced by a previous version can't be read
> after upgrade.
> The header change made by IGNITE-24260 (protocol version byte removed) is
> *not* a problem: the reader masks the version byte.
> h3. Impact
> {{CheckpointMarkersStorage#initialize}} catches the failure, logs
> {noformat}
> Failed to unmarshal earliest checkpoint map snapshot
> {noformat}
> deletes the file and rebuilds the earliest checkpoint map from checkpoint
> markers. No data is lost, but the optimization is silently lost on the first
> start after an upgrade and the node logs an error. PDS files must stay
> backward compatible, so the old format has to be readable.
> h3. Reproducer
> {{CheckpointMapSnapshotCompatibilityTest}} in {{ignite-compatibility}}
> (attached): starts a 2.17.0 / 2.18.0 node with persistence, waits for
> {{cpMapSnapshot.bin}}, restarts on the same PDS with the current version and
> asserts the error above isn't logged.
> Test output (same for 2.17.0 and 2.18.0):
> {noformat}
> class org.apache.ignite.IgniteCheckedException: Failed to deserialize object
> with given class loader: null
> at
> org.apache.ignite.marshaller.jdk.JdkMarshallerImpl.unmarshal0(JdkMarshallerImpl.java:128)
> at
> org.apache.ignite.marshaller.jdk.JdkMarshallerImpl.unmarshal0(JdkMarshallerImpl.java:135)
> at
> org.apache.ignite.marshaller.AbstractNodeNameAwareMarshaller.unmarshal(AbstractNodeNameAwareMarshaller.java:74)
> at
> org.apache.ignite.internal.processors.cache.persistence.checkpoint.CheckpointMarkersStorage.initialize(CheckpointMarkersStorage.java:209)
> at
> org.apache.ignite.internal.processors.cache.persistence.checkpoint.CheckpointManager.initializeStorage(CheckpointManager.java:325)
> at
> org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.readMetastore(GridCacheDatabaseSharedManager.java:854)
> at
> org.apache.ignite.internal.processors.cache.persistence.GridCacheDatabaseSharedManager.notifyMetaStorageSubscribersOnReadyForRead(GridCacheDatabaseSharedManager.java:3075)
> at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:1122)
> at
> org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:1713)
> at
> org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1635)
> at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1077)
> at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:587)
> at
> org.apache.ignite.testframework.junits.GridAbstractTest.startGrid(GridAbstractTest.java:1307)
> at
> org.apache.ignite.compatibility.testframework.junits.IgniteCompatibilityAbstractTest.startGrid(IgniteCompatibilityAbstractTest.java:344)
> at
> org.apache.ignite.testframework.junits.GridAbstractTest.startGrid(GridAbstractTest.java:1246)
> at
> org.apache.ignite.testframework.junits.GridAbstractTest.startGrid(GridAbstractTest.java:1222)
> at
> org.apache.ignite.testframework.junits.GridAbstractTest.startGrid(GridAbstractTest.java:1053)
> at
> org.apache.ignite.compatibility.persistence.CheckpointMapSnapshotCompatibilityTest.testSnapshotFileReadableByCurrentVersion(CheckpointMapSnapshotCompatibilityTest.java:121)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at
> java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
> at
> java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.base/java.lang.reflect.Method.invoke(Method.java:569)
> at
> org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
> at
> org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
> at
> org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
> at
> org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
> at
> org.apache.ignite.testframework.junits.GridAbstractTest$6.run(GridAbstractTest.java:2464)
> at java.base/java.lang.Thread.run(Thread.java:840)
> Caused by: java.io.EOFException
> at
> java.base/java.io.ObjectInputStream$BlockDataInputStream.readBoolean(ObjectInputStream.java:3374)
> at
> java.base/java.io.ObjectInputStream.readBoolean(ObjectInputStream.java:1062)
> at
> org.apache.ignite.internal.dto.IgniteDataTransferObjectInput.readBoolean(IgniteDataTransferObjectInput.java:99)
> at
> org.apache.ignite.internal.util.CommonUtils.readUuid(CommonUtils.java:682)
> at
> org.apache.ignite.internal.processors.cache.persistence.checkpoint.EarliestCheckpointMapSnapshotSerializer.readExternal(EarliestCheckpointMapSnapshotSerializer.java:84)
> at
> org.apache.ignite.internal.processors.cache.persistence.checkpoint.EarliestCheckpointMapSnapshotSerializer.readExternal(EarliestCheckpointMapSnapshotSerializer.java:38)
> at
> org.apache.ignite.internal.dto.IgniteDataTransferObject.readIgniteDataTransferObject(IgniteDataTransferObject.java:84)
> at
> org.apache.ignite.internal.dto.IgniteDataTransferObject.readExternal(IgniteDataTransferObject.java:67)
> at
> java.base/java.io.ObjectInputStream.readExternalData(ObjectInputStream.java:2306)
> at
> java.base/java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2255)
> at
> java.base/java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1733)
> at
> java.base/java.io.ObjectInputStream.readObject(ObjectInputStream.java:509)
> at
> java.base/java.io.ObjectInputStream.readObject(ObjectInputStream.java:467)
> at
> org.apache.ignite.marshaller.jdk.JdkMarshallerImpl.unmarshal0(JdkMarshallerImpl.java:120)
> ... 27 more
> {noformat}
> h3. Possible fix
> Make the generated code (or a hand-written {{readExternalData}} in
> {{EarliestCheckpointMapSnapshot}}) able to read the legacy layout, e.g. keep
> {{writeObject/readObject}} for this class or detect the old format. Add the
> compatibility test to the suite.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
