AMashenkov commented on code in PR #5293: URL: https://github.com/apache/ignite-3/pull/5293#discussion_r1977683185
########## modules/catalog/README.md: ########## @@ -81,8 +81,65 @@ version stored by `catalog.version` key, and apply those updates entries once ag #### Update log entries serialization -Update log entries are serialized by custom marshallers (see +Update log entry contains an hierarchical structure of objects. + +For example, `NewSchemaEntry` contains `CatalogSchemaDescriptor` which +contains `CatalogTableDescriptor` which contains +`CatalogTableColumnDescriptor` and so on. + +<details> + <summary>NewSchemaEntry hierarchy example</summary> + +* NewSchemaEntry + * CatalogSchemaDescriptor + * CatalogTableDescriptor[] + * CatalogTableSchemaVersions + * CatalogTableColumnDescriptor + * CatalogTableColumnDescriptor\[\] + * ... + * ... + * CatalogIndexDescriptor\[\] +</details> + +To simplify versioning of catalog objects, each serializable catalog object must implement +[MarshallableEntry](src/main/java/org/apache/ignite/internal/catalog/storage/serialization/MarshallableEntry.java) +which allows the serializer to understand what type is being (de)serialized. + +For each serializable object, there must be an external serializer that implements the +[CatalogObjectSerializer](src/main/java/org/apache/ignite/internal/catalog/storage/serialization/CatalogObjectSerializer.java) +interface and is marked with the +[CatalogSerializer](src/main/java/org/apache/ignite/internal/catalog/storage/serialization/CatalogSerializer.java) annotation. +This annotation specifies the serializer version and is used to dynamically build a registry of all existing serializers +(see [CatalogEntrySerializerProvider](src/main/java/org/apache/ignite/internal/catalog/storage/serialization/CatalogEntrySerializerProvider.java)). + +When serializing an object, a header is written for it consisting of the object type (2 bytes) and the serializer version (1-3 bytes). + [UpdateLogMarshaller](src/main/java/org/apache/ignite/internal/catalog/storage/serialization/UpdateLogMarshaller.java) -and [MarshallableEntry](src/main/java/org/apache/ignite/internal/catalog/storage/serialization/MarshallableEntry.java) -for details). At the moment, backward compatibility is preserved by increasing the version of the protocol, -but more sophisticated approach may be introduced later. \ No newline at end of file +is the entry point for catalog object serialization. + +Overall serialization format looks as follow + +| Field description | type | +|-------------------------------------|--------| +| PROTOCOL VERSION<sup>[1]</sup> | short | +| Object type | short | +| Object serialization format version | varint | +| Object payload | ... | + +<sup>[1]</sup> The current description corresponds to protocol version 2. + +##### Serialization versioning rules + +- The serializer version is incremented by 1. + +- Each serializer must be annotated with Review Comment: Seems, "type=N" is not mandatory option. Is this description still actual? Please, recheck. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@ignite.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
