xtern commented on code in PR #5293: URL: https://github.com/apache/ignite-3/pull/5293#discussion_r1979620918
########## modules/catalog/src/main/java/org/apache/ignite/internal/catalog/descriptors/CatalogSortedIndexDescriptorSerializers.java: ########## @@ -0,0 +1,79 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.ignite.internal.catalog.descriptors; + +import static org.apache.ignite.internal.catalog.storage.serialization.CatalogSerializationUtils.readList; +import static org.apache.ignite.internal.catalog.storage.serialization.CatalogSerializationUtils.writeList; + +import java.io.IOException; +import java.util.List; +import org.apache.ignite.internal.catalog.storage.serialization.CatalogEntrySerializerProvider; +import org.apache.ignite.internal.catalog.storage.serialization.CatalogObjectSerializer; +import org.apache.ignite.internal.catalog.storage.serialization.CatalogSerializer; +import org.apache.ignite.internal.catalog.storage.serialization.MarshallableEntryType; +import org.apache.ignite.internal.util.io.IgniteDataInput; +import org.apache.ignite.internal.util.io.IgniteDataOutput; + +/** + * Serializers for {@link CatalogSortedIndexDescriptor}. + */ +public class CatalogSortedIndexDescriptorSerializers { Review Comment: Thanks, dropped `CatalogIndexColumnDescriptorSerializers`. ########## modules/catalog/README.md: ########## @@ -81,8 +81,65 @@ version stored by `catalog.version` key, and apply those updates entries once ag #### Update log entries serialization -Update log entries are serialized by custom marshallers (see +Update log entry contains an hierarchical structure of objects. + +For example, `NewSchemaEntry` contains `CatalogSchemaDescriptor` which +contains `CatalogTableDescriptor` which contains +`CatalogTableColumnDescriptor` and so on. + +<details> + <summary>NewSchemaEntry hierarchy example</summary> + +* NewSchemaEntry + * CatalogSchemaDescriptor + * CatalogTableDescriptor[] + * CatalogTableSchemaVersions + * CatalogTableColumnDescriptor + * CatalogTableColumnDescriptor\[\] + * ... + * ... + * CatalogIndexDescriptor\[\] +</details> + +To simplify versioning of catalog objects, each serializable catalog object must implement +[MarshallableEntry](src/main/java/org/apache/ignite/internal/catalog/storage/serialization/MarshallableEntry.java) +which allows the serializer to understand what type is being (de)serialized. + +For each serializable object, there must be an external serializer that implements the +[CatalogObjectSerializer](src/main/java/org/apache/ignite/internal/catalog/storage/serialization/CatalogObjectSerializer.java) +interface and is marked with the +[CatalogSerializer](src/main/java/org/apache/ignite/internal/catalog/storage/serialization/CatalogSerializer.java) annotation. +This annotation specifies the serializer version and is used to dynamically build a registry of all existing serializers +(see [CatalogEntrySerializerProvider](src/main/java/org/apache/ignite/internal/catalog/storage/serialization/CatalogEntrySerializerProvider.java)). + +When serializing an object, a header is written for it consisting of the object type (2 bytes) and the serializer version (1-3 bytes). + [UpdateLogMarshaller](src/main/java/org/apache/ignite/internal/catalog/storage/serialization/UpdateLogMarshaller.java) -and [MarshallableEntry](src/main/java/org/apache/ignite/internal/catalog/storage/serialization/MarshallableEntry.java) -for details). At the moment, backward compatibility is preserved by increasing the version of the protocol, -but more sophisticated approach may be introduced later. \ No newline at end of file +is the entry point for catalog object serialization. + +Overall serialization format looks as follow + +| Field description | type | +|-------------------------------------|--------| +| PROTOCOL VERSION<sup>[1]</sup> | short | +| Object type | short | +| Object serialization format version | varint | +| Object payload | ... | + +<sup>[1]</sup> The current description corresponds to protocol version 2. + +##### Serialization versioning rules + +- The serializer version is incremented by 1. + +- Each serializer must be annotated with Review Comment: Thanks, fixed. ########## modules/catalog/src/main/java/org/apache/ignite/internal/catalog/CatalogManagerImpl.java: ########## @@ -461,12 +461,16 @@ public List<SystemView<?>> systemViews() { class OnUpdateHandlerImpl implements OnUpdateHandler { @Override - public CompletableFuture<Void> handle(UpdateLogEvent event, HybridTimestamp metaStorageUpdateTimestamp, long causalityToken) { + public CompletableFuture<Void> handle(MarshallableEntry event, HybridTimestamp metaStorageUpdateTimestamp, long causalityToken) { Review Comment: Done. `UpdateLogEvent` returned back. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: notifications-unsubscr...@ignite.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
