This is an automated email from the ASF dual-hosted git repository. chaokunyang pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/fory-site.git
commit 12e2d4bcc50032ce29ce7edb664f9ad87826ea8b Author: chaokunyang <[email protected]> AuthorDate: Sat May 16 15:31:42 2026 +0000 🔄 synced local 'docs/compiler/' with remote 'docs/compiler/' --- docs/compiler/compiler-guide.md | 24 +++-- docs/compiler/generated-code.md | 207 ++++++++++++++++++++++++++++++++++------ docs/compiler/index.md | 6 +- docs/compiler/schema-idl.md | 40 ++++++-- 4 files changed, 227 insertions(+), 50 deletions(-) diff --git a/docs/compiler/compiler-guide.md b/docs/compiler/compiler-guide.md index 8cefb9abf5..6fff34d0b3 100644 --- a/docs/compiler/compiler-guide.md +++ b/docs/compiler/compiler-guide.md @@ -56,7 +56,6 @@ Compile options: | ------------------------------------- | ----------------------------------------------------- | ------------- | | `--lang` | Comma-separated target languages | `all` | | `--output`, `-o` | Output directory | `./generated` | -| `--package` | Override package name from Fory IDL file | (from file) | | `-I`, `--proto_path`, `--import_path` | Add directory to import search path (can be repeated) | (none) | | `--java_out=DST_DIR` | Generate Java code in DST_DIR | (none) | | `--python_out=DST_DIR` | Generate Python code in DST_DIR | (none) | @@ -68,6 +67,7 @@ Compile options: | `--swift_out=DST_DIR` | Generate Swift code in DST_DIR | (none) | | `--dart_out=DST_DIR` | Generate Dart code in DST_DIR | (none) | | `--scala_out=DST_DIR` | Generate Scala 3 code in DST_DIR | (none) | +| `--kotlin_out=DST_DIR` | Generate Kotlin code in DST_DIR | (none) | | `--go_nested_type_style` | Go nested type naming: `camelcase` or `underscore` | `underscore` | | `--swift_namespace_style` | Swift namespace style: `enum` or `flatten` | `enum` | | `--emit-fdl` | Emit translated FDL (for non-FDL inputs) | `false` | @@ -125,7 +125,7 @@ foryc schema.fdl **Compile for specific languages:** ```bash -foryc schema.fdl --lang java,python,csharp,javascript,swift,dart +foryc schema.fdl --lang java,python,csharp,javascript,swift,dart,kotlin ``` **Specify output directory:** @@ -134,12 +134,6 @@ foryc schema.fdl --lang java,python,csharp,javascript,swift,dart foryc schema.fdl --output ./src/generated ``` -**Override package name:** - -```bash -foryc schema.fdl --package com.myapp.models -``` - **Compile multiple files:** ```bash @@ -178,13 +172,16 @@ foryc src/main.fdl -I libs/common,libs/types --proto_path third_party/ foryc schema.fdl --java_out=./src/main/java # Generate multiple languages to different directories -foryc schema.fdl --java_out=./java/gen --python_out=./python/src --go_out=./go/gen --csharp_out=./csharp/gen --javascript_out=./javascript/src --swift_out=./swift/gen --dart_out=./dart/gen +foryc schema.fdl --java_out=./java/gen --python_out=./python/src --go_out=./go/gen --csharp_out=./csharp/gen --javascript_out=./javascript/src --swift_out=./swift/gen --dart_out=./dart/gen --kotlin_out=./kotlin/gen # Combine with import paths foryc schema.fdl --java_out=./gen/java -I proto/ -I common/ # Generate Scala 3 code to a specific directory foryc schema.fdl --scala_out=./src/main/scala + +# Generate Kotlin code to a specific directory +foryc schema.fdl --kotlin_out=./src/main/kotlin ``` When using `--{lang}_out` options: @@ -262,6 +259,7 @@ Compiling src/main.fdl... | Swift | `swift` | `.swift` | Fory Swift model macros | | Dart | `dart` | `.dart` | `@ForyStruct` classes with annotations | | Scala | `scala` | `.scala` | Scala 3 models with macro derivation | +| Kotlin | `kotlin` | `.kt` | Kotlin models with KSP serializers | ## Output Structure @@ -275,12 +273,12 @@ generated/ ├── User.java ├── Order.java ├── Status.java - └── ExampleForyRegistration.java + └── ExampleForyModule.java ``` - One file per type (enum or message) - Package structure matches Fory IDL package -- Registration helper class generated +- Schema module class generated ### Python @@ -400,7 +398,7 @@ generated/ ├── User.scala ├── Status.scala ├── Animal.scala - └── ExampleForyRegistration.scala + └── ExampleForyModule.scala ``` - One Scala 3 source file per generated type @@ -409,7 +407,7 @@ generated/ - `optional T` fields use `Option[T]` - Enums use Scala 3 `enum` - Unions use Scala 3 ADT `enum` with `@ForyUnion`, `@ForyCase`, and an `UnknownCase` -- Registration helper object included +- Schema module object included ### C# IDL Matrix Verification diff --git a/docs/compiler/generated-code.md b/docs/compiler/generated-code.md index 0d8183b6d9..60d7ce7893 100644 --- a/docs/compiler/generated-code.md +++ b/docs/compiler/generated-code.md @@ -21,7 +21,7 @@ license: | This document explains generated code for each target language. -Fory IDL generated types are idiomatic in host languages and can be used directly as domain objects. Generated types also include `to/from bytes` helpers and registration helpers. +Fory IDL generated types are idiomatic in host languages and can be used directly as domain objects. Generated types also include `to/from bytes` helpers and registration helpers or modules. ## Reference Schemas @@ -118,7 +118,11 @@ For `package addressbook`, Java output is generated under: - `<java_out>/addressbook/` - Type files: `AddressBook.java`, `Person.java`, `Dog.java`, `Cat.java`, `Animal.java` -- Registration helper: `AddressbookForyRegistration.java` +- Schema module: `AddressbookForyModule.java` + +For schemas without a Java package, the schema module name is derived from the +source file stem, for example `main.fdl` generates `MainForyModule.java`. +Java import graphs cannot mix default-package schemas with named Java packages. ### Type Generation @@ -177,20 +181,28 @@ public final class Animal extends Union { } ``` -### Registration +### Schema Module -Generated registration helper: +Each JVM schema generates a `ForyModule`. Imported schema modules are installed +through `fory.register(...)`, so shared imports are deduplicated by the runtime. ```java -public static void register(Fory fory) { +public final class AddressbookForyModule implements org.apache.fory.ForyModule { + public static final AddressbookForyModule INSTANCE = new AddressbookForyModule(); + + static ThreadSafeFory getFory() { ... } + + @Override + public void install(Fory fory) { org.apache.fory.resolver.TypeResolver resolver = fory.getTypeResolver(); - resolver.registerUnion(Animal.class, 106L, new org.apache.fory.serializer.UnionSerializer(fory, Animal.class)); + resolver.registerUnion(Animal.class, 106L, new org.apache.fory.serializer.UnionSerializer(resolver, Animal.class)); resolver.register(Person.class, 100L); resolver.register(Person.PhoneType.class, 101L); resolver.register(Person.PhoneNumber.class, 102L); resolver.register(Dog.class, 104L); resolver.register(Cat.class, 105L); resolver.register(AddressBook.class, 103L); + } } ``` @@ -198,9 +210,9 @@ For schemas without explicit `[id=...]`, generated registration uses computed nu ```java resolver.register(Status.class, 1124725126L); -resolver.registerUnion(Wrapper.class, 1471345060L, new org.apache.fory.serializer.UnionSerializer(fory, Wrapper.class)); +resolver.registerUnion(Wrapper.class, 1471345060L, new org.apache.fory.serializer.UnionSerializer(resolver, Wrapper.class)); resolver.register(Envelope.class, 3022445236L); -resolver.registerUnion(Envelope.Detail.class, 1609214087L, new org.apache.fory.serializer.UnionSerializer(fory, Envelope.Detail.class)); +resolver.registerUnion(Envelope.Detail.class, 1609214087L, new org.apache.fory.serializer.UnionSerializer(resolver, Envelope.Detail.class)); resolver.register(Envelope.Payload.class, 2862577837L); ``` @@ -212,7 +224,7 @@ resolver.registerUnion( Holder.class, "myapp.models", "Holder", - new org.apache.fory.serializer.UnionSerializer(fory, Holder.class)); + new org.apache.fory.serializer.UnionSerializer(resolver, Holder.class)); ``` ### Usage @@ -1042,6 +1054,131 @@ void main() { } ``` +## Kotlin + +The Kotlin target emits Kotlin source only. The compiler does not generate Java +files. + +### Output Layout + +For source file `addressbook.fdl` with `package addressbook`, Kotlin output is +generated under: + +- `<kotlin_out>/addressbook/` +- Type files: `AddressBook.kt`, `Person.kt`, `Dog.kt`, `Cat.kt`, `Animal.kt` +- Schema module: `AddressbookForyModule.kt` + +The schema module name is derived from the source file stem. Schemas in the same +Kotlin package need distinct generated file names; duplicate generated Kotlin +file paths are rejected before files are written. + +If `option kotlin_package = "...";` is present, the output path and Kotlin +package use that option. Otherwise Kotlin uses the FDL package. A Kotlin import +graph cannot mix default-package schemas with named Kotlin packages. +Registration still uses the FDL package so cross-language type names stay +stable. + +### Type Generation + +Messages generate Kotlin `data class` declarations by default: + +```kotlin +@ForyStruct +public data class Person( + @field:ForyField(id = 1) + public val name: String, + + @field:ForyField(id = 7) + public val phones: List<PersonPhoneNumber>, + + @field:ForyField(id = 8) + public val pet: Animal, +) { + public fun toBytes(): ByteArray = AddressbookForyModule.getFory().serialize(this) + + public companion object { + public fun fromBytes(bytes: ByteArray): Person = + AddressbookForyModule.getFory().deserialize(bytes, Person::class.java) + } +} +``` + +Messages that participate in compiler-detected construction cycles generate +normal mutable classes so the generated serializer can publish the instance +before reading back-references: + +```kotlin +@ForyStruct +public class Node() { + @ForyField(id = 1) + public var id: String = "" + + @Ref + @ForyField(id = 2) + public var parent: Node? = null +} +``` + +Generated Kotlin IDL sources express nullability with Kotlin `?`, not Fory +`@Nullable`, including mutable classes emitted for compiler-detected +construction cycles. + +Enums generate Kotlin enum classes with stable Fory enum IDs. Unions generate +sealed classes with `@ForyUnion`; case ID `0` is the unknown-case carrier and +schema-defined cases hold a single `value` property. + +```kotlin +@ForyUnion +public sealed class Animal { + @ForyCase(id = 0) + public data class UnknownCase(public val caseId: Int, public val value: Any?) : Animal() + + @ForyCase(id = 1) + public data class DogCase(public val value: Dog) : Animal() +} +``` + +Kotlin `int32`, `int64`, `uint32`, and `uint64` fields use xlang varint +encoding by default, so generated Kotlin does not emit `@VarInt` for the +default case. It emits `@Fixed` or `@Tagged` only when the schema requests that +non-default encoding. `duration` maps to `kotlin.time.Duration`, and infinite +durations are rejected when encoded. Dense `array<float16>` and +`array<bfloat16>` use the Java core `Float16Array` and `BFloat16Array` +carriers. Generated Kotlin IDL uses `@ArrayType ByteArray` for `array<int8>`, +including nested positions. + +### Schema Module + +Generated schema modules register schema types and resolve KSP-generated +serializers from the target class name. The package-owned helper runtime uses +`ForyKotlin.builder()` with the schema module installed, so message +`toBytes`/`fromBytes` helpers work without caller-managed runtime setup. For +`addressbook.fdl`: + +```kotlin +public object AddressbookForyModule : ForyModule { + private val fory: ThreadSafeFory by lazy { + ForyKotlin.builder() + .withXlang(true) + .withCompatible(true) + .withRefTracking(true) + .withModule(this) + .buildThreadSafeFory() + } + + internal fun getFory(): ThreadSafeFory = fory + + override fun install(fory: Fory) { + KotlinSerializers.registerType(fory, Person::class.java, 100L) + KotlinSerializers.registerSerializer(fory, Person::class.java) + KotlinSerializers.registerUnion(fory, Animal::class.java, 106L) + } +} +``` + +`registerUnion` discovers the generated `<Target>_ForySerializer`; callers do +not pass a serializer instance. + ## Scala The Scala target emits Scala 3 source only. The `fory-scala` runtime artifact @@ -1054,7 +1191,12 @@ For `package addressbook`, Scala output is generated under: - `<scala_out>/addressbook/` - Type files: `AddressBook.scala`, `Person.scala`, `Dog.scala`, `Cat.scala`, `Animal.scala` -- Registration helper: `AddressbookForyRegistration.scala` +- Schema module: `AddressbookForyModule.scala` + +For schemas without a Scala package, the schema module name is derived from the +source file stem, for example `main.fdl` generates `MainForyModule.scala`. +Scala import graphs cannot mix default-package schemas with named Scala +packages. ### Type Generation @@ -1070,7 +1212,15 @@ final case class Person( @ForyField(id = 3) email: Option[String], @ForyField(id = 7) phones: List[Person.PhoneNumber], @ForyField(id = 8) pet: Animal -) derives ForySerializer +) derives ForySerializer { + def toBytes(): Array[Byte] = + AddressbookForyModule.getFory.serialize(this) +} + +object Person { + def fromBytes(bytes: Array[Byte]): Person = + AddressbookForyModule.getFory.deserialize(bytes).asInstanceOf[Person] +} ``` Messages in circular construction cycles generate normal classes with mutable @@ -1132,15 +1282,26 @@ enum Animal derives ForySerializer { `@Ref` on the field or constructor parameter. Nested element/value references use type-use annotations such as `List[Node @Ref]`. -### Registration +### Schema Module -Generated registration helpers register Scala serializers, enums, structs, and -unions for `Fory` and `ThreadSafeFory`: +Generated schema modules register schema serializers, enums, structs, and +unions. The package-owned helper runtime uses `ForyScala.builder()` with the +schema module installed, so message `toBytes`/`fromBytes` helpers work without +caller-managed runtime setup: ```scala -object AddressbookForyRegistration { - def register(fory: Fory): Unit = { - ScalaSerializers.registerSerializers(fory) +object AddressbookForyModule extends org.apache.fory.ForyModule { + private lazy val fory: ThreadSafeFory = + ForyScala.builder() + .withXlang(true) + .withCompatible(true) + .withRefTracking(true) + .withModule(this) + .buildThreadSafeFory() + + private[addressbook] def getFory: ThreadSafeFory = fory + + override def install(fory: Fory): Unit = { ScalaSerializers.registerEnum(fory, classOf[Person.PhoneType], 101L) ForySerializer.register(fory, classOf[Person.PhoneNumber], 102L) ForySerializer.register(fory, classOf[Person], 100L) @@ -1149,16 +1310,6 @@ object AddressbookForyRegistration { } ``` -Run the end-to-end Scala IDL matrix with: - -```bash -cd integration_tests/idl_tests -./run_scala_tests.sh -``` - -The runner regenerates Scala fixtures, runs Scala 3 IDL tests, and then runs the -Java peer matrix with `IDL_PEER_LANG=scala`. - ## Cross-Language Notes ### Type ID Behavior @@ -1186,6 +1337,8 @@ Java peer matrix with `IDL_PEER_LANG=scala`. | Language | Helpers | | ---------- | ------------------------- | | Java | `toBytes` / `fromBytes` | +| Kotlin | `toBytes` / `fromBytes` | +| Scala | `toBytes` / `fromBytes` | | Python | `to_bytes` / `from_bytes` | | Rust | `to_bytes` / `from_bytes` | | C++ | `to_bytes` / `from_bytes` | diff --git a/docs/compiler/index.md b/docs/compiler/index.md index e0317ed22b..d0c9f2e5e7 100644 --- a/docs/compiler/index.md +++ b/docs/compiler/index.md @@ -21,7 +21,8 @@ license: | Fory IDL is a schema definition language for Apache Fory that enables type-safe cross-language serialization. Define your data structures once and generate -native data structure code for Java, Python, Go, Rust, C++, C#, Swift, JavaScript, Dart, and Scala. +native data structure code for Java, Python, Go, Rust, C++, C#, Swift, +JavaScript, Dart, Scala, and Kotlin. ## Example Schema @@ -105,6 +106,7 @@ Generated code uses native language constructs: - Swift: Fory model macros with field/case metadata and registration helpers - Dart: `@ForyStruct` classes with `@ForyField` annotations and registration helpers - Scala: Scala 3 `case class`, normal class, enum, and ADT enum models with macro-derived serializers +- Kotlin: Kotlin `data class`, enum, and sealed class models with KSP-generated serializers ## Quick Start @@ -142,7 +144,7 @@ message Person { foryc example.fdl --output ./generated # Generate for specific languages -foryc example.fdl --lang java,python,csharp,javascript,swift,dart,scala --output ./generated +foryc example.fdl --lang java,python,csharp,javascript,swift,dart,scala,kotlin --output ./generated ``` ### 4. Use Generated Code diff --git a/docs/compiler/schema-idl.md b/docs/compiler/schema-idl.md index 784bb9610f..50364d6c27 100644 --- a/docs/compiler/schema-idl.md +++ b/docs/compiler/schema-idl.md @@ -103,6 +103,7 @@ package com.example.models alias models_v1; | C# | Namespace | | JavaScript | Module name (last segment) | | Dart | Library name (package segments) | +| Kotlin | Kotlin package | ## File-Level Options @@ -173,6 +174,28 @@ message Payment { - Output path follows namespace segments (`MyCorp/Payment/V1/` under `--csharp_out`) - Type registration still uses the Fory IDL package (`payment`) for cross-language compatibility +### Kotlin Package Option + +Override the Kotlin package for generated source: + +```protobuf +package payment; +option kotlin_package = "com.mycorp.payment.v1"; + +message Payment { + string id = 1; +} +``` + +**Effect:** + +- Generated Kotlin files are written under `com/mycorp/payment/v1/` +- Kotlin source uses `package com.mycorp.payment.v1` +- Type registration still uses the Fory IDL package (`payment`) for cross-language compatibility + +If `kotlin_package` is absent, Kotlin uses the FDL package. A Kotlin import +graph cannot mix default-package schemas with named Kotlin packages. + ### Go Nested Type Style Option Control Go naming for nested message/enum/union types: @@ -376,9 +399,9 @@ For protobuf extension options, see For language-specific packages/namespaces: -1. Command-line package override (highest priority) -2. Language-specific option (`java_package`, `go_package`, `csharp_namespace`) -3. Fory IDL package declaration (fallback) +1. Language-specific option (`java_package`, `go_package`, `csharp_namespace`, + `kotlin_package`) +2. Fory IDL package declaration (fallback) **Example:** @@ -387,11 +410,10 @@ package myapp.models; option java_package = "com.example.generated"; ``` -| Scenario | Java Package Used | -| ------------------------- | ------------------------- | -| No override | `com.example.generated` | -| CLI: `--package=override` | `override` | -| No java_package option | `myapp.models` (fallback) | +| Scenario | Java Package Used | +| ---------------------- | ------------------------- | +| `java_package` present | `com.example.generated` | +| No `java_package` | `myapp.models` (fallback) | ### Cross-Language Type Registration @@ -1329,6 +1351,8 @@ For handwritten Dart models, `array<bool>` requires `BoolList` plus `list<bool>`. For handwritten Java models, unsigned primitive arrays use type-use annotations on the element type, for example `private @UInt32Type int[] ids;`. +For generated Kotlin models, `array<int8>` uses `@ArrayType ByteArray`, +including nested collection and map positions. #### Map --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
