This is an automated email from the ASF dual-hosted git repository. chaokunyang pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/fory-site.git
commit ab712abd79431f908483265e84cb6a9e3a9a32ef Author: chaokunyang <[email protected]> AuthorDate: Tue Apr 21 17:49:40 2026 +0000 🔄 synced local 'docs/specification/' with remote 'docs/specification/' --- docs/specification/xlang_serialization_spec.md | 88 ++++++++++++++++++++++++-- docs/specification/xlang_type_mapping.md | 4 +- 2 files changed, 83 insertions(+), 9 deletions(-) diff --git a/docs/specification/xlang_serialization_spec.md b/docs/specification/xlang_serialization_spec.md index da344a902..5019c70d8 100644 --- a/docs/specification/xlang_serialization_spec.md +++ b/docs/specification/xlang_serialization_spec.md @@ -73,8 +73,8 @@ This specification defines the Fory xlang binary format. The format is dynamic r - duration: an absolute length of time, independent of any calendar/timezone, as a count of nanoseconds. - timestamp: a point in time, independent of any calendar/timezone, encoded as seconds (int64) and nanoseconds (uint32) since the epoch at UTC midnight on January 1, 1970. -- date: a naive date without timezone. The count is days relative to an epoch at UTC midnight on Jan 1, 1970. -- decimal: exact decimal value represented as an integer value in two's complement. +- date: a naive date without timezone, encoded as a signed varint64 count of days since the Unix epoch. +- decimal: an exact decimal value encoded as a signed `scale` and an exact `unscaled` integer. - binary: an variable-length array of bytes. - array: only allow 1d numeric components. Other arrays will be taken as List. The implementation should support the interoperability between array and list. @@ -203,8 +203,8 @@ Named types (`NAMED_*`) do not embed a user ID; their names are carried in metad | 36 | NONE | Empty/unit type (no data) | | 37 | DURATION | Time duration (seconds + nanoseconds) | | 38 | TIMESTAMP | Point in time (seconds + nanoseconds since epoch) | -| 39 | DATE | Date without timezone (days since epoch) | -| 40 | DECIMAL | Arbitrary precision decimal | +| 39 | DATE | Date without timezone (signed varint64 days) | +| 40 | DECIMAL | Arbitrary precision decimal (scale + unscaled) | | 41 | BINARY | Raw binary data | | 42 | ARRAY | Generic array type | | 43 | BOOL_ARRAY | 1D boolean array | @@ -1248,12 +1248,86 @@ This is a fixed-size 12-byte payload (8 bytes seconds + 4 bytes nanos). ### date -Date represents a date without timezone. It is encoded as an `int32` count of days since the Unix epoch -(1970-01-01). This is a fixed-size 4-byte payload. +Date represents a date without timezone. It is encoded as: + +- `days` (varint64): signed count of days since the Unix epoch (`1970-01-01`) + +The value is reconstructed as `LocalDate.ofEpochDay(days)` or the equivalent calendar-date constructor in +the target runtime. + +This `varint64` encoding applies to xlang serialization only. Native, language-specific local-date +encodings are unchanged. ### decimal -Not supported for now. +A decimal value is encoded as: + +1. `scale`: signed varint32 +2. `unscaledHeader`: unsigned varint64 +3. optional `payload`: present only for large unscaled values + +The mathematical value is: + +`value = unscaled × 10^-scale` + +#### Scale + +- `scale` is encoded as signed varint32. +- `scale` carries no extra flags or mode bits. + +#### Unscaled Header + +`unscaledHeader` selects the encoding of `unscaled`: + +- If `(unscaledHeader & 1) == 0`, the value uses the small encoding. +- If `(unscaledHeader & 1) == 1`, the value uses the big encoding. + +#### Small Encoding + +For small values, `unscaled` must fit in signed 64-bit range and the zigzag-encoded value must fit in 63 bits. + +Encoding: + +- `unscaledHeader = zigzag(unscaled) << 1` +- no payload is written + +Decoding: + +- `unscaled = zigzagDecode(unscaledHeader >>> 1)` + +#### Big Encoding + +For big values, `unscaled` is encoded as sign plus magnitude bytes. + +Encoding: + +- `sign = 0` if `unscaled >= 0`, otherwise `1` +- `magnitude = abs(unscaled)` +- `len = byte length of magnitude in canonical minimal little-endian form` +- `meta = (len << 1) | sign` +- `unscaledHeader = (meta << 1) | 1` +- `payload = magnitude as canonical minimal little-endian bytes` + +Decoding: + +- `meta = unscaledHeader >>> 1` +- `sign = meta & 1` +- `len = meta >>> 1` +- read `len` bytes as little-endian unsigned magnitude +- `unscaled = magnitude` if `sign == 0`, otherwise `-magnitude` + +#### Canonical Rules + +- Zero must use the small encoding. +- Big encoding must not be used for zero. +- In big encoding, `payload` must be the minimal little-endian representation. +- Therefore, for big encoding, `len > 0` and `payload[len - 1] != 0`. + +#### Final Value + +After decoding `scale` and `unscaled`, the decimal value is reconstructed as: + +`value = unscaled × 10^-scale` ### struct diff --git a/docs/specification/xlang_type_mapping.md b/docs/specification/xlang_type_mapping.md index 748cb4db9..ee62b1423 100644 --- a/docs/specification/xlang_type_mapping.md +++ b/docs/specification/xlang_type_mapping.md @@ -86,8 +86,8 @@ When reading type IDs: | none | 36 | null | None | null | `std::monostate` | nil | `()` | | duration | 37 | Duration | timedelta | Number | duration | Duration | Duration | | timestamp | 38 | Instant | datetime | Number | std::chrono::nanoseconds | Time | DateTime | -| date | 39 | Date | datetime | Number | fory::serialization::Date | Time | DateTime | -| decimal | 40 | BigDecimal | Decimal | bigint | / | / | / | +| date | 39 | LocalDate | datetime.date | Date | fory::serialization::Date | fory.Date | chrono::NaiveDate | +| decimal | 40 | BigDecimal | Decimal | Decimal | / | fory.Decimal | fory::Decimal | | binary | 41 | byte[] | bytes | / | `uint8_t[n]/vector<T>` | `[n]uint8/[]T` | `Vec<uint8_t>` | | array | 42 | array | np.ndarray | / | / | array/slice | Vec | | bool_array | 43 | bool[] | ndarray(np.bool\_) | / | `bool[n]` | `[n]bool/[]T` | `Vec<bool>` | --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
