This is an automated email from the ASF dual-hosted git repository.
jamesbognar pushed a commit to branch docs
in repository https://gitbox.apache.org/repos/asf/juneau.git
The following commit(s) were added to refs/heads/docs by this push:
new ea59f8f2e2 feat(marshall): protobuf binary wire-format codec +
binary-format conformance hardening (cbor/msgpack/bson/parquet)
ea59f8f2e2 is described below
commit ea59f8f2e2402e9304179b509c22c1186d301925
Author: James Bognar <[email protected]>
AuthorDate: Wed Jun 17 07:00:40 2026 -0400
feat(marshall): protobuf binary wire-format codec + binary-format
conformance hardening (cbor/msgpack/bson/parquet)
---
pages/release-notes/10.0.0.md | 40 +++++++
pages/topics/02.34.07.ProtobufBinaryBasics.md | 153 ++++++++++++++++++++++++++
sidebars.ts | 5 +
3 files changed, 198 insertions(+)
diff --git a/pages/release-notes/10.0.0.md b/pages/release-notes/10.0.0.md
index 39b40854c5..98ecfd34e3 100644
--- a/pages/release-notes/10.0.0.md
+++ b/pages/release-notes/10.0.0.md
@@ -328,6 +328,46 @@ MarshalledNode.of(doc).set("/foo/bar/-", "z"); //
appends -> ['x','y','z']
Both `MarshalledNode` and `JsonPointer` are marked **Beta — API subject to
change**. See the new [Tree Model & RFC 6901
JSON-Pointer](/docs/topics/MarshalledNode) topic page.
+#### Protobuf binary wire-format codec
+
+Juneau 10.0 adds a native, bean-driven **Protocol Buffers binary** serializer
and parser in the new
+`org.apache.juneau.marshall.protobuf` package — distinct from the existing
text-format `marshall.proto`
+codec (`text/protobuf`). It emits the compact, non-self-describing protobuf
binary wire format used by
+`protoc`-generated code, with **no `.proto` schema file or code generation
required**: the
+field-number/type schema is derived directly from Juneau bean metadata.
+
+```java
+public class Person {
+ public String name;
+ public int age;
+ public Person() {}
+ public Person(String name, int age) { this.name = name; this.age = age; }
+}
+
+byte[] protobuf = Protobuf.of(new Person("Alice", 30));
+Person parsed = Protobuf.to(protobuf, Person.class); // target type
required on read
+```
+
+It is **interop-capable but bean-driven**: lossless out of the box for
Juneau-to-Juneau messaging, and
+upgradable field-by-field to true external `protoc` interop via the new
`@Protobuf(fieldNumber=…, type=…)`
+annotation.
+
+- **Media types:** produces `application/protobuf`; accepts
`application/protobuf` and
+ `application/x-protobuf` (no collision with text `text/protobuf`).
+- **Field numbers:** explicit `@Protobuf(fieldNumber=N)` wins; otherwise
auto-assigned sequentially from 1
+ ordered alphabetically by property name, skipping the reserved 19000–19999
band.
+- **Scalar mapping:** sensible Java→proto defaults (`int`→int32, `long`→int64,
`String`→string,
+ `enum`→int32 ordinal, `byte[]`→bytes, dates/`BigInteger`→lossless string),
each overridable via
+ `@Protobuf(type=…)` (zigzag `sint*`, `fixed*`/`sfixed*`, `uint*`,
enum-by-name).
+- **Containers:** packed repeated scalars, repeated tagged
strings/bytes/messages, length-delimited nested
+ messages, and `Map<K,V>` as proto `map<>` entry sub-messages. Parsing
accepts both packed and unpacked
+ repeated forms.
+- **Deferred (assess-only):** `oneof`, full proto2 presence, unknown-field
retention, and well-known types
+ (`Timestamp`/`Duration`/`Any`/wrappers).
+
+See the new [Protobuf Binary Format Basics](/docs/topics/ProtobufBinaryBasics)
topic page for the full
+scalar/container mapping tables, field-number rules, and round-trip guarantees.
+
### Bug Fixes
_TBD — to be filled in as development continues._
diff --git a/pages/topics/02.34.07.ProtobufBinaryBasics.md
b/pages/topics/02.34.07.ProtobufBinaryBasics.md
new file mode 100644
index 0000000000..644fc7bff6
--- /dev/null
+++ b/pages/topics/02.34.07.ProtobufBinaryBasics.md
@@ -0,0 +1,153 @@
+---
+title: "Protobuf Binary Format Basics"
+slug: ProtobufBinaryBasics
+---
+
+Juneau supports converting arbitrary POJOs to and from the Protocol Buffers
**binary** wire format using native, bean-driven serializers and parsers. No
`.proto` schema file or code generation is required — the
field-number/type schema is derived directly from Juneau bean metadata.
+
+The <a
href="/site/apidocs/org/apache/juneau/marshall/protobuf/ProtobufSerializer.html"
target="_blank">ProtobufSerializer</a> converts POJOs directly to protobuf
binary bytes, and the <a
href="/site/apidocs/org/apache/juneau/marshall/protobuf/ProtobufParser.html"
target="_blank">ProtobufParser</a> creates POJOs directly from protobuf binary
bytes.
+
+This is **distinct from** the text-format <a
href="/docs/topics/ProtobufBasics">Protobuf Text Format</a> marshaller
(`text/protobuf`). This codec emits the compact, non-self-describing protobuf
binary wire format used by `protoc`-generated code.
+
+It is **interop-capable but bean-driven**: it works losslessly out of the box
for Juneau-to-Juneau messaging, and upgrades field-by-field to true external
`protoc` interop when you supply explicit field numbers and scalar types via
the `@Protobuf` annotation.
+
+##### Maven Dependency
+
+Protobuf binary support is included in `juneau-marshall`:
+
+```xml
+<dependency>
+ <groupId>org.apache.juneau</groupId>
+ <artifactId>juneau-marshall</artifactId>
+ <version>${juneau.version}</version>
+</dependency>
+```
+
+##### Quick Start
+
+```java
+// A simple bean - properties are sorted alphabetically: id=1, name=2
+public class Person {
+ public String name;
+ public int age;
+ public Person() {}
+ public Person(String name, int age) { this.name = name; this.age = age; }
+}
+
+// Serialize a bean to protobuf binary bytes
+Person alice = new Person("Alice", 30);
+byte[] protobuf = Protobuf.of(alice);
+
+// Parse protobuf binary bytes back into a bean (target type is required)
+Person parsed = Protobuf.to(protobuf, Person.class);
+```
+
+##### Serialization
+
+```java
+// Use the default serializer
+byte[] protobuf = ProtobufSerializer.DEFAULT.serialize(someObject);
+
+// Use the marshaller for convenience
+byte[] protobuf = Protobuf.of(someObject);
+
+// Serialize to OutputStream
+try (OutputStream os = new FileOutputStream("data.pb")) {
+ Protobuf.of(someObject, os);
+}
+```
+
+##### Parsing
+
+Protobuf binary is **not self-describing**, so the target type is always
required on read.
+
+```java
+// Parse protobuf binary into a bean
+Person p = Protobuf.to(protobuf, Person.class);
+
+// Parse from InputStream
+try (InputStream is = new FileInputStream("data.pb")) {
+ Person p = Protobuf.to(is, Person.class);
+}
+```
+
+##### Java-to-Protobuf Scalar Mapping
+
+Defaults are derived from the Java property type. All are overridable
per-property via `@Protobuf(type=...)`.
+
+| Java Type | Default Proto Type | Wire Type | Notes |
+|-----------|--------------------|-----------|-------|
+| `boolean`/`Boolean` | bool | VARINT | |
+| `byte`/`short`/`int`/`Integer` | int32 | VARINT | Plain (not zigzag) |
+| `long`/`Long` | int64 | VARINT | Plain |
+| `float`/`Float` | float | I32 | IEEE 754, little-endian |
+| `double`/`Double` | double | I64 | IEEE 754, little-endian |
+| `String` | string | LEN | UTF-8 |
+| `byte[]` | bytes | LEN | Native binary |
+| `enum` | int32 | VARINT | Ordinal by default |
+| `BigInteger`/`BigDecimal`/`char` | string | LEN | Lossless string form (no
native proto scalar) |
+| `Date`/`Calendar`/`Temporal`/`Duration`/`Period` | string | LEN | ISO-8601
(lossless) |
+
+The following proto scalar types are available via `@Protobuf(type=...)` for
`protoc` interop: `SINT32`/`SINT64` (zigzag), `FIXED32`/`FIXED64`,
`SFIXED32`/`SFIXED64`, `UINT32`/`UINT64`, and `ENUM_STRING` (encode enums by
name instead of ordinal).
+
+##### Container Encodings
+
+These follow the proto3 wire conventions so they interoperate with real
`protoc`-generated messages:
+
+| Java Container | Protobuf Encoding |
+|----------------|-------------------|
+| Repeated numeric / bool / enum (`int[]`, `List<Integer>`, ...) | **Packed**
(single length-delimited block) |
+| Repeated string / bytes / message | Repeated tagged entries (cannot be
packed) |
+| Nested bean | Length-delimited embedded message |
+| `Map<K,V>` | Repeated `entry { key=1; value=2 }` sub-messages (proto `map<>`
shape) |
+
+On parse, repeated scalar fields accept **both** the packed and the unpacked
(repeated-tagged) forms for interop tolerance.
+
+##### Field-Number Assignment
+
+Protobuf wire bytes carry field **numbers**, not names, so every property is
assigned a stable number:
+
+1. **Explicit numbers win.** `@Protobuf(fieldNumber=N)` claims number `N`.
+2. **Auto-assignment** fills the remaining properties sequentially starting at
`1`, **ordered alphabetically by property name**, skipping any
explicitly-claimed numbers.
+3. The reserved **19000–19999** band is always skipped.
+
+```java
+public class Order {
+ @Protobuf(fieldNumber=10)
+ public String id; // pinned to field 10
+
+ @Protobuf(type=ProtobufScalarType.SINT32)
+ public int delta; // zigzag-encoded; auto field number
+
+ public String customer; // auto field number
+}
+```
+
+Reordering properties in source does **not** change field numbers; renaming a
property may shift its auto-assigned number (an explicit source change). For
long-term external interop, pin numbers with `@Protobuf(fieldNumber=...)`.
+
+##### Presence / Null / Round-Trip Guarantees
+
+- A field is emitted only if its value is **non-null**; an absent field leaves
the property at its default/null on parse.
+- Round-tripping is lossless for nullable properties and for all
scalar/container shapes above.
+- **proto3 zero-value caveat:** an unboxed primitive left at its zero value
(e.g. `int` `0`, `boolean` `false`) is emitted, but on the wire a zero value is
indistinguishable from "unset". Use boxed types (`Integer`, `Boolean`) if you
need to distinguish "zero" from "absent".
+- **uint64:** a `long`-typed field carries the raw 64-bit pattern (surfaced
via `Long.toUnsignedString`); a `BigInteger`-typed field carries the full
unsigned magnitude losslessly.
+
+##### Media Types
+
+- **Produces**: `application/protobuf`
+- **Accepts**: `application/protobuf`, `application/x-protobuf`
+
+(No collision with the text-format marshaller, which uses `text/protobuf`.)
+
+##### Known Limitations
+
+- **Parse requires the target type.** There is no schema in the bytes, so
parsing arbitrary unknown protobuf into a generic map is a non-goal.
+- **Unknown fields are skipped, not preserved.** Unknown field numbers are
skipped correctly by wire type (parsing never breaks) but are not retained on
re-serialization.
+- **Deferred (assess-only):** `oneof`, full proto2 presence/required/defaults,
unknown-field retention, well-known types
(`Timestamp`/`Duration`/`Any`/wrappers), and groups (deprecated) are not
implemented in this iteration.
+
+##### See Also
+
+- <a href="https://protobuf.dev/programming-guides/encoding/";
target="_blank">Protocol Buffers Encoding</a>
+- <a href="/docs/topics/ProtobufBasics">Protobuf Text Format Basics</a>
+- <a href="/site/apidocs/org/apache/juneau/marshall/marshaller/Protobuf.html"
target="_blank">Protobuf marshaller</a>
+- <a
href="/site/apidocs/org/apache/juneau/marshall/protobuf/package-summary.html"
target="_blank">org.apache.juneau.marshall.protobuf package</a>
diff --git a/sidebars.ts b/sidebars.ts
index 82f7157a41..f23d57bde2 100644
--- a/sidebars.ts
+++ b/sidebars.ts
@@ -665,6 +665,11 @@ const sidebars: SidebarsConfig = {
id:
'topics/02.34.05.ProtobufBasics',
label:
'2.34.5.1. Protobuf Text Format Basics',
},
+ {
+ type:
'doc',
+ id:
'topics/02.34.07.ProtobufBinaryBasics',
+ label:
'2.34.5.2. Protobuf Binary Format Basics',
+ },
{
type:
'doc',
id:
'topics/02.34.06.ParquetBasics',
