This is an automated email from the ASF dual-hosted git repository.
jamesbognar pushed a commit to branch docs
in repository https://gitbox.apache.org/repos/asf/juneau.git
The following commit(s) were added to refs/heads/docs by this push:
new 3dc2a6c027 Bean streaming docs
3dc2a6c027 is described below
commit 3dc2a6c0275d2b5e9590cbef10b0b3cd1c1e5f44
Author: James Bognar <[email protected]>
AuthorDate: Mon Apr 6 08:41:54 2026 -0400
Bean streaming docs
---
.settings/.gitignore | 1 +
pages/release-notes/9.2.1.md | 97 ++++++-
pages/topics/02.17a.LargeDatasetStreaming.md | 416 +++++++++++++++++++++++++++
sidebars.ts | 5 +
4 files changed, 518 insertions(+), 1 deletion(-)
diff --git a/.settings/.gitignore b/.settings/.gitignore
new file mode 100644
index 0000000000..b853d18584
--- /dev/null
+++ b/.settings/.gitignore
@@ -0,0 +1 @@
+/org.sonarlint.eclipse.core.prefs
diff --git a/pages/release-notes/9.2.1.md b/pages/release-notes/9.2.1.md
index 9ae953978c..01be36dfd5 100644
--- a/pages/release-notes/9.2.1.md
+++ b/pages/release-notes/9.2.1.md
@@ -6,7 +6,7 @@ title: "Release 9.2.1"
**Date:** TBD
-Juneau 9.2.1 is a minor release with native TOML and YAML support, BSON
(Binary JSON) support for MongoDB-interoperable binary serialization, CBOR
(Concise Binary Object Representation) per RFC 8949 for IoT and constrained
environments, full CSV serializer/parser support, JCS (JSON Canonicalization
Scheme) per RFC 8785 for deterministic hashing and signing, RDF/THRIFT and
RDF/PROTO binary format support, native serialization support for
lazy-evaluated sequence types, full support for Jav [...]
+Juneau 9.2.1 is a minor release with native TOML and YAML support, BSON
(Binary JSON) support for MongoDB-interoperable binary serialization, CBOR
(Concise Binary Object Representation) per RFC 8949 for IoT and constrained
environments, full CSV serializer/parser support, JCS (JSON Canonicalization
Scheme) per RFC 8785 for deterministic hashing and signing, RDF/THRIFT and
RDF/PROTO binary format support, native serialization support for
lazy-evaluated sequence types, large-dataset stream [...]
### juneau-marshall
@@ -479,6 +479,101 @@ String json3 =
Json5Serializer.DEFAULT.serialize(iterable);
The POJO categories table has been updated. `Iterator`, `Iterable`,
`Enumeration`, and `Stream` are no longer in the "swapped objects" category
(group 4b). They are now in a new group (2c) alongside Collections and arrays
as natively serializable sequence types.
+#### Large-Dataset Streaming APIs
+
+New interfaces and annotations enable streaming serialization and parsing of
arbitrarily large
+datasets without loading all elements into memory. Every serializer and parser
format supports
+these APIs transparently.
+
+##### New Classes
+
+- **`BeanSupplier<T>`** — Serialization-side streaming interface (extends
`Iterable<T>`).
+ Provides beans lazily from a cursor or query via `begin()`, `iterator()`,
`onError()`, and `complete()` lifecycle methods.
+- **`BeanConsumer<T>`** — Parse-side streaming interface (extends
`ThrowingConsumer<T>`).
+ Receives beans one-at-a-time from any parser for insertion or processing,
with the same lifecycle methods.
+- **`BeanChannel<T>`** — Combines `BeanSupplier` and `BeanConsumer` for
round-trip marshalling
+ on a single property. The same object drives both serialization and parsing.
+- **`ListBeanChannel<T>`** — Built-in in-memory `BeanChannel` backed by an
`ArrayList`, useful
+ for testing and small in-memory datasets.
+- **`BeanFactory<T>`** — Universal `@FunctionalInterface` for creating
instances of any type,
+ enabling integration with dependency-injection frameworks such as Spring.
+
+##### New Annotations
+
+- **`@Bean(factory=X.class)`** — Specifies the `BeanFactory` class used to
instantiate a type
+ during parsing. The factory is resolved from the `BeanStore` (e.g. a Spring
`ApplicationContext`).
+- **`@Beanp(factory=X.class)`** — Property-level factory for a specific bean
property value.
+- **`@Beanp(elementType=Y.class)`** — Declares the element type for generic
streaming properties
+ (`Stream<Y>`, `BeanSupplier<Y>`, `BeanConsumer<Y>`, `BeanChannel<Y>`) to
overcome Java type
+ erasure. Also supports narrowing to concrete implementation types.
+
+##### New API
+
+- **`BeanContext.Builder.beanStore(BeanStore)`** — Wires a `BeanStore` (e.g.
`SpringBeanStore`)
+ into the marshalling context so `@Bean(factory=)` factories can be resolved
from a DI container.
+- **`ParserSession.parseToBeanConsumer(Object, BeanConsumer<T>, Class<T>)`** —
Parses an input
+ source directly into a `BeanConsumer`, driving the full lifecycle
automatically.
+
+##### Lifecycle
+
+All three interfaces share the same three-phase lifecycle: `begin()` for
setup, the transfer
+method (`iterator()` or `acceptThrows()`) for element-by-element data flow,
`onError()` for
+error handling, and `complete()` for cleanup. `complete()` is always called
regardless of
+whether an error occurred.
+
+##### Key Behaviors
+
+- **Direction validation**: using a `BeanConsumer` during serialization or a
`BeanSupplier`
+ during parsing throws an informative exception recommending `BeanChannel`
for bidirectional use.
+- **Getter-only `BeanConsumer` properties**: parsers reuse a `BeanConsumer`
exposed via a getter
+ (no setter required), calling `acceptThrows()` on the existing instance —
matching the existing
+ behavior for mutable `Collection` properties.
+- **`Supplier<T>` unwrapping**: the standard JDK `Supplier<T>` (not
`BeanSupplier`) is
+ transparently unwrapped by serializers as a single lazy value, recursively
up to depth 10.
+- **`Stream<T>` auto-close**: `java.util.stream.Stream` objects used during
serialization are
+ automatically closed via try-with-resources after iteration completes.
+
+##### Example
+
+```java
+// Serialize a large table from a JDBC cursor — no memory materialization.
+@Bean(factory=EmployeeSupplier.Factory.class)
+public class EmployeeSupplier implements BeanSupplier<Employee> {
+ private Connection conn;
+ private ResultSet rs;
+
+ @Override public void begin() throws Exception {
+ conn = dataSource.getConnection();
+ rs = conn.prepareStatement("SELECT * FROM employee").executeQuery();
+ }
+ @Override public Iterator<Employee> iterator() { return
Employee.cursorIterator(rs); }
+ @Override public void complete() throws Exception { rs.close();
conn.close(); }
+}
+
+// Bulk-insert with batch commits — no memory materialization.
+@Bean(factory=EmployeeConsumer.Factory.class)
+public class EmployeeConsumer implements BeanConsumer<Employee> {
+ private Connection conn;
+ private PreparedStatement stmt;
+ private int count;
+
+ @Override public void begin() throws Exception {
+ conn = dataSource.getConnection();
+ conn.setAutoCommit(false);
+ stmt = conn.prepareStatement("INSERT INTO employee (name, dept) VALUES
(?,?)");
+ }
+ @Override public void acceptThrows(Employee e) throws Exception {
+ stmt.setString(1, e.getName()); stmt.setString(2, e.getDepartment());
+ stmt.executeUpdate();
+ if (++count % 500 == 0) conn.commit();
+ }
+ @Override public void onError(Exception e) throws Exception {
conn.rollback(); throw e; }
+ @Override public void complete() throws Exception { conn.commit();
stmt.close(); conn.close(); }
+}
+```
+
+See [Large-Dataset Streaming](/docs/topics/LargeDatasetStreaming) for full
documentation.
+
#### Java Records Support
Java records are now fully supported for both serialization and parsing across
all serializers and parsers.
diff --git a/pages/topics/02.17a.LargeDatasetStreaming.md
b/pages/topics/02.17a.LargeDatasetStreaming.md
new file mode 100644
index 0000000000..a8040f416c
--- /dev/null
+++ b/pages/topics/02.17a.LargeDatasetStreaming.md
@@ -0,0 +1,416 @@
+---
+title: "Large-Dataset Streaming"
+slug: LargeDatasetStreaming
+---
+
+Juneau supports streaming serialization and parsing of arbitrarily large
datasets without loading
+all elements into memory. The feature is built directly into the serializer
and parser APIs, so
+every format (JSON, XML, YAML, CSV, MessagePack, …) works transparently.
+
+## Key Interfaces
+
+| Interface | Extends | Direction | Purpose |
+|---|---|---|---|
+| `BeanSupplier<T>` | `Iterable<T>` | Serialization | Provides beans to
serializers lazily from a cursor or query |
+| `BeanConsumer<T>` | `ThrowingConsumer<T>` | Parsing | Receives beans
one-at-a-time from a parser for insertion/processing |
+| `BeanChannel<T>` | `BeanSupplier<T>` + `BeanConsumer<T>` | Both | Combines
both roles for round-trip marshalling on a single property |
+| `ListBeanChannel<T>` | `BeanChannel<T>` | Both | Built-in in-memory
implementation backed by an `ArrayList` |
+| `BeanFactory<T>` | — | Factory | Creates instances of any type `T`, enabling
DI-framework integration |
+
+## Lifecycle
+
+All three interfaces share the same three-phase lifecycle, driven by the
framework:
+
+| Phase | BeanSupplier (serialization) | BeanConsumer (parsing) |
+|---|---|---|
+| **Setup** | `begin()` — open cursor, execute query | `begin()` — open
connection, prepare statement |
+| **Transfer** | `iterator()` — yield one bean per call | `acceptThrows(T)` —
receive one bean per call |
+| **Error** | `onError(Exception)` — rollback / log; rethrow to stop |
`onError(Exception)` — rollback / log; absorb to skip-and-continue |
+| **Cleanup** | `complete()` — close cursor, connection | `complete()` — final
commit, close statement |
+
+`complete()` is **always** called — even when `onError()` rethrows — so it is
safe to use for
+resource cleanup in all cases.
+
+---
+
+## BeanSupplier — Serializing from a Database Cursor
+
+The following example streams a large `Employee` table from a JDBC cursor
directly to an HTTP
+response, one row at a time, without ever loading the full table into memory.
+
+```java
+@Bean(factory=EmployeeSupplier.Factory.class)
+public class EmployeeSupplier implements BeanSupplier<Employee> {
+
+ private final DataSource ds;
+ private Connection conn;
+ private ResultSet rs;
+
+ public EmployeeSupplier(DataSource ds) {
+ this.ds = ds;
+ }
+
+ @Override
+ public void begin() throws Exception {
+ conn = ds.getConnection();
+ var stmt = conn.prepareStatement(
+ "SELECT id, name, department FROM employee ORDER BY id");
+ rs = stmt.executeQuery();
+ }
+
+ @Override
+ public Iterator<Employee> iterator() {
+ return new Iterator<>() {
+ @Override public boolean hasNext() { return
ResultSetIterator.hasNext(rs); }
+ @Override public Employee next() { return Employee.fromRow(rs); }
+ };
+ }
+
+ @Override
+ public void onError(Exception e) throws Exception {
+ throw e; // propagate; complete() will still close the cursor
+ }
+
+ @Override
+ public void complete() throws Exception {
+ if (rs != null) rs.close();
+ if (conn != null) conn.close();
+ }
+
+ // Factory is retrieved from Spring ApplicationContext via BeanStore.
+ public static class Factory implements BeanFactory<EmployeeSupplier> {
+ private final DataSource ds;
+ public Factory(DataSource ds) { this.ds = ds; }
+
+ @Override
+ public EmployeeSupplier create() {
+ return new EmployeeSupplier(ds);
+ }
+ }
+}
+```
+
+In a Spring REST resource:
+
+```java
+@Rest
+public class EmployeeResource extends BasicRestServlet {
+
+ @Inject
+ private EmployeeSupplier.Factory supplierFactory;
+
+ @RestGet("/employees")
+ public EmployeeSupplier getEmployees() {
+ return supplierFactory.create(); // framework calls begin(), iterates,
calls complete()
+ }
+}
+```
+
+The serializer calls `begin()` before iterating, serializes each `Employee` as
it arrives, and
+always calls `complete()` afterward — whether serialization succeeded or
failed.
+
+---
+
+## BeanConsumer — Parsing into a Database Table
+
+The following example accepts a large JSON array from an HTTP request body and
bulk-inserts the
+rows via JDBC, committing every 500 rows.
+
+```java
+@Bean(factory=EmployeeConsumer.Factory.class)
+public class EmployeeConsumer implements BeanConsumer<Employee> {
+
+ private static final int BATCH_SIZE = 500;
+
+ private final DataSource ds;
+ private Connection conn;
+ private PreparedStatement stmt;
+ private int count;
+
+ public EmployeeConsumer(DataSource ds) {
+ this.ds = ds;
+ }
+
+ @Override
+ public void begin() throws Exception {
+ conn = ds.getConnection();
+ conn.setAutoCommit(false);
+ stmt = conn.prepareStatement(
+ "INSERT INTO employee (name, department) VALUES (?, ?)");
+ }
+
+ @Override
+ public void acceptThrows(Employee emp) throws Exception {
+ stmt.setString(1, emp.getName());
+ stmt.setString(2, emp.getDepartment());
+ stmt.executeUpdate();
+ if (++count % BATCH_SIZE == 0)
+ conn.commit(); // periodic batch commit
+ }
+
+ @Override
+ public void onError(Exception e) throws Exception {
+ conn.rollback();
+ throw e; // stop parsing; complete() will still close resources
+ }
+
+ @Override
+ public void complete() throws Exception {
+ conn.commit(); // final commit for the last partial batch
+ stmt.close();
+ conn.close();
+ }
+
+ public static class Factory implements BeanFactory<EmployeeConsumer> {
+ private final DataSource ds;
+ public Factory(DataSource ds) { this.ds = ds; }
+
+ @Override
+ public EmployeeConsumer create() {
+ return new EmployeeConsumer(ds);
+ }
+ }
+}
+```
+
+Parsing via `parseToBeanConsumer`:
+
+```java
+// Direct API — framework calls begin(), acceptThrows() per element,
complete().
+var consumer = consumerFactory.create();
+JsonParser.DEFAULT.getSession().parseToBeanConsumer(inputStream, consumer,
Employee.class);
+```
+
+In a Spring REST resource:
+
+```java
+@Rest
+public class EmployeeResource extends BasicRestServlet {
+
+ @Inject
+ private EmployeeConsumer.Factory consumerFactory;
+
+ @RestPost("/employees/bulk")
+ public void importEmployees(RestRequest req) throws Exception {
+ var consumer = consumerFactory.create();
+ req.getBody().parseToBeanConsumer(consumer, Employee.class);
+ }
+}
+```
+
+### Fault-Tolerant Ingestion (Skip-and-Continue)
+
+Override `onError()` to absorb exceptions instead of rethrowing. Parsing then
continues to the
+next element, skipping only the bad record:
+
+```java
+BeanConsumer<Employee> consumer = new BeanConsumer<>() {
+ @Override
+ public void acceptThrows(Employee emp) throws Exception {
+ validateAndInsert(emp);
+ }
+ @Override
+ public void onError(Exception e) {
+ log.warn("Skipping invalid record: {}", e.getMessage());
+ // absorb — parsing continues to the next element
+ }
+};
+```
+
+---
+
+## BeanChannel — Round-Trip Marshalling
+
+`BeanChannel<T>` extends both `BeanSupplier<T>` and `BeanConsumer<T>`,
allowing the same object
+to drive both serialization and parsing. The implementation decides which
direction is active at
+runtime.
+
+### In-Memory: ListBeanChannel
+
+`ListBeanChannel<T>` is a built-in implementation backed by an `ArrayList`. It
requires no
+factory setup and is useful for testing, small datasets, or property-level
round-trips:
+
+```java
+public class EmployeeCollection {
+
+ private final ListBeanChannel<Employee> employees = new
ListBeanChannel<>();
+
+ // No setter needed — parser calls acceptThrows() on the existing instance.
+ @Beanp(elementType=Employee.class)
+ public ListBeanChannel<Employee> getEmployees() { return employees; }
+}
+
+// Serialize — channel iterated as a sequence.
+String json = Json5Serializer.DEFAULT.serialize(collection);
+
+// Parse — channel populated via acceptThrows().
+Json5Parser.DEFAULT.parse(json, EmployeeCollection.class);
+```
+
+### Database-Backed Channel
+
+```java
+@Bean(factory=EmployeeChannel.Factory.class)
+public class EmployeeChannel implements BeanChannel<Employee> {
+
+ private final DataSource ds;
+ private Connection conn;
+ private ResultSet rs;
+ private PreparedStatement insertStmt;
+ private int insertCount;
+
+ public EmployeeChannel(DataSource ds) {
+ this.ds = ds;
+ }
+
+ // ---- BeanSupplier side (serialization) ----
+
+ @Override
+ public void begin() throws Exception {
+ conn = ds.getConnection();
+ insertStmt = conn.prepareStatement(
+ "INSERT INTO employee (name, department) VALUES (?, ?)");
+ conn.setAutoCommit(false);
+ var qStmt = conn.prepareStatement(
+ "SELECT name, department FROM employee ORDER BY id");
+ rs = qStmt.executeQuery();
+ }
+
+ @Override
+ public Iterator<Employee> iterator() {
+ return new Iterator<>() {
+ @Override public boolean hasNext() { return
ResultSetIterator.hasNext(rs); }
+ @Override public Employee next() { return Employee.fromRow(rs); }
+ };
+ }
+
+ // ---- BeanConsumer side (parsing) ----
+
+ @Override
+ public void acceptThrows(Employee emp) throws Exception {
+ insertStmt.setString(1, emp.getName());
+ insertStmt.setString(2, emp.getDepartment());
+ insertStmt.executeUpdate();
+ if (++insertCount % 500 == 0) conn.commit();
+ }
+
+ // ---- Shared lifecycle ----
+
+ @Override
+ public void onError(Exception e) throws Exception {
+ if (conn != null) conn.rollback();
+ throw e;
+ }
+
+ @Override
+ public void complete() throws Exception {
+ if (conn != null) {
+ conn.commit();
+ if (rs != null) rs.close();
+ if (insertStmt != null) insertStmt.close();
+ conn.close();
+ }
+ }
+
+ public static class Factory implements BeanFactory<EmployeeChannel> {
+ private final DataSource ds;
+ public Factory(DataSource ds) { this.ds = ds; }
+
+ @Override
+ public EmployeeChannel create() { return new EmployeeChannel(ds); }
+ }
+}
+```
+
+---
+
+## Spring Integration via BeanFactory and BeanStore
+
+Spring-managed beans (e.g. a `DataSource`) can be injected into streaming
implementations
+without reflection-based construction scanning. Register the factories as
Spring beans, then
+wire a `SpringBeanStore` into the Juneau context.
+
+### Spring Configuration
+
+```java
+@Configuration
+public class JuneauStreamingConfig {
+
+ @Bean
+ public EmployeeSupplier.Factory employeeSupplierFactory(DataSource ds) {
+ return new EmployeeSupplier.Factory(ds);
+ }
+
+ @Bean
+ public EmployeeConsumer.Factory employeeConsumerFactory(DataSource ds) {
+ return new EmployeeConsumer.Factory(ds);
+ }
+
+ @Bean
+ public EmployeeChannel.Factory employeeChannelFactory(DataSource ds) {
+ return new EmployeeChannel.Factory(ds);
+ }
+}
+```
+
+### Wiring SpringBeanStore
+
+```java
+@Rest
+public class EmployeeResource extends BasicSpringRestServlet {
+
+ @Inject
+ private ApplicationContext appCtx;
+
+ @Override
+ protected BeanContext.Builder createBeanContext(RestContext.Builder
rcBuilder) {
+ return super.createBeanContext(rcBuilder)
+ .beanStore(new SpringBeanStore(appCtx));
+ }
+}
+```
+
+With this wiring in place, any class annotated with `@Bean(factory=X.class)`
is automatically
+instantiated by retrieving the factory from the Spring `ApplicationContext`
and calling
+`BeanFactory.create()`. No manual construction is needed in individual REST
methods.
+
+---
+
+## Supporting Annotations
+
+| Annotation | Target | Purpose |
+|---|---|---|
+| `@Bean(factory=X.class)` | Class | Specifies the `BeanFactory` class used to
instantiate this type during parsing. The factory is resolved from the
`BeanStore`. |
+| `@Beanp(factory=X.class)` | Bean property | Specifies a property-level
factory for a specific bean property value. |
+| `@Beanp(elementType=Y.class)` | Bean property | Declares the element type
for generic streaming properties (`Stream<Y>`, `BeanSupplier<Y>`, etc.),
overcoming Java type erasure. Also supports narrowing to concrete
implementation types. |
+
+---
+
+## Supplier<T> Single-Value Unwrapping
+
+The standard JDK `Supplier<T>` (not `BeanSupplier`) is treated as a
single-value lazy wrapper.
+Serializers call `get()` and serialize the result transparently. Nested
`Supplier` chains are
+unwrapped recursively up to a depth of 10.
+
+:::note
+`BeanSupplier` is **not** unwrapped — it is treated as an `Iterable` sequence,
not a
+single-value wrapper.
+:::
+
+```java
+// Serialized as the string "hello" — Supplier is unwrapped.
+Supplier<String> lazy = () -> "hello";
+String json = Json5Serializer.DEFAULT.serialize(lazy); // 'hello'
+
+// Nested Supplier chains are also unwrapped.
+Supplier<Supplier<Integer>> nested = () -> () -> 42;
+json = Json5Serializer.DEFAULT.serialize(nested); // 42
+```
+
+---
+
+## Direction Validation
+
+Using a `BeanConsumer` during serialization or a `BeanSupplier` during parsing
throws an
+appropriate exception with a message recommending `BeanChannel` for
bidirectional use cases.
+`BeanChannel` is accepted in both directions.
diff --git a/sidebars.ts b/sidebars.ts
index 7bc5f37251..e5ca2e66ac 100644
--- a/sidebars.ts
+++ b/sidebars.ts
@@ -297,6 +297,11 @@ const sidebars: SidebarsConfig = {
id:
'topics/02.17.ReadingContinuousStreams',
label: '2.17. Reading
Continuous Streams',
},
+ {
+ type: 'doc',
+ id:
'topics/02.17a.LargeDatasetStreaming',
+ label: '2.17a.
Large-Dataset Streaming',
+ },
{
type: 'doc',
id:
'topics/02.18.MarshallingUris',
