Hi Victor,
We migrated away from Java Serialisation in stages, admittedly it was a
significant task, but after watching your video, I thought I'd share
some of my experiences.
The first step we made was to create a deserialiser that used
constructors, serialisation still used OOS, we reimplemented OIS, I
retained the existing object serial form. I posted about it here
previously: Explicit Serialization API and Security
<https://mail.openjdk.org/pipermail/core-libs-dev/2015-January/030724.html>
Source:AtomicSerial xref
<https://pfirmstone.github.io/JGDMS/xref/org/apache/river/api/io/AtomicSerial.html>
I built a framework around deserialisation and tools to defensively
validate invariants prior to object construction, it didn't support
circular object graphs.
For the second stage, I implemented OOS and serialisers for Java
Collections and classes we needed but couldn't control, eg Java platform
classes. One of the decisions I made in the design, was Collections
were immutable, and designed to be copied during invariant validation,
there's an important reason for this I will focus on later.
The API was originally a constructor signature with an annotation, it
grew to include two static methods for the developer to declare class
schema and serialise the object.
The constructor signature is standardised with a caller sensitive
parameter, allowing implementations to retrieve parameter arguments, but
only for the calling class's private namespace. This parameter argument
can then be passed to a super class constructor. This constructor
signature is designed for class inheritance hierarchy evolution and
refactoring, including changes in implementations from inheritance to
encapsulation, and intra class invariant checks.
We don't think of the object's serial form as serialised fields; they
are serialised parameter arguments used to create new objects.
The third stage was to implement a new serialisation wire protocol.
One of the issues we have is our security model is designed to safely
run foreign code, audited using static analysis, but untrusted, using SM
with PoLP generated policy, but OpenJDK's is now based on trusted code,
having since removed SM, for now we've forked OpenJDK while we try to
reduce our dependency on dynamic code, inherited from its Jini origins.
JERI - Jini Extensible Remote Invocation, this is similar to other
remote invocation frameworks like RMI, but unlike RMI it's extensible,
it has multiple layers, each with pluggable implementations, which
allowed Java Serialisation to be replaced.
Something that may not be obvious, is class resolution during
deserialisation. Mike Warres's 2006 paper "Class Loading Issues in
Java™ RMI and Jini™ Network Technology", section 6.2/6.3, to pry apart
the multiple functions ClassLoaders conflate, we implemented a per-proxy
dedicated ClassLoader model, which is a working instance of this
separation, basically we replaced the duplication of class resolution
that RMIClassLoader created and instead assigned a ClassLoader to the
stream, that determines class visibility. This has been a very clean
solution to class resolution.
Now about the new serialisation protocol, Java Serialization is opaque
and non deterministic, and JavaSpaces used it as though it were
deterministic, serial form was used for equality in JavaSpaces, to avoid
deserialization, creating issues with equality due to non determinism.
For our serialisation protocol we chose canonical DER ASN.1 encoding,
and we developed rules around collections to ensure determinism, such
that we have cross platform collections, with identical serial form.
This allows us to make equality comparisons of collections in Java
Spaces, no matter which programming language the object originated
from. Objects are serialised with their schema, this is checked at
parsing, which is performed defensively. Schema is not designed for
evolution, it is exact and specific, any differences between schema and
serialised objects is exceptional and will throw.
To reduce the dependency on dynamic code, we are implementing
Turing-incomplete remote functions. These functions are incapable of
arbitrary side effects; no reflection, no arbitrary method dispatch,
nothing beyond an expression grammar which includes a subset of CEL
syntax and mathematical operations, that will allow JavaSpace searches
to use logical comparison and simple calculations.
Furthermore the schema and DER encoding are fully transparent,
parameters can be deeply introspected back to their string and primitive
values, providing fodder for functions.
--
Regards,
Peter
