I fail to see a fundamental difference in level (high vs. low) - we should consider extensibility, ergonomics, and how people are likely to use the API.
"It gets in the way of laziness" is a valid reason to have JsonObject not be a Map<String, Json> (though I still want to interrogate it), "the idea is to use delegation" and a desire for "DOP over OOP" is not. This is part of why fromUntyped/toUntyped is uncomfortable - it's choosing one particular encoding of JSON and making it central to the API. Is that really the most important encoding? And I get a lot of mileage out of Json extending JsonEncodable - both in terms of ergonomics for constructing Json structures (being able to turn a List<Person> into Json using the same overload as you would a List<Json>, etc.) as well as an extension point for interacting with other json libraries (easy to teach jackson and co. to treat something implementing an interface differently). On Fri, May 16, 2025 at 2:26 PM <fo...@univ-mlv.fr> wrote: > > > ------------------------------ > > *From: *"Ethan McCue" <et...@mccue.dev> > *To: *"Remi Forax" <fo...@univ-mlv.fr> > *Cc: *"Paul Sandoz" <paul.san...@oracle.com>, "core-libs-dev" < > core-libs-dev@openjdk.org> > *Sent: *Friday, May 16, 2025 1:44:52 AM > *Subject: *Re: Towards a JSON API for the JDK > > I present for your consideration the library I made when spiraling about > this problem space a few years ago > > https://github.com/bowbahdoe/json > > > https://javadoc.io/doc/dev.mccue/json/latest/dev.mccue.json/dev/mccue/json/package-summary.html > > Notably missing during the design process here were patterns, hence the > JsonDecoder design. I haven't been able to evaluate how patterns affect > that on account of them not being out. > > I will more thoroughly peruse the draft of java.util.json at a later date, > but my initial observations/comments: > > * I am not sure having JsonValue be distinct from Json has value. > * toUntyped feels a little strange to me - the only type information > presumably lost is the sealed-ness of the hierarchy. The interplay between > that and toNumber is also a little unnerving. > * One notion that I found helpful was that a class could be "json > encodable," meaning there is a method to call to obtain a canonical json > representation. > > record Person(String name) implements JsonEncodable { > @Override > public Json toJson() { > return Json.objectBuilder() > .put("namen", name) > .build(); > } > } > > Which helper methods like Json#of(List<? extends JsonEncodable>) could > make use of. Json itself (JsonValue in your prototype) could then have a > vacuous implementation. > > * Terminology wise - I went with reading/writing for the actual > parsing/generation of json and encoding/decoding for the mapping of those > representations to/from specific classes. The merits are not top of mind, > just noting the difference. read/write vs parse/toString+toDisplayString > * One thing I did was make the helper methods in Json null tolerant and > the ones in the specific subtypes like JsonString not. This was because > from what I saw of usages of javax.json/jakarta.json that nullability was a > footgun and correcting for it required changes to code structure (breaking > up builder chains with if (x != null) checks) > * The functionality you want from JsonNumber could be achieved by making > it just extend Number ( > https://github.com/bowbahdoe/json/blob/main/src/main/java/dev/mccue/json/JsonNumber.java) > instead of a bespoke toNumber. You need the extra methods to go to big > decimal and co, but it's just an extension to the behavior of Number at > that point. > * JsonObject and JsonArray could implement Map<String, Json> and > List<Json> respectively. This lowers the need for toUntyped() - since > presumably one of the use cases for that is turning the json tree into > something that more generic map/list traversal code can handle. It also > complicates any lazy loading somewhat. > * Assuming patterns can be placed on interfaces, you might want to > consider something similar to JsonDecoder, but with a pattern instead of > a method that throws an exception. > > // Where here fromJson would box up the logic for testing and extracting > from each element in the array. > List<Person> people = array(json, Person::fromJson); > > * I don't think there is sufficient cause for anything to be non-sealed at > this point. > * JsonBoolean and JsonNull do not have reasonable alternative > implementations - as far as I can imagine, maybe i'm wrong - so maybe those > can just be final classes? > * If you seal up the whole hierarchy then its pretty trivial to make it > serializable ( > https://github.com/bowbahdoe/json/blob/main/src/main/java/dev/mccue/json/serialization/JsonSerializationProxy.java > ) > > > Hello, > I think your library is more high-level than java.util.json. > Moreover, Paul proposes a design closer to DOP (Data Oriented Programming) > than OOP. > > JsonNumber should not extend Number, JsonObject and JsonArray should not > implements Map and List, the idea is to use delegation instead. > (BTW extending Number will become an issue in the future if the VM tries > to flatten filed typed with a value capable abstract class like Number) > > JSON is fundamentally untyped and recursive, but also describe data > structures, for me, the proposed API exposes that structure (the subtypes > of JsonValue) and make it easy to move from a JSON text to Java objects or > vice-versa. > The java.util.json API is very simple, > - Json.fromUntyped() see a Java object to a structured JSON value (a > JsonValue), > - Json.toUntyped() unwrap a structured JSON value to extract the > underlying Java object. > - Json.parse() from String to a structured JSON value > - Json.toDisplayString() (i would prefer "prettyString") goes from a > structured JSON value to a pretty string. > > Having a JsonEncoder and JsonDecoder, JsonDecodable, etc is more high > level than what is proposed here, so you can write objects like > JsonEncoder/Decoder on top of it. > Your example of JsonEncodable can be re-written to use the java.util.json > API like this > > record Person(String name) implements JsonEncodable { > @Override > public JsonValue toJson() { > return Json.fromUntyped(Map.of("name", name)); > } > } > > Compared to your code, java.util.json uses immutable data structures > instead of a mutable builder (and yes, you lose the order, until > SequenceMap.of() exists). > > regards, > Rémi > > > > > On Thu, May 15, 2025 at 11:29 PM Remi Forax <fo...@univ-mlv.fr> wrote: > >> Hi Paul, >> yes, not having a simple JSON API in Java is an issue for beginners. >> >> It's not clear to me why JsonArray (for example) has to be an interface >> instead of a record ? >> >> I understand why Json.parse() only works on String and char[] but the API >> make it too easy to have many performance issues. >> I think you need versions using a Reader and a Path. >> Bonus point, if there is a method walk() that also returns a JsonValue >> but the List/Map inside JsonArray/JsonObject are populated lazily. >> >> Minor point: Json.toDisplayString() should takes a second parameters >> indicating the number of spaces used for the indentation (like >> JSON.stringify in JS). >> >> regards, >> Rémi >> >> ----- Original Message ----- >> > From: "Paul Sandoz" <paul.san...@oracle.com> >> > To: "core-libs-dev" <core-libs-dev@openjdk.org> >> > Sent: Thursday, May 15, 2025 10:30:42 PM >> > Subject: Towards a JSON API for the JDK >> >> > Hi, >> > >> > We would like to share with you our thoughts and plans towards a JSON >> API for >> > the JDK. >> > Please see the document below. >> > >> > - >> > >> > We have had the pleasure of using a clone of this API in some >> experiments we are >> > conducting with >> > ONNX and code reflection [1]. Using the API we were able to quickly >> write code >> > to ingest and convert >> > a JSON document representing ONNX operation schema into instances of >> records >> > modeling the schema >> > (see here [2]). >> > >> > The overall out-of-box experience with such a minimal "batteries >> included” API >> > has so far been positive. >> > >> > Thanks, >> > Paul. >> > >> > [1] https://openjdk.org/projects/babylon/ >> > [2] >> > >> https://github.com/openjdk/babylon/blob/code-reflection/cr-examples/onnx/opgen/src/main/java/oracle/code/onnx/opgen/OpSchemaParser.java#L87 >> > >> > # Towards a JSON API for the JDK >> > >> > One of the most common requests for the JDK is an API for parsing and >> generating >> > JSON. While JSON originated as a text-based serialization format for >> JSON >> > objects ("JSON" stands for "JavaScript Object Notation"), because of >> its simple >> > and flexible syntax, it eventually found use outside the JavaScript >> ecosystem as >> > a general data interchange format, such as framework configuration >> files and web >> > service requests/response formats. >> > >> > While the JDK cannot, and should not, provide libraries for every >> conceivable >> > file format or protocol, the JDK philosophy is one of "batteries >> included", >> > which is to say we should be able to write basic programs that use >> common >> > protocols such as HTTP, without having to appeal to third party >> libraries. >> > The Java ecosystem already has plenty of JSON libraries, so inclusion in >> > the JDK is largely meant to be a convenience, rather than needing to be >> the "one >> > true" JSON library to meet the needs of all users. Users with specific >> needs >> > are always free to select one of the existing third-party libraries. >> > >> > ## Goals and requirements >> > >> > Our primary goal is that the library be simple to use for parsing, >> traversing, >> > and generating conformant JSON documents. Advanced features, such as >> data >> > binding or path-based traversal should be possible to implement as >> layered >> > features, but for simplicity are not included in the core API. We adopt >> a goal >> > that the performance should be "good enough", but where performance >> > considerations conflict with simplicity and usability, we will choose >> in favor >> > of the latter. >> > >> > ## API design approach >> > >> > The description of JSON at `https:://json.org` describes a JSON >> document using >> > the familiar "railroad diagram": >> >  >> > >> > This diagram describes an algebraic data type (a sum of products), >> which we >> > model directly with a set of Java interfaces: >> > >> > ``` >> > interface JsonValue { } >> > interface JsonArray extends JsonValue { List<JsonValue> values(); } >> > interface JsonObject extends JsonValue { Map<String, JsonValue> >> members(); } >> > interface JsonNumber extends JsonValue { Number toNumber(); } >> > interface JsonString extends JsonValue { String value(); } >> > interface JsonBoolean extends JsonValue { boolean value(); } >> > interface JsonNull extends JsonValue { } >> > ``` >> > >> > These interfaces have (hidden) companion implementation classes that >> admit >> > greater flexibility of implementation than modeling them directly with >> records >> > would permit. >> > Further, these interfaces are unsealed. We compromise on the sealed sum >> of >> > products to enable >> > alternative implementations, for example to support alternative formats >> that >> > encode the same information in a JSON document but in a more efficient >> form than >> > text. >> > >> > The API has static methods for parsing strings into a `JsonValue`, >> conversion to >> > and from purely untyped representations (lists and maps), and factory >> methods >> > for building JSON documents. We apply composition consistently, e.g, a >> > JsonString has a string, a JsonObject has a map of string to JsonValue, >> as >> > opposed to extension for structural JSON values. >> > >> > It turns out that this simple API is almost all we need for traversal. >> It gives >> > us an immutable representation of a document, and we can use pattern >> matching to >> > answer the myriad questions that will come up (Does this object have >> key X? Does >> > it map to a number? Is that number representable as an integer?) when >> going >> > from an untyped format like JSON to a more strongly typed domain model. >> > Given a simple document like: >> > >> > ``` >> > { >> > "name": "John”, >> > "age": 30 >> > } >> > ``` >> > >> > we can parse and traverse the document as follows: >> > >> > ``` >> > JsonValue doc = Json.parse(inputString); >> > if (doc instanceof JsonObject o >> > && o.members().get("name") instanceof JsonString s >> > && s.value() instanceof String name >> > && o.members().get("age") instanceof JsonNumber n >> > && n.toNumber() instanceof Long l && l instanceof int age) { >> > // use "name" and "age" >> > } >> > ``` >> > >> > Later, when the language acquires the ability to expose deconstruction >> patterns >> > for arbitrary interfaces (similar to today's record patterns, see >> > >> https://openjdk.org/projects/amber/design-notes/patterns/towards-member-patterns >> ), >> > this will be simplifiable to: >> > >> > ``` >> > JsonValue doc = Json.parse(inputString); >> > if (doc instanceof JsonObject(var members) >> > && members.get("name") instanceof JsonString(String name) >> > && members.get("age") instanceof JsonNumber(int age)) { >> > // use "name" and "age" >> > } >> > ``` >> > >> > So, overtime, as more pattern matching features are introduced we >> anticipate >> > improved use of the API. This is a primary reason why the API is so >> minimal. >> > Convenience methods we add today, such as a method that accesses a JSON >> > object component as say a JSON string or throws an exception, will >> become >> > redundant in the future. >> > >> > ## JSON numbers >> > >> > The specification of JSON number makes no explicit distinction between >> integral >> > and decimal numbers, nor specifies limits on the size of those numbers. >> > This is a common source of interoperability issues when consuming JSON >> > documents. Generally users cannot always but often do assume JSON >> numbers are >> > parsable, without loss of precision, to IEEE double-precision floating >> point >> > numbers or 32-bit signed integers. >> > >> > In this respect the API provides three means to operate on the JSON >> number, >> > giving the user full control: >> > >> > 1. Underlying string representation can be obtained, if preserving >> syntactic >> > details such as leading or trailing zeros is important. >> > 2. The string representation can be parsed to an instance of >> `BigDecimal`, using >> > `toBigDecimal` if preserving decimal numbers is important. >> > 3. The string representation can be parsed into an instance of `Long`, >> `Double`, >> > `BigInteger`, or `BigDecimal`, using `toNumber`. The result of this >> method >> > depends on how the representation can be parsed, possibly losing >> precision, >> > choosing a suitably convenient numeric type that can then be pattern >> > matched on. >> > >> > Primitive pattern matching will help as will further pattern matching >> features >> > enabling the user to partially match. >> > >> > ## Prototype implementation >> > >> > The prototype implementation is currently located into the JDK sandbox >> > repository >> > under the `json` branch, see >> > here >> > >> https://github.com/openjdk/jdk-sandbox/tree/json/src/java.base/share/classes/java/util/json >> > The prototype API javadoc generated from the repository is also >> available at >> > >> https://cr.openjdk.org/~naoto/json/javadoc/api/java.base/java/util/json/package-summary.html >> > >> > ### Testing and conformance >> > >> > The prototype implementation passes all conformance test cases but two, >> > available >> > on https://github.com/nst/JSONTestSuite. The two exceptions are the >> ones which >> > the >> > prototype specifically prohibits, i.e, duplicated names in JSON objects >> > (https://cr.openjdk.org/~naoto/json/conformance/results/parsing.html#35 >> ). >> > >> > ### Performance >> > >> > Our main focus so far has been on the API design and a functional >> > implementation. >> > Hence, there has been less focus on performance even though we know >> there are a >> > number of performance enhancements we can make eventually. >> > We are reasonably happy with the current performance. The >> > implementation performs well when compared to other JSON implementations >> > parsing from string instances and traversing documents. >> > >> > An example of where we may choose simplicity and usability over >> performance >> > is the rejection of JSON documents containing objects that in turn >> contain >> > members >> > with duplicate names. That may increase the cost of parsing, but >> simplifies the >> > user >> > experience for the majority of cases since if we reasonably assume >> JsonObjects >> > are >> > map-like, what should the user do with such members, pick one the last >> one? >> > merge >> > the values? or reject? >> > >> > ## A JSON JEP? >> > >> > We plan to draft JEP when we are ready. Attentive readers will observe >> that >> > a JEP already exists, JEP 198: Light-Weight JSON API >> > (https://openjdk.org/jeps/198). We will >> > either update this JEP, or withdraw it and draft a new one. >> > >
