Hi Steve, hi all, I played with Steve's suggestion, and I think it's the right direction. Here are the main ideas:
1) Ship both Jackson 2 and Jackson 3 during a transition period. The runtime jars would bundle both, until Jackson 2 is removed. 2) Introduce compatibility tests to verify that the switch carries no regressions. 3) Remove Jackson 2 types from the Json serde layer. This requires introducing 2-3 Iceberg-owned shims on top of Jackson types, as Steve suggested. The change is mostly mechanical, but touches a lot of files. 4) Remove Jackson 2 types from the REST layer. We'd introduce more shims to replace exposure points like RESTSerializers.registerAll() and HTTPClient.Builder.withObjectMapper(). Would the community agree on this approach? I think it's a sane one, and has the benefit of cleaning up the public API, which is a good thing to do anyway. If so, we just need to figure out how to proceed. The full change would touch more than 100 files, especially item 3 above. It's certainly possible to split it into a few PRs. What are your thoughts? Thanks, Alex On Tue, Jul 28, 2026 at 12:22 PM Steve Loughran <[email protected]> wrote: > > > I'd go for > > leave jackson 2 as is, deprecate, with a goal to cut completely at some point > accompany an api which hides the json binding, > Move to jackson 3 internally, never expose this, either remove the jackson 2 > apis or move to a iceberg-core-jackson2-legacy module for expllicit use and > make clear it's only there for one more release > > Maybe moving the jackson2 stuff to a legacy module from the outset makes > clear its doomed, but if there's one lesson from OSS, it's "unless you are > ruthless and make clear it will go, people don't believe you". > At then same time: make changes incompatible at the binary level and people > don't upgrade to your latest release, so don't pick up the latest releases, > with direct and indirect CVE fixes. > > On that topic, there's a new jackson 2 release out: no release notes. That > implies "security fix but they don't want to announce the CVE just yet" > > > On Mon, 27 Jul 2026 at 14:52, Alexandre Dutra <[email protected]> wrote: >> >> Hi Steve, hi all, >> >> We discussed this topic briefly in the last sync, in the context of >> iceberg-core specifically. Here is a quick summary: >> >> - The exposure of the Jackson 2 API in iceberg-core often appears accidental. >> >> - If exposure is unavoidable, introducing dual exposure via method >> overloads has been proposed. >> >> - Another option under consideration is providing two separate >> artifacts, one for Jackson 2 and one for Jackson 3. >> >> - Ultimately, we need more time to evaluate the optimal path forward. >> >> To help guide this decision, I did a deeper investigation into where >> these Jackson types actually leak out. The exposure broadly falls into >> four categories: >> >> 1) Parser classes (SchemaParser, MetadataUpdateParser, etc.) >> 2) The JsonUtil utility class. >> 3) The HTTPRequest.mapper() method. >> 4) RESTSerializers.registerAll(ObjectMapper) and >> HTTPClient.Builder.withObjectMapper(ObjectMapper). >> >> Based on these findings, I suggest the following approach for each area: >> >> - For the parser classes (1), we should ideally restrict visibility to >> package-private if feasible (implies revapi exceptions). >> >> - For JsonUtil (2), I recommend either transitioning it to >> package-private or deprecating it for removal. >> >> - For HTTPRequest.mapper() (3), the exposure was an unfortunate design >> choice: we should deprecate it for removal and re-implement the logic >> within a package-private helper class. >> >> Item (4) represents the biggest challenge since consumers legitimately >> rely on them to configure their object mappers. Removing them entirely >> might be too disruptive; a better approach would be to accept the >> tight coupling and introduce parallel Jackson 3 overloads instead. Or, >> as Steve mentions, add a layer of indirection to decouple the API >> completely. >> >> In conclusion, if we put the extension points (4) aside for a moment, >> it looks like we have a clear opportunity to perform some API cleanup >> before we fully commit to the Jackson 3 migration, by reducing the >> Jackson 2 exposure to its minimum. >> >> What do you all think? >> >> Thanks, >> Alex >> >> On Mon, Jul 20, 2026 at 2:57 PM Steve Loughran <[email protected]> wrote: >> > >> > >> > the public APIs are the pain point, and adding jackson 3 overloads is >> > possibly going to replicate the same problem. >> > >> > I'd propose >> > - identify those things where json parsing/generation takes place and are >> > published >> > - see how they could extended such that jackson 3 supported could be done >> > without adding jackson 3 to their signatures >> > >> > I'm thinking of how the various parsers have toJson() methods which take >> > jackson generators >> > >> > public static void toJson(TableIdentifier identifier, JsonGenerator >> > generator) >> > >> > if another layer of indirection is added (JsonBinding?) that supplied the >> > generator then an overridden entry point could be supplied >> > >> > public static void toJson(TableIdentifier identifier, JsonBinding >> > binding) { >> > if (binding instance of Jackson2Binding jackson2binding) { >> > return toJson(identifier, jackson2binding.generator()); >> > } else { >> > return ((jackson3binding)bindiong)....) >> > >> > These could be added for the apis today, with the jackson2 stuff tagged as >> > deprecated, and new apis only coming though the new overload >> > >> > I'm glossing over the many implementation details here, but the key thing >> > is: exposing someone else's library as part your public API inevitably >> > becomes something you end up regretting. >> > >> > >> > On Thu, 16 Jul 2026 at 15:53, Alexandre Dutra <[email protected]> wrote: >> >> >> >> Hi all, >> >> >> >> As more frameworks (like Spring Boot and Quarkus) migrate to Jackson >> >> 3, I wanted to open a conversation about what a Jackson 3 migration >> >> would look like for Iceberg and what trade-offs we should think >> >> through together. >> >> >> >> Iceberg currently depends on Jackson 2 in several modules, most >> >> significantly iceberg-core. Other modules with direct Jackson >> >> dependencies include iceberg-aws, iceberg-nessie, iceberg-snowflake, >> >> and iceberg-kafka-connect. Most modules could be migrated separately. >> >> >> >> The migration is largely mechanical but pervasive: artifact >> >> coordinates and Java package names change across the board, several >> >> core classes and methods are renamed, ObjectMapper construction moves >> >> to a builder pattern, and exceptions become unchecked. JDK 17 is >> >> required, which Iceberg already enforces. >> >> >> >> The blast radius for iceberg-core would be roughly: >> >> >> >> - Spark and Flink runtime JARs: low impact. Both already shade Jackson >> >> 2; adding a relocation rule for Jackson 3 is a one-line fix per >> >> runtime variant. >> >> >> >> - Kafka Connect: also low impact, although the Kafka Connect >> >> distribution uses unshaded JARs. >> >> >> >> - Downstream users of iceberg-core: this is the most significant >> >> concern. Jackson types leak into the public API of many XYZParser >> >> classes, JsonUtil, and the REST HTTP layer. >> >> >> >> The honest answer is that icebegr-core's public API is permanently >> >> Jackson-coupled by design. We should imho acknowledge this fact; the >> >> best migration approach is to add Jackson 3 overloads wherever >> >> applicable, and make Jackson 2 and 3 coexist for some time. >> >> >> >> Looking forward to hearing your thoughts on this topic. >> >> >> >> Thanks, >> >> Alex
