Hi all, Following up on this. After discussing with Matteo, we decided the upgrade should be far less invasive for Pulsar users, and that it was worth spending the effort to find out whether that was possible. It was, and the PR now meets that goal: existing client code does not have to change.
The reason it is possible is that in Pulsar's case the application already hands us the class. When you call Schema.AVRO(MyPojo.class), that class - not some schema arriving over the wire - is the authoritative description of what will be serialized. Pulsar derives the Avro schema from it anyway, so it can walk that derived schema and trust exactly the types Avro will resolve from it: the nested records and enums it references, including ones in other packages, and the collection and @Stringable types its fields declare. Nothing is guessed and nothing is granted wholesale. The paths where this happens are the ones where a Class is supplied: * Schema.AVRO(Class) * Schema.AVRO(SchemaDefinition) built with SchemaDefinition.builder().withPojo(...) * Schema.PROTOBUF(Class), where the message class is declared before the schema is derived Pulsar Functions and connectors are covered by the same mechanism, because they build their schemas through those same calls, and so is the broker for the types it writes to its own system topics. Schema.JSON needs nothing declared either: it encodes and decodes through Jackson rather than through Avro reflection. What is deliberately not covered is a class named only by a schema that arrived from the schema registry, since that is exactly the case the Avro allow-list exists to constrain. Schema.AUTO_CONSUME() is unaffected in the ordinary case: a record schema is decoded generically into a GenericRecord and no class is resolved at all, nested records included. The one gap is a topic whose schema is not a record at the top level, which is not typical of existing Pulsar usage - a producer has to have created the topic with something like Schema.AVRO(SomeEnum.class). For those topics Pulsar decodes with Avro's reflection-based reader, which loads a Java class for every record, enum and fixed type the schema names. Putting the type inside an array or a map does not avoid this; the class is loaded as soon as there is an element to read. Only schemas that name no such type at all, a string or an array of strings for instance, are unaffected. An application consuming such a topic declares the class itself through org.apache.pulsar.client.schema.AvroTrustedClasses, which is also there for anything else that needs widening. Since topics of that shape are unusual to begin with, the risk of this breaking an existing Pulsar application is minor. Older Avro versions are handled too. org.apache.avro.util.ClassSecurityValidator only exists from Avro 1.12.2, and an application on pulsar-client-original (the unshaded client) can pin an older Avro. Where the class is absent there is nothing to enforce, so declaring trust is a no-op rather than a NoClassDefFoundError, and this is covered by a test. Reviewers: please see https://github.com/apache/pulsar/pull/24992 for the details. Thanks, Lari On Thu, 20 Aug 2026 at 16:04, Lari Hotari <[email protected]> wrote: > > Hi all, > > While upgrading Avro from 1.12.0 to 1.12.2 (PR #24992, still in > review) I ran into a behaviour change that I think everyone working on > schemas, Functions or connectors should know about, because it also > affects user applications on upgrade and not just our own code. > > Short version: as of Avro 1.12.2, Avro refuses to reflect over any > class that is not explicitly trusted. Pulsar's AVRO schemas are built > on Avro's reflection API, so without an allow-list every > Schema.AVRO(MyPojo.class) encode and decode throws SecurityException. > (Schema.JSON is unaffected - it goes through Jackson.) > > > WHAT CHANGED, AND IN WHICH RELEASE > ---------------------------------- > > It is worth separating the two releases, since the change that causes > this landed in 1.12.2 rather than in 1.12.1. > > Avro 1.12.1 (October 2025) shipped 4 security fixes. One of them > removed the default trusted packages, which used to be: > > java.lang, java.math, java.io, java.net, org.apache.avro.reflect > > But in 1.12.1 the allow-list was still consulted in exactly one place, > SpecificDatumReader.findStringClass. That means it only guarded > classes named by the "java-class" / "java-key-class" schema > properties. Record, enum and fixed types were still resolved with no > check at all. The class org.apache.avro.util.ClassSecurityValidator > does not exist in 1.12.1. > > Avro 1.12.2 is where the scope changed. AVRO-4189 introduced > ClassSecurityValidator and moved the check into ClassUtils.forName, so > it now guards *every* reflective class resolution. What remains > trusted by default is 16 hardcoded JDK classes and no packages at all: > > Boolean, Byte, Character, CharSequence, Double, Enum, Float, Integer, > Long, Number, Object, Short, String, Void, BigDecimal, BigInteger > > That matters because SpecificData.getClass(Schema) resolves a class > for every RECORD, ENUM and FIXED schema, and > ReflectData.populateEncoderCache calls it via getCustomEncoding on > both ReflectDatumWriter.write and ReflectDatumReader.read. (Those > getCustomEncoding call sites are themselves new in 1.12.1, from > AVRO-4165.) So both the produce and the consume path go through the > check now, for every named type in the schema. > > This is a significant tightening: in 1.12.1 the allow-list applied > only to classes named by java-class / java-key-class, and in 1.12.2 it > applies to every reflective class resolution. Plan for it as a > deliberate hardening that we adapt to. > > Note that staying on 1.12.1 is not a workaround: AVRO-3940 in 1.12.1 > introduced a StackOverflowError in ReflectData.getSchema for any POJO > with a recursive field (AVRO-4209), and that is only fixed in 1.12.2. > > > WHAT ACTUALLY BREAKS > -------------------- > > If you do nothing, on Avro 1.12.2: > > * Producing or consuming with Schema.AVRO(MyPojo.class) fails with > > java.lang.SecurityException: Forbidden com.example.MyPojo! > This class is not trusted to be included in Avro schemas. > > wrapped in SchemaSerializationException. This hits both > directions, not just produce. > > Schema.JSON is NOT affected: it derives an Avro schema for the > SchemaInfo but reads and writes through Jackson, so it never resolves > a class reflectively. Neither is Schema.AUTO_CONSUME over a record > schema, which stays generic. Schema.PROTOBUF is affected, but at > schema-construction time rather than on produce - avro-protobuf > resolves the generated message classes while building the schema. > > * The failure is per named type, so it is not enough to trust the > top-level POJO. Every nested record and every enum reachable from it > is resolved separately, including types from other packages. > > * A field's *declared* collection type is recorded by Avro as a > "java-class" property on the generated array/map schema and resolved > reflectively too. So a field declared List<Foo> needs java.util.List > trusted, a HashSet field needs java.util.HashSet, and so on. > > * Nested classes have to be named with the binary '$' form. Avro > derives the schema name as com.example.Outer.Inner but resolves the > class by retrying with '$' separators, and the validator matches on > Class.getName(). An allow-list entry written with dots silently never > matches. > > * @Stringable types are affected as before, but the previous > defaults are gone, so java.net.URI, java.net.URL and java.io.File now > need trusting explicitly. BigDecimal, BigInteger and Integer are still > covered by the hardcoded set. > > * One non-obvious case: Schema.AUTO_CONSUME() against a topic whose > schema is Schema.JSON(SomeEnum.class). Nearly every POJO produces a > top-level Avro RECORD, and records are handled generically without > resolving any class. An enum is the exception - it produces a > top-level ENUM, and for a JSON-typed schema AutoConsumeSchema resolves > the class named in the *writer's* schema in order to rebuild a reader > schema for it. So the consumer needs the enum trusted, and the error > names it with the binary '$' form even though the schema spells it > with dots: > > java.lang.SecurityException: Forbidden com.example.Outer$Colour! > > Bare String or Integer POJOs are not affected - they produce plain > "string" / "int" schemas with nothing to resolve - and an AVRO-typed > enum schema takes a different branch that does not resolve either. It > is a narrow case, but not one anybody would predict from "Avro schemas > need trusted classes". > > Two further 1.12.1 changes are worth flagging while we are here: > > * AVRO-3230 enabled the fast reader by default. It reports malformed > data as AvroTypeException where the classic path raised > IndexOutOfBoundsException, so code that catches narrow exception types > around an Avro decode can now leak an Avro exception to callers. > > * AVRO-4133 makes avro-protobuf emit a "default" attribute inside > generated enum schemas, which changes the schema JSON produced for a > protobuf class. > > > WHAT PULSAR DOES ABOUT IT (PR #24992) > ------------------------------------- > > Pulsar cannot know an application's classes, so the PR only takes > responsibility for what Pulsar itself serializes and for the code > Pulsar runs on the deployment's behalf. > > * A new public class, > org.apache.pulsar.client.schema.AvroTrustedClasses, owns the trust and > installs it into Avro's global validator. It composes with whatever > validator is already installed, so Avro's own defaults, the > SERIALIZABLE_* system properties and any policy the application > installed itself all keep working. Its signatures use only Class, > String, ClassLoader and java.util.function.Predicate - deliberately no > Avro types, so it behaves the same in the shaded and unshaded clients. > > * BrokerAvroTrustedClasses declares the broker's own Avro-serialized > types: transaction buffer snapshots, topic policy events on > __change_events, and the metadata events used by the metadata > synchronizer. > > * FunctionAvroTrustedClasses trusts the class loader Pulsar created > for a deployed function or connector, so one using Schema.AVRO over > its own POJOs keeps working with no configuration, on all three > runtimes. Scoping to the class loader is deliberately narrower than a > wildcard: only the classes actually deployed become trusted, not > everything on the class path. > > > WHAT CLIENT APPLICATIONS HAVE TO DO > ----------------------------------- > > If your application uses Schema.AVRO with your own POJOs, declare them: > > import org.apache.pulsar.client.schema.AvroTrustedClasses; > > // Once, before the first producer or consumer. > AvroTrustedClasses.trust(Order.class); > > trust(Class...) walks the schema Avro derives, so one call also covers > the nested records and enums it references - including ones in other > packages - and the declared collection and @Stringable types its > fields carry. That last part matters more than it sounds: a POJO with > a List field needs java.util.List trusted too, which is why "just > trust my model package" is usually not enough on its own. There is > also trustExactly(Class...) when you want only what you named, > trustClasses(String...) for classes you cannot reference at compile > time, trustPackages(String...), trustClassLoader(...) for plugin code > you load yourself, and trust(Predicate<Class<?>>) as an escape hatch. > > A warning about Avro's own advice, because it is the first thing users > will read. The SecurityException message says to set > org.apache.avro.SERIALIZABLE_CLASSES / SERIALIZABLE_PACKAGES "or set > them via the API (see org.apache.avro.util.ClassSecurityValidator)". > For anyone on the shaded client - pulsar-client or pulsar-client-all, > which is the usual dependency - BOTH of those are wrong: > > * The class is not on their class path. Shading relocates it to > org.apache.pulsar.shade.org.apache.avro.util.ClassSecurityValidator, > so the snippet does not compile. Adding org.apache.avro:avro to make > it compile is worse: it compiles, runs, configures a second copy of > the class and the SecurityException fires unchanged. > > * The property name is relocated with it, so > -Dorg.apache.avro.SERIALIZABLE_PACKAGES is silently inert there. > > So a shaded-client user who follows the exception gets a > byte-identical failure and concludes the upgrade is broken. That is > the main reason the PR adds AvroTrustedClasses rather than pointing at > Avro's API: we cannot change Avro's message, but we can give people > something that works either way. > > The system properties do work if you use the right name for the artifact: > > # pulsar-client / pulsar-client-all (shaded - the usual dependency) > > -Dorg.apache.pulsar.shade.org.apache.avro.SERIALIZABLE_PACKAGES=com.example.model > > # pulsar-client-original (unshaded), broker, proxy and other components > -Dorg.apache.avro.SERIALIZABLE_PACKAGES=com.example.model > > SERIALIZABLE_CLASSES takes individual class names instead. Both are > read in a static initializer, so they have to be set before anything > in the JVM has touched Avro - a command-line -D always works, and > System.setProperty() works only if it runs first. Note that a package > on its own will not cover the java.util.List in the example above; you > would need SERIALIZABLE_CLASSES for that too, which is exactly the > bookkeeping trust(Class...) does for you. > > It is also possible to turn the check off entirely, by trusting every > package - again with a different property name depending on the > artifact: > > # pulsar-client / pulsar-client-all (shaded) > -Dorg.apache.pulsar.shade.org.apache.avro.SERIALIZABLE_PACKAGES=* > > # pulsar-client-original (unshaded), broker, proxy and other components > -Dorg.apache.avro.SERIALIZABLE_PACKAGES=* > > Avro treats "*" as trust-all, so this restores the pre-1.12.2 > behaviour. It is there if the upgrade breaks an application and you > need it working again while you work out which classes to trust. > Whether to keep it that way is a policy decision for each deployment > to make on its own assessment. > > This needs a release note, and I think it needs a documentation page > too. Feedback on the plan is welcome. > > Thanks, Lari
