Wojciech Trefon created NIFI-16349:
--------------------------------------
Summary: StandardProtobufReader fails to compile Confluent
Protobuf schemas with implicit field_meta options
Key: NIFI-16349
URL: https://issues.apache.org/jira/browse/NIFI-16349
Project: Apache NiFi
Issue Type: Bug
Reporter: Wojciech Trefon
h2. Summary
{{StandardProtobufReader}} fails to compile Protobuf schemas retrieved from
Confluent Schema Registry when a schema contains the {{(confluent.field_meta)}}
custom option without an explicit {{confluent/meta.proto}} import.
The failure occurs during schema compilation, before the message payload is
decoded.
h2. Background
Confluent Schema Registry treats {{confluent/meta.proto}} as a built-in known
dependency. Schemas generated from Kafka Connect logical types can therefore
contain {{confluent.field_meta}} without listing {{confluent/meta.proto}} as a
normal Schema Registry reference or explicit import.
h2. Example schema
{code:protobuf}
syntax = "proto3";
package example;
message Value {
optional int32 enabled = 31 [(confluent.field_meta) = {
params: [
{
key: "connect.type",
value: "int16"
}
]
}];
}
{code}
h2. Steps to reproduce
# Store or generate a Protobuf schema in Confluent Schema Registry that
contains {{confluent.field_meta}} without an explicit {{confluent/meta.proto}}
import or schema reference.
# Serialize a message using the Confluent Protobuf wire format and the affected
schema.
# Configure {{StandardProtobufReader}} to use
{{ConfluentEncodedSchemaReferenceReader}}, {{ConfluentSchemaRegistry}}, and
{{ConfluentProtobufMessageNameResolver}}.
# Read the serialized message.
h2. Actual behavior
Schema compilation fails with an error similar to:
{code}
com.squareup.wire.schema.SchemaException: unable to resolve option
confluent.field_meta
for field enabled
in message example.Value
{code}
The message payload is not decoded.
h2. Expected behavior
{{StandardProtobufReader}} should be able to compile and read a schema accepted
by Confluent Schema Registry when the schema uses Confluent's built-in metadata
options.
h2. Impact
Messages using affected schemas cannot be consumed as records. Existing
messages remain affected because their Confluent wire headers continue to
reference the original schema identifiers.
h2. Technical context
{{ProtobufSchemaCompiler}} provides selected Wire runtime definitions for
standard Protobuf types. It does not currently make {{confluent/meta.proto}} or
its {{google/protobuf/descriptor.proto}} dependency available during
compilation. Wire also requires the source schema to import the file that
defines a custom option.
Observed with the {{StandardProtobufReader}} implementation using Square Wire
6.4.6.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
