[
https://issues.apache.org/jira/browse/KAFKA-3910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
John Hofman updated KAFKA-3910:
-------------------------------
Description:
Cyclic schema's are not supported by ConnectSchema or SchemaBuilder.
Subsequently the AvroConverter (confluentinc/schema-registry) hits a stack
overflow when converting a cyclic avro schema, e.g:
{code}
{"type":"record",
"name":"list","fields":[{"name":"value","type":"int"},{"name":"next","type":["null","list"]}]}
{code}
This is a blocking issue for all connectors running on the connect framework
with data containing cyclic references. The AvroConverter cannot support cyclic
schema's until the underlying ConnectSchema and SchemaBuilder do.
To reproduce the stack-overflow (Confluent-3.0.0):
Produce some cyclic data:
{code}
bin/kafka-avro-console-producer --broker-list localhost:9092 --topic test
--property value.schema='{"type":"record",
"name":"list","fields":[{"name":"value","type":"int"},{"name":"next","type":["null","list"]}]}'
{"value":1,"next":null}
{"value":1,"next":{"list":{"value":2,"next":null}}}
{code}
Then try to consume it with connect:
{code:title=connect-console-sink.properties}
name=local-console-sink
connector.class=org.apache.kafka.connect.file.FileStreamSinkConnector
tasks.max=1
topics=test
{code}
{code}
./bin/connect-standalone
./etc/schema-registry/connect-avro-standalone.properties
connect-console-sink.properties
… start up logging …
java.lang.StackOverflowError
at org.apache.avro.JsonProperties.getJsonProp(JsonProperties.java:54)
at org.apache.avro.JsonProperties.getProp(JsonProperties.java:45)
at io.confluent.connect.avro.AvroData.toConnectSchema(AvroData.java:1055)
at io.confluent.connect.avro.AvroData.toConnectSchema(AvroData.java:1103)
at io.confluent.connect.avro.AvroData.toConnectSchema(AvroData.java:1137)
at io.confluent.connect.avro.AvroData.toConnectSchema(AvroData.java:1103)
at io.confluent.connect.avro.AvroData.toConnectSchema(AvroData.java:1137)
{code}
was:
Cyclic schema's are not supported by ConnectSchema or SchemaBuilder.
Subsequently the AvroConverter (confluentinc/schema-registry) hits a stack
overflow when converting a cyclic avro schema, e.g:
{code}
{"type":"record",
"name":"list","fields":[{"name":"value","type":"int"},{"name":"next","type":["null","list"]}]}
{code}
This is a blocking issue for all connectors running on the connect framework
and with data containing cyclic references. The AvroConverter cannot support
cyclic schema's until the underlying ConnectSchema and SchemaBuilder do.
To reproduce the stack-overflow (Confluent-3.0.0):
Produce some cyclic data:
{code}
bin/kafka-avro-console-producer --broker-list localhost:9092 --topic test
--property value.schema='{"type":"record",
"name":"list","fields":[{"name":"value","type":"int"},{"name":"next","type":["null","list"]}]}'
{"value":1,"next":null}
{"value":1,"next":{"list":{"value":2,"next":null}}}
{code}
Then try to consume it with connect:
{code:title=connect-console-sink.properties}
name=local-console-sink
connector.class=org.apache.kafka.connect.file.FileStreamSinkConnector
tasks.max=1
topics=test
{code}
{code}
./bin/connect-standalone
./etc/schema-registry/connect-avro-standalone.properties
connect-console-sink.properties
… start up logging …
java.lang.StackOverflowError
at org.apache.avro.JsonProperties.getJsonProp(JsonProperties.java:54)
at org.apache.avro.JsonProperties.getProp(JsonProperties.java:45)
at io.confluent.connect.avro.AvroData.toConnectSchema(AvroData.java:1055)
at io.confluent.connect.avro.AvroData.toConnectSchema(AvroData.java:1103)
at io.confluent.connect.avro.AvroData.toConnectSchema(AvroData.java:1137)
at io.confluent.connect.avro.AvroData.toConnectSchema(AvroData.java:1103)
at io.confluent.connect.avro.AvroData.toConnectSchema(AvroData.java:1137)
{code}
> Cyclic schema support in ConnectSchema and SchemaBuilder
> --------------------------------------------------------
>
> Key: KAFKA-3910
> URL: https://issues.apache.org/jira/browse/KAFKA-3910
> Project: Kafka
> Issue Type: Improvement
> Components: KafkaConnect
> Affects Versions: 0.10.0.0
> Reporter: John Hofman
> Assignee: Ewen Cheslack-Postava
> Priority: Blocker
>
> Cyclic schema's are not supported by ConnectSchema or SchemaBuilder.
> Subsequently the AvroConverter (confluentinc/schema-registry) hits a stack
> overflow when converting a cyclic avro schema, e.g:
> {code}
> {"type":"record",
> "name":"list","fields":[{"name":"value","type":"int"},{"name":"next","type":["null","list"]}]}
> {code}
> This is a blocking issue for all connectors running on the connect framework
> with data containing cyclic references. The AvroConverter cannot support
> cyclic schema's until the underlying ConnectSchema and SchemaBuilder do.
> To reproduce the stack-overflow (Confluent-3.0.0):
> Produce some cyclic data:
> {code}
> bin/kafka-avro-console-producer --broker-list localhost:9092 --topic test
> --property value.schema='{"type":"record",
> "name":"list","fields":[{"name":"value","type":"int"},{"name":"next","type":["null","list"]}]}'
> {"value":1,"next":null}
> {"value":1,"next":{"list":{"value":2,"next":null}}}
> {code}
> Then try to consume it with connect:
> {code:title=connect-console-sink.properties}
> name=local-console-sink
> connector.class=org.apache.kafka.connect.file.FileStreamSinkConnector
> tasks.max=1
> topics=test
> {code}
> {code}
> ./bin/connect-standalone
> ./etc/schema-registry/connect-avro-standalone.properties
> connect-console-sink.properties
> … start up logging …
> java.lang.StackOverflowError
> at org.apache.avro.JsonProperties.getJsonProp(JsonProperties.java:54)
> at org.apache.avro.JsonProperties.getProp(JsonProperties.java:45)
> at io.confluent.connect.avro.AvroData.toConnectSchema(AvroData.java:1055)
> at io.confluent.connect.avro.AvroData.toConnectSchema(AvroData.java:1103)
> at io.confluent.connect.avro.AvroData.toConnectSchema(AvroData.java:1137)
> at io.confluent.connect.avro.AvroData.toConnectSchema(AvroData.java:1103)
> at io.confluent.connect.avro.AvroData.toConnectSchema(AvroData.java:1137)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
