[
https://issues.apache.org/jira/browse/KAFKA-3910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15710509#comment-15710509
]
ASF GitHub Bot commented on KAFKA-3910:
---------------------------------------
GitHub user shikhar opened a pull request:
https://github.com/apache/kafka/pull/2196
KAFKA-3910: prototype of another approach to cyclic schemas
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/shikhar/kafka KAFKA-3910
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/kafka/pull/2196.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #2196
----
commit 03a85d774e9d6a2feebba1ae1f50967619857040
Author: Shikhar Bhushan <shik...@confluent.io>
Date: 2016-12-01T01:39:22Z
KAFKA-3910: prototype of another approach to cyclic schemas
----
> Cyclic schema support in ConnectSchema and SchemaBuilder
> --------------------------------------------------------
>
> Key: KAFKA-3910
> URL: https://issues.apache.org/jira/browse/KAFKA-3910
> Project: Kafka
> Issue Type: Improvement
> Components: KafkaConnect
> Affects Versions: 0.10.0.0
> Reporter: John Hofman
> Assignee: Shikhar Bhushan
>
> Cyclic schema's are not supported by ConnectSchema or SchemaBuilder.
> Subsequently the AvroConverter (confluentinc/schema-registry) hits a stack
> overflow when converting a cyclic avro schema, e.g:
> {code}
> {"type":"record",
> "name":"list","fields":[{"name":"value","type":"int"},{"name":"next","type":["null","list"]}]}
> {code}
> This is a blocking issue for all connectors running on the connect framework
> with data containing cyclic references. The AvroConverter cannot support
> cyclic schema's until the underlying ConnectSchema and SchemaBuilder do.
> To reproduce the stack-overflow (Confluent-3.0.0):
> Produce some cyclic data:
> {code}
> bin/kafka-avro-console-producer --broker-list localhost:9092 --topic test
> --property value.schema='{"type":"record",
> "name":"list","fields":[{"name":"value","type":"int"},{"name":"next","type":["null","list"]}]}'
> {"value":1,"next":null}
> {"value":1,"next":{"list":{"value":2,"next":null}}}
> {code}
> Then try to consume it with connect:
> {code:title=connect-console-sink.properties}
> name=local-console-sink
> connector.class=org.apache.kafka.connect.file.FileStreamSinkConnector
> tasks.max=1
> topics=test
> {code}
> {code}
> ./bin/connect-standalone
> ./etc/schema-registry/connect-avro-standalone.properties
> connect-console-sink.properties
> … start up logging …
> java.lang.StackOverflowError
> at org.apache.avro.JsonProperties.getJsonProp(JsonProperties.java:54)
> at org.apache.avro.JsonProperties.getProp(JsonProperties.java:45)
> at io.confluent.connect.avro.AvroData.toConnectSchema(AvroData.java:1055)
> at io.confluent.connect.avro.AvroData.toConnectSchema(AvroData.java:1103)
> at io.confluent.connect.avro.AvroData.toConnectSchema(AvroData.java:1137)
> at io.confluent.connect.avro.AvroData.toConnectSchema(AvroData.java:1103)
> at io.confluent.connect.avro.AvroData.toConnectSchema(AvroData.java:1137)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
