[
https://issues.apache.org/jira/browse/FLINK-4050?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15385958#comment-15385958
]
Stephan Ewen commented on FLINK-4050:
-------------------------------------
Could you drop a code snippet of the API as you would want it to be?
We may be able to adopt that, or some of that, into a newer version of the
FlinkKafkaConsumer
> FlinkKafkaProducer API Refactor
> -------------------------------
>
> Key: FLINK-4050
> URL: https://issues.apache.org/jira/browse/FLINK-4050
> Project: Flink
> Issue Type: Improvement
> Components: Kafka Connector
> Affects Versions: 1.0.3
> Reporter: Elias Levy
>
> The FlinkKafkaProducer API seems more difficult to use than it should be.
> The API requires you pass it a SerializationSchema or a
> KeyedSerializationSchema, but the Kafka producer already has a serialization
> API. Requiring a serializer in the Flink API precludes the use of the Kafka
> serializers. For instance, they preclude the use of the Confluent
> KafkaAvroSerializer class that makes use of the Confluent Schema Registry.
> Ideally, the serializer would be optional, so as to allow the Kafka producer
> serializers to handle the task.
> In addition, the KeyedSerializationSchema conflates message key extraction
> with key serialization. If the serializer were optional, to allow the Kafka
> producer serializers to take over, you'd still need to extract a key from the
> message.
> And given that the key may not be part of the message you want to write to
> Kafka, an upstream step may have to package the key with the message to make
> both available to the sink, for instance in a tuple. That means you also need
> to define a method to extract the message to write to Kafka from the element
> passed into the sink by Flink.
> In summary, there should be separation of extraction of the key and message
> from the element passed into the sink from serialization, and the
> serialization step should be optional.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
