Hi Team, 

We have a few use cases where nested record with array needs to be flattened / 
exploded to multiple rows and write it to destination like Cassandra. 

Now current Transformation interface in Kafka connect takes one record and 
returns one / zero record.
Also current flatten ignores the array 
https://issues.apache.org/jira/browse/KAFKA-12305.

We would like to know about your thought on enhancing the transformation 
interface to produce more than one record , 1 -> Many or Many -> Many. 

We understand an alternative is to use an upstream KStream pipeline but that 
has multiple challenges like adding extra hop, more pipelines to maintain etc.

Enhancing the Transformation interface would allow us to have a Generic SMT to 
handle this like  Explode, similar to 
https://github.com/apache/spark/blob/007c4593058537251d83a4cb44efe31e394aee22/sql/core/src/main/scala/org/apache/spark/sql/functions.scala#L4542
Thanks 
Rabin

Reply via email to