[
https://issues.apache.org/jira/browse/KAFKA-7189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16550752#comment-16550752
]
ASF GitHub Bot commented on KAFKA-7189:
---------------------------------------
elarib opened a new pull request #5405: KAFKA-7189 : Add a Merge Transformer
for Kafka Connect
URL: https://github.com/apache/kafka/pull/5405
*More detailed description of your change,
if necessary. The PR title and PR message become
the squashed commit message, so use a separate
comment to ping reviewers.*
Like the flatten, there is the need too for a merge transformer.
Example transformation :
We want to add the offset and the partition for each record, and after that
merge them into one field _metadata:
```"transforms":"AddOffset, AddPartition, MergeFields",
"transforms.AddOffset.type":"org.apache.kafka.connect.transforms.InsertField$Value",
"transforms.AddOffset.offset.field":"offset!",
"transforms.AddPartition.type":"org.apache.kafka.connect.transforms.InsertField$Value",
"transforms.AddPartition.partition.field":"partition!",
"transforms.MergeFields.type":"org.apache.kafka.connect.transforms.Merge$Value",
"transforms.MergeFields.field.list":"offset,partition",
"transforms.MergeFields.field.root":"_metadata"
```
*Summary of testing strategy (including rationale)
for the feature or bug fix. Unit and/or integration
tests are expected for any behaviour change and
system tests should be considered for larger changes.*
1. Test topLevelStrcuteRequired for records with Schema
2. Test topLevelMapRequired for schemaless records
3. Test Merge fields for a record value with schema for the 4 cases.
4. Test Merge fields for a schemaless record key.
### Committer Checklist (excluded from commit message)
- [ ] Verify design and implementation
- [ ] Verify test coverage and CI build status
- [ ] Verify documentation (including upgrade notes)
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Add a Merge Transformer for Kafka Connect
> -----------------------------------------
>
> Key: KAFKA-7189
> URL: https://issues.apache.org/jira/browse/KAFKA-7189
> Project: Kafka
> Issue Type: Improvement
> Components: KafkaConnect
> Reporter: Abdelhamide EL ARIB
> Priority: Minor
>
> Like the
> [flatten|https://docs.confluent.io/current/connect/transforms/flatten.html#flatten],
> there is the need too for a merge transformer.
> Example transformation :
> We want to add the offset and the partition for each record, and after that
> merge them into one field _metadata :
>
> {code:java}
> "transforms":"AddOffset, AddPartition, MergeFields",
> "transforms.AddOffset.type":"org.apache.kafka.connect.transforms.InsertField$Value",
> "transforms.AddOffset.offset.field":"offset!",
> "transforms.AddPartition.type":"org.apache.kafka.connect.transforms.InsertField$Value",
> "transforms.AddPartition.partition.field":"partition!",
> "transforms.MergeFields.type":"org.apache.kafka.connect.transforms.Merge$Value",
> "transforms.MergeFields.field.list":"offset,partition",
> "transforms.MergeFields.field.root":"_metadata"
> {code}
>
> This is our MR for this transformer
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
