MOBIN-F commented on PR #3865:
URL: https://github.com/apache/flink-cdc/pull/3865#issuecomment-2819931254
> Thanks for your fix. But there is now a user scenario where users want to
add fields of `__namespace_name__ || '.' || __schema_name__ || '.' ||
__table_name__ identifier_name` and `__data_event_type__ AS op_type` to certain
tables, but they do not need these two fields for other tables(or only one
`__data_event_type__ AS op_type` field). and their yaml file may looks like
this:
>
> ```
> transform:
> - source-table: mydb.web_order[0-9]+
> projection: id, order_id, __namespace_name__ || '.' || __schema_name__
|| '.' || __table_name__ identifier_name, __data_event_type__ AS op_type
> - source-table: \.*.\.*
> projection: \*, __data_event_type__ AS op_type
> ```
>
> Because we do not have table.exclude configuration in transform, making it
difficult to configure such information for the rest of tables if we require
the same schema after each transform rule.
maybe we can use regular expressions to exclude mydb.web_order[0-9]+
```
- source-table: \.*.\.* Change to - source-table: ^(?!mydb\.web_order\d+$).\*
```
Suppose we support the following transform (i.e. the first projection is
used first when there are multiple projections for the same table)
```
transform:
- source-table: mydb.web_order[0-9]+
projection: id, order_id, __namespace_name__ || '.' || __schema_name__
|| '.' || __table_name__ identifier_name, __data_event_type__ AS op_type
- source-table: \.*.\.*
projection: \*, __data_event_type__ AS op_type
```
Then the following transform will also become legal, but it is difficult to
determine the user's intention from the transform rules, which can easily cause
confusion
```
transform:
- source-table: default_namespace.default_schema.mytable2
projection: \*
filter: age < 18
- source-table: default_namespace.default_schema.mytable2
projection: id,UPPER(name) AS name
filter: age >= 18
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
