I can only speak to my view on it, but I view logical types as truly generic. The point is that they can be handled as the underlying type safely, processed as such as needed, but if the logical type is properly preserved they can properly translate through as the more "complicated" type.
To me, this would mean that we try to preserve logical type names and parameters where possible, such that use cases like you're suggesting would be feasible -- we might just handle them as int32, but by carrying along the relevant type/parameter info, we would enable other apps/systems to handle them in a more nuanced way. If you want to encode the column type, I would recommend using the same approach Connect does for logical types (specifying the schema name as the class that implements it), but in general using any name & parameters *should* work, modulo any bugs in converters that fail to preserve that information. -Ewen On Tue, Mar 6, 2018 at 3:29 AM, Gunnar Morling <gun...@hibernate.org> wrote: > Hi, > > A user of the Debezium CDC Kafka Connect connectors has asked whether we > could provide information about the original source type of captured table > columns. > > Usually the type info we provide by using the Kafka Connect types and some > custom semantic types is good enough. But there are some cases where > additional type info would help: e.g. in case of MySQL, MEDIUMINT and INT > columns are transmitted as Connect Int32 (as that's the smallest type which > covers their value range). But from that, a consumer can't tell wether an > INT or MEDIUMINT column should be created in a downstream database. > > Now my question is: would it be a reasonable thing for us to encode the > original column type as an additional parameter of the Kafka Connect > schemas (using a special parameter name), or would this be bending the > concept of schema parameters too much? Admittedly, this metadata would be > kind of source-specific, but I can see how it'd be beneficial in some use > cases. > > Thanks for any advice, > > --Gunnar >
