Hi there,
Currently Catalyst supports encoding custom classes represented as Java Beans (among others). This Java Bean implementation depends internally on Catalyst’s ObjectType extension of DataType. Currently, this class is private to the sql package [1], which is sensible, as it is only necessary for representing objects in the encoder. However, its private scope makes it more difficult (if not impossible) to write full custom encoders for other classes, themselves perhaps composed of additional objects. Could the definition of the ObjectType be made public in order to support writing custom Encoders? I can't see any particular danger in this. In order to implement an encoder for Avro-specified classes, I have to work off a fork where the ObjectType definition has be liberalized as described. [1] -- https://github.com/apache/spark/blob/master/sql/ catalyst/src/main/scala/org/apache/spark/sql/types/ObjectType.scala#L39 Thanks, Aleksander Eskilson
