Re: Questions about the future of UDTs and Encoders

Hi Michael, Having faced the same limitation, I have found these two libraries to be helpful: - Frameless (https://github.com/typelevel/frameless ) - struct-type-encoder (https://benfradet.github.io/blog/2017/06/14/Deriving-Spark-Dataframe-schemas-with-S

Thoughts on extedning ML exporting in Spark?

Hi folks, I've been giving a bit of thought to trying to improve ML exporting in Spark to support a wider variety of formats. If you implement pipeline stages, or you've added your own export logic, I'd especially love your input. A quick little draft of what I've been thinking about (after jumpi