I would categorize these as bugs. We should (but probably don't fully yet) support arbitrary nesting as long as you use basic collections / case classes / primitives. Please do open JIRAs as you find problems.
On Thu, Oct 27, 2016 at 1:05 PM, Koert Kuipers <ko...@tresata.com> wrote: > well i was using Aggregators that returned sequences of structs, or > structs with sequences inside etc. and got compilation errors on the > codegen. > > i didnt bother trying to reproduce it so far, or post it, since what i did > was beyond the supposed usage anyhow. > > > On Thu, Oct 27, 2016 at 4:02 PM, Herman van Hövell tot Westerflier < > hvanhov...@databricks.com> wrote: > >> What kind of difficulties are you experiencing? >> >> On Thu, Oct 27, 2016 at 9:57 PM, Koert Kuipers <ko...@tresata.com> wrote: >> >>> i have been pushing my luck a bit and started using ExpressionEncoder >>> for more complex types like sequences of case classes etc. (where the case >>> classes only had primitives and Strings). >>> >>> it all seems to work but i think the wheels come off in certain cases in >>> the code generation. i guess this is not unexpected, after all what i am >>> doing is not yet supported. >>> >>> is there a planned path forward to support more complex types with >>> encoders? it would be nice if we can at least support all types that >>> spark-sql supports in general for DataFrame. >>> >>> best, koert >>> >> >> >
