Re: encoders for more complex types

2016-10-27 Thread Koert Kuipers
https://issues.apache.org/jira/browse/SPARK-18147 On Thu, Oct 27, 2016 at 4:55 PM, Koert Kuipers wrote: > ok will do > > On Thu, Oct 27, 2016 at 4:51 PM, Michael Armbrust > wrote: > >> I would categorize these as bugs. We should (but probably don't fully >> yet) support arbitrary nesting as lo

Re: encoders for more complex types

2016-10-27 Thread Koert Kuipers
ok will do On Thu, Oct 27, 2016 at 4:51 PM, Michael Armbrust wrote: > I would categorize these as bugs. We should (but probably don't fully > yet) support arbitrary nesting as long as you use basic collections / case > classes / primitives. Please do open JIRAs as you find problems. > > On Thu

Re: encoders for more complex types

2016-10-27 Thread Michael Armbrust
I would categorize these as bugs. We should (but probably don't fully yet) support arbitrary nesting as long as you use basic collections / case classes / primitives. Please do open JIRAs as you find problems. On Thu, Oct 27, 2016 at 1:05 PM, Koert Kuipers wrote: > well i was using Aggregators

Re: encoders for more complex types

2016-10-27 Thread Koert Kuipers
well i was using Aggregators that returned sequences of structs, or structs with sequences inside etc. and got compilation errors on the codegen. i didnt bother trying to reproduce it so far, or post it, since what i did was beyond the supposed usage anyhow. On Thu, Oct 27, 2016 at 4:02 PM, Herm

Re: encoders for more complex types

2016-10-27 Thread Herman van Hövell tot Westerflier
What kind of difficulties are you experiencing? On Thu, Oct 27, 2016 at 9:57 PM, Koert Kuipers wrote: > i have been pushing my luck a bit and started using ExpressionEncoder for > more complex types like sequences of case classes etc. (where the case > classes only had primitives and Strings). >

encoders for more complex types

2016-10-27 Thread Koert Kuipers
i have been pushing my luck a bit and started using ExpressionEncoder for more complex types like sequences of case classes etc. (where the case classes only had primitives and Strings). it all seems to work but i think the wheels come off in certain cases in the code generation. i guess this is n