Ah, I missed that bit of documentation my bad :). That totally explains
the behavior!
Thanks a lot!
---
Regards,
Andy
On Sat, Jan 7, 2017 at 10:11 AM, Liang-Chi Hsieh wrote:
>
> Hi Andy,
>
> Thanks for sharing the code snippet.
>
> I am not sure if you miss something in the snippet, becau
Hi Andy,
Thanks for sharing the code snippet.
I am not sure if you miss something in the snippet, because some function
signature are not matched, e.g.,
@Override
public StructType bufferSchema() {
return new UserDefineType(schema, unboundedEncoder);
}
Maybe you define a c
Hi Liang-Chi,
The snippet of code is below. If I bind the encoder early (the schema
doesn't change throughout the execution), the final result is a list of the
same entries.
@RequiredArgsConstructor
public class UDAF extends UserDefinedAggregateFunction {
// Do not resolve and bind this expr
Can you show how you use the encoder in your UDAF?
Andy Dang wrote
> One more question about the behavior of ExpressionEncoder
>
> .
>
> I have a UDAF that has ExpressionEncoder
>
> as a member variable.
>
> However, if call resolveAndBind() eagerly on this encoder, it appears to
> break th
One more question about the behavior of ExpressionEncoder.
I have a UDAF that has ExpressionEncoder as a member variable.
However, if call resolveAndBind() eagerly on this encoder, it appears to
break the UDAF. Bascially somehow the deserialized row are all the same
during the merge step. Is it t
Perfect. The API in Java is bit clumsy though
What I ended up doing in Java (the val is from lombok, if anyone's
wondering):
val attributes =
JavaConversions.asJavaCollection(schema.toAttributes()).stream().map(Attribute::toAttribute).collect(Collectors.toList());
val encoder =
Row
You need to resolve and bind the encoder.
ExpressionEncoder enconder = RowEncoder.apply(struct).resolveAndBind();
Andy Dang wrote
> Hi all,
> (cc-ing dev since I've hit some developer API corner)
>
> What's the best way to convert an InternalRow to a Row if I've got an
> InternalRow and the co
