Oh, now I get it
Thanks, it works fine.
> On Jan 25, 2016, at 9:07 PM, Chiwan Park <chiwanp...@apache.org> wrote:
>
> I think that it is not related to Zeppelin. Because your function
> `unionByName` is not method of `DataFrame`, you have to call it like
> following:
>
> val unionized = unionByName(raw1.select(columns: _*), raw3.select(columns:
> _*))
>
> Regards,
> Chiwan Park
>
>> On Jan 25, 2016, at 9:00 PM, 진상형 <jsh90...@gmail.com> wrote:
>>
>> I want to use custom function, but it doesn't work.
>>
>> Here are codes and outs
>>
>> /// codes
>> def unionByName(a: DataFrame, b:DataFrame): DataFrame = {
>> val columns = a.dtypes.intersect(b.dtypes).map{case (c, _) => col(c)}
>> a.select(columns: _*).unionAll(b.select(columns: _*))
>> }
>> raw1.select(columns: _*).unionByName(raw3.select(columns: _*)).count()
>>
>> /// outputs
>> unionByName: (a: org.apache.spark.sql.DataFrame, b:
>> org.apache.spark.sql.DataFrame)org.apache.spark.sql.DataFrame
>> <console>:141: error: value unionByName is not a member of
>> org.apache.spark.sql.DataFrame
>> raw1.select(columns: _*).unionByName(raw3.select(columns:
>> _*)).count()
>>
>>
>>
>> p.s. Is there any function to check types of RDD or else? (e.g. dtypes for
>> checking types of column)
>
