Hi Mark,
Thank you for helping out.
The items I got back from Spark SQL has the type information as follows:
scala> items
res16: org.apache.spark.sql.Row = [WrappedArray([1,orange],[2,apple])]
I tried to iterate the items as you suggested but no luck.
Best Regards,
Jerry
On Mon, Dec 15, 2014 at 2:18 PM, Mark Hamstra <m...@clearstorydata.com>
wrote:
>
> scala> val items = Row(1 -> "orange", 2 -> "apple")
>
> items: org.apache.spark.sql.catalyst.expressions.Row =
> [(1,orange),(2,apple)]
>
>
> If you literally want an iterator, then this:
>
>
> scala> items.toIterator.count { case (user_id, name) => user_id == 1 }
>
> res0: Int = 1
>
>
> ...else:
>
>
> scala> items.count { case (user_id, name) => user_id == 1 }
>
> res1: Int = 1
>
> On Mon, Dec 15, 2014 at 11:04 AM, Jerry Lam <chiling...@gmail.com> wrote:
>>
>> Hi spark users,
>>
>> Do you know how to access rows of row?
>>
>> I have a SchemaRDD called user and register it as a table with the
>> following schema:
>>
>> root
>> |-- user_id: string (nullable = true)
>> |-- item: array (nullable = true)
>> | |-- element: struct (containsNull = false)
>> | | |-- item_id: string (nullable = true)
>> | | |-- name: string (nullable = true)
>>
>>
>> val items=sqlContext.sql("select items from user where user_id = 1").first
>>
>> The type of items is org.apache.spark.sql.Row. I want to iterate through
>> the items and count how many items that user_id = 1 has.
>>
>> I could not find a method in which I can do that. The farthest I can get
>> to is to convert items.toSeq. The type information I got back is:
>>
>> scala> items.toSeq
>> res57: Seq[Any] = [WrappedArray([1,orange],[2,apple])]
>>
>> Any suggestion?
>>
>> Best Regards,
>>
>> Jerry
>>
>
