Sorry ignore my last reply.
Rusty,
I am very thankful for your help. Actually, I am facing difficulty in
objects. My plan is that, I have an object list containing list of User
objects. After parallelizing it through spark context, I apply comparator
on user.getUserName(). As usernames are sorted, their related user object
are sorted according to user names.
In the end when I apply top, I get the whole object of user .
Some like this:
public static Comparator<User> UserComparator
= new Comparator<User>() {
public int compare(User usr1, User usr2) {
String userName1 = usr1.getUserName().toUpperCase();
String userName2 = usr1.getUserName().toUpperCase();
//ascending order
return userName1.compareTo(userName2);
//descending order
//return fruitName2.compareTo(fruitName1);
}
};
JavaRDD<User> rdd = context.parallelize(usersList);
List<User> top =rdd.top(1, UserComparator);// but it is giving me
serialization issues.
Can you guide me that how can I sort out this issue?
I also applied your code, it runs fine but I want it with object (You know
Object gives me tough time).
Best
Hafsa
2015-07-07 16:54 GMT+02:00 Hafsa Asif <[email protected]>:
> Rusty,
>
> I am very thankful for your help. Actually, I am facing difficulty in
> objects. My plan is that, I have an object list containing list of User
> objects. After parallelizing it through spark context, I apply comparator
> on user.getUserName(). As usernames are sorted, their related user object
> are sorted according to user names.
> In the end when I apply top, I get the whole object of user .
>
> Some like this:
>
> public static Comparator<User> UserComparator
> = new Comparator<User>() {
>
> public int compare(User usr1, User usr2) {
> String userName1 = usr1.getUserName().toUpperCase();
> String userName2 = usr1.getUserName().toUpperCase();
>
> //ascending order
> return userName1.compareTo(userName2);
>
> //descending order
> //return fruitName2.compareTo(fruitName1);
> }
>
> };
>
> JavaRDD<User> rdd = context.parallelize(usersList);
>
> 2015-07-07 16:05 GMT+02:00 rusty [via Apache Spark User List] <[hidden
> email] <http:///user/SendEmail.jtp?type=node&node=23688&i=0>>:
>
>> JavaRDD<String> lines2 = ctx.parallelize(Arrays.asList("3", "6", "2",
>> "5", "8", "6", "7"));
>> List<String> top =lines2.top(7, new
>> CustomComaprator<String>());
>> for (String integer : top) {
>> System.out.println(integer);
>> }
>>
>>
>>
>>
>> class CustomComaprator<T> implements Serializable, Comparator<T> {
>> /**
>> *
>> */
>> public CustomComaprator() {
>> // TODO Auto-generated constructor stub
>>
>> }
>>
>> private static final long serialVersionUID =
>> 2004092520677431781L;
>>
>> @Override
>> public int compare(T o11, T o12) {
>> int o1 = Integer.parseInt(String.valueOf(o11));
>> int o2 = Integer.parseInt(String.valueOf(o12));
>>
>> return o1 > o2 ? 1 : o1 == o2 ? 0 : -1;
>> }
>>
>> }
>>
>>
>> ------------------------------
>> If you reply to this email, your message will be added to the
>> discussion below:
>>
>> http://apache-spark-user-list.1001560.n3.nabble.com/How-to-implement-top-and-filter-on-object-List-for-JavaRDD-tp23669p23684.html
>> To unsubscribe from How to implement top() and filter() on object List
>> for JavaRDD, click here.
>> NAML
>> <http://apache-spark-user-list.1001560.n3.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml>
>>
>
>
> ------------------------------
> View this message in context: Re: How to implement top() and filter() on
> object List for JavaRDD
> <http://apache-spark-user-list.1001560.n3.nabble.com/How-to-implement-top-and-filter-on-object-List-for-JavaRDD-tp23669p23688.html>
>
> Sent from the Apache Spark User List mailing list archive
> <http://apache-spark-user-list.1001560.n3.nabble.com/> at Nabble.com.
>
