"toArray","first" get the different result from one element RDD

[buring]

Hi
        Recently I have some problems about rdd behaviors.It's about
"RDD.first","RDD.toArray" method when RDD only has one element. 
        I get the different result in different method from one element RDD 
where i
should have the same result. I will give more detail after the code.
        My code was as follows:
        //get and rdd with just one row ，RDD[(Long,Array[Byte])]        
        val alsresult =
sc.sequenceFile(args(0)+"/als",classOf[LongWritable],classOf[BytesWritable]).map{case(uid,sessions)=>
      sessions.setCapacity(sessions.getLength)
      (uid.get(),sessions.getBytes)
    }.filter{line=>
      line._1 == userindex.value //specified from arguments
    }
    //log information really surprised me
    logger.info("alsInformation:%d".format(alsresult.count()))
   
alsresult.toArray().foreach(e=>logger.info("alstoarray:%d\t%s".format(e._1,e._2.mkString("
"))))
   
alsresult.take(1).foreach(e=>logger.info("take1result:%d\t%s".format(e._1,e._2.mkString("
"))))
   
logger.info("firstInformation:%d\t%s".format(alsresult.first()._1,alsresult.first()._2.mkString("
")))
   
alsresult.collect().foreach(e=>logger.info("alscollectresult:%d\t%s".format(e._1,e._2.mkString("
"))))
   
alsresult.take(3).foreach(e=>logger.info("alstake3result:%d\t%s".format(e._1,e._2.mkString("
")))) //3 is big than the rdd.count()

    I get a RDD which just have one element. But use the different method ,I
got the different element. My print information as follows:

argument userindex      33057172                                                
        28116855                                                        3814772 
                                3209314
alsInformation          1                                                       
                1                                                               
        1                                               1 
alstoarray                      1612242 0 22 47 37 6 19...      3337442 16 32 0 
22 13 49...     
3697319 16 32 0 22 13 49...      3       0 22 47 37 6 19...
take1result                     1612242 21 24 3 56 16 27...     3337442 16 52 
31 42 29 36 ...
3697319 39 21 34 56 3 37...      3   34 10 18 28 38 11...
firstInformation        1612242 21 24 3 56 16 27...     3337442 16 52 31 42 29 
36 ...
3697319 39 21 34 56 3 37...      3   34 10 18 28 38 11...
alscollectresult        1612242 0 22 47 37 6 19...      3337442 16 32 0 22 13 
49...     
3697319 16 32 0 22 13 49...      3   0 22 47 37 6 19...
alstake3result          1612242 0 22 47 37 6 19...              3337442 16 32 0 
22 13 49...     
3697319 16 32 0 22 13 49...      3   0 22 47 37 6 19...
 I filter the rdd and guarantee the RDD.count() equal 1.,I think different
"userindex.value"arguments should get different alsresult ,
but "RDD.toArray","RDD.collect","RDD.take(3)" ,have the same result and
under the same argument "toArray" ,"take(1)","take(3)" 
have the different resultmethod ,It's really surpurised me. The arguments is
random specified.

Can anyone explain it or give me some reference?

Thanks 




--
View this message in context: 
http://apache-spark-user-list.1001560.n3.nabble.com/toArray-first-get-the-different-result-from-one-element-RDD-tp20734.html
Sent from the Apache Spark User List mailing list archive at Nabble.com.

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscr...@spark.apache.org
For additional commands, e-mail: user-h...@spark.apache.org