Re: specify output format using pyspark

2014-02-26 Thread Chengi Liu
Cool.Thanks On Wed, Feb 26, 2014 at 9:48 AM, Ewen Cheslack-Postava wrote: > You need to convert it to the format you want yourself. The output you're > seeing is just the automatic conversion of your data by unicode(). > > -Ewen > > Chengi Liu > February 26, 2014 at 9:43 AM > Hi, > How do

specify output format using pyspark

2014-02-26 Thread Chengi Liu
Hi, How do we save data to hdfs using pyspark in "right" format. I use: counts = counts.saveAsTextFile("hdfs://localhost:1234//foo") But when I look into the data... It is always in tuple format (1245,23) (1235,99) How do i specify output format in pyspark. Thanks

Re: specify output format using pyspark

2014-02-26 Thread Ewen Cheslack-Postava
You need to convert it to the format you want yourself. The output you're seeing is just the automatic conversion of your data by unicode(). -Ewen Chengi Liu February 26, 2014 at 9:43 AM Hi,  How do we save data to hdfs using pyspark in "right" format.I use:counts = cou