Re: pyspark EOFError after calling map

2016-04-22 Thread Pete Werner
Oh great, thank you for clearing that up. On Fri, Apr 22, 2016 at 5:15 PM, Davies Liu wrote: > This exception is already handled well, just noisy, should be muted. > > On Wed, Apr 13, 2016 at 4:52 PM, Pete Werner > wrote: > >> Hi >> >> I am new to spark & pyspark. >> >> I am reading a small csv

Re: pyspark EOFError after calling map

2016-04-22 Thread Davies Liu
This exception is already handled well, just noisy, should be muted. On Wed, Apr 13, 2016 at 4:52 PM, Pete Werner wrote: > Hi > > I am new to spark & pyspark. > > I am reading a small csv file (~40k rows) into a dataframe. > > from pyspark.sql import functions as F > df = > sqlContext.read.forma