Pls checke the specific ERORR lines of the text file . Chaces are are few Columns are not properly delimited in specific rows.
Regards Prakash On Fri, Sep 7, 2018, 3:41 AM dimitris plakas <dimitrisp...@gmail.com> wrote: > Hello everyone, I am new in Pyspark and i am facing an issue. Let me > explain what exactly is the problem. > > I have a dataframe and i apply on this a map() function > (dataframe2=datframe1.rdd.map(custom_function()) > dataframe = sqlContext.createDataframe(dataframe2) > > when i have > > dataframe.show(30,True) it shows the result, > > when i am using dataframe.show(60, True) i get the error. The Error is in > the attachement Pyspark_Error.txt. > > Could you please explain me what is this error and how to overpass it? > > > --------------------------------------------------------------------- > To unsubscribe e-mail: user-unsubscr...@spark.apache.org
