Loading a text file

2022-03-14 Thread Hinko Kocevar
I have a standalone spark 3.2.0 cluster with two workers started on PC_A and want to run a pyspark job from PC_B. The job wants to load a text file. I keep getting file not found error messages when I execute the job. Folder/file "/home/bddev/parrot/words.txt" exists on PC_B but not on PC_A. tr

Re: Python performance

2022-02-06 Thread Hinko Kocevar
2022 at 5:02 PM Hinko Kocevar wrote: I'm looking into using Python interface with Spark and came across this [1] chart showing some performance hit when going with Python RDD. Data is ~ 7 years and for older version of Spark. Is this still the case with more recent Spark releases?

Python performance

2022-02-04 Thread Hinko Kocevar
I'm looking into using Python interface with Spark and came across this [1] chart showing some performance hit when going with Python RDD. Data is ~ 7 years and for older version of Spark. Is this still the case with more recent Spark releases? I'm trying to understand what to expect from Pytho

reading file containing flatbuffers serialized data

2021-11-14 Thread Hinko Kocevar
Is it possible to use/read a file containing flatbuffers serialized records that were written to a data file? Thanks, Hinko - To unsubscribe e-mail: user-unsubscr...@spark.apache.org