I am talking about spark here.
On Sat, Feb 22, 2020, 4:19 PM rahul c wrote:
> Hi,
>
> df.write.csv()
> Will ideally give you a csv file which can be used in further processing.
> I am not that much aware of raw_csv function of pandas.
>
> On Sat, 22 Feb, 2020, 4:09 PM Ksh
e node and write from
> there to get a single file out you could simple move/rename the file that
> spark creates or write the csv yourself with your library of preference?
>
> On Sat, 22 Feb 2020 at 10:39, Kshitij wrote:
>
>> Is there any way to save it as raw_csv file as we do i
Is there any way to save it as raw_csv file as we do in pandas? I have a
script that uses the CSV file for further processing.
On Sat, 22 Feb 2020 at 14:31, rahul c wrote:
> Hi Kshitij,
>
> There are option to suppress the metadata files from get created.
> Set the below proper
Hi,
There is no dataframe spark API which writes/creates a single file instead
of directory as a result of write operation.
Below both options will create directory with a random file name.
df.coalesce(1).write.csv()
df.write.csv()
Instead of creating directory with standard files (_SUCCESS
