Hi , We have ~ 1TB of data to process , but our cluster doesn't have sufficient memory for such data set. ( we have 5-10 machine cluster). Is it possible to process 1TB data using ON DISK options using spark?
If yes where can I read about the configuration for ON DISK executions. Thanks Oleg.
