Re: Refreshing Data in Spark Memory (DataFrames)

2020-11-13 Thread Arti Pande
sources like RDBMS, Distributed data store, file system etc as batch sources. Please correct if required. Thanks & regards, Arti Pande On Sat, Nov 14, 2020 at 12:11 AM Lalwani, Jayesh wrote: > Is this a streaming application or a batch application? > > > > Normally, for batch app

Refreshing Data in Spark Memory (DataFrames)

2020-11-13 Thread Arti Pande
ere a solution / workaround for this? Thanks & regards, Arti Pande

Spark on Kubernetes

2020-11-13 Thread Arti Pande
%20Spark%20applications . Apparently there are open JIRA issues for spark that talk about problems on K8s with dynamic allocation etc. Are there any workarounds? Are there any other issues? What is the official recommendation? Thanks & regards, Arti Pande