Question about Spark, PySpark data frames and JDBC connections to TSQL databases

Hi, I am currently working on my first Spark app (V2.4.3, running as standalone on a single node cluster, configured as 6 workers). I am coding this app in Python/PySpark. This app is to be used to create a daily delta data set from a TSQL database (sybase ASE). This data set returns new, updat

Re: Start a standalone server as root and use it with user accounts

I hate to be "that guy", but I'd like to know myself. I tried to setup something similar, except I created a "service" account which starts the Spark service, but like you, I kept on getting file permission errors when submitting jobs under my own login. My current workaround was to su to the ser