Re: Start a standalone server as root and use it with user accounts

2020-02-12 Thread WranglingData
I hate to be "that guy", but I'd like to know myself. I tried to setup something similar, except I created a "service" account which starts the Spark service, but like you, I kept on getting file permission errors when submitting jobs under my own login. My current workaround was to su to the ser

Question about Spark, PySpark data frames and JDBC connections to TSQL databases

2020-01-30 Thread WranglingData
Hi, I am currently working on my first Spark app (V2.4.3, running as standalone on a single node cluster, configured as 6 workers). I am coding this app in Python/PySpark. This app is to be used to create a daily delta data set from a TSQL database (sybase ASE). This data set returns new, updat