Overirde Jackson jar - Spark Submit

Hi, I am having similar issue as mentioned in the below link but was not able to resolve. any other solutions please? https://stackoverflow.com/questions/57329060/exclusion-of-dependency-of-spark-core-in-cdh Thanks, Asmath

Python level of knowledge for Spark and PySpark

Hi gurus, I have knowledge of Java, Scala and good enough knowledge of Spark, Spark SQL and Spark Functional programing with Scala. I have started using Python with Spark PySpark. Wondering, in order to be proficient in PySpark, how much good knowledge of Python programing is needed? I know the a

Re: Python level of knowledge for Spark and PySpark

The answer always is "it depends". At the outset it seems you are in pretty good shape and have all the key skills you need. All I can suggest is try to take inherent benefits of the language and hone your coding practices On Thu, 15 Apr 2021 at 2:25 am, ashok34...@yahoo.com.INVALID wrote: > Hi

Re: Python level of knowledge for Spark and PySpark

There is no good answer to the question “Have I learnt enough”. You can never learn enough. You have to constrantly learn Practically, if you want to make a career out of using technology XYZ, you only need to learn enough XYZ to get a job doing XYZ. Once you get a job doing XYZ, other people a

Re: Python level of knowledge for Spark and PySpark

Hi, I believe both Ayan and Jayesh have made valuable points. From my experience the companies that traditionally have a Python team in house would like to add Spark capabilities to their inventory not least because 1. There is more Python code running around that will benefit from modern

Spark submit hbase issue

Hi, Spark submit is connecting to local host instead of zookeeper mentioned in hbase-site.xml. This same program works in ide which gets connected to hbase-site.xml. What am I missing in spark submit? > >  > spark-submit --driver-class-path > C:\Users\mdkha\bitbucket\clx-spark-scripts\src\te

Re: Spark submit hbase issue

Try adding hbase-site.xml file to %SPARK_HOME%\conf and see if it works HTH view my Linkedin profile *Disclaimer:* Use it at your own risk. Any and all responsibility for any loss, damage or destruction of data or any other pro