Re: Issue while creating spark app

Thanks for all these useful info Hi all What is the current trend. Is it Spark on Scala with intellij or Spark on python with pycharm.  I am curious because I have moderate experience with Spark on both Scala and python and want to focus on Scala OR python going forward with the intention of jo

Re: Issue while creating spark app

Might as well update the artefacts to the correct versions hopefully. Downloaded scala 2.12.8 scala -version Scala code runner version 2.12.8 -- Copyright 2002-2018, LAMP/EPFL and Lightbend, Inc. Edited the pom.xml as below http://maven.apache.org/POM/4.0.0"; xmlns:xsi="http://www.w3

Re: Issue while creating spark app

Rajat Kumer wrote: "Cannot find project Scala library 2.12.12 for module SparkSimpleApp" So when I google this error message I find scala project maven sync failed søn. 27. feb. 2022 kl. 21:59 skrev Mich Talebzadeh < mich.talebza...@gmail.com>:

Re: Issue while creating spark app

sorry which error? view my Linkedin profile https://en.everybodywiki.com/Mich_Talebzadeh *Disclaimer:* Use it at your own risk. Any and all responsibility for any loss, damage or destruction of data or any other property which m

Re: Issue while creating spark app

Anyway I did google on your error and found this scala project maven sync failed Is this the same as the one you are getting? søn. 27. feb. 2022 kl. 21:16 skrev Mich Talebzadeh < mich.talebza...@gmail.com>: > Thanks Bjorn. I am aware of that. I j

Re: Issue while creating spark app

Thanks Bjorn. I am aware of that. I just really wanted to create the uber jar files with both sbt and maven in Intellij. cheers view my Linkedin profile https://en.everybodywiki.com/Mich_Talebzadeh *Disclaimer:* Use it at you

Re: Issue while creating spark app

Mitch: You are using scala 2.11 to do this. Have a look at Building Spark "Spark requires Scala 2.12/2.13; support for Scala 2.11 was removed in Spark 3.0.0." søn. 27. feb. 2022 kl. 20:55 skrev Mich Talebzadeh < mich.talebza...@gmail.com>:

Re: Issue while creating spark app

OK I decided to give a try to maven. Downloaded maven and unzipped the file WSL-Ubuntu terminal as unzip apache-maven-3.8.4-bin.zip Then added to Windows env variable as MVN_HOME and added the bin directory to path in windows. Restart intellij to pick up the correct path. Again on the command li

Re: Issue while creating spark app

Got curious with this intellij stuff. I recall using sbt rather than MVN so go to terminal in your intellij and verify what is installed sbt -version sbt version in this project: 1.3.4 sbt script version: 1.3.4 scala -version Scala code runner version 2.11.7 -- Copyright 2002-2013, LAMP/EPFL

Re: Issue while creating spark app

I don't think any of that is related, no. How are you dependencies set up? manually with IJ, or in a build file (Maven, Gradle)? Normally you do the latter and dependencies are taken care of for you, but you app would definitely have to express a dependency on Scala libs. On Sat, Feb 26, 2022 at 4

Re: Issue while creating spark app

Java SDK installed? On Sun, Feb 27, 2022 at 5:39 AM Sachit Murarka wrote: > Hello , > > Thanks for replying. I have installed Scala plugin in IntelliJ first then > also it's giving same error > > Cannot find project Scala library 2.12.12 for module SparkSimpleApp > > Thanks > Rajat > > On Sun,

Re: Issue while creating spark app

Hello , Thanks for replying. I have installed Scala plugin in IntelliJ first then also it's giving same error Cannot find project Scala library 2.12.12 for module SparkSimpleApp Thanks Rajat On Sun, Feb 27, 2022, 00:52 Bitfox wrote: > You need to install scala first, the current version for

Re: Issue while creating spark app

You need to install scala first, the current version for spark is 2.12.15 I would suggest you install scala by sdk which works great. Thanks On Sun, Feb 27, 2022 at 12:10 AM rajat kumar wrote: > Hello Users, > > I am trying to create spark application using Scala(Intellij). > I have installed S