Re: Self contained Spark application with local master without spark-submit

Hi Colin, Yes, you can. You only need to define master="local" and build the jar with Main class defined in the manifest. [image: facebook] [image: twitter] [image: linkedin] [image: instagram]

Re: Choice of IDE for Spark

Disclaimer: I'm developer avocado for data engineering at JetBrains, so I'm definitely biased. And if someone likes Zeppelin — there is an awesome integration of Zeppelin into IDEA via Big Data Tools plugin — one can perform any explorations they want/need and then extract all their work into real

Re: Drop-In Virtual Office Hour round 2 :)

Hey Holden, Thank you for the opportunity! Didn't you think about calendly appointments? Looks like they could work for this and many other goals easily (I'm not affiliated with them in any way, just customer) --- Best wishes, Pasha Finkelsheyn Developer advocate for data engineering @ JetBrains

Re: How can I read ftp

We have solved it using an orchestrator, which copied data from FTP to HDFS. But of course, you can just use a Java FTP client to just read files, put them somewhere, and then read with Spark. пн, 9 авг. 2021 г. в 06:39, Sean Owen : > FTP is definitely not supported. Read the files to distributed