I have seen many data engineering teams start out with Scala because technically it is the best choice for many given reasons and basically it is what Spark is. I also concur that Python is more popular than Scala because of the advent of data science. A majority of use cases we see these days are data science or related use cases where people mostly do Python. Most Cloud Data warehouses offer embedded modeling tools that rely extensively on Python packages. So, if you need those two worlds to share code e and even handover code, you do not want the ideological battle of Scala vs Python. Often we chose python for the sake of everybody speaking the same language.
With regard to Spark docs showing Python code in the first tab etc, in my opinion it is a moot point. The laws of diminishing return may indicate that the time spent on changing the order may not worth it HTH view my Linkedin profile <https://www.linkedin.com/in/mich-talebzadeh-ph-d-5205b2/> https://en.everybodywiki.com/Mich_Talebzadeh *Disclaimer:* Use it at your own risk. Any and all responsibility for any loss, damage or destruction of data or any other property which may arise from relying on this email's technical content is explicitly disclaimed. The author will in no case be liable for any monetary damages arising from such loss, damage or destruction. On Wed, 22 Feb 2023 at 21:00, Allan Folting <afolting...@gmail.com> wrote: > Hi all, > > I would like to propose that we show Python code examples first in the > Spark documentation where we have multiple programming language examples. > An example is on the Quick Start page: > https://spark.apache.org/docs/latest/quick-start.html > > I propose this change because Python has become more popular than the > other languages supported in Apache Spark. There are a lot more users of > Spark in Python than Scala today and Python attracts a broader set of new > users. > For Python usage data, see https://www.tiobe.com/tiobe-index/ and > https://insights.stackoverflow.com/trends?tags=r%2Cscala%2Cpython%2Cjava. > > Also, this change aligns with Python already being the first tab on our > home page: > https://spark.apache.org/ > > Anyone who wants to use another language can still just click on the other > tabs. > > I created a draft PR for the Spark SQL, DataFrames and Datasets Guide page > as a first step: > https://github.com/apache/spark/pull/40087 > > > I would appreciate it if you could share your thoughts on this proposal. > > > Thanks a lot, > Allan Folting >
