Hi,
This is basically a simple module
from pyspark import SparkContext
from pyspark.sql import SQLContext
from pyspark.sql import HiveContext
from pyspark.sql import SparkSession
from pyspark.sql import Row
from pyspark.sql.types import StringType, ArrayType
from pyspark.sql.functions import udf, col
import random
import string
import math
*spark = SparkSession.builder.appName("sparkp"*
*).enableHiveSupport().getOrCreate()*
and comes back with the following error
Traceback (most recent call last):
File "C:/Users/whg220/PycharmProjects/sparkp/venv/Scripts/sparkp.py",
line 43, in <module>
spark =
SparkSession.builder.appName("sparkp").enableHiveSupport().getOrCreate()
File
"C:\Users\whg220\spark\spark-2.3.4-bin-hadoop2.6\python\lib\pyspark.zip\pyspark\sql\session.py",
line 173, in getOrCreate
File
"C:\Users\whg220\spark\spark-2.3.4-bin-hadoop2.6\python\lib\pyspark.zip\pyspark\context.py",
line 363, in getOrCreate
File
"C:\Users\whg220\spark\spark-2.3.4-bin-hadoop2.6\python\lib\pyspark.zip\pyspark\context.py",
line 129, in __init__
File
"C:\Users\whg220\spark\spark-2.3.4-bin-hadoop2.6\python\lib\pyspark.zip\pyspark\context.py",
line 312, in _ensure_initialized
File
"C:\Users\whg220\spark\spark-2.3.4-bin-hadoop2.6\python\lib\pyspark.zip\pyspark\java_gateway.py",
line 46, in launch_gateway
File
"C:\Users\whg220\spark\spark-2.3.4-bin-hadoop2.6\python\lib\pyspark.zip\pyspark\java_gateway.py",
line 101, in _launch_gateway
File "C:\Program Files\Anaconda3\lib\subprocess.py", line 707, in __init__
restore_signals, start_new_session)
File "C:\Program Files\Anaconda3\lib\subprocess.py", line 990, in
_execute_child
startupinfo)
FileNotFoundError: [WinError 2] The system cannot find the file specified
very frustrating. Any help will be appreciated.
Thanks
*Disclaimer:* Use it at your own risk. Any and all responsibility for any
loss, damage or destruction of data or any other property which may arise
from relying on this email's technical content is explicitly disclaimed.
The author will in no case be liable for any monetary damages arising from
such loss, damage or destruction.
