Hi Team,
1. Can someone please guide on how to change the following transformation
change for the below data in Spark 2.0?
{"name":"Michael", "age":30, "state":AZ, "Job": SoftwareDev}
{"name":"Andy", "age":30, "state":WA, "Job": Mana}
{"name":"Justin", "age":19, , "state":VIC, "Job": DBAdmin
Hi, All.
As a continuation of SPARK-20682(Support a new faster ORC data source based on
Apache ORC), I would like to suggest to make the default ORCFileFormat
configurable between sql/hive and sql/core for the followings.
spark.read.orc(...)
spark.write.orc(...)
CREATE TABLE t
Hi everyone,
For the last few months I've been working on static type annotations for
PySpark. For those of you, who are not familiar with the idea, typing
hints have been introduced by PEP 484
(https://www.python.org/dev/peps/pep-0484/) and further extended with
PEP 526 (https://www.python.org/de
