There is a correctness in the following code snippet. (https://issues.apache.org/jira/browse/SPARK-35211)
``` spark.conf.set("spark.sql.execution.arrow.pyspark.enabled", "false") from pyspark.testing.sqlutils import ExamplePoint import pandas as pd pdf = pd.DataFrame({'point': pd.Series([ExamplePoint(1, 1), ExamplePoint(2, 2)])}) df = spark.createDataFrame(pdf, verifySchema=False) df.show() ``` I created two pr to resolve it: PR 1 of 2: for inferred schema, also perform schema verification https://github.com/apache/spark/pull/32320 PR 2 of 2: with schema verification disabled, do number conversion properly https://github.com/apache/spark/pull/32327 Hope to get them reviewed. BTW And for UDT Support in PySpark, besides correctness issue, arrow support is also missing. (https://issues.apache.org/jira/browse/SPARK-34771) I've created a PR to solve it.