[
https://issues.apache.org/jira/browse/SPARK-60148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nicolas Lu updated SPARK-60148:
-------------------------------
Description:
{{pyspark/traceback_utils.py}} has no dedicated unit tests. It is only
exercised indirectly through RDD and DataFrame actions
({{{}core/context.py{}}}, {{{}core/rdd.py{}}},
{{{}sql/classic/dataframe.py{}}}), so several branches are never checked
directly.
This ticket adds {{python/pyspark/tests/test_traceback_utils.py}} and registers
it in {{{}dev/sparktestsupport/modules.py{}}}. It covers:
* {{{}first_spark_call(){}}}: empty stack returns {{{}None{}}}; returns the
user's file and line for the first Spark frame; only the first Spark frame is
used; Spark frame at the bottom of the stack.
* {{{}SCCallSiteSync{}}}: sets and clears the call site; fallback message when
the call site is unknown; nested usage sets the call site only at the outermost
level; the call site is cleared when the block raises.
was:
I'm working on this and will open a PR shortly
{{pyspark/traceback_utils.py}} has no dedicated unit tests. It is only
exercised indirectly through RDD and DataFrame actions
({{{}core/context.py{}}}, {{{}core/rdd.py{}}},
{{{}sql/classic/dataframe.py{}}}), so several branches are never checked
directly.
This ticket adds {{python/pyspark/tests/test_traceback_utils.py}} and registers
it in {{{}dev/sparktestsupport/modules.py{}}}. It covers:
* {{{}first_spark_call(){}}}: empty stack returns {{{}None{}}}; returns the
user's file and line for the first Spark frame; only the first Spark frame is
used; Spark frame at the bottom of the stack.
* {{{}SCCallSiteSync{}}}: sets and clears the call site; fallback message when
the call site is unknown; nested usage sets the call site only at the outermost
level; the call site is cleared when the block raises.
> Add unit tests for pyspark.traceback_utils
> ------------------------------------------
>
> Key: SPARK-60148
> URL: https://issues.apache.org/jira/browse/SPARK-60148
> Project: Spark
> Issue Type: Test
> Components: PySpark
> Affects Versions: 4.4.0
> Reporter: Nicolas Lu
> Priority: Minor
>
> {{pyspark/traceback_utils.py}} has no dedicated unit tests. It is only
> exercised indirectly through RDD and DataFrame actions
> ({{{}core/context.py{}}}, {{{}core/rdd.py{}}},
> {{{}sql/classic/dataframe.py{}}}), so several branches are never checked
> directly.
> This ticket adds {{python/pyspark/tests/test_traceback_utils.py}} and
> registers it in {{{}dev/sparktestsupport/modules.py{}}}. It covers:
> * {{{}first_spark_call(){}}}: empty stack returns {{{}None{}}}; returns the
> user's file and line for the first Spark frame; only the first Spark frame is
> used; Spark frame at the bottom of the stack.
> * {{{}SCCallSiteSync{}}}: sets and clears the call site; fallback message
> when the call site is unknown; nested usage sets the call site only at the
> outermost level; the call site is cleared when the block raises.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
