Github user BryanCutler commented on the pull request:
https://github.com/apache/spark/pull/12304#issuecomment-208507385
@jkbradley I noticed a couple of potential issues with current JavaCallable
that I took care of here. First, trying to detach the Java object on `__del__`
could cause a problem if the Python object was copied and one instance gets
garbage collected. This would cause the remaining Python objects to have an
invalid Java object reference, so I removed the call to detach. Second,
assigning the Spark context in the constructor as `self._sc =
SparkContext._active_spark_context` could be a problem if the Python object is
created outside an active SparkContext and then the user does something to
`_call_java` within a valid context. So I think it is better to only use
`SparkContext._active_spark_context` when needed and calls will fail then if
outside a valid context.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
