Sean Owen created SPARK-17193:
---------------------------------
Summary: HadoopRDD NPE at DEBUG log level when getLocationInfo ==
null
Key: SPARK-17193
URL: https://issues.apache.org/jira/browse/SPARK-17193
Project: Spark
Issue Type: Improvement
Components: Spark Core
Affects Versions: 2.0.0
Reporter: Sean Owen
Assignee: Sean Owen
Priority: Trivial
When I set the log level to "DEBUG" in one of my apps that reads from Parquet,
I notice several NullPointerExceptions logged from
HadoopRDD.getPreferredLocations.
It doesn't affect executions as it just results in "no preferred locations". It
happens when InputSplitWithLocationInfo.getLocationInfo produces null, which it
may. The code just dereferences it however.
It's cleaner to check this directly (and maybe tighten up the code slightly)
and avoid polluting the log, though, it's just at debug level. No big deal, but
enough of an annoyance when I was debugging something that it's probably worth
zapping.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
