Lantao Jin created SPARK-20816:
----------------------------------
Summary: MetricsConfig doen't trim the properties file cause the
exception very confused
Key: SPARK-20816
URL: https://issues.apache.org/jira/browse/SPARK-20816
Project: Spark
Issue Type: Improvement
Components: Spark Core
Affects Versions: 2.1.1
Reporter: Lantao Jin
Priority: Minor
Spark Metrics System use a {{Properties File}} to load the configurations but
doesn't trim the keys and values. It might cause the exception very confused if
the property is a class name.
For example below, you must do not notice there is a space at the line end.
{quote}
*.sink.ganglia.class=org.apache.spark.metrics.sink.GangliaSink
{quote}
Unfortunately, the {{ClassNotFoundException}} throwing from Driver also doesn't
tell me what happens and confuses me because I am sure the related jar is in
the CLASSPATH.
{quote}
17/05/20 12:47:04 ERROR SparkContext: Error initializing SparkContext.
java.lang.ClassNotFoundException: org.apache.spark.metrics.sink.GangliaSink
at
scala.reflect.internal.util.AbstractFileClassLoader.findClass(AbstractFileClassLoader.scala:62)
{quote}
As a reference, I check the code of Log4j, a classic {{Properties}} using
library. It do the trim when load the properties.
{code:title=org.apache.log4j.filter.PropertyFilter.java|borderStyle=solid}
private Hashtable parseProperties(String props) {
Hashtable hashTable = new Hashtable();
StringTokenizer pairs = new StringTokenizer(props, ",");
while (pairs.hasMoreTokens()) {
StringTokenizer entry = new
StringTokenizer(pairs.nextToken(), "=");
hashTable.put(entry.nextElement().toString().trim(),
entry.nextElement().toString().trim());
}
return hashTable;
}
{code}
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
