Tathagata Das created KAFKA-2295:
------------------------------------
Summary: Dynamically loaded classes (encoders, etc.) may not be
found by Kafka Producer
Key: KAFKA-2295
URL: https://issues.apache.org/jira/browse/KAFKA-2295
Project: Kafka
Issue Type: Bug
Components: producer
Reporter: Tathagata Das
Assignee: Jun Rao
Kafka Producer (via CoreUtils.createObject) effectively uses Class.forName to
load encoder classes. Class.forName is by design finds classes only in the
defining classloader of the enclosing class (which is often the bootstrap class
loader). It does not use the current thread context class loader. This can lead
to problems in environments where classes are dynamically loaded and therefore
may not be present in the bootstrap classloader.
This leads to ClassNotFound Exceptions in environments like Spark where classes
are loaded dynamically using custom classloaders. Issues like this have
reported. E.g. -
https://www.mail-archive.com/[email protected]/msg30951.html
Other references regarding this issue with Class.forName
http://stackoverflow.com/questions/21749741/though-my-class-was-loaded-class-forname-throws-classnotfoundexception
This is a problem we have faced repeatedly in Apache Spark and we solved it by
explicitly specifying the class loader to use. See
https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/util/Utils.scala#L178
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
