Hi, Hao Sun For debugging the `ClassNotFoundException`, maybe the Arthas[1] tool can help.
[1] Arthas <https://github.com/alibaba/arthas>: https://github.com/alibaba/arthas Hao Sun <ha...@zendesk.com> 于2019年1月3日周四 下午10:08写道： > > I am on Flink 1.7.1 and K8S. > I said "suddenly" because my program worked fine until I added a new > MapFunction. > I do not know the details, but I think I know what is causing it > > === Start of Program === > val stream: DataStream[MaxwellEvent] = <get the stream> > steam.map(new ProblemFunction()) will cause the issue > class ProblemFunction(stringParam: String)(implicit datadog: > DatadogClient) extends MapFunction[MaxwellEvent, MaxwellEvent] > === End of Program === > > I think the class taking curry params caused the issue, after I give up on > the curry format, the error disappeared. > > I am using https://github.com/sbt/sbt-assembly to assemble the fat jar. > There might be some issue, or config issue with that as well. > > I am reading this article, it is a good start for me as well > > https://heapanalytics.com/blog/engineering/missing-scala-class-noclassdeffounderror > > > Hao Sun > Team Lead > 1019 Market St. 7F > San Francisco, CA 94103 > > > On Thu, Jan 3, 2019 at 1:08 AM Timo Walther <twal...@apache.org> wrote: > >> Hi Hao, >> >> which Flink version are you using? What do you mean with "suddenly", did >> it work before? >> >> Regards, >> Timo >> >> >> Am 03.01.19 um 07:13 schrieb Hao Sun: >> >> Yep, javap shows the class is there, but FlinkUserCodeClassLoaders >> somehow could not find it suddenly >> >> javap -cp /opt/flink/lib/zendesk-fps-core-assembly-0.1.0.jar >> 'com.zendesk.fraudprevention.examples.ConnectedStreams$$anon$90$$anon$45' >> Compiled from "ConnectedStreams.scala" >> public final class >> com.zendesk.fraudprevention.examples.ConnectedStreams$$anon$90$$anon$45 >> extends >> org.apache.flink.api.scala.typeutils.CaseClassSerializer<com.zendesk.fraudprevention.datatypes.MaxwellEvent> >> { >> public com.zendesk.fraudprevention.datatypes.MaxwellEvent >> createInstance(java.lang.Object[]); >> public >> org.apache.flink.api.scala.typeutils.CaseClassSerializer<com.zendesk.fraudprevention.datatypes.MaxwellEvent> >> createSerializerInstance(java.lang.Class<com.zendesk.fraudprevention.datatypes.MaxwellEvent>, >> org.apache.flink.api.common.typeutils.TypeSerializer<?>[]); >> public org.apache.flink.api.java.typeutils.runtime.TupleSerializerBase >> createSerializerInstance(java.lang.Class, >> org.apache.flink.api.common.typeutils.TypeSerializer[]); >> public java.lang.Object createInstance(java.lang.Object[]); >> public >> com.zendesk.fraudprevention.examples.ConnectedStreams$$anon$90$$anon$45(com.zendesk.fraudprevention.examples.ConnectedStreams$$anon$90, >> org.apache.flink.api.common.typeutils.TypeSerializer[]); >> } >> >> Hao Sun >> Team Lead >> 1019 Market St. 7F >> San Francisco, CA 94103 >> >> >> On Wed, Jan 2, 2019 at 6:04 PM qi luo <luoqi...@gmail.com> wrote: >> >>> Hi Hao, >>> >>> Since Flink is using Child-First class loader, you may try search for >>> the class >>> "*com.zendesk.fraudprevention.examples.ConnectedStreams$$anon$90$$anon$45” >>> in your fat JAR. Is that an inner class?* >>> >>> *Best,* >>> *Qi* >>> >>> On Jan 3, 2019, at 7:01 AM, Hao Sun <ha...@zendesk.com> wrote: >>> >>> Hi, >>> >>> I am wondering if there are any protips to figure out what class is not >>> found? >>> >>> ===== Logs ==== >>> org.apache.flink.streaming.runtime.tasks.StreamTaskException: Could not >>> instantiate chained outputs. >>> at >>> org.apache.flink.streaming.api.graph.StreamConfig.getChainedOutputs(StreamConfig.java:324) >>> at >>> org.apache.flink.streaming.runtime.tasks.OperatorChain.createOutputCollector(OperatorChain.java:292) >>> at org.apache.flink.streaming.runtime.tasks.OperatorChain.<init>( >>> http://OperatorChain.java:133) >>> at >>> org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:267) >>> at org.apache.flink.runtime.taskmanager.Task.run(Task.java:704) >>> at java.lang.Thread.run(Thread.java:748) >>> *Caused by: java.lang.ClassNotFoundException: >>> com.zendesk.fraudprevention.examples.ConnectedStreams$$anon$90$$anon$45* >>> at java.net.URLClassLoader.findClass(URLClassLoader.java:381) >>> at java.lang.ClassLoader.loadClass(ClassLoader.java:424) >>> at >>> org.apache.flink.runtime.execution.librarycache.FlinkUserCodeClassLoaders$ChildFirstClassLoader.loadClass(FlinkUserCodeClassLoaders.java:129) >>> at java.lang.ClassLoader.loadClass(ClassLoader.java:357) >>> at java.lang.Class.forName0(Native Method) >>> at java.lang.Class.forName(Class.java:348) >>> at >>> org.apache.flink.util.InstantiationUtil$ClassLoaderObjectInputStream.resolveClass(InstantiationUtil.java:77) >>> at >>> java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1866) >>> at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1749) >>> at >>> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2040) >>> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1571) >>> at >>> java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2285) >>> at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2209) >>> at >>> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2067) >>> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1571) >>> at >>> java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2285) >>> at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2209) >>> at >>> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2067) >>> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1571) >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:431) >>> at java.util.ArrayList.readObject(ArrayList.java:797) >>> at sun.reflect.GeneratedMethodAccessor26.invoke(Unknown Source) >>> at >>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >>> at java.lang.reflect.Method.invoke(Method.java:498) >>> at >>> java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1158) >>> at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2176) >>> at >>> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2067) >>> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1571) >>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:431) >>> at >>> org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:524) >>> at >>> org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:510) >>> at >>> org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:498) >>> at >>> org.apache.flink.util.InstantiationUtil.readObjectFromConfig(InstantiationUtil.java:459) >>> at >>> org.apache.flink.streaming.api.graph.StreamConfig.getChainedOutputs(StreamConfig.java:321) >>> ... 5 more >>> >>> >>> Hao Sun >>> Team Lead >>> 1019 Market St. 7F >>> San Francisco, CA 94103 >>> >>> >>> >> -- Blog:http://www.klion26.com GTalk:qcx978132955 一切随心
