Thanks Congxian for the tip. Arthas looks great Hao Sun Team Lead 1019 Market St. 7F San Francisco, CA 94103
On Fri, Jan 4, 2019 at 5:42 PM Congxian Qiu <qcx978132...@gmail.com> wrote: > Hi, Hao Sun > > For debugging the `ClassNotFoundException`, maybe the Arthas[1] tool can > help. > > [1] Arthas <https://github.com/alibaba/arthas>: > https://github.com/alibaba/arthas > <https://github.com/alibaba/arthas> > > Hao Sun <ha...@zendesk.com> 于2019年1月3日周四 下午10:08写道： > >> >> I am on Flink 1.7.1 and K8S. >> I said "suddenly" because my program worked fine until I added a new >> MapFunction. >> I do not know the details, but I think I know what is causing it >> >> === Start of Program === >> val stream: DataStream[MaxwellEvent] = <get the stream> >> steam.map(new ProblemFunction()) will cause the issue >> class ProblemFunction(stringParam: String)(implicit datadog: >> DatadogClient) extends MapFunction[MaxwellEvent, MaxwellEvent] >> === End of Program === >> >> I think the class taking curry params caused the issue, after I give up >> on the curry format, the error disappeared. >> >> I am using https://github.com/sbt/sbt-assembly >> <https://github.com/sbt/sbt-assembly> to assemble >> the fat jar. >> There might be some issue, or config issue with that as well. >> >> I am reading this article, it is a good start for me as well >> >> https://heapanalytics.com/blog/engineering/missing-scala-class-noclassdeffounderror >> <https://heapanalytics.com/blog/engineering/missing-scala-class-noclassdeffounderror> >> >> >> Hao Sun >> Team Lead >> 1019 Market St. 7F >> San Francisco, CA 94103 >> >> >> On Thu, Jan 3, 2019 at 1:08 AM Timo Walther <twal...@apache.org> wrote: >> >>> Hi Hao, >>> >>> which Flink version are you using? What do you mean with "suddenly", did >>> it work before? >>> >>> Regards, >>> Timo >>> >>> >>> Am 03.01.19 um 07:13 schrieb Hao Sun: >>> >>> Yep, javap shows the class is there, but FlinkUserCodeClassLoaders >>> somehow could not find it suddenly >>> >>> javap -cp /opt/flink/lib/zendesk-fps-core-assembly-0.1.0.jar >>> 'com.zendesk.fraudprevention.examples.ConnectedStreams$$anon$90$$anon$45' >>> Compiled from "ConnectedStreams.scala" >>> public final class >>> com.zendesk.fraudprevention.examples.ConnectedStreams$$anon$90$$anon$45 >>> extends >>> org.apache.flink.api.scala.typeutils.CaseClassSerializer<com.zendesk.fraudprevention.datatypes.MaxwellEvent> >>> { >>> public com.zendesk.fraudprevention.datatypes.MaxwellEvent >>> createInstance(java.lang.Object[]); >>> public >>> org.apache.flink.api.scala.typeutils.CaseClassSerializer<com.zendesk.fraudprevention.datatypes.MaxwellEvent> >>> createSerializerInstance(java.lang.Class<com.zendesk.fraudprevention.datatypes.MaxwellEvent>, >>> org.apache.flink.api.common.typeutils.TypeSerializer<?>[]); >>> public org.apache.flink.api.java.typeutils.runtime.TupleSerializerBase >>> createSerializerInstance(java.lang.Class, >>> org.apache.flink.api.common.typeutils.TypeSerializer[]); >>> public java.lang.Object createInstance(java.lang.Object[]); >>> public >>> com.zendesk.fraudprevention.examples.ConnectedStreams$$anon$90$$anon$45(com.zendesk.fraudprevention.examples.ConnectedStreams$$anon$90, >>> org.apache.flink.api.common.typeutils.TypeSerializer[]); >>> } >>> >>> Hao Sun >>> Team Lead >>> 1019 Market St. 7F >>> San Francisco, CA 94103 >>> >>> >>> On Wed, Jan 2, 2019 at 6:04 PM qi luo <luoqi...@gmail.com> wrote: >>> >>>> Hi Hao, >>>> >>>> Since Flink is using Child-First class loader, you may try search for >>>> the class >>>> "*com.zendesk.fraudprevention.examples.ConnectedStreams$$anon$90$$anon$45” >>>> in your fat JAR. Is that an inner class?* >>>> >>>> *Best,* >>>> *Qi* >>>> >>>> On Jan 3, 2019, at 7:01 AM, Hao Sun <ha...@zendesk.com> wrote: >>>> >>>> Hi, >>>> >>>> I am wondering if there are any protips to figure out what class is not >>>> found? >>>> >>>> ===== Logs ==== >>>> org.apache.flink.streaming.runtime.tasks.StreamTaskException: Could not >>>> instantiate chained outputs. >>>> at >>>> org.apache.flink.streaming.api.graph.StreamConfig.getChainedOutputs(StreamConfig.java:324) >>>> at >>>> org.apache.flink.streaming.runtime.tasks.OperatorChain.createOutputCollector(OperatorChain.java:292) >>>> at org.apache.flink.streaming.runtime.tasks.OperatorChain.<init>( >>>> http://OperatorChain.java:133 >>>> <http://OperatorChain.java:133>) >>>> at >>>> org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:267) >>>> at org.apache.flink.runtime.taskmanager.Task.run(Task.java:704) >>>> at java.lang.Thread.run(Thread.java:748) >>>> *Caused by: java.lang.ClassNotFoundException: >>>> com.zendesk.fraudprevention.examples.ConnectedStreams$$anon$90$$anon$45* >>>> at java.net.URLClassLoader.findClass(URLClassLoader.java:381) >>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:424) >>>> at >>>> org.apache.flink.runtime.execution.librarycache.FlinkUserCodeClassLoaders$ChildFirstClassLoader.loadClass(FlinkUserCodeClassLoaders.java:129) >>>> at java.lang.ClassLoader.loadClass(ClassLoader.java:357) >>>> at java.lang.Class.forName0(Native Method) >>>> at java.lang.Class.forName(Class.java:348) >>>> at >>>> org.apache.flink.util.InstantiationUtil$ClassLoaderObjectInputStream.resolveClass(InstantiationUtil.java:77) >>>> at >>>> java.io.ObjectInputStream.readNonProxyDesc(ObjectInputStream.java:1866) >>>> at java.io.ObjectInputStream.readClassDesc(ObjectInputStream.java:1749) >>>> at >>>> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2040) >>>> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1571) >>>> at >>>> java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2285) >>>> at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2209) >>>> at >>>> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2067) >>>> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1571) >>>> at >>>> java.io.ObjectInputStream.defaultReadFields(ObjectInputStream.java:2285) >>>> at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2209) >>>> at >>>> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2067) >>>> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1571) >>>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:431) >>>> at java.util.ArrayList.readObject(ArrayList.java:797) >>>> at sun.reflect.GeneratedMethodAccessor26.invoke(Unknown Source) >>>> at >>>> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) >>>> at java.lang.reflect.Method.invoke(Method.java:498) >>>> at >>>> java.io.ObjectStreamClass.invokeReadObject(ObjectStreamClass.java:1158) >>>> at java.io.ObjectInputStream.readSerialData(ObjectInputStream.java:2176) >>>> at >>>> java.io.ObjectInputStream.readOrdinaryObject(ObjectInputStream.java:2067) >>>> at java.io.ObjectInputStream.readObject0(ObjectInputStream.java:1571) >>>> at java.io.ObjectInputStream.readObject(ObjectInputStream.java:431) >>>> at >>>> org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:524) >>>> at >>>> org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:510) >>>> at >>>> org.apache.flink.util.InstantiationUtil.deserializeObject(InstantiationUtil.java:498) >>>> at >>>> org.apache.flink.util.InstantiationUtil.readObjectFromConfig(InstantiationUtil.java:459) >>>> at >>>> org.apache.flink.streaming.api.graph.StreamConfig.getChainedOutputs(StreamConfig.java:321) >>>> ... 5 more >>>> >>>> >>>> Hao Sun >>>> Team Lead >>>> 1019 Market St. 7F >>>> San Francisco, CA 94103 >>>> >>>> >>>> >>> > > -- > Blog:http://www.klion26.com > <http://www.klion26.com> > GTalk:qcx978132955 > 一切随心 >
