Which version of slf4j you using ? for this error you need to put SLF4J 1.7.2 also in classpath as org.slf4j.spi.LocationAwareLogger.log is available in 1.7.2 of slf4j.
From: Dongliang Sun [mailto:[email protected]] Sent: Monday, January 14, 2013 11:54 AM To: Connor Woodson; Chhaya Vishwakarma; [email protected] Subject: Re: Could not instantiate class org.apache.flume.clients.log4jappender It does not work, -Dpig.additional.jars={REALPATH}/lib/*. Also tried Chhaya said, I put all the libraries from FLUME_HOME/lib/* to PIG_HOME/lib/, got following error: 13/01/14 14:19:03 WARN api.NettyAvroRpcClient: Invalid value for batchSize: 0; Using default value. Exception in thread "main" java.lang.NoSuchMethodError: org.slf4j.spi.LocationAwareLogger.log(Lorg/slf4j/Marker;Ljava/lang/String;ILjava/lang/String;[Ljava/lang/Object;Ljava/lang/Throwable;)V at org.apache.commons.logging.impl.SLF4JLocationAwareLog.error(SLF4JLocationAwareLog.java:211) at org.apache.pig.impl.util.LogUtils.writeLog(LogUtils.java:125) at org.apache.pig.impl.util.LogUtils.writeLog(LogUtils.java:79) at org.apache.pig.Main.run(Main.java:589) at org.apache.pig.Main.main(Main.java:111) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.util.RunJar.main(RunJar.java:156) 2013/1/14 Connor Woodson <[email protected]<mailto:[email protected]>> Alright so it is just before you weren't adding them properly to the classpath. I am not quite sure how to do that; a look at Pig's run script (bin/pig) might yield insight into an option to add them. Putting the files in pig's lib's folder almost works; the issue is those libraries have more dependencies that aren't in the classpath yet. Looking at their (log4jappender and sdk) poms could tell you exactly which libs you need to include, but let's not do that. I mistyped the -D command from before. Try this: -Dpig.additional.jars=${FLUME_HOME}/lib/* FLUME_HOME obviously needs to be declared beforehand, or just replace that with the real path (and make sure the log4jappender and sdk jars, and all other jars, are in the libs folder) On Sun, Jan 13, 2013 at 10:05 PM, Dongliang Sun <[email protected]<mailto:[email protected]>> wrote: REGISTER and -Dpig.additional.libraries both do not work, same error, could not find the class. If put the particular jars in pig's lib, the following error: Error before Pig is launched ---------------------------- ERROR 2998: Unhandled internal error. org/jboss/netty/channel/ChannelFactory java.lang.NoClassDefFoundError: org/jboss/netty/channel/ChannelFactory at org.apache.flume.api.RpcClientFactory.getDefaultInstance(RpcClientFactory.java:168) at org.apache.flume.api.RpcClientFactory.getDefaultInstance(RpcClientFactory.java:128) at org.apache.flume.clients.log4jappender.Log4jAppender.activateOptions(Log4jAppender.java:184) at org.apache.log4j.config.PropertySetter.activate(PropertySetter.java:257) at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:133) at org.apache.log4j.config.PropertySetter.setProperties(PropertySetter.java:97) at org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:689) at org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:647) at org.apache.log4j.PropertyConfigurator.parseCatsAndRenderers(PropertyConfigurator.java:568) at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:442) at org.apache.log4j.PropertyConfigurator.configure(PropertyConfigurator.java:367) at org.apache.pig.Main.configureLog4J(Main.java:678) at org.apache.pig.Main.run(Main.java:337) at org.apache.pig.Main.main(Main.java:111) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.util.RunJar.main(RunJar.java:156) Caused by: java.lang.ClassNotFoundException: org.jboss.netty.channel.ChannelFactory at java.net.URLClassLoader$1.run(URLClassLoader.java:202) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:190) at java.lang.ClassLoader.loadClass(ClassLoader.java:306) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) at java.lang.ClassLoader.loadClass(ClassLoader.java:247) ... 19 more 2013/1/14 Connor Woodson <[email protected]<mailto:[email protected]>> You could try calling 'REGISTER <log4jappender jar>' at the start of your script, or on the command line by adding -Dpig.additional.libraries=<log4jappender jar> to your Pig call. I don't know if that will work or not though. You could try placing it in pig's lib folder. You also need to include the flume-ng-sdk jar the same way as the log4jappender one, as the sdk is a dependency of the appender. On Sun, Jan 13, 2013 at 9:24 PM, Dongliang Sun <[email protected]<mailto:[email protected]>> wrote: Actually I have append the FLUME_HOME/lib/ to my CLASSPATH. Whether there is some problem in my PIG log4j.properties? Do you have any experience using log4jappender on PIG? Thanks, Dongliang 2013/1/14 Chhaya Vishwakarma <[email protected]<mailto:[email protected]>> Put flume-log4j-appender<https://repository.cloudera.com/content/repositories/releases/com/cloudera/flume-log4j-appender/0.9.4-cdh3u1/flume-log4j-appender-0.9.4-cdh3u1.jar>.jar in CLASSPATH. Jar can be found in FLUME_HOME/lib/ From: 孙东亮 [mailto:[email protected]<mailto:[email protected]>] Sent: Monday, January 14, 2013 8:55 AM To: [email protected]<mailto:[email protected]> Subject: Could not instantiate class org.apache.flume.clients.log4jappender I'm a newbie for the Flume, and I just set up flume for a test. I want to use the log4jappender to get log info from PIG scripts, the log4j.properties is: log4j.appender.flume = org.apache.flume.clients.log4jappender.Log4jAppender log4j.appender.flume.Hostname = localhost log4j.appender.flume.Port = 41414 log4j.logger.org.apache.pig=DEBUG,flume And for the flume.config: a1.sources = r1 a1.sinks = k1 a1.channels = c1 # Describe/configure the source a1.sources.r1.type = avro a1.sources.r1.bind = localhost a1.sources.r1.port = 41414 # Describe the sink a1.sinks.k1.type = logger # Use a channel which buffers events in memory a1.channels.c1.type = memory a1.channels.c1.capacity = 1000 a1.channels.c1.transactionCapacity = 100 # Bind the source and sink to the channel a1.sources.r1.channels = c1 a1.sinks.k1.channel = c1 But I got the following error when run a pig script: java.lang.ClassNotFoundException: org.apache.flume.clients.log4jappender.Log4jAppender at java.net.URLClassLoader$1.run(URLClassLoader.java:202) at java.security.AccessController.doPrivileged(Native Method) at java.net.URLClassLoader.findClass(URLClassLoader.java:190) at java.lang.ClassLoader.loadClass(ClassLoader.java:306) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) at java.lang.ClassLoader.loadClass(ClassLoader.java:247) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:169) at org.apache.log4j.helpers.Loader.loadClass(Loader.java:179) at org.apache.log4j.helpers.OptionConverter.instantiateByClassName(OptionConverter.java:320) at org.apache.log4j.helpers.OptionConverter.instantiateByKey(OptionConverter.java:121) at org.apache.log4j.PropertyConfigurator.parseAppender(PropertyConfigurator.java:664) at org.apache.log4j.PropertyConfigurator.parseCategory(PropertyConfigurator.java:647) at org.apache.log4j.PropertyConfigurator.parseCatsAndRenderers(PropertyConfigurator.java:568) at org.apache.log4j.PropertyConfigurator.doConfigure(PropertyConfigurator.java:442) at org.apache.log4j.PropertyConfigurator.configure(PropertyConfigurator.java:367) at org.apache.pig.Main.configureLog4J(Main.java:678) at org.apache.pig.Main.run(Main.java:337) at org.apache.pig.Main.main(Main.java:111) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.util.RunJar.main(RunJar.java:156) log4j:ERROR Could not instantiate appender named "flume". Could you please help me find what I miss or where is incorrect for the configuration. Thanks a lot! Dongliang ________________________________ The contents of this e-mail and any attachment(s) may contain confidential or privileged information for the intended recipient(s). Unintended recipients are prohibited from taking action on the basis of information in this e-mail and using or disseminating the information, and must notify the sender and delete it from their system. L&T Infotech will not accept responsibility or liability for the accuracy or completeness of, or the presence of any virus or disabling code in this e-mail"
