sure :) ... Maybe we could also commit the fix on our end to. would love a chance to dive into the flume code some more
On Mon, May 13, 2013 at 11:39 AM, Brock Noland <[email protected]> wrote: > Hmm that shouldn't cause an NPE. Could you find a JIRA for this? > > > On Mon, May 13, 2013 at 10:11 AM, Jay Vyas <[email protected]> wrote: > >> Ahhh I had no batchSize parameter. So the fix was to add batchSize = 1 >> to my file and now its working. Anyone wants more details just ping me >> >> >> On Sun, May 12, 2013 at 7:13 PM, Jay Vyas <[email protected]> wrote: >> >>> Hi ! Thanks --- Here's my config >>> >>> a1.sources = r1 >>> a1.sinks = myFileSystem >>> a1.channels = c1 >>> >>> # Describe/configure the source >>> a1.sources.r1.type = exec >>> a1.sources.r1.command = tail -F /root/xyz/flumeinput.tmp >>> a1.sources.r1.channels = c1 >>> a1.sources = r1 >>> a1.channels = c1 >>> a1.sinks = k1 >>> a1.sinks.k1.type = file_roll >>> a1.sinks.k1.channel = c1 >>> a1.sinks.k1.sink.directory = /mnt/xyz/flume >>> >>> #Do we need this? >>> agent1.sinks.sink1.hdfs.filePrefix = LogCreateTest >>> >>> # Use a channel which buffers events in memory >>> a1.channels.c1.type = file >>> #a1.channels.c1.capacity = 100 >>> #a1.channels.c1.transactionCapacity = 10 >>> >>> # Bind the source and sink to the channel >>> a1.sources.r1.channels = c1 >>> a1.sinks.myFileSystem.channel = c1 >>> >>> >>> >>> On Sun, May 12, 2013 at 8:57 AM, Alexander Alten-Lorenz < >>> [email protected]> wrote: >>> >>>> Hi, >>>> >>>> Could you attach the config too? >>>> >>>> - Alex >>>> >>>> On May 11, 2013, at 11:52 PM, Jay Vyas <[email protected]> wrote: >>>> >>>> > Hi flume : >>>> > >>>> > I've setup a file to file source/sink , just as a test. Im using the >>>> exec source, with tail -F to follow a simple tmp file that is being catted >>>> to. >>>> > >>>> > However, I've started seeing the following exception in the flume >>>> logs: >>>> > >>>> > java.lang. >>>> > IllegalStateException: Channel closed [channel=c1]. Due to >>>> java.lang.NullPointerException: null >>>> > at >>>> org.apache.flume.channel.file.FileChannel.createTransaction(FileChannel.java:353) >>>> > at >>>> org.apache.flume.channel.BasicChannelSemantics.getTransaction(BasicChannelSemantics.java:122) >>>> > at >>>> org.apache.flume.sink.RollingFileSink.process(RollingFileSink.java:183) >>>> > at >>>> org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68) >>>> > at >>>> org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147) >>>> > at java.lang.Thread.run(Thread.java:662) >>>> > Caused by: java.lang.NullPointerException >>>> > at org.apache.flume.channel.file.Log.writeCheckpoint(Log.java:895) >>>> > at org.apache.flume.channel.file.Log.replay(Log.java:406) >>>> > at >>>> org.apache.flume.channel.file.FileChannel.start(FileChannel.java:303) >>>> > at >>>> org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:236) >>>> > at >>>> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439) >>>> > at >>>> java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317) >>>> > at >>>> java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150) >>>> > at >>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98) >>>> > at >>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:180) >>>> > at >>>> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:204) >>>> > at >>>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895) >>>> > at >>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918) >>>> > ... 1 more >>>> > >>>> > Any thoughts on where this NullPointerException is coming from? It >>>> appears from scanning the code that maybe it related to a missing folder or >>>> directory. But I cant find the exact line on the git hub branches. >>>> > >>>> > This is using apache-flume-1.3.1.23-... >>>> > >>>> > >>>> > -- >>>> > Jay Vyas >>>> > http://jayunit100.blogspot.com >>>> >>>> -- >>>> Alexander Alten-Lorenz >>>> http://mapredit.blogspot.com >>>> German Hadoop LinkedIn Group: http://goo.gl/N8pCF >>>> >>>> >>> >>> >>> -- >>> Jay Vyas >>> http://jayunit100.blogspot.com >>> >> >> >> >> -- >> Jay Vyas >> http://jayunit100.blogspot.com >> > > > > -- > Apache MRUnit - Unit testing MapReduce - http://mrunit.apache.org > -- Jay Vyas http://jayunit100.blogspot.com
