Anyone have any ideas on this? I can't even find the class throwing the exception to try and see what it is doing. I would really like to use this on Windows, but would like to know at least if there's some compatibility issue so I can move on.
thanks, Paul ________________________________ From: Paul Chavez [mailto:pcha...@verticalsearchworks.com] Sent: Thursday, April 11, 2013 3:15 PM To: user@flume.apache.org Subject: spoolDir source problem Hello, I've run into a problem with the spoolDir source, on Windows, and am not sure how to proceed. The agent starts fine and the source is created without issue and is apparently ready. After agent start a .flumespool directory is created in the path the source is watching. This directory remains empty as long as the agent is idle. However, as soon as I drop a file into the spool directory (parent to the .flumespool directory), I get a series of errors in the flume log and a file named '.flumespool-main.meta<string of numbers>.tmp' is created in that .flumespool directory at the rate of one per second. The file in the spool directory is never touched as far as I can tell and the /metrics web page shows no movement on the channel or sink. A possibly related note is that the sources don't show in the metrics page, even though the logs say the source(s) are started. All I have done so far is set the directory security to be 'Everyone/Full Control', basically the windows version of 'chmod 777' Any help is appreciated! thanks, Paul Here's what the log shows. 11 Apr 2013 15:11:48,092 INFO [conf-file-poller-0] (org.apache.flume.node.Application.startAllComponents:184) - Starting Source spool_WebLogs 11 Apr 2013 15:11:48,092 INFO [conf-file-poller-0] (org.apache.flume.node.Application.startAllComponents:184) - Starting Source http_Default 11 Apr 2013 15:11:48,092 INFO [lifecycleSupervisor-1-0] (org.apache.flume.source.SpoolDirectorySource.start:66) - SpoolDirectorySource source starting with directory: c:\flume_data\spool\web 11 Apr 2013 15:11:48,124 INFO [conf-file-poller-0] (org.mortbay.log.Slf4jLog.info:67) - Logging to org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 11 Apr 2013 15:11:48,139 INFO [conf-file-poller-0] (org.mortbay.log.Slf4jLog.info:67) - jetty-6.1.26 11 Apr 2013 15:11:48,155 INFO [conf-file-poller-0] (org.mortbay.log.Slf4jLog.info:67) - Started SocketConnector@0.0.0.0:41414<mailto:SocketConnector@0.0.0.0:41414> 11 Apr 2013 15:11:48,202 INFO [lifecycleSupervisor-1-2] (org.mortbay.log.Slf4jLog.info:67) - jetty-6.1.26 11 Apr 2013 15:11:48,217 INFO [lifecycleSupervisor-1-2] (org.mortbay.log.Slf4jLog.info:67) - Started SocketConnector@0.0.0.0:6240<mailto:SocketConnector@0.0.0.0:6240> 11 Apr 2013 15:11:48,389 INFO [lifecycleSupervisor-1-1] (org.apache.flume.sink.AvroSink.start:253) - Avro sink avro_Default started. 11 Apr 2013 15:11:48,404 ERROR [pool-6-thread-1] (org.apache.flume.client.avro.ReliableSpoolingFileEventReader.getNextFile:442) - Exception opening file: c:\flume_data\spool\web\u_ex130411.log-201304111500.log java.io.IOException: Unable to delete existing meta file c:\flume_data\spool\web\.flumespool\.flumespool-main.meta at org.apache.flume.serialization.DurablePositionTracker.getInstance(DurablePositionTracker.java:96) at org.apache.flume.client.avro.ReliableSpoolingFileEventReader.getNextFile(ReliableSpoolingFileEventReader.java:423) at org.apache.flume.client.avro.ReliableSpoolingFileEventReader.readEvents(ReliableSpoolingFileEventReader.java:212) at org.apache.flume.source.SpoolDirectorySource$SpoolDirectoryRunnable.run(SpoolDirectorySource.java:154) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439) at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:204) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662) 11 Apr 2013 15:11:48,919 ERROR [pool-6-thread-1] (org.apache.flume.client.avro.ReliableSpoolingFileEventReader.getNextFile:442) - Exception opening file: c:\flume_data\spool\web\u_ex130411.log-201304111500.log java.io.IOException: Unable to delete existing meta file c:\flume_data\spool\web\.flumespool\.flumespool-main.meta at org.apache.flume.serialization.DurablePositionTracker.getInstance(DurablePositionTracker.java:96) at org.apache.flume.client.avro.ReliableSpoolingFileEventReader.getNextFile(ReliableSpoolingFileEventReader.java:417) at org.apache.flume.client.avro.ReliableSpoolingFileEventReader.readEvents(ReliableSpoolingFileEventReader.java:212) at org.apache.flume.source.SpoolDirectorySource$SpoolDirectoryRunnable.run(SpoolDirectorySource.java:154) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:439) at java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:317) at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:150) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$101(ScheduledThreadPoolExecutor.java:98) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.runPeriodic(ScheduledThreadPoolExecutor.java:180) at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:204) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang.Thread.run(Thread.java:662)
