Hi Andrew, You may try lowering transactionCapacity here. The transactionCapacity should be set to the value of the largest batch size that will be used to store or remove events from that channel. You currently have it equal to the capacity of the channel. So essentially the channel *could be* filled with one transaction depending on how you are batching with your client.
Also it may be useful to turn up jmx monitoring and watch the channel counters using jconsole. This way you can see exactly how many events are placed in the channel. To do this you will need to Set the following Java system properties located at, /etc/flume-ng/conf/flume-env.sh. com.sun.management.jmxremote com.sun.management.jmxremote.port=8081 com.sun.management.jmxremote.authenticate=false com.sun.management.jmxremote.ssl=false You should than be able to connect with jconsole hostname:8081 -Jeff On Thu, Jan 17, 2013 at 9:36 AM, Andrew Otto <o...@wikimedia.org> wrote: > > I took those two border sequence numbers and extracted all of the log > lines generated by cp1044 on the nearby host (not using Flume). I should > be able to compare the number of lines here with the number of lines in the > 12 files I extracted from HDFS and Flume. If they are the same, then Flume > and UDPSource is working! > > Oh, I meant to link to a Gist with my current flume.conf and the commands > I executed to investigate this. Here it is: > > https://gist.github.com/4557178 > > > >
