Hi all,
I started trying to write some code on this, and realized there are a
number of issues that need to be discussed in order to really design this
feature effectively. The requirements that have been discussed thus far are:
1. Fetching data from S3 periodically
2. Fetching data from multiple
Please send mailto:user-unsubscr...@flume.apache.org
On Wed, Aug 6, 2014 at 8:14 AM, Xiaobo Liu wrote:
>
--
thanks
ashish
Blog: http://www.ashishpaliwal.com/blog
My Photo Galleries: http://www.pbase.com/ashishpaliwal
Sharing some random thoughts
1. Download the file using S3 SDK and let the SpoolDirectory implementation
take care of rest. Like a Decorator in front of SpoolDirectory
2. Use S3 SDK to create InputStream of S3 objects directly in code and
create events out of it.
Would be great to reuse an exist
Agree to the feedback provided by Ashish.
I have started writing one which is similar to the ExecSource, but I like
the idea of doing something where spooldir takes over most of the hard work
of spitting out events to sinks. Let me think more on how to structure
that.
Quick thinking out loud, I c
Hi,
I think that it is not possible to simply use SpoolDirectorySource. Maybe
it will be possible to use some elements of SpoolDirectory but without
touching it's code I think SpoolDirectory is not a good base. At the very
beginning SpoolDirectorySource does this:
File directory = new File(spoolD
