You are probably ok. There are a bunch of sinks that already do this (Asynchbase sink/Morphline Solr sink etc).
On Sun, Aug 24, 2014 at 10:17 PM, terreyshih <terreys...@gmail.com> wrote: > Thanks for answering my question. > > No, the transaction will not be shared by different threads. The > transaction is committed after I convert the events to some data structure. > > I would only have a Timer/Executor thread that fires off at specific > intervals to send these converted data structures (from events) to > customized storage/consumer for processing. > > Do you see an issue with this approach ? > > thanks, > Terrey. > > > > On Aug 24, 2014, at 9:59 PM, Hari Shreedharan <hshreedha...@cloudera.com> > wrote: > > If you create a large number of threads in a sink - that could affect the > performance of the agent - simply because there are too many threads. Also, > transactions are thread-local - so make sure each transaction is completely > owned and run by the same thread. > > > On Sun, Aug 24, 2014 at 7:28 PM, terreyshih <terreys...@gmail.com> wrote: > >> Hi, >> >> If in my customized sink, I create some threads and schedule it to do >> some queueing. For example, I would like to drop the events off every 10 >> seconds. If I start the thread or an executor in the process method of the >> customized sink class, would have have some negative impact to the >> threading model in Flume ? >> >> thanks, >> -Terrey > > > >
