thanks On Aug 24, 2014, at 10:20 PM, Hari Shreedharan <hshreedha...@cloudera.com> wrote:
> You are probably ok. There are a bunch of sinks that already do this > (Asynchbase sink/Morphline Solr sink etc). > > > On Sun, Aug 24, 2014 at 10:17 PM, terreyshih <terreys...@gmail.com> wrote: > Thanks for answering my question. > > No, the transaction will not be shared by different threads. The transaction > is committed after I convert the events to some data structure. > > I would only have a Timer/Executor thread that fires off at specific > intervals to send these converted data structures (from events) to customized > storage/consumer for processing. > > Do you see an issue with this approach ? > > thanks, > Terrey. > > > > On Aug 24, 2014, at 9:59 PM, Hari Shreedharan <hshreedha...@cloudera.com> > wrote: > >> If you create a large number of threads in a sink - that could affect the >> performance of the agent - simply because there are too many threads. Also, >> transactions are thread-local - so make sure each transaction is completely >> owned and run by the same thread. >> >> >> On Sun, Aug 24, 2014 at 7:28 PM, terreyshih <terreys...@gmail.com> wrote: >> Hi, >> >> If in my customized sink, I create some threads and schedule it to do some >> queueing. For example, I would like to drop the events off every 10 >> seconds. If I start the thread or an executor in the process method of the >> customized sink class, would have have some negative impact to the threading >> model in Flume ? >> >> thanks, >> -Terrey >> > >
