Are you talking about FLUME-1491 patch 7 or some other patch? If yes, then
it does listen for change notifications from Zookeeper, via Curator's
NodeCache.
On Sun, Aug 3, 2014 at 10:21 PM, Paweł Róg wrote:
> Hi,
> I fetched a patch and see that zookeeper configuration provider doesn't
> listen
One of the hdfs compression codecs had a memory leak. dont recall which
right now. if you are using one, try changing to a diff codec and see if
the leak goes away.
-roshan
On Sat, Aug 2, 2014 at 6:19 AM, Christopher Shannon
wrote:
> I do want to add that the Windows agents in our configuration
The one we are using is the bzip2 codec. That is something we could test.
On Monday, August 4, 2014, Roshan Naik wrote:
> One of the hdfs compression codecs had a memory leak. dont recall which
> right now. if you are using one, try changing to a diff codec and see if
> the leak goes away.
> -ro
This was fixed in FLUME-2416. Codecs using direct memory were leaking
direct buffers if full GC did not happen (since the direct buffers are
actually cleaned up only on full gc).
On Mon, Aug 4, 2014 at 2:57 PM, Christopher Shannon
wrote:
> The one we are using is the bzip2 codec. That is someth
Roshan, Hari,
This explains a lot of the problems we have been experiencing.
Thank you very much!
All the best,
Chris Shannon.
On Monday, August 4, 2014, Hari Shreedharan
wrote:
> This was fixed in FLUME-2416. Codecs using direct memory were leaking
> direct buffers if full GC did not happen
What about using a workflow tool like Oozie, Azkaban, or Amazon Data
Pipeline? Set them to be triggered as soon as the s3 bucket is available
and execute the ALTER TABLE command.
On Thursday, July 31, 2014, Viral Bajaria wrote:
> Any suggestions on this ? Still trying to figure out how do I get
I was planning to use the notify/callback to add an entry in some external
queue system or trigger a workflow. Without knowing what partition just got
written to is very tough.
Plus not everything is always written to a S3 bucket. Some files are
actually written to HDFS too.
How will a workflow t
