Hi,
On Fri, Aug 1, 2014 at 4:52 AM, Jonathan Natkins
wrote:
> Hey all,
>
> I created a JIRA for this:
> https://issues.apache.org/jira/browse/FLUME-2437
>
Thanks! Should Fix Version be set to the next Flume release version?
I thought I'd start working on one myself, which can hopefully be
> c
Hi Sharninder and Ashnish,
Thanks for your nice suggestions. I agree one good solution would be
writing some tools to glue libpcap, Avro and Flume.
2014-08-01 14:27 GMT+08:00 Sharninder :
> Liu, you first need to figure out what TCP data you want to collect. Is
> there a possibility that this d
The result is the same that I put below, it inserts 2 values into one column:
column3
col1val: firstPart
col2val: This is the first part of the result
From: Jonathan Natkins [mailto:na...@streamsets.com]
Sent: Thursday, July 31, 2014 7:02 PM
To: user@flume.apache.org
Subject: Re: Flume to Hbase
Hi,
Jonathan how should we interpret your last e-mail? You opened an JIRA issue
and want to start implementing this and do you have any estimate how long
it will take?
I think the biggest challenge here is to have dynamic configuration of
Flume. It doesn't seem to be part of FLUME-2437 issue. Am I
Hi,
Currently I see no option to configure Flume dynamically.
I have a case where I have to be able to
* add or remove Source
* attach Source to a Channel
* attach a Channel to a Sink or several Sinks
All of this things are possible but only with configuration file which
means restart of applicat
If you change the configuration file, Flume reloads it, unless you tell it
not to by specifying no-reload-conf option.
On Fri, Aug 1, 2014 at 3:35 PM, Paweł wrote:
> Hi,
> Currently I see no option to configure Flume dynamically.
>
> I have a case where I have to be able to
> * add or remove So
Hi,
Thanks for information. Please tell me what happens with existing sources,
channels and sinks when configuration is reloaded? Are they destroyed and
started once again?
Zookeeper would be nice here. Can you tell me if there are any estimates
for time needed to close the issue or to have any st
On Fri, Aug 1, 2014 at 4:40 PM, Paweł wrote:
> Hi,
> Thanks for information. Please tell me what happens with existing sources,
> channels and sinks when configuration is reloaded? Are they destroyed and
> started once again?
>
During reload Flume shall stop existing ones and the reload the conf
Have a look at Flume Client SDK, this would make implementation very easy.
Basically you implement a wrapper for libpcap, run it anywhere and you can
send Avro events to Avro source.
On Fri, Aug 1, 2014 at 1:27 PM, Blade Liu wrote:
> Hi Sharninder and Ashnish,
>
> Thanks for your nice suggestio
Hey Pawel,
My intention is to start working on it, but I don't know exactly how long
it will take, and I'm not a committer, so time estimates would have to be
taken with a grain of salt regardless. If this is something that you need
urgently, it may not be ideal to wait for me to start building so
Let's be a little more explicit, since this result doesn't make a lot of
sense to me:
What is the value you have configured for
agent.sinks.hbaseSink.serializer.colNames
and agent.sinks.hbaseSink.serializer.regex, and can you give an example of
the exact event that is coming into Flume?
Is it sti
Hi all,
I'm playing with Flume for the first time, so perhaps I'm misunderstanding
something, but I can't find in the documentation any command to stop or
kill or a Flume job - other than shutting the whole machine down, which is
a bit of overkill when you're tinkering and changing things.
Can an
Charles,
To kill the Flume job AND invoke the shutdown hooks for a clean shutdown,
use kill -HUP or kill -TERM. Any other signal will kill the JVM without
invoking the shutdown hooks.
On Fri, Aug 1, 2014 at 1:14 PM, Charles Robertson <
charles.robert...@gmail.com> wrote:
> Hi all,
>
> I'm playi
Also, flume will notice that you changed the configuration file and will
reload itself with the changes.
On Fri, Aug 1, 2014 at 1:16 PM, Christopher Shannon
wrote:
> Charles,
>
> To kill the Flume job AND invoke the shutdown hooks for a clean shutdown,
> use kill -HUP or kill -TERM. Any other
+1 on an S3 Source. I would gladly review.
Jonathan Natkins wrote:
Hey Pawel,
My intention is to start working on it, but I don't know exactly how
long it will take, and I'm not a committer, so time estimates would
have to be taken with a grain of salt regardless. If this is something
that you
Christopher,
Perfect, thank you!
Charles
On 1 August 2014 19:17, Christopher Shannon wrote:
> Also, flume will notice that you changed the configuration file and will
> reload itself with the changes.
>
>
> On Fri, Aug 1, 2014 at 1:16 PM, Christopher Shannon > wrote:
>
>> Charles,
>>
>> To
Hi,
Thanks for explanation Jonathan. I think I will also start working on it.
When you have any patch (even draft) I'd be glad if you can attach it in
JIRA. I'll do the same.
What do you think?
--
Paweł Róg
2014-08-01 20:19 GMT+02:00 Hari Shreedharan :
> +1 on an S3 Source. I would gladly review
Hi,
In a recent thread (
http://search-hadoop.com/m/DEeB4aM4aA&subj=Re+AWS+S3+flume+source ) there
was talk of configuring Flume dynamically.
One option suggested was to just change the config file stored on the FS
because Flume will automatically reload it when it's changed.
But there was also
Eh, for anyone else wondering about this - it looks like there is a 2 years
old issue with a patch at https://issues.apache.org/jira/browse/FLUME-1491
Is this something one of the committers could review and get in the next
Flume release?
Thanks,
Otis
--
Performance Monitoring * Log Analytics * S
Is that a custom Flume build or from some distro ? Hard to say without
additional info. anything interesting in the logs when it crashes ?
On Tue, Jul 29, 2014 at 10:15 AM, Christopher Shannon wrote:
> For development and testing, I sometimes have to run multiple agents on
> the same server /
20 matches
Mail list logo
