Hi,
I am new to flume.
I am doing cat a file using exec source into hdfs.
While running it manually, I am able to see the file transferred
completely. But still flume in is running state.
How do I find when the complete transfer would be done.
Example:
My flume.conf
myAgent.sources.mySource
, I don't think
there's a really good way other than checking the file itself, or
looking to see if the cat command is still running.
Does that help?
Thanks,
Natty
On Thu, Jul 24, 2014 at 2:00 AM, Anandkumar Lakshmanan
mailto:an...@orzota.com>> wrote:
Hi,
I am new to fl
, use the spool directory
source which is much more reliable. If you do want to use the exec
source, no point using cat since that's as good as cp'ing the file the
HDFS, use tail -f rather.
--
Sharninder
On Sat, Jul 26, 2014 at 9:34 AM, Anandkumar Lakshmanan
mailto:an...@orzota.c
Hi,
Config files are independent.
Only the agent name in the file matters.
Anand.
On 09/01/2014 08:06 AM, Blade Liu wrote:
Hi,
I have a simple question about config file. In a distributed log
collection environment, is it required for all hosts to use same
config file? If yes, it indicates
Hi,
You can decide the file size to be stored in HDFS by using the following
properties:
* hdfs.rollInterval ---> Number of seconds to wait before rolling
current file(0 = never roll based on time interval) and Default value is
30 seconds.
* hdfs.rollSize ---> File size to trigger roll, in bytes
Hi,
Try increasing connect-timeout for avro sink. set it to 10ms and
also add request-timeout=10.
Anand.
On 09/18/2014 07:04 AM, centerqi hu wrote:
Hi all
My flume version is 1.5.0
Flume Avro sink failure does not retry.
When the network timeout, Flume Avro sink failed, but he will
Anandkumar Lakshmanan <mailto:an...@orzota.com>>:
Hi,
Try increasing connect-timeout for avro sink. set it to 10ms
and also add request-timeout=10.
Anand.
On 09/18/2014 07:04 AM, centerqi hu wrote:
Hi all
My flume version is 1.5.0
Flume
Hi Ed,
By using cat, Flume will cat the file, but then the source will become
useless, because the command will have finished, and there's no way that
I'm aware of to get an agent to start a new command.
As for figuring out when the transfer is complete, I don't think there's
a really good w
Congrats Roshan.
Thanks.
Anand.
On 11/05/2014 03:42 AM, Arvind Prabhakar wrote:
On behalf of Apache Flume PMC, it is my pleasure to announce that
Roshan Naik has been elected to the Flume Project Management
Committee. Roshan has been active with the project for many years and
has been a commi
Hi,
You can use even hadoop-1.2.0. It is simple one.
Anand.
On 04/17/2015 03:57 PM, mahendran m wrote:
Can anyone tell to which version hadoop can i ingest data using flume ?
Thanks.
Hi Bijoy,
Because of short callTimeout, where the HDFS cluster does not complete
the call in the time for which the HDFS sink in Flume waits for the call
to complete.
So Flume retries the entire transaction, and events that were written as
part of the previous failed transaction, are again wr
Hi Nik,
Please verify the firewall settings. It blocks the connection it seems.
Thanks
Anand.
On 08/07/2015 02:07 AM, Nikhil Gs wrote:
Hello Team,
Facing the below error very alternatively even though worked with
different port numbers. I have pasted my flume config file along with
the err
e spool
directory and it was getting into my Hbase table. But now I don't
see any data in my destination tables. I don't know why. Any
suggestions.
Thanks in advance for your time and reply.
Regards,
Nik.
On Thu, Aug 6, 2015 at 11:43 PM, Anandkumar Lakshmanan
s,
Nik.
On Fri, Aug 7, 2015 at 9:21 AM, Anandkumar Lakshmanan
mailto:an...@orzota.com>> wrote:
Hi,
Flush the firewall rules and start flume.
What OS are you using?
Thanks
Anand.
On 08/07/2015 07:24 PM, Nikhil Gs wrote:
Hello AnandKumar,
The issue is aga
14 matches
Mail list logo
