Hi Flume Experts,
I want to copy my local data to hdfs using flume in a single machine which
isrunning hadoop, How can I do that, please help me.
--
Thanks,
Kishore.
Dear Sharminder,
Thanks for your reply, yes I am playing with flume, as you suggested i am
using spool directory source, My configuration file looks like
*tier1.sources = source1
tier1.channels = channel1
tier1.sinks= sink1
tier1.sources.source1.type = spooldir
tier1.sources.source1.spo
what could be the problem, please help me.
On Sun, Jun 15, 2014 at 10:21 PM, Mohit Durgapal
wrote:
> Replace -n agent with -n tier1
>
>
> On Sunday, June 15, 2014, kishore alajangi
> wrote:
>
>> Dear Sharminder,
>>
>> Thanks for your reply, yes I am playing wi
checked, nothing is written in hdfs.
On Mon, Jun 16, 2014 at 10:10 AM, Sharninder wrote:
> That just means source has done its work and is waiting for more data to
> read. Did you check hdfs to see if all data has been written?
>
>
>
> On Mon, Jun 16, 2014 at 11:34 AM
Do I need to create the /flume/messages/ directories?
On Mon, Jun 16, 2014 at 10:14 AM, kishore alajangi <
alajangikish...@gmail.com> wrote:
> checked, nothing is written in hdfs.
>
>
> On Mon, Jun 16, 2014 at 10:10 AM, Sharninder wrote:
>
>> That just means sour
I created the /flume/messages directories, but still nothing is written
with flume in those directories. please help me.
On Mon, Jun 16, 2014 at 10:15 AM, kishore alajangi <
alajangikish...@gmail.com> wrote:
> Do I need to create the /flume/messages/ directories?
>
>
>
> On
Instead just mentioning hdsfs.path = /flume/messages/, do i need to mention
something else?
On Mon, Jun 16, 2014 at 10:25 AM, kishore alajangi <
alajangikish...@gmail.com> wrote:
> I created the /flume/messages directories, but still nothing is written
> with flume in those directo
could anybody help me ?
On Mon, Jun 16, 2014 at 10:27 AM, kishore alajangi <
alajangikish...@gmail.com> wrote:
> Instead just mentioning hdsfs.path = /flume/messages/, do i need to
> mention something else?
>
>
> On Mon, Jun 16, 2014 at 10:25 AM, kishore alajangi <
>
nt
> Bartley Wood Business Park, Hook, Hampshire RG27 9UP
> D 01256 75 3362
>
> I welcome VSRE emails. Learn more at http://vsre.info/
>
>
> --
> *From:* kishore alajangi [mailto:alajangikish...@gmail.com]
> *Sent:* 16 June 2014 08:04
> *T
Hi Flummers,
I copied this code from on-line to do experiments with flume,
a1.sources = r1
a1.sinks = k1 k2
a1.channels = c1
# Describe/configure the source
a1.sources.r1.type = exec
a1.sources.r1.shell = /bin/bash -c
a1.sources.r1.command = cat /Users/david/flume/testdata.txt
a1.sources.r1.in
where the error logs stored? I installed flume 1.5.0, with tar.gz.
On Mon, Jun 16, 2014 at 1:25 PM, Nitin Pawar
wrote:
> can you share full error log ?
>
>
> On Mon, Jun 16, 2014 at 2:49 PM, kishore alajangi <
> alajangikish...@gmail.com> wrote:
>
>> Hi Flummers,
Hi Ashish,
Do i need to use both avrosink and hdfssink to write the text file in avro
format into hdfs?
On Wed, Jun 18, 2014 at 12:13 PM, Ashish wrote:
> For reading -
> http://flume.apache.org/FlumeUserGuide.html#spooling-directory-source
>
> For writing to HDFS -
> http://flume.apache.org/Fl
The flume is writing to hdfs when I start flume manually through config
file like
flume-ng agent -c /etc/flume-ng/conf -f /etc/flume-ng/conf/flume.conf -n
but not start by
service flume-ng-agent start
what could be the reason, the docs says we can start flume either 1st or
second method, why i
Hi flumers,
I deleted all files and directories which flume wrote in hdfs, and used new
spooldirectory source, and why i am facing the error "uncaught exception
spooldirectorysource thread" when i starts flume? when I checked ps -ef |
grep flume*, there are so many flume processes are running, I k
I want to store avro files into hdfs using flume, could anybody suggest me
the configuration? I am trying with the below configuration, but it throws
"flume.EventDeliveryException: Could not find schema for event" error,
please help me
first I copied the schema file to /etc/flume.conf/schemas(crea
Flume 1.4.0-cdh4.5.0
On Wed, Jun 25, 2014 at 7:12 AM, Mike Percy wrote:
> Can you please provide details on the errors you are seeing? What version
> of Flume?
>
>
> On Thu, Jun 19, 2014 at 12:43 AM, kishore alajangi <
> alajangikish...@gmail.com> wrote:
>
>>
t;
> Sent from my iPhone
>
> On Jun 24, 2014, at 10:44 PM, kishore alajangi
> wrote:
>
> Flume 1.4.0-cdh4.5.0
>
>
> On Wed, Jun 25, 2014 at 7:12 AM, Mike Percy wrote:
>
>> Can you please provide details on the errors you are seeing? What version
>> o
Hi Flume Experts,
Could anybody help me to store avro files located in my localfilesystem
into hdfs using flume, frequently new files will be generated so "spooldir"
work for it as source,
--
Thanks,
Kishore.
In document the avro source agent example is given below, where is the
source file to write its content into sink, please help me.
a1.sources = r1a1.channels = c1a1.sources.r1.type =
avroa1.sources.r1.channels = c1a1.sources.r1.bind =
0.0.0.0a1.sources.r1.port = 4141
--
Thanks,
Kishore.
r
On Wed, Jun 25, 2014 at 2:07 PM, Sharninder wrote:
> Did you try using the spooldir source with an hdfs sink? What problems did
> you face?
>
> --
> Sharninder
>
>
>
> On Wed, Jun 25, 2014 at 12:15 PM, kishore alajangi <
> alajangikish...@gmail.com> wrote:
>
the new files are generated, I think if i used "spooldir" as source
for flume agent it looks for new files and write into hdfs sink, but when i
tried with spooldir as a source the above error i am facing, please help me
to solve this problem.
On Thu, Jun 26, 2014 at 10:45 AM, kishor
HI Experts,
My Avro file contains fields year, date and day. how to set the hdfs path
to create year, date and day partitions.Please help me.
--
Thanks,
Kishore.
Hi,
I am trying to build flume-souces-1.0-SNAPSHOT.jar to analyze twitter data
on hue from "https://github.com/cloudera/cdh-twitter-example";
the error I am getting is "[ERROR] Failed to execute goal on project
flume-sources: Could not resolve dependencies for project
com.cloudera:flume-sources:ja
23 matches
Mail list logo
