RE: question using multiplexing and the the same serializer for multiple sinks to multiple hbase tables

[Paul Chavez]

There is a configuration error in your multiplexing channel selector section. 
You are referencing ‘server-agent.sources.avor-Src.’ and it should be 
‘server-agent.sources.mySrc.’. Otherwise, the configuration looks good and 
should satisfy your requirements.

From: terrey shih [mailto:terreys...@gmail.com]
Sent: Friday, August 08, 2014 1:20 PM
To: user@flume.apache.org
Subject: Re: question using multiplexing and the the same serializer for 
multiple sinks to multiple hbase tables

Hi,

One more thing.  I would like to know if one serializer can be used for two 
table.

thanks,
-Gary
server-agent.sources = mySrc
server-agent.sinks = hbase-sink1 hbase-sink2
server-agent.channels = C1 C2

# Describe/configure the source
server-agent.sources.mySrc.type = avro
server-agent.sources.mySrc.bind = 127.0.0.1
server-agent.sources.mySrc.port = 5000

# Use a channel which buffers events in memory
server-agent.channels.C1.type = memory
server-agent.channels.C1.capacity = 1000
server-agent.channels.C1.transactionCapacity = 100
server-agent.sinks.hbase-sink1.type = asynchbase
server-agent.sinks.hbase-sink1.table = table1
server-agent.sinks.hbase-sink1.columnFamily = fam1
server-agent.sinks.hbase-sink1.batchSize = 1000
server-agent.sinks.hbase-sink1.serializer = 
com.test.flume.server.HBaseSinkSerializer
server-agent.sinks.hbase-sink1.channel = C1
server-agent.sinks.hbase-sink1.serializer.columns = col1,col2,col3

server-agent.channels.C2.type = memory
server-agent.channels.C2.capacity = 1000
server-agent.channels.C2.transactionCapacity = 100
server-agent.sinks.hbase-sink2.type = asynchbase
server-agent.sinks.hbase-sink2.table = table2
server-agent.sinks.hbase-sink2.columnFamily = fam2
server-agent.sinks.hbase-sink2.batchSize = 1000
server-agent.sinks.hbase-sink2.serializer = 
com.test.flume.server.HBaseSinkSerializer
server-agent.sinks.hbase-sink2.channel = C2
server-agent.sinks.hbase-sink2.serializer.columns = table2Col

# Bind the source and sink to the channel
server-agent.sources.mySrc.channels = C1 C2
server-agent.sources.avor-Src.selector.type = multiplexing
server-agent.sources.avor-Src.selector.header = DataSrc
server-agent.sources.avor-Src.selector.mapping.ABC = C1
server-agent.sources.avor-Src.selector.mapping.BCD = C2

On Fri, Aug 8, 2014 at 11:19 AM, terrey shih 
<terreys...@gmail.com<mailto:terreys...@gmail.com>> wrote:
HI,

Here is my sample config.



On Fri, Aug 8, 2014 at 10:52 AM, Hari Shreedharan 
<hshreedha...@cloudera.com<mailto:hshreedha...@cloudera.com>> wrote:
Can you please send your config? That would make it easier to understand.


terrey shih wrote:


I have a fanning out operation where I have one source and and based
on the event headers (headers are added from the source input), I
would like to channel the event to different Hbase tables.  I am using
the same serializer for the hbase tables.