Hi Andras, Sure, to confirm you can use the mongodb plugin same as mysql plugin, ie. instantiate multiple plugins with different names, separate config directives for each, etc.
Cheers, Paolo On Thu, Jul 04, 2013 at 11:08:29AM +0200, Andras Horvai wrote: > Paolo, > > I think this was not clear for me: > Can I use mongodb plugin the same way as mysql plugin? Like > mysql[in],mysql[out]? > So can I create a config like this: > plugins: mongodb[conn], mongodb[in], mongodb[out] > > mongo_table[conn]: acctv5.netflowsender_hist_%Y_%m_%d > mongo_table[in]: acctv5.netflowsender_in_%Y_%m_%d > mongo_table[out]: acctv5.netflowsender_out_%Y_%m_%d > > aggregate[conn]: src_host,dst_host,src_port,dst_port,proto > aggregate[in]: dst_host > aggregate[out]: src_host > > mongo_history: 5m > mongo_time_roundoff: m > mongo_refresh_time:300 > > But it makes sense if it would be possible :) (I have never tried to be > honest). > > Thanks, > Andras > > > > On Thu, Jul 4, 2013 at 1:47 AM, Paolo Lucente <[email protected]> wrote: > > > Hi Andras, > > > > Inline: > > > > On Wed, Jul 03, 2013 at 01:29:38PM +0200, Andras Horvai wrote: > > > > > So in mongodb I have the historical data of connection and in mysql I > > have > > > the data what I can use for accounting (how many bytes a host receives > > and > > > sends). I would like to use one database backands but I did not find > > info > > > how to do this with mongodb only. > > > > What precisely (which behaviour or so) you do manage to achieve with > > MySQL that you cannot with MongoDB? Was reading your config and though > > it was hourly counters (sql_history: 1h) refreshing them every 5 mins > > (sql_refresh_time: 300) which would make sense: we do not support the > > update of records in MongoDB (but you can indeed always make sums). But > > then i spotted in your config a sql_dont_try_update set to true .. so > > i'm a bit puzzled. > > > > > Another thing is that I restart the pmacct everyday at midnight and > > create > > > a separate table (naming with the actual date) in every database. So for > > > every day I have the data in a separate table. > > > > This you can avoid by using dynamic table names, ie. to recall your > > MongoDB table name "acctv5.netflowsender_%Y_%m_%d". With MongoDB this > > is it since it's schemaless. With MySQL you can achieve the same but > > please read docs on how to feed the schema accordingly. Please refer > > to sql_table and sql_table_schema entries of the CONFIG-KEYS doc. > > > > Cheers, > > Paolo > > > > > > > On Tue, Jun 18, 2013 at 6:16 PM, Andras Horvai <[email protected] > > >wrote: > > > > > > > Thanks Edward, > > > > > > > > I will check triggers.sql. > > > > > > > > Andras > > > > > > > > > > > > On Tue, Jun 18, 2013 at 5:04 PM, Edward van Kuik <[email protected] > > >wrote: > > > > > > > >> Hi > > > >> > > > >> What I do is I use triggers which create some summary tables for me > > as it > > > >> inserts. > > > >> I've attached my trigger sql in case you find it helpful. > > > >> > > > >> I insert it into tables with partitioning by range. But the system is > > > >> generally still quite slow. (I currently have 1.3 billion rows) > > > >> I am thinking of inserting the data and summary data into another > > type of > > > >> database, like mongo. > > > >> > > > >> Edward > > > >> > > > >> > > > >> > > > >> > > > >> On 18/06/2013 16:34, Andras Horvai wrote: > > > >> > > > >> Hello, > > > >> > > > >> > > > >> This is what I would like to achieve: > > > >> I would like to have a history of ip connections of my network, let's > > > >> say > > > >> back to 90 days. So to achieve this I understand that I have to > > aggregate > > > >> in > > > >> nfacctd. > > > >> Here is the aggregate what I tried: > > > >> > > > >> aggregate: src_host,dst_host,src_port,dst_port,proto > > > >> > > > >> The problem is when I do this type of aggregate my system slows > > > >> down very much. (I cannot run a simple query in mysql so I cannot > > > >> get the information what I need... so the collection is useless). > > > >> > > > >> I tried to distinguish the in and out traffic with plugins, > > > >> but did not help me regarding the performance. > > > >> The only way what does not slows down my system if I aggregate only > > > >> src or dst addresses (and separate these into different tables). > > > >> But with this only aggregate, I lost the information about the > > src_port, > > > >> dst_port and protocol (per ip pairs) > > > >> > > > >> I would like to monitor the traffic of 400 hosts. > > > >> Is it possible with a server with > > > >> 4 cores (Intel(R) Xeon(R) CPU E5603 @ 1.60GHz) > > > >> and with 4 GB of rams. > > > >> > > > >> Flow exporter is a L3 cisco siwtch in the core of the network. It has > > > >> vlans defined to them. I am interested in the traffic between local > > vlanand local > > > >> vlan, and between local vlan and Internet (or external network). > > > >> > > > >> Thanks, > > > >> > > > >> Andras > > > >> > > > >> > > > >> _______________________________________________ > > > >> pmacct-discussion mailing listhttp://www.pmacct.net/#mailinglists > > > >> > > > >> > > > >> > > > >> -- > > > >> Edward van Kuik > > > >> Nitric Software Laboratory > > > >> 78 Strand Street > > > >> Tel: 021 300 1073 > > > >> Cell: 083 645 6443http://nitric.co.za/ > > > >> > > > >> > > > >> _______________________________________________ > > > >> pmacct-discussion mailing list > > > >> http://www.pmacct.net/#mailinglists > > > >> > > > > > > > > > > > > > _______________________________________________ > > > pmacct-discussion mailing list > > > http://www.pmacct.net/#mailinglists > > > > > > _______________________________________________ > > pmacct-discussion mailing list > > http://www.pmacct.net/#mailinglists > > _______________________________________________ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
