Paolo, I think this was not clear for me: Can I use mongodb plugin the same way as mysql plugin? Like mysql[in],mysql[out]? So can I create a config like this: plugins: mongodb[conn], mongodb[in], mongodb[out]
mongo_table[conn]: acctv5.netflowsender_hist_%Y_%m_%d mongo_table[in]: acctv5.netflowsender_in_%Y_%m_%d mongo_table[out]: acctv5.netflowsender_out_%Y_%m_%d aggregate[conn]: src_host,dst_host,src_port,dst_port,proto aggregate[in]: dst_host aggregate[out]: src_host mongo_history: 5m mongo_time_roundoff: m mongo_refresh_time:300 But it makes sense if it would be possible :) (I have never tried to be honest). Thanks, Andras On Thu, Jul 4, 2013 at 1:47 AM, Paolo Lucente <[email protected]> wrote: > Hi Andras, > > Inline: > > On Wed, Jul 03, 2013 at 01:29:38PM +0200, Andras Horvai wrote: > > > So in mongodb I have the historical data of connection and in mysql I > have > > the data what I can use for accounting (how many bytes a host receives > and > > sends). I would like to use one database backands but I did not find > info > > how to do this with mongodb only. > > What precisely (which behaviour or so) you do manage to achieve with > MySQL that you cannot with MongoDB? Was reading your config and though > it was hourly counters (sql_history: 1h) refreshing them every 5 mins > (sql_refresh_time: 300) which would make sense: we do not support the > update of records in MongoDB (but you can indeed always make sums). But > then i spotted in your config a sql_dont_try_update set to true .. so > i'm a bit puzzled. > > > Another thing is that I restart the pmacct everyday at midnight and > create > > a separate table (naming with the actual date) in every database. So for > > every day I have the data in a separate table. > > This you can avoid by using dynamic table names, ie. to recall your > MongoDB table name "acctv5.netflowsender_%Y_%m_%d". With MongoDB this > is it since it's schemaless. With MySQL you can achieve the same but > please read docs on how to feed the schema accordingly. Please refer > to sql_table and sql_table_schema entries of the CONFIG-KEYS doc. > > Cheers, > Paolo > > > > On Tue, Jun 18, 2013 at 6:16 PM, Andras Horvai <[email protected] > >wrote: > > > > > Thanks Edward, > > > > > > I will check triggers.sql. > > > > > > Andras > > > > > > > > > On Tue, Jun 18, 2013 at 5:04 PM, Edward van Kuik <[email protected] > >wrote: > > > > > >> Hi > > >> > > >> What I do is I use triggers which create some summary tables for me > as it > > >> inserts. > > >> I've attached my trigger sql in case you find it helpful. > > >> > > >> I insert it into tables with partitioning by range. But the system is > > >> generally still quite slow. (I currently have 1.3 billion rows) > > >> I am thinking of inserting the data and summary data into another > type of > > >> database, like mongo. > > >> > > >> Edward > > >> > > >> > > >> > > >> > > >> On 18/06/2013 16:34, Andras Horvai wrote: > > >> > > >> Hello, > > >> > > >> > > >> This is what I would like to achieve: > > >> I would like to have a history of ip connections of my network, let's > > >> say > > >> back to 90 days. So to achieve this I understand that I have to > aggregate > > >> in > > >> nfacctd. > > >> Here is the aggregate what I tried: > > >> > > >> aggregate: src_host,dst_host,src_port,dst_port,proto > > >> > > >> The problem is when I do this type of aggregate my system slows > > >> down very much. (I cannot run a simple query in mysql so I cannot > > >> get the information what I need... so the collection is useless). > > >> > > >> I tried to distinguish the in and out traffic with plugins, > > >> but did not help me regarding the performance. > > >> The only way what does not slows down my system if I aggregate only > > >> src or dst addresses (and separate these into different tables). > > >> But with this only aggregate, I lost the information about the > src_port, > > >> dst_port and protocol (per ip pairs) > > >> > > >> I would like to monitor the traffic of 400 hosts. > > >> Is it possible with a server with > > >> 4 cores (Intel(R) Xeon(R) CPU E5603 @ 1.60GHz) > > >> and with 4 GB of rams. > > >> > > >> Flow exporter is a L3 cisco siwtch in the core of the network. It has > > >> vlans defined to them. I am interested in the traffic between local > vlanand local > > >> vlan, and between local vlan and Internet (or external network). > > >> > > >> Thanks, > > >> > > >> Andras > > >> > > >> > > >> _______________________________________________ > > >> pmacct-discussion mailing listhttp://www.pmacct.net/#mailinglists > > >> > > >> > > >> > > >> -- > > >> Edward van Kuik > > >> Nitric Software Laboratory > > >> 78 Strand Street > > >> Tel: 021 300 1073 > > >> Cell: 083 645 6443http://nitric.co.za/ > > >> > > >> > > >> _______________________________________________ > > >> pmacct-discussion mailing list > > >> http://www.pmacct.net/#mailinglists > > >> > > > > > > > > > _______________________________________________ > > pmacct-discussion mailing list > > http://www.pmacct.net/#mailinglists > > > _______________________________________________ > pmacct-discussion mailing list > http://www.pmacct.net/#mailinglists >
_______________________________________________ pmacct-discussion mailing list http://www.pmacct.net/#mailinglists
