Umm , I use inotifywait with simple sed awk scripting its seems to be a similar idea as far is i understood you :)
But. I will give a try to Flex for sure. Thnx :) On 04/02/2010 06:29 AM, Nikos Balkanas wrote: > Flex (aka lex) is the front end parser to bison (yacc). It uses C, and > it is even used by kannel. It is faster in pattern maching than C since > it uses optimized tables for matching instead of if statements. Patterns > are very similar to regexp. > > If you open the log as a stream, instead of parsing the whole thing at > once, you can parse it real time, one line at a time as it comes in. So > it is both very light and fast. > > Advisory: For the trained professionals only! > > Nikos > ----- Original Message ----- From: "seikath" <seik...@gmail.com> > To: <users@kannel.org> > Sent: Friday, April 02, 2010 2:40 AM > Subject: Re: Too many dlr at once > > >> Ok, now I want to know what is flexing a log-stream ?:) >> >> cheers >> >> >> On 04/01/2010 08:30 PM, Nikos Balkanas wrote: >>> Just to clarify: >>> >>> Lighty is lighttpd, and Seikath's ISS is actually IIS ;-) >>> >>> @Seikath: You have not tried the best of all: flexing a log-stream! >>> Fastest and lighter than all the rest. You can even do all the db >>> pooling you want by batching inserts together in real time (of course it >>> is up to you to code this part) ;-) >>> >>> Nikos >>> ----- Original Message ----- From: "seikath" <seik...@gmail.com> >>> To: <users@kannel.org> >>> Sent: Thursday, April 01, 2010 8:16 PM >>> Subject: Re: Too many dlr at once >>> >>> >>>> In general DLR is not so important info to be injected right away into >>>> the database. >>>> if you have high load of MO/DLR, consider db pooling and even better, >>>> drop the http requests. >>>> The Apache or Lighty or even ISS can handle the traffic you have >>>> mentioned with no issues. >>>> What I do for high load of MO/DLR, is either use sqlbox to handle it, >>>> either simply write directly to simple xml files. >>>> OR, you may parse the kannel logs, which will require some regexp >>>> skills. >>>> I used to implement all of the above, according to the specific >>>> projects. >>>> >>>> The XML files easily can be loaded later in a queue in the database. >>>> >>>> >>>> On 04/01/2010 06:33 PM, Gabor Maros wrote: >>>>> >>>>> Thanks Nikos, >>>>> >>>>> it may help but there is another problem i haven't mentioned before. >>>>> We have >>>>> a webapplication that receives dlrs from kannel. If kannel gets 10k >>>>> dlr in >>>>> one sec then kannel tries to send all of them in the same sec to the >>>>> app. >>>>> This behaviour kills the app (and the database behind it) because it >>>>> gets >>>>> 10000 http connections in one sec which is quite huge amount >>>>> according to >>>>> our peaktime when there is 25 SMs/sec. >>>>> Unfortunately we are not the NASA with unimaginable computing >>>>> capacity, so >>>>> the ideal solution for us would be a parameter that tells kannel how >>>>> many >>>>> connections are allowed in one sec. >>>>> >>>>> Bye, >>>>> Gabor >>>>> >>>>> >>>>> >>>>> Nikos Balkanas wrote: >>>>>> >>>>>> Hi, >>>>>> >>>>>> Check if you havd /etc/hosts, and if you do you should have >>>>>> specified your >>>>>> gateway. >>>>>> >>>>>> Also check if named is running (Linux) >>>>>> >>>>>> BR, >>>>>> Nikos >>>>>> ----- Original Message ----- From: "Gabor Maros" >>>>>> <gabor.ma...@erstebank.hu> >>>>>> To: <users@kannel.org> >>>>>> Sent: Thursday, April 01, 2010 12:58 PM >>>>>> Subject: Too many dlr at once >>>>>> >>>>>> >>>>>> >>>>>> Hi, >>>>>> >>>>>> I've got a kannel install with emi smsc connection. >>>>>> When we send lots of sms to the smsc at once the delivery >>>>>> notifications >>>>>> only >>>>>> come at the end when kannel's queue is empty. Smsc only accepts 10-15 >>>>>> SM/sec >>>>>> but can send back horrible amount at once. This is a problem for us >>>>>> because >>>>>> kannel gets thousands of dlrs in one second and ERROR messages >>>>>> appear in >>>>>> smsbox.log: >>>>>> >>>>>> 2010-04-01 08:21:17 [4834] [4] INFO: Starting delivery report <sms> >>>>>> from >>>>>> <0036303444481> >>>>>> 2010-04-01 08:21:17 [4834] [4] INFO: Starting delivery report <sms> >>>>>> from >>>>>> <0036303444481> >>>>>> 2010-04-01 08:21:17 [4834] [4] INFO: Starting delivery report <sms> >>>>>> from >>>>>> <0036303444481> >>>>>> 2010-04-01 08:21:17 [4834] [4] INFO: Starting delivery report <sms> >>>>>> from >>>>>> <0036303444481> >>>>>> 2010-04-01 08:21:17 [4834] [4] INFO: Starting delivery report <sms> >>>>>> from >>>>>> <0036303444481> >>>>>> >>>>>> …after thousands of such normal logrecords we can see >>>>>> thousands of the >>>>>> following: >>>>>> >>>>>> 2010-04-01 08:21:18 [4834] [9] ERROR: Error while gw_gethostbyname >>>>>> occurs. >>>>>> 2010-04-01 08:21:18 [4834] [9] ERROR: System error 2: No such file or >>>>>> directory >>>>>> 2010-04-01 08:21:18 [4834] [9] ERROR: gethostbyname failed >>>>>> 2010-04-01 08:21:18 [4834] [9] ERROR: error connecting to server >>>>>> `xxxx' at >>>>>> port `yyy' >>>>>> 2010-04-01 08:21:18 [4834] [9] ERROR: Couldn't send request to >>>>>> <https://xyz> >>>>>> 2010-04-01 08:21:18 [4834] [9] ERROR: Error while gw_gethostbyname >>>>>> occurs. >>>>>> 2010-04-01 08:21:18 [4834] [9] ERROR: System error 2: No such file or >>>>>> directory >>>>>> 2010-04-01 08:21:18 [4834] [9] ERROR: gethostbyname failed >>>>>> 2010-04-01 08:21:18 [4834] [9] ERROR: error connecting to server >>>>>> `xxxx' at >>>>>> port `yyy' >>>>>> 2010-04-01 08:21:18 [4834] [9] ERROR: Couldn't send request to >>>>>> <https://xyz> >>>>>> 2010-04-01 08:21:18 [4834] [9] ERROR: Error while gw_gethostbyname >>>>>> occurs. >>>>>> 2010-04-01 08:21:18 [4834] [9] ERROR: System error 2: No such file or >>>>>> directory >>>>>> 2010-04-01 08:21:18 [4834] [9] ERROR: gethostbyname failed >>>>>> 2010-04-01 08:21:18 [4834] [9] ERROR: error connecting to server >>>>>> `xxxx' at >>>>>> port `yyy' >>>>>> 2010-04-01 08:21:18 [4834] [9] ERROR: Couldn't send request to >>>>>> <https://xyz> >>>>>> 2010-04-01 08:21:18 [4834] [9] ERROR: Error while gw_gethostbyname >>>>>> occurs. >>>>>> 2010-04-01 08:21:18 [4834] [9] ERROR: System error 2: No such file or >>>>>> directory >>>>>> 2010-04-01 08:21:18 [4834] [9] ERROR: gethostbyname failed >>>>>> 2010-04-01 08:21:18 [4834] [9] ERROR: error connecting to server >>>>>> `xxxx' at >>>>>> port `yyy' >>>>>> 2010-04-01 08:21:18 [4834] [9] ERROR: Couldn't send request to >>>>>> <https://xyz> >>>>>> 2010-04-01 08:21:18 [4834] [9] ERROR: Error while gw_gethostbyname >>>>>> occurs. >>>>>> 2010-04-01 08:21:18 [4834] [9] ERROR: System error 2: No such file or >>>>>> directory >>>>>> >>>>>> Is there a configuration parameter that change this behavior and we >>>>>> can >>>>>> slow it down? >>>>>> I don’t know why it is happen but there must be some kind of >>>>>> limit (I >>>>>> think >>>>>> it is not an open file issue but something similar). >>>>>> Maybe there is another side effect (but I’m not sure yet) in >>>>>> connection >>>>>> with >>>>>> DLR database because the number of SMs that are not in the end phase >>>>>> (delivered or can’t be delivered) are growing. >>>>>> >>>>>> Thanks, >>>>>> Gabor >>>>>> >>>>>> -- >>>>>> View this message in context: >>>>>> http://old.nabble.com/Too-many-dlr-at-once-tp28106589p28106589.html >>>>>> Sent from the Kannel - User mailing list archive at Nabble.com. >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>> >>>> >>>> >>> >>> >>> >>> >> > >
