Re: regular expressions was: Kernel Oops

2011-03-10 Thread Steven Champeon
on Wed, Mar 09, 2011 at 11:00:34AM +1100, Erik de Castro Lopo wrote: > My idea was to autogenerate the complex regexes using > something like this: > > 178.183.237.0.dsl.dynamic.eranet.pl > 183.246.69.111.dynamic.snap.net.nz > 188.146.109.136.nat.umts.dynamic.eranet.pl > > as input.

Re: regular expressions was: Kernel Oops

2011-03-09 Thread Stan Hoeppner
Steve put forth on 3/8/2011 5:12 PM: > Maybe using if/endif conditions like Stan Hoeppner has done on his pcre map > could speedup things even more? -> http://www.hardwarefreak.com/fqrdns.pcre You're giving me too much credit. ;) Again, I'm not the original author of that table. That person cr

Re: regular expressions was: Kernel Oops

2011-03-09 Thread Stan Hoeppner
mouss put forth on 3/8/2011 5:03 PM: > [WARNING: Steven CC'd] > > things. so I'd say, do not consider performances as a primary target. go > for catching spammers first. only tune after you get the irght rules, > and only if needed (I personally don't tune anything here. I'm happy to > focus on c

Re: regular expressions was: Kernel Oops

2011-03-08 Thread Erik de Castro Lopo
Noel Jones wrote: > Many years ago I worked on a system with a 32k limit on pcre > expressions. Ever since then, everything I've checked has > been 64k, and then I gave up checking. I expect any > non-ancient system will support 64k, and some maybe even more. > (To clarify for others follo

Re: regular expressions was: Kernel Oops

2011-03-08 Thread Noel Jones
On 3/8/2011 6:00 PM, Erik de Castro Lopo wrote: Noel Jones wrote: The pattern length limit is controlled by the pcre library you're using. I think most implementations limit single expressions to 64k characters. Obviously something that needs testing. Many years ago I worked on a system wi

Re: regular expressions was: Kernel Oops

2011-03-08 Thread Steven Champeon
on Wed, Mar 09, 2011 at 12:03:27AM +0100, mouss wrote: > [WARNING: Steven CC'd] :-) > Le 08/03/2011 21:29, Stan Hoeppner a écrit : > > That makes me wonder why Enemies List[1] uses complex expressions, > > each one precisely matching a specific rDNS pattern, given EL > > matches 65k+ patterns to

Re: regular expressions was: Kernel Oops

2011-03-08 Thread Erik de Castro Lopo
Noel Jones wrote: > The pattern length limit is controlled by the pcre library > you're using. I think most implementations limit single > expressions to 64k characters. Obviously something that needs testing. > It's unclear to me if a single huge complex expression will > evaluate faster th

Re: regular expressions was: Kernel Oops

2011-03-08 Thread Wietse Venema
mouss: [ Charset ISO-8859-1 unsupported, converting... ] > Le 08/03/2011 23:49, Erik de Castro Lopo a ?crit : > > Wietse Venema wrote: > > > >> If you must match a very large numbers of patterns, you need an > >> implementation that transforms N patterns into one deterministic > >> automaton. This

Re: regular expressions was: Kernel Oops

2011-03-08 Thread Erik de Castro Lopo
Steve wrote: > > If not, it would be possible to convert this (3 only, but could be > > hundreds or even thousands): > > > >/^([0-9]{1,3}\.){4}\.dsl\.dynamic\.eranet\.pl$/ > >/^([0-9]{1,3}\.){4}\.dynamic\.snap\.net\.nz$/ > >/^([0-9]{1,3}\.){4}\.nat\.umts\.dynamic\.eranet\.pl$/ > > >

Re: regular expressions was: Kernel Oops

2011-03-08 Thread mouss
Le 08/03/2011 23:49, Erik de Castro Lopo a écrit : > Wietse Venema wrote: > >> If you must match a very large numbers of patterns, you need an >> implementation that transforms N patterns into one deterministic >> automaton. This can match 1 pattern in the same time as N patterns. >> Once the auto

Re: regular expressions was: Kernel Oops

2011-03-08 Thread Noel Jones
On 3/8/2011 4:49 PM, Erik de Castro Lopo wrote: Wietse Venema wrote: If you must match a very large numbers of patterns, you need an implementation that transforms N patterns into one deterministic automaton. This can match 1 pattern in the same time as N patterns. Once the automaton is built (

Re: regular expressions was: Kernel Oops

2011-03-08 Thread Steve
Original-Nachricht > Datum: Wed, 9 Mar 2011 09:49:21 +1100 > Von: Erik de Castro Lopo > An: postfix-users@postfix.org > Betreff: Re: regular expressions was: Kernel Oops > Wietse Venema wrote: > > > If you must match a very large numbers

Re: regular expressions was: Kernel Oops

2011-03-08 Thread mouss
[WARNING: Steven CC'd] Le 08/03/2011 21:29, Stan Hoeppner a écrit : > Wietse Venema put forth on 3/8/2011 10:39 AM: >> Stan Hoeppner: >>> So, the question is, which form of expression processes the "does not >>> match" case faster? The fully qualified expression, or the simple >>> expression? No

Re: regular expressions was: Kernel Oops

2011-03-08 Thread Erik de Castro Lopo
Wietse Venema wrote: > If you must match a very large numbers of patterns, you need an > implementation that transforms N patterns into one deterministic > automaton. This can match 1 pattern in the same time as N patterns. > Once the automaton is built (which takes some time) it is blindingly > f

Re: regular expressions was: Kernel Oops

2011-03-08 Thread Victor Duchovni
On Tue, Mar 08, 2011 at 02:29:23PM -0600, Stan Hoeppner wrote: > So this would mean the simpler expressions would be faster? That makes > me wonder why Enemies List[1] uses complex expressions, each one > precisely matching a specific rDNS pattern, To avoid false positives by matching in the wro

Re: regular expressions was: Kernel Oops

2011-03-08 Thread Stan Hoeppner
Wietse Venema put forth on 3/8/2011 10:39 AM: > Stan Hoeppner: >> So, the question is, which form of expression processes the "does not >> match" case faster? The fully qualified expression, or the simple >> expression? Noel mentioned that the fully qualified expressions will >> tend to process f

Re: regular expressions was: Kernel Oops

2011-03-08 Thread Wietse Venema
Stan Hoeppner: > So, the question is, which form of expression processes the "does not > match" case faster? The fully qualified expression, or the simple > expression? Noel mentioned that the fully qualified expressions will > tend to process faster. Is this true? Is it true for both the > "ma

Re: regular expressions was: Kernel Oops

2011-03-08 Thread Stan Hoeppner
mouss put forth on 3/7/2011 5:45 PM: > Le 07/03/2011 15:13, Stan Hoeppner a écrit : >> Ok, so if I'm doing what I've heard called a "fully qualified regular >> expression", WRT FQrDNS matching, should I use the anchors or not? >> postmap -q says these all work (the actuals with action and text tha