Re: [PERFORM] optimized counting of web statistics

2005-06-28 Thread Matthew Nuzum
On 6/29/05, Rudi Starcevic <[EMAIL PROTECTED]> wrote: > Hi, > > >I do my batch processing daily using a python script I've written. I > >found that trying to do it with pl/pgsql took more than 24 hours to > >process 24 hours worth of logs. I then used C# and in memory hash > >tables to drop the ti

Re: [PERFORM] optimized counting of web statistics

2005-06-28 Thread Rudi Starcevic
Hi, >I do my batch processing daily using a python script I've written. I >found that trying to do it with pl/pgsql took more than 24 hours to >process 24 hours worth of logs. I then used C# and in memory hash >tables to drop the time to 2 hours, but I couldn't get mono installed >on some of my ol

Re: [PERFORM] optimized counting of web statistics

2005-06-28 Thread Matthew Nuzum
On 6/28/05, Billy extyeightysix <[EMAIL PROTECTED]> wrote: > Hola folks, > > I have a web statistics Pg database (user agent, urls, referrer, etc) > that is part of an online web survey system. All of the data derived > from analyzing web server logs is stored in one large table with each > record

Re: [PERFORM] optimized counting of web statistics

2005-06-28 Thread Billy extyeightysix
> The bottleneck in the > whole process is actually counting each data point (how many times a > url was visited, or how many times a url referred the user to the > website). So more specifically I am wondering if there is way to store > and retrieve the data such that it speeds up the counting of

[PERFORM] optimized counting of web statistics

2005-06-28 Thread Billy extyeightysix
Hola folks, I have a web statistics Pg database (user agent, urls, referrer, etc) that is part of an online web survey system. All of the data derived from analyzing web server logs is stored in one large table with each record representing an analyzed webserver log entry. Currently all reports a