[GENERAL] Data partitioning?

2003-10-07 Thread psql-mail
hi, I am becoming more and more convinced that in order to achieve the required performance and scalability I need to split my data amoungst many backend machines. Ideally I would start with about 10 machine and have 1/10th of the data on each. As the data set grows I would then buy additional

[GENERAL] Slow SELECT

2003-10-03 Thread psql-mail
I am running a SELECT to get all tuples within a given date range. This query is much slwoer than i expected - am i missing something? I have a table 'meta' with a column 'in_date' of type timestamp(0), i am trying to select all records within a given date range. I have an index on 'in_date' and

[GENERAL] Tsearch2 Causing Backend Crash

2003-10-03 Thread psql-mail
After applying the patches supplied so far and also trying the lastest stable tar.gz for tsearch2 ( downloaded 24th of september) I am still experiencing the same issue as previously described: I try to do a SELECT to_tsvector( 'default', 'some text' ) The backend crashes. SELECT to_tsvector(

[GENERAL] query distribution

2003-09-19 Thread psql-mail
Is postgresql have an axisting solution for distibuting the databse amoungst several servers, so that each one holds a subset of the data and queries are passed to each one and then collated by a master server? I have heard erServer mentioned but i got the impression this might just be for bac

Re: [GENERAL] postmaster crashing

2003-09-05 Thread psql-mail
Tom Lane writes: > That has nothing whatever to do with how much memory the kernel will let > any one process have. Check what ulimit settings the postmaster is > running under (particularly -d, -m, -v). My ulimit settings you requested look ok (others included for info) ulimit -d, -m, -v : un

Re: [GENERAL] postmaster crashing

2003-09-04 Thread psql-mail
First - apologies for the stuff about "i don't understand why there's only one core file", i now have a post-it note now saying "ulimit gets reset at reboot" (i assume thats what happened). So please find below a potentially more useful core file gdb output: Core was generated by `postgres: mat

Re: [GENERAL] postmaster crashing

2003-09-04 Thread psql-mail
> [EMAIL PROTECTED] writes: > > I have set "ulimit -c unlimited" as you sugested, > > i then copied postmaster to /home/postgres > > and ran it as postgres from there... > > but still no core files. Where should they appear? > > In $PGDATA/base/yourdbnumber/core (under some OSes the file name mi

Re: [GENERAL] postmaster crashing

2003-09-04 Thread psql-mail
> From: Tom Lane <[EMAIL PROTECTED]> > > [EMAIL PROTECTED] writes: > > How do i get the core files to examine? There never seem to be any > > produced, even outside the debuggers. > > Most likely you have launched the postmaster under "ulimit -c 0", which > prevents core dumps. This seems to

[GENERAL] postmaster crashing

2003-09-03 Thread psql-mail
I have been trying to find out more about the postmaster crashing, but things seem to be getting stranger! I am experiencing problems running postmaster in gdb too (see end of message) I will put all the information in this posting for completness, apologies for the duplicated sections. I am r

[GENERAL] Bit String Manipulation

2003-08-18 Thread psql-mail
Hi, I am having problems manipulating bit strings. CREATE TABLE lookup( fname TEXT PRIMARY KEY, digest BIT VARYING ); I am trying to construct another bit string based on the length of the first: SELECT b'1'::bit( bit_length( digest ) ) FROM lookup; This doesn't work as i had hoped, where am I

[GENERAL] FTI problems

2003-08-14 Thread psql-mail
I am trying to use the fti module to search my text. Searching through the raw text using ILIKE takes 3 seconds, searching using fti takes 212 seconds. Then i tried to turn off seq_scan to see what happens, the planner still does a seq_scan. Why does the planner not use the index? Are there any o

Re: [GENERAL] Tsearch2 custom dictionaries

2003-08-14 Thread psql-mail
> On Thu, 7 Aug 2003 [EMAIL PROTECTED] wrote: > > > Part1. > > > > I have created a dictionary called 'webwords' which checks all words > > and curtails them to 300 chars (for now) > > > > after running > > make > > make install > > > > I then copied the lib_webwords.so into my $libdir > > > > I

Re: [GENERAL] tsearch2: Error: Word is too long

2003-08-06 Thread psql-mail
Bad form to reply to my own posting i know but - I notice that the integer dictionary can accept MAXLEN for the longest number that is considered a valid integer. Can i set MAXLEN for the en dictionary to be the longest word i want indexed? I think i'd need to create a new dictionary...? >

[GENERAL] tsearch2 on postgresql 7.3.4

2003-08-06 Thread psql-mail
I am trying to setup tsearch2 on postgresql 7.3.4 on a Redhat9 system, installed from rpms. There seemed to be some files required for installation of tsearch missing so I downloaded the src bundle too. Tsearch2 then compiled ok but now the command: psql mydb < tsearch2.sql fails with a messa

[GENERAL] # Re: Monthly table partitioning for fast purges?

2003-08-05 Thread psql-mail
Below is the EXPLAIN ANALYZE output of a typical current query. I have just begun looking at tsearch2 to index the header and body fields. I have also been using 'atop' to see I/O stats on the disk, i am now pretty sure thats where the current bottleneck is. As soon as a query is launched the

Re: [GENERAL] Monthly table partitioning for fast purges?

2003-08-04 Thread psql-mail
I am looking at ways to speed up queries, the most common way by for queries to be constrianed is by date range. I have indexed the date column. Queries are still slower than i would like. Would there be any performance increase for these types of queries if the tables were split by month as de

Re: [GENERAL] Hardware selection

2003-07-28 Thread psql-mail
Ron thank you for your comments, sorry of the slow response - i actually replied to you on saturday but i think the list was having trouble again?! Your questions are answered below... > On Fri, 2003-07-25 at 07:42, [EMAIL PROTECTED] wrote: > > As mentioned previously I have a large text databa

[GENERAL] Hardware selection

2003-07-25 Thread psql-mail
As mentioned previously I have a large text database with upwards of 40GB of data and 8 million tuples. The time has come to buy some real hardware for it. Having read around the subject online I see the general idea is to get as much memory and the fastest I/O possible. The buget for the serv

Re: [GENERAL] libpq.so.2 problems

2003-07-14 Thread psql-mail
Ok - discovered the solution in pgsql-php, repeated below for reference: From: "Peter De Muer (Work)" <[EMAIL PROTECTED]> To: <[EMAIL PROTECTED]> Subject: Re: 7.3.1 update gives PHP libpq.so.2 problem Date: Tue, 4 Feb 2003 14:06:04 +0100 try making a soft

[GENERAL] libpq.so.2 problems

2003-07-14 Thread psql-mail
Hi, I'm having trouble with libpg.so.2. Specifically: Can't load '/usr/lib/perl5/site_perl/5.8.0/i386-linux-thread-multi/auto/ Pg/Pg.so' for module Pg: libpq.so.2: cannot open shared object file: No such file or directory at /usr/lib/perl5/5.8.0/i386-linux-thread-multi/ DynaLoader.pm line 229. I

[GENERAL] Concatenating two Text fields from the same tuple

2003-07-03 Thread psql-mail
Apologies if this is a repost - I tried sending it yesterday and haven' t seen it in the forum yet. I am currently writing a perl script to convert the string a user supplies to a search engine into SQL. The user supplies a string in the same foramt as google uses - e.g. "cat -dog" finds records