Re: [GENERAL] Postgres 8.2 Memory Tuning

2009-06-15 Thread CM J
Hi, I need to measure the memory used by postgres under various loads in my application.How to calculate the exact memory used by postgres for OS:Windows as well as linux ? Thanks, CM J On Mon, Jun 15, 2009 at 5:37 PM, Massa, Harald Armin wrote: > CMJ, > > same system.Even under modera

Re: [GENERAL] integer only sposix/regex

2009-06-15 Thread Steve Atkins
On Jun 15, 2009, at 8:21 PM, rodeored wrote: I'm trying to search phone numbers for digits. Unfortunately, the validation has been sloppy and the numbers are not just numbers, they also have dashes and probably other stuff. I would like the search to ignore anything but integers WHERE (a.phone

[GENERAL] integer only sposix/regex

2009-06-15 Thread rodeored
I'm trying to search phone numbers for digits. Unfortunately, the validation has been sloppy and the numbers are not just numbers, they also have dashes and probably other stuff. I would like the search to ignore anything but integers WHERE (a.phone1 ~* '.*626.*' OR a.phone2 ~* '.*626.*' OR a.phon

Re: [GENERAL] Custom Fields Database Architecture

2009-06-15 Thread Sim Zacks
Custom fields are a fact of life, and used in many, many business critical applications. EAV sucks, as you mentioned, but that doesn't take away from the requirement to build that kind of system. >From the user's perspective: If you design an application for me and I want to add a new data field o

Re: [GENERAL] pl/sql resources for pl/pgsql?

2009-06-15 Thread Pavel Stehule
Hello documentation is very good http://www.postgresql.org/docs/8.3/static/plpgsql.html and some articles: http://www.postgres.cz/index.php/PL/pgSQL_%28en%29 regards Pavel Stehule 2009/6/16 gvimrc : > I'm fairly new to PostgreSQL and completely new to using pl/pgsql though > I've used MySQL's p

Re: [GENERAL] Perf differences between timestamp and timestamp with timezone

2009-06-15 Thread Bruce Momjian
Mike Christensen wrote: > Awesome! One more followup question.. > > If I modify an existing table from timestamp to timestamptz, will it use the > current system timezone? If so, how can I modify all the rows to convert to > UTC time (basically add 8 hrs to everything).. I think you just cast i

Re: [GENERAL] Perf differences between timestamp and timestamp with timezone

2009-06-15 Thread Mike Christensen
Awesome! One more followup question.. If I modify an existing table from timestamp to timestamptz, will it use the current system timezone? If so, how can I modify all the rows to convert to UTC time (basically add 8 hrs to everything).. On Mon, Jun 15, 2009 at 6:52 PM, Bruce Momjian wrote: >

Re: [GENERAL] Amazon EC2 | Any recent developments

2009-06-15 Thread Just Someone
Hi, I've seen both - some unknown reason for it to die (mostly related to the underlying hardware having issues). We also see instance failure from time to time with advanced notice. Just like a regular machine dies from time to time, so do cloud instances. I'd say it's bit more common on the clou

Re: [GENERAL] Perf differences between timestamp and timestamp with timezone

2009-06-15 Thread Bruce Momjian
Mike Christensen wrote: > Hi all - > > I'm considering changing all my "timestamp" columns to "timestamp with > timezone" columns instead. The reason is I want to use UTC time for > everything in the DB and on the web server, and only ever convert to local > time on the client itself. I could us

[GENERAL] Perf differences between timestamp and timestamp with timezone

2009-06-15 Thread Mike Christensen
Hi all - I'm considering changing all my "timestamp" columns to "timestamp with timezone" columns instead. The reason is I want to use UTC time for everything in the DB and on the web server, and only ever convert to local time on the client itself. I could use a timestamp and just "know" that t

[GENERAL] pl/sql resources for pl/pgsql?

2009-06-15 Thread gvimrc
I'm fairly new to PostgreSQL and completely new to using pl/pgsql though I've used MySQL's procedural language a little. I heard pl/pgsql is similar to Oracle's pl/sql so would it be possible, given that pl/pgsql literature is a bit thin on the ground, to use books on pl/sql for developing pl/p

Re: [GENERAL] DB Migration 8.4 -> 8.3

2009-06-15 Thread Eoghan Murray
2009/6/15 Glyn Astill : > Try the 8.4 pg_restore against the 8.3 server > Thanks!, that worked! FWIW I set up an ssh connection with the reverse/remote forwarding option -R Also the error message on pg_restore should have read: 8.3: pg_restore: [archiver] unsupported version (1.11) in file h

Re: [GENERAL] DB Migration 8.4 -> 8.3

2009-06-15 Thread Tom Lane
Eoghan Murray writes: > How can I migrate a database from 8.4 to 8.3? The only way is to dump to text (no -Fc or -Ft) with 8.4's pg_dump, and then manually edit the file until 8.3 will take it. >8.3: pg_restore: [archiver] input file does not appear to be a > valid archive It's possible her

Re: [GENERAL] Amazon EC2 | Any recent developments

2009-06-15 Thread David Kerr
On Mon, Jun 15, 2009 at 12:11:54PM -0700, Just Someone wrote: - Hi, - - I have more than a few Postgres instances on EC2. For reliability I - use EBS, and take regular snapshots while also streaming the WAL files - to S3. So far, the few times that my machine died, I had no issue with - getting it

[GENERAL] How can I interpolate psql variables in function bodies?

2009-06-15 Thread J. Greg Davidson
Hi dear colleagues, I'm trying to pull some platform-specific constants out of my code by using psql variables, e.g.: $ psql -v TypeLength=4 # CREATE TYPE tref ( INTERNALLENGTH = :TRefTypeLength, INPUT = tref_in, OUTPUT = tref_out, PASSEDBYVALUE ); which works fine, but when I need such

Re: [GENERAL] DB Migration 8.4 -> 8.3

2009-06-15 Thread Glyn Astill
--- On Mon, 15/6/09, Eoghan Murray wrote: > From: Eoghan Murray > Subject: [GENERAL] DB Migration 8.4 -> 8.3 > To: pgsql-general@postgresql.org > Date: Monday, 15 June, 2009, 10:19 PM > I unintentionally installed 8.4beta2 > on a server (using yum), while I > run 8.3.7 on my dev machine. > Th

[GENERAL] DB Migration 8.4 -> 8.3

2009-06-15 Thread Eoghan Murray
I unintentionally installed 8.4beta2 on a server (using yum), while I run 8.3.7 on my dev machine. The 8.3 version of pg_dump does not work with the server, even with the -i option: 8.3: pg_dump: Error message from server: ERROR: column "reltriggers" does not exist the 8.4 pg_dump works okay, b

Re: [GENERAL] Custom Fields Database Architecture

2009-06-15 Thread David Fetter
On Mon, Jun 15, 2009 at 10:37:04PM +0200, Stefan Keller wrote: > @David: You wrote in the links cited "The "flexibility" stems from > fear of making a design decision.". That's an important note. > Nevertheless, there are use cases where you *can not* know in > advance what the name is of the attr

Re: [GENERAL] 10 TB database

2009-06-15 Thread Brent Wood
Hi Artur, Some general comments: I'd look at partitioning and tablespaces to better manage the files where the data is stored, but also look at some efficiently parallelised disks behind the filesystems. You might also look at optimising the filesystem &OS parameters to increase efficiency as

Re: [GENERAL] Custom Fields Database Architecture

2009-06-15 Thread Stefan Keller
@David: You wrote in the links cited "The "flexibility" stems from fear of making a design decision.". That's an important note. Nevertheless, there are use cases where you *can not* know in advance what the name is of the attribute! To me that's not fear but adaptiveness, modesty and knowing when

Re: [GENERAL] Select ranges based on sequential breaks

2009-06-15 Thread David Wilson
On Mon, Jun 15, 2009 at 2:23 PM, Mike Toews wrote: > Hi, > > I'm having difficulty constructing a query that will find breaks where data > change in a time-series. I've done some searching for this too, but I > haven't found anything. > > Here is my example situation, consider my source table: > da

Re: [GENERAL] interval is ignored

2009-06-15 Thread Tom Lane
rodeored writes: > On Jun 11, 10:14 am, rodeored wrote: >> SELECT INTERVAL '1' MONTH = 00:00:00 > Never mind, > its > SELECT NOW(),NOW()- INTERVAL '1 MONTH' as onemonthago Just for completeness --- the INTERVAL '1' MONTH syntax is supported as of 8.4. regards, tom lane

[GENERAL] interval is ignored

2009-06-15 Thread rodeored
SELECT INTERVAL '1' MONTH = 00:00:00 Therefore now=onemonthago SELECT NOW(),NOW()- INTERVAL '1' MONTH as onemonthago How do I get the timestamp for one month ago? -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your subscription: http://www.postgresql.or

Re: [GENERAL] interval is ignored

2009-06-15 Thread rodeored
On Jun 11, 10:14 am, rodeored wrote: > SELECT INTERVAL '1' MONTH = 00:00:00 >  Therefore now=onemonthago > SELECT NOW(),NOW()- INTERVAL '1' MONTH as onemonthago > > How do I get the timestamp for one month ago? Never mind, its SELECT NOW(),NOW()- INTERVAL '1 MONTH' as onemonthago -- Sent via p

[GENERAL] Select ranges based on sequential breaks

2009-06-15 Thread Mike Toews
Hi, I'm having difficulty constructing a query that will find breaks where data change in a time-series. I've done some searching for this too, but I haven't found anything. Here is my example situation, consider my source table: datebin 2009-01-01 red 2009-01-02 red 2009-01-03

Re: [GENERAL] cygwin and postgresql

2009-06-15 Thread James B. Byrne
On Mon, June 15, 2009 13:02, Scott Marlowe wrote: > > So, for sure something like: > > alter user dbuser with createdb; > > didn't fix the problem? > I have removed the windows installation and can no longer check this. I maintained the pg roles via pgadmin3 and, to the best of my ability to re

Re: [GENERAL] Amazon EC2 | Any recent developments

2009-06-15 Thread Just Someone
Hi, I have more than a few Postgres instances on EC2. For reliability I use EBS, and take regular snapshots while also streaming the WAL files to S3. So far, the few times that my machine died, I had no issue with getting it back from EBS or the EBS volume. I also take tar backups every day, and I

Re: [GENERAL] 10 TB database

2009-06-15 Thread Dann Corbit
> -Original Message- > From: pgsql-general-ow...@postgresql.org [mailto:pgsql-general- > ow...@postgresql.org] On Behalf Of Artur > Sent: Monday, June 15, 2009 5:00 AM > To: pgsql-general@postgresql.org > Subject: [GENERAL] 10 TB database > > Hi! > > We are thinking to create some stocks

Re: [GENERAL] Amazon EC2 | Any recent developments

2009-06-15 Thread David Kerr
On Mon, Jun 15, 2009 at 11:12:32AM -0700, AJAY A wrote: - Hello All, - - I am investigating the possibility of hosting pgsql 8.3 on Amazon EC2 - & implementing a simple HA solution. My search of postgresql & amazon - cloud has produced little result. Just wondering if there has been - any recent

Re: [GENERAL] horizontal sharding

2009-06-15 Thread Scott Marlowe
On Mon, Jun 15, 2009 at 11:36 AM, John R Pierce wrote: > Jim Mlodgenski wrote: >> >> >>    also is it possible to paritition without changing client code >> >> Yes, but it depends on the SQL in your client code. If you are just using >> simple SQL with no stored functions, you should be able to sli

[GENERAL] Amazon EC2 | Any recent developments

2009-06-15 Thread AJAY A
Hello All, I am investigating the possibility of hosting pgsql 8.3 on Amazon EC2 & implementing a simple HA solution. My search of postgresql & amazon cloud has produced little result. Just wondering if there has been any recent development with EBS etc. and anybody would care to share their exp

Re: [GENERAL] 10 TB database

2009-06-15 Thread Alvaro Herrera
Artur wrote: > Hi! > > We are thinking to create some stocks related search engine. > It is experimental project just for fun. > > The problem is that we expect to have more than 250 GB of data every month. > This data would be in two tables. About 50.000.000 new rows every month. Sounds a bit lik

Re: [GENERAL] horizontal sharding

2009-06-15 Thread Jim Mlodgenski
what about queries that need to do joins or aggregate reporting across the partitions?!? I can't see how that could be done transparently short of something like Oracle RAC. GridSQL actually does a nice job of breaking up the query and optimizing it appropriately to handle cross node joins. There

Re: [GENERAL] horizontal sharding

2009-06-15 Thread John R Pierce
Jim Mlodgenski wrote: also is it possible to paritition without changing client code Yes, but it depends on the SQL in your client code. If you are just using simple SQL with no stored functions, you should be able to slip in either solution without changing the client code. what ab

Re: [GENERAL] horizontal sharding

2009-06-15 Thread Jim Mlodgenski
what is a good way to horizontal shard in postgresql > 1. pgpool 2 > 2. gridsql > > which is a better way to use sharding > Both are good methods of sharding, but it depends on your goals. GridSQL is better in reporting applications where as PG Pool2 is better in transactional situations. > > a

Re: [GENERAL] Trigger Function and backup

2009-06-15 Thread Merlin Moncure
On Mon, Jun 15, 2009 at 4:29 AM, Nishkarsh wrote: > Hello every one, > > I am new to databases. I am using Postgres 8.2 (Migrating to 8.3.7 in few > days) on windows platform. > > I had tried using Slony-I for replication and was not able to create a > cluster. > > After struggling for some time i

Re: [GENERAL] cygwin and postgresql

2009-06-15 Thread Scott Marlowe
On Mon, Jun 15, 2009 at 8:22 AM, James B. Byrne wrote: > On: 15 Jun 2009 12:08:22 GMT, Jasen Betts >> >> On Sun, June 14, 2009 15:45, Scott Marlowe wrote: >>> >>> Is there a reason you're not using the native windows postgresql >>> packages? >>> >> >> Because for some reason, processes running in

Re: [GENERAL] String Manipulation

2009-06-15 Thread Christine Penner
Alban, That was exactly what I was looking for. Thanks Christine At 03:45 AM 13/06/2009, you wrote: On Jun 13, 2009, at 12:35 AM, Christine Penner wrote: Sam, The problem with making it a numeric field is that I have seen things like A123, #123a or 23-233. This is only here to make most sor

Re: [GENERAL] Custom Fields Database Architecture

2009-06-15 Thread David Goodenough
On Monday 15 June 2009, Gnanam wrote: > Hi, > > I'm designing a database schema in which I should allow user to create > custom fields at the application level. My application is a web-based > system and it has multiple companies in a single database. So this means > that each company can create

Re: [GENERAL] Trigger Function and backup

2009-06-15 Thread Vick Khera
On Mon, Jun 15, 2009 at 5:50 AM, Havasvölgyi Ottó wrote: > Hi, > > I have found the following strangeness on Windows: > > create table round_test (id int primary key, value double precision); > insert into round_test(id, value) values(1, 1.5); > insert into round_test(id, value) values(2, -1.5); >

Re: [GENERAL] horizontal sharding

2009-06-15 Thread David Fetter
On Mon, Jun 15, 2009 at 12:32:15AM -0700, mobiledream...@gmail.com wrote: > Hey guys > what is a good way to horizontal shard in postgresql "Sharding" is not really a technical term, so it's not really possible to answer this question meaningfully as posed. What is it that you actually want to ac

Re: [GENERAL] Custom Fields Database Architecture

2009-06-15 Thread David Fetter
On Mon, Jun 15, 2009 at 06:04:25AM -0700, Gnanam wrote: > > Hi, > > I'm designing a database schema in which I should allow user to create > custom fields at the application level. This is called EAV (Entity-Attribute-Value), and it's a multi-decade-old mistake. Re-think your design. http://ar

Re: [GENERAL] Rounding incompatibility

2009-06-15 Thread Dave Page
On Mon, Jun 15, 2009 at 3:33 PM, Tom Lane wrote: > =?ISO-8859-1?Q?Havasv=F6lgyi_Ott=F3?= writes: >> I have found the following strangeness on Windows versions: > > Were your two versions built different ways (perhaps with different > compilers)?  This comes down to what the system-supplied rint()

[GENERAL] Custom Fields Database Architecture

2009-06-15 Thread Gnanam
Hi, I'm designing a database schema in which I should allow user to create custom fields at the application level. My application is a web-based system and it has multiple companies in a single database. So this means that each company can create their own custom fields. A custom field creat

[GENERAL] horizontal sharding

2009-06-15 Thread mobiledreamers
Hey guys what is a good way to horizontal shard in postgresql 1. pgpool 2 2. gridsql which is a better way to use sharding also is it possible to paritition without changing client code thanks -- Bidegg worlds best auction site http://bidegg.com

Re: [GENERAL] Rounding incompatibility

2009-06-15 Thread Gurjeet Singh
On Mon, Jun 15, 2009 at 4:28 PM, Havasvölgyi Ottó < havasvolgyi.o...@gmail.com> wrote: > Hi, > > I have found the following strangeness on Windows versions: > > create table round_test (id int primary key, value double precision); > insert into round_test(id, value) values(1, 1.5); > insert into r

Re: [GENERAL] Rounding incompatibility

2009-06-15 Thread Tom Lane
=?ISO-8859-1?Q?Havasv=F6lgyi_Ott=F3?= writes: > I have found the following strangeness on Windows versions: Were your two versions built different ways (perhaps with different compilers)? This comes down to what the system-supplied rint() function does. regards, tom lane

Re: [GENERAL] cygwin and postgresql

2009-06-15 Thread James B. Byrne
On: 15 Jun 2009 12:08:22 GMT, Jasen Betts > > On Sun, June 14, 2009 15:45, Scott Marlowe wrote: >> >> Is there a reason you're not using the native windows postgresql >> packages? >> > > Because for some reason, processes running in the cygwin environment > could not create databases in the postgr

Re: [GENERAL] help with data recovery from injected UPDATE

2009-06-15 Thread Merlin Moncure
On Sun, Jun 14, 2009 at 10:32 AM, Gus Gutoski wrote: > Merlin Moncure wrote: >>> postgresql  8.1 supports pitr archiving.  you can >>> do continuous backups and restore the database to just before the bad >>> data. > > I tried using point-in-time-recovery to restore the state of the > database imme

Re: [GENERAL] 10 TB database

2009-06-15 Thread Whit Armstrong
I have a 300GB database, and I would like to look at partitioning as a possible way to speed it up a bit. I see the partitioning examples from the documentation: http://www.postgresql.org/docs/8.3/static/ddl-partitioning.html Is anyone aware of additional examples or tutorials on partitioning? T

Re: [GENERAL] 10 TB database

2009-06-15 Thread Grzegorz Jaśkiewicz
On Mon, Jun 15, 2009 at 1:00 PM, Artur wrote: > Hi! > > We are thinking to create some stocks related search engine. > It is experimental project just for fun. > > The problem is that we expect to have more than 250 GB of data every month. > This data would be in two tables. About 50.000.000 new ro

[GENERAL] 10 TB database

2009-06-15 Thread Artur
Hi! We are thinking to create some stocks related search engine. It is experimental project just for fun. The problem is that we expect to have more than 250 GB of data every month. This data would be in two tables. About 50.000.000 new rows every month. We want to have access to all the date m

Re: [GENERAL] running pg_dump from python

2009-06-15 Thread Jasen Betts
On 2009-06-14, Garry Saddington wrote: > I ahve the following python file that I am running as an external method > in Zope. > > def backup(): > import os > os.popen("c:/scholarpack/postgres/bin/pg_dump scholarpack > > c:/scholarpack/ancillary/scholarpack.sql") are you sure you're usi

Re: [GENERAL] cygwin and postgresql

2009-06-15 Thread Jasen Betts
On 2009-06-14, James B. Byrne wrote: > > On Sun, June 14, 2009 15:45, Scott Marlowe wrote: > why it would not form part of the cygwin >>> installation? >> >> Is there a reason you're not using the native windows postgresql >> packages? >> > > Because for some reason, processes running in the cygwi

Re: [GENERAL] Postgres 8.2 Memory Tuning

2009-06-15 Thread Massa, Harald Armin
CMJ, same system.Even under moderate load in my application, each of the postgres > process occupies some 25 MB memory leading to total postgres memory usage of > 500+ MB. Now, how do i limit the memory used by each how did you measure those 25MB and those 500+MB MEMORY usage? I guess you are

[GENERAL] Postgres 8.2 Memory Tuning

2009-06-15 Thread CM J
Hi , I have installed Postgres 8.2 in windows with default configuration.The windows machine has dual processor(2 GHZ,1GHZ) with 1 GB RAM.I use my java application with postgres and my database connection pool has some 25 connections.I understand from the documentation that postgres start

[GENERAL] Rounding incompatibility

2009-06-15 Thread Havasvölgyi Ottó
Hi, I have found the following strangeness on Windows versions: create table round_test (id int primary key, value double precision); insert into round_test(id, value) values(1, 1.5); insert into round_test(id, value) values(2, -1.5); insert into round_test(id, value) values(3, 3.5); select round

Re: [GENERAL] Trigger Function and backup

2009-06-15 Thread Havasvölgyi Ottó
Hi, I have found the following strangeness on Windows: create table round_test (id int primary key, value double precision); insert into round_test(id, value) values(1, 1.5); insert into round_test(id, value) values(2, -1.5); insert into round_test(id, value) values(3, 3.5); select round(value) f

[GENERAL] Trigger Function and backup

2009-06-15 Thread Nishkarsh
Hello every one, I am new to databases. I am using Postgres 8.2 (Migrating to 8.3.7 in few days) on windows platform. I had tried using Slony-I for replication and was not able to create a cluster. After struggling for some time i decide to implement a way around to take differential backup. A

Re: [GENERAL] WITH RECURSIVE clause -- all full and partial paths

2009-06-15 Thread aryoo
Dear David and Harald, Thanks both for your help. Good day. Aryé.

Re: [GENERAL] Postgres connectivity problem.

2009-06-15 Thread John R Pierce
Bhujbal, Santosh wrote: Hi All, Postgres ‘psql’ client is getting hang in ‘stat’ call while connecting to postgres server. Pstack output: 11017: ./bin/psql -U postgres configdb stat (ff3f5640, ffbff578) My machine details: SunOS my_machine 5.10 Generic_120011-14 sun4u sparc SUNW

[GENERAL] Postgres connectivity problem.

2009-06-15 Thread Bhujbal, Santosh
Hi All, Postgres 'psql' client is getting hang in 'stat' call while connecting to postgres server. Pstack output: 11017: ./bin/psql -U postgres configdb stat (ff3f5640, ffbff578) My machine details: SunOS my_machine 5.10 Generic_120011-14 sun4u sparc SUNW,Sun-Fire-V245