Re: [GENERAL] Why does splitting $PGDATA and xlog yield a performance benefit?

2015-08-25 Thread David Kerr
> On Aug 25, 2015, at 10:45 AM, Bill Moran wrote: > > On Tue, 25 Aug 2015 10:08:48 -0700 > David Kerr wrote: > >> Howdy All, >> >> For a very long time I've held the belief that splitting PGDATA and xlog on >> linux systems fairly universally

Re: [GENERAL] Why does splitting $PGDATA and xlog yield a performance benefit?

2015-08-25 Thread David Kerr
On Tue, Aug 25, 2015 at 10:16:37AM PDT, Andomar wrote: > >However, I know from experience that's not entirely true, (although it's not > >always easy to measure all aspects of your I/O bandwith). > > > >Am I missing something? > > > Two things I can think of: > > Transaction writes are entirely s

[GENERAL] Why does splitting $PGDATA and xlog yield a performance benefit?

2015-08-25 Thread David Kerr
Howdy All, For a very long time I've held the belief that splitting PGDATA and xlog on linux systems fairly universally gives a decent performance benefit for many common workloads. (i've seen up to 20% personally). I was under the impression that this had to do with regular fsync()'s from the

Re: [GENERAL] Replication fell out of sync

2015-03-02 Thread David Kerr
On Mon, Mar 02, 2015 at 04:06:02PM PDT, Adrian Klaver wrote: > On 03/02/2015 03:25 PM, David Kerr wrote: > >Howdy, > > > >I had an instance where a replica fell out of sync with the master. > > > >Now it's in in a state where it's unable to catch up bec

Re: [GENERAL] Replication fell out of sync

2015-03-02 Thread David Kerr
On Mon, Mar 02, 2015 at 03:33:22PM PDT, Joshua D. Drake wrote: > > On 03/02/2015 03:25 PM, David Kerr wrote: > > > >Howdy, > > > >I had an instance where a replica fell out of sync with the master. > > > >Now it's in in a state where it's u

[GENERAL] Replication fell out of sync

2015-03-02 Thread David Kerr
Howdy, I had an instance where a replica fell out of sync with the master. Now it's in in a state where it's unable to catch up because the master has already removed the WAL segment. (logs) Mar 2 23:10:13 db13 postgres[11099]: [3-1] user=,db=,host= LOG: streaming replication successfully co

Re: [GENERAL] Monitoring number of backends

2013-10-23 Thread David Kerr
On Wed, Oct 23, 2013 at 12:11:39PM -0500, andy wrote: - On 10/23/2013 11:07 AM, David Kerr wrote: - >On Tue, Oct 22, 2013 at 12:41:58PM -0500, andy wrote: - >- Hi all. - >- - >- My website is about to get a little more popular. I'm trying to add in - >- some measurements t

Re: [GENERAL] Monitoring number of backends

2013-10-23 Thread David Kerr
On Tue, Oct 22, 2013 at 12:41:58PM -0500, andy wrote: - Hi all. - - My website is about to get a little more popular. I'm trying to add in - some measurements to determine an upper limit of how many concurrent - database connections I'm currently using. - - I've started running this: - - SELE

Re: [GENERAL] PostgreSQL vs Mongo

2013-10-17 Thread David Kerr
On Wed, Oct 16, 2013 at 09:30:59AM -0600, CS DBA wrote: - All; - - One of our clients is talking about moving to Mongo for their - reporting/data mart. I suspect the real issue is the architecture of - their data mart schema, however I don't want to start pushing back if I - can't back it up.

Re: [GENERAL] oids on disk not in pg_class

2013-10-07 Thread David Kerr
On Mon, Oct 07, 2013 at 06:32:57PM -0400, Guy Rouillier wrote: - On 10/7/2013 5:58 PM, Steve Atkins wrote: - > - >On Oct 7, 2013, at 2:48 PM, Guy Rouillier - >wrote: - > - >>We have a fairly large (1 TB) database we put on all SSDs because - >>of a very high insert and update rate (38 million rows

Re: [GENERAL] Trouble installing psycopg2

2013-09-26 Thread David Kerr
On Thu, Sep 26, 2013 at 02:56:14PM -0400, Augori wrote: - Thanks, for the replies Chris and David. - - Chris, I couldn't find any psycopg files under my Python installs, so I - decided to try David's advice and yum install python-psycopg2 - It reported success, but it installed it under - ../usr/l

Re: [GENERAL] Trouble installing psycopg2

2013-09-26 Thread David Kerr
On Thu, Sep 26, 2013 at 01:01:54PM -0400, Laura Tateosian wrote: - Hi, I'm trying to install psycopg2 on a Centos 5, 64-bit machine. I have - both 2.4 and 2.7 Python versions on this machine. I attempted to install - using - easy-install2.7 psycopg2 - - The install is not working. (I can't im

Re: [GENERAL] SQL Path in psql

2013-09-06 Thread David Kerr
On Fri, Sep 06, 2013 at 10:45:26AM -0700, David Johnston wrote: - lup wrote - >> - >> - > I wonder if this would at least get the full path on-screen for a c/p - > \! for d in $SQLPATH; do find $d -name - > - > ; done - > - > That said, I would down-vote this suggestion. I tend to put sql

Re: [GENERAL] About postgres scale out

2013-07-17 Thread David Kerr
On Wed, Jul 17, 2013 at 03:10:37PM +0800, Xiang Jun Wu wrote: - Hello, - - I'd like to ask a common question about scale out for postgres. - - Our current data volume is about 500GB ~ 1TB in one pg cluster(postgres 9.2). We've set up master/slave replication to keep sync. - To reach better perf

Re: [GENERAL] Build RPM from Postgres Source

2013-07-15 Thread David Kerr
On Fri, Jul 12, 2013 at 02:37:19PM -0700, ktewari1 wrote: - Hi, - I need to have some different settings(like NAMEDATALEN etc.) and - that's why I'm trying to build postgres from the source and to create an rpm - to be send for install. - - Now, the build works fine but, I don't see a way to

Re: [GENERAL] LONG delete with LOTS of FK's

2013-05-16 Thread David Kerr
On Thu, May 16, 2013 at 06:01:51PM -0500, Larry Rosenman wrote: - On 2013-05-16 17:52, David Kerr wrote: - >On Fri, May 10, 2013 at 11:01:15AM -0500, Larry Rosenman wrote: - >- On 2013-05-10 10:57, Tom Lane wrote: - >- >Larry Rosenman writes: - >- >On 2013-05-10 09:

Re: [GENERAL] LONG delete with LOTS of FK's

2013-05-16 Thread David Kerr
On Fri, May 10, 2013 at 11:01:15AM -0500, Larry Rosenman wrote: - On 2013-05-10 10:57, Tom Lane wrote: - >Larry Rosenman writes: - >On 2013-05-10 09:14, Tom Lane wrote: - >... and verify you get a cheap plan for each referencing table. - > - >We don't :( - > - >Ugh. I bet the problem is that in s

Re: [GENERAL] AWS and postgres issues

2013-04-08 Thread David Kerr
On Apr 8, 2013, at 5:52 PM, Tatsuo Ishii wrote: >> 2013/4/9 Tatsuo Ishii : While debugging this with a coworker we figured out that pg_ctl was attaching to the tty and then it clicked that we needed to be using '-t' where I was using -T or (neither). >>> >>> Are you sure? I chec

Re: [GENERAL] AWS and postgres issues

2013-04-08 Thread David Kerr
On Mon, Apr 08, 2013 at 04:24:45PM -0700, David Kerr wrote: - On Mon, Apr 08, 2013 at 02:59:56PM -0700, David Kerr wrote: - - On Mon, Apr 08, 2013 at 02:09:42PM -0700, David Kerr wrote: - - - On Mon, Apr 08, 2013 at 02:14:14PM -0600, Quentin Hartman wrote: - - - - What version of pgpool are you

Re: [GENERAL] AWS and postgres issues

2013-04-08 Thread David Kerr
On Mon, Apr 08, 2013 at 02:59:56PM -0700, David Kerr wrote: - On Mon, Apr 08, 2013 at 02:09:42PM -0700, David Kerr wrote: - - On Mon, Apr 08, 2013 at 02:14:14PM -0600, Quentin Hartman wrote: - - - What version of pgpool are you using? - - - - - - Are there other commands you have a problem with

Re: [GENERAL] AWS and postgres issues

2013-04-08 Thread David Kerr
On Mon, Apr 08, 2013 at 02:09:42PM -0700, David Kerr wrote: - On Mon, Apr 08, 2013 at 02:14:14PM -0600, Quentin Hartman wrote: - - What version of pgpool are you using? - - - - Are there other commands you have a problem with? I would suspect that the - - restart is causing the postgres server to

Re: [GENERAL] AWS and postgres issues

2013-04-08 Thread David Kerr
On Mon, Apr 08, 2013 at 02:14:14PM -0600, Quentin Hartman wrote: - What version of pgpool are you using? - - Are there other commands you have a problem with? I would suspect that the - restart is causing the postgres server to go away, pgpool decides to - disconnect, and then it has to be manuall

[GENERAL] AWS and postgres issues

2013-04-08 Thread David Kerr
Howdy, I'm having a couple of problems that I believe are related to AWS and I'm wondering if anyone's seen them / overcome them. Brief background, I'm running PG 9.2.4 in a VPC on Amazon Linux. I'm also (attempting) to use PgPool for load balancing/failover. The overall problem is that it seem

Re: [GENERAL] PG9.2.3. Query hanging: SELECT count(*) FROM pg_catalog.pg_class...

2013-02-18 Thread David Kerr
On Sat, Feb 16, 2013 at 10:30:44AM -0800, Kevin Grittner wrote: - David Kerr wrote: - - > Also, if anyone else stumbles upon this, it only seems to happen with jruby. - - > I have standard ruby programs where this does not occur. - - It sounds like it is at least possible that it is the

Re: [GENERAL] PG9.2.3. Query hanging: SELECT count(*) FROM pg_catalog.pg_class...

2013-02-15 Thread David Kerr
appen with jruby. I have standard ruby programs where this does not occur. - On Friday, February 15, 2013 01:58:55 PM David Kerr wrote: - > Howdy! - > - > This query is coming from PgPool I believe. - > - > SELECT count(*) FROM pg_catalog.pg_class AS c WHERE c.relname = -

[GENERAL] PG9.2.3. Query hanging: SELECT count(*) FROM pg_catalog.pg_class...

2013-02-15 Thread David Kerr
Howdy! This query is coming from PgPool I believe. SELECT count(*) FROM pg_catalog.pg_class AS c WHERE c.relname = 'import_jobs' AND c.relpersistence = 'u' This is a very small database, like 10/15 tables, it's basically empty. If i run this query manually, it comes back immediatly. However

Re: [GENERAL] Calculating Replication Lag - units

2012-06-26 Thread David Kerr
On Tue, Jun 26, 2012 at 09:13:44AM -0700, Steve Crawford wrote: - On 06/26/2012 08:16 AM, David Kerr wrote: - >On 06/26/2012 05:11 AM, Stuart Bishop wrote: - >>On Tue, Jun 26, 2012 at 6:21 AM, David Kerr wrote: - >>>On Mon, Jun 25, 2012 at 02:17:22PM -0700, Steve Crawford wrot

Re: [GENERAL] Calculating Replication Lag - units

2012-06-26 Thread David Kerr
On 06/26/2012 05:11 AM, Stuart Bishop wrote: On Tue, Jun 26, 2012 at 6:21 AM, David Kerr wrote: On Mon, Jun 25, 2012 at 02:17:22PM -0700, Steve Crawford wrote: - On 06/25/2012 01:17 PM, David Kerr wrote: ->Howdy, -> ->When calculating Replication lag, I know that we have to co

Re: [GENERAL] Calculating Replication Lag - units

2012-06-26 Thread David Kerr
On 06/26/2012 05:11 AM, Stuart Bishop wrote: On Tue, Jun 26, 2012 at 6:21 AM, David Kerr wrote: On Mon, Jun 25, 2012 at 02:17:22PM -0700, Steve Crawford wrote: - On 06/25/2012 01:17 PM, David Kerr wrote: ->Howdy, -> ->When calculating Replication lag, I know that we have to co

Re: [GENERAL] Calculating Replication Lag - units

2012-06-25 Thread David Kerr
On 6/25/2012 9:55 PM, Raghavendra wrote: On Tue, Jun 26, 2012 at 1:47 AM, David Kerr mailto:d...@mr-paradox.net>> wrote: Howdy, When calculating Replication lag, I know that we have to compare the pg_current_xlog_location to pg_last_xlog_receive_location, etc. but wh

Re: [GENERAL] Calculating Replication Lag - units

2012-06-25 Thread David Kerr
On Mon, Jun 25, 2012 at 02:17:22PM -0700, Steve Crawford wrote: - On 06/25/2012 01:17 PM, David Kerr wrote: - >Howdy, - > - >When calculating Replication lag, I know that we have to compare the - >pg_current_xlog_location - >to pg_last_xlog_receive_location, etc. but what I'm

[GENERAL] Calculating Replication Lag - units

2012-06-25 Thread David Kerr
Howdy, When calculating Replication lag, I know that we have to compare the pg_current_xlog_location to pg_last_xlog_receive_location, etc. but what I'm trying to figure out is what are the units that I'm left with after the calculation. (i.e., does the xlog_location imply some time value?) He

[GENERAL] select current_setting('transaction_isolation')

2012-05-29 Thread David Kerr
Howdy, I recently did a log_min_duration_statement=0 run on my app, and found ~3million copies of "select current_setting('transaction_isolation')" I'm a Java + Hibernate stack. Does anyone know if this is a Hibernate artifact? or a jdbc artifact? or something else (implicit to some query patte

[GENERAL] haproxy / pgpool / rhcs

2012-05-22 Thread David Kerr
Hello I'm implementing HA/failover for my PG nodes. I'm using PG9.0 and async replication and linux. Typical problem - if node 1 fails I want the mirror to become active and take over for the master. The solution should be able to initiate the failover of the standby and start re-directing tr

Re: [GENERAL] postgresql.conf evaluation of duplicate keys

2012-03-21 Thread David Kerr
On 03/21/2012 07:02 AM, Martin Gerdes wrote: I've got a question relating to how the postgres configuration is parsed: If I write into the following into postgresql.conf: shared_buffers = 24MB shared_buffers = 32MB and start up postgres, the command 'show shared_buffers;' answers '32MB'. That

Re: [GENERAL] huge price database question..

2012-03-20 Thread David Kerr
On 03/20/2012 07:26 PM, Jim Green wrote: On 20 March 2012 22:21, David Kerr wrote: I'm imagining that you're loading the raw file into a temporary table that you're going to use to process / slice new data data into your 7000+ actual tables per stock. Thanks! would "slic

Re: [GENERAL] huge price database question..

2012-03-20 Thread David Kerr
On 03/20/2012 07:08 PM, Jim Green wrote: On 20 March 2012 22:03, David Kerr wrote: \copy on 1.2million rows should only take a minute or two, you could make that table "unlogged" as well to speed it up more. If you could truncate / drop / create / load / then index the table each

Re: [GENERAL] huge price database question..

2012-03-20 Thread David Kerr
On 03/20/2012 06:50 PM, Jim Green wrote: On 20 March 2012 21:40, David Kerr wrote: On 03/20/2012 04:27 PM, Jim Green wrote: Greetings list! I am pretty new to postgresql from mysql and did a fairly extensive search of the list and came up with a few good ones but didn't find the exact

Re: [GENERAL] huge price database question..

2012-03-20 Thread David Kerr
On 03/20/2012 04:27 PM, Jim Green wrote: Greetings list! I am pretty new to postgresql from mysql and did a fairly extensive search of the list and came up with a few good ones but didn't find the exact same situation as I have now. so I am venturing asking here. I have daily minute stock price

Re: [GENERAL] : postgres: archiver process failed on 0000000100000F72000000F0

2011-11-11 Thread David Kerr
On Fri, Nov 11, 2011 at 09:58:56PM +0530, Venkat Balaji wrote: - Hello, - - WAL Archive process in our production is not working. - - [postgres@hostname]$ ps -ef | grep archive - postgres 12077 16015 0 10:19 pts/400:00:00 grep archive - postgres 31126 27607 0 Nov10 ?00:01:18 postgre

Re: [GENERAL] Foreign Keys and Deadlocks

2011-11-10 Thread David Kerr
On Thu, Nov 10, 2011 at 09:09:06AM +0100, Csaba Nagy wrote: - Hi David, - - On Wed, 2011-11-09 at 09:52 -0800, David Kerr wrote: - > So, aside from removing the PKs do i have any other options? - - Sure you have: order the inserts by primary key inside each transaction. - Then you will not

Re: [GENERAL] Foreign Keys and Deadlocks

2011-11-09 Thread David Kerr
On Wed, Nov 09, 2011 at 11:11:23AM -0300, Alvaro Herrera wrote: - - Excerpts from David Kerr's message of vie nov 04 13:01:29 -0300 2011: - - > I did more digging and found some good discussions on the subject in general, but - > most of the examples out there contain explicit updates (which is

Re: [GENERAL] Foreign Keys and Deadlocks

2011-11-04 Thread David Kerr
On Thu, Nov 03, 2011 at 03:30:20PM -0700, David Kerr wrote: - Howdy, - - We have a process that's deadlocking frequently. It's basically multiple threads inserting data into a single table. - - That table has FK constraints to 3 other tables. - - I understand how an FK check wi

[GENERAL] Foreign Keys and Deadlocks

2011-11-03 Thread David Kerr
Howdy, We have a process that's deadlocking frequently. It's basically multiple threads inserting data into a single table. That table has FK constraints to 3 other tables. I understand how an FK check will cause a sharelock to be acquired on the reference table and in some instances that lea

[GENERAL] OOM Killer / PG9 / RHEL 6.1

2011-11-02 Thread David Kerr
Howdy, just a quick check, is vm.overcommit_memory = 2 vm.swappiness = 0 Still the way to go with PG9.0 / RHEL 6.1 (64bit) ? I know we gained some control over the OOM Killer in newer kernels and remember reading that maybe postgres could handle it in a different way now. Thanks Dave -- Se

Re: [GENERAL] Server hitting 100% CPU usage, system comes to a crawl.

2011-10-31 Thread David Kerr
On Thu, Oct 27, 2011 at 02:09:51PM -0600, Brian Fehrle wrote: - On 10/27/2011 01:48 PM, Scott Marlowe wrote: - >On Thu, Oct 27, 2011 at 12:39 PM, Brian Fehrle - > wrote: - >>Looking at top, I see no SWAP usage, very little IOWait, and there are a - >>large number of postmaster processes at 100% cp

Re: [GENERAL] What's the impact of archive_command failing?

2011-10-18 Thread David Kerr
On 10/18/2011 09:44 AM, Simon Riggs wrote: On Tue, Oct 18, 2011 at 4:58 PM, David Kerr wrote: I have postgres setup for streaming replication and my slave box went down. My question is, how long can that box stay down before it causes a material impact on the master? The archive_command

[GENERAL] What's the impact of archive_command failing?

2011-10-18 Thread David Kerr
I have postgres setup for streaming replication and my slave box went down. My question is, how long can that box stay down before it causes a material impact on the master? The archive_command that I use will not archive logs while the slave is down. I know the obvious problems: * you're no

Re: [GENERAL] Calculating memory allocaiton per process

2011-04-14 Thread David Kerr
On Thu, Apr 14, 2011 at 03:00:07PM -0400, Jerry Sievers wrote: - David Kerr writes: - - > Howdy, - > - > Is there a doc somewhere that has a formula for how much memory PG - > backend process will use? - > - > I'm looking to get something like total_mem = max_conne

[GENERAL] Calculating memory allocaiton per process

2011-04-14 Thread David Kerr
Howdy, Is there a doc somewhere that has a formula for how much memory PG backend process will use? I'm looking to get something like total_mem = max_connections * ( work_mem + temp_buffers ) // I know it's more complicated than that, which is why I'm asking =) Something similar to Table 17-2

Re: [GENERAL] Reordering a table

2011-02-22 Thread David Kerr
On Tue, Feb 22, 2011 at 04:40:36PM +, Howard Cole wrote: - Hi, - - a puzzle to solve... - - I have a table with a primary key, and a timestamp, e.g. - - idstamp - 1 2011-02-01 10:00 - 2 2011-02-01 09:00 - 3 2011-02-01 11:00 - - Now for reasons too painful to go into, I need

Re: [GENERAL] find column name that has under score (_)

2011-02-17 Thread David Kerr
On Thu, Feb 17, 2011 at 01:55:46PM -0500, akp geek wrote: - Hi all - - - I am trying to write a query to find all the column names in - database that has a underscore in it (_) example souce_id. I know like will - not work , if where column_name like '%_%' Can you please help? - - Regar

Re: [GENERAL] pg_dump: schema with OID 58698 does not exist

2011-02-15 Thread David Kerr
On Fri, Feb 11, 2011 at 03:17:51PM -0500, Tom Lane wrote: - David Kerr writes: - > So i removed the 5 entries from pg_class, but i still get that error - > when trying to pg_dump: - - > pg_dump: schema with OID 58698 does not exist - - > Any other ideas where i could look? - - We

Re: [GENERAL] pg_dump: schema with OID 58698 does not exist

2011-02-11 Thread David Kerr
On 02/09/2011 11:23 AM, David Kerr wrote: On Wed, Feb 09, 2011 at 02:15:06PM -0500, Tom Lane wrote: - David Kerr writes: -> Ok, I found the bad entries, 2 tables a sequence and 2 primary key indexes are associated -> with the wrong (invalid / nonexistant ) schema. - -> However,

Re: [GENERAL] pg_dump: schema with OID 58698 does not exist

2011-02-09 Thread David Kerr
On Wed, Feb 09, 2011 at 02:15:06PM -0500, Tom Lane wrote: - David Kerr writes: - > Ok, I found the bad entries, 2 tables a sequence and 2 primary key indexes are associated - > with the wrong (invalid / nonexistant ) schema. - - > However, there are correct entries for those objects as

Re: [GENERAL] pg_dump: schema with OID 58698 does not exist

2011-02-09 Thread David Kerr
On Wed, Feb 09, 2011 at 09:42:36AM -0800, David Kerr wrote: - On Tue, Feb 08, 2011 at 10:16:02PM -0500, Tom Lane wrote: - - David Kerr writes: - - > I'm getting the above error in one of my dev DBs. - - - - Would you poke around in the system catalogs and find where the dangling - - refe

Re: [GENERAL] pg_dump: schema with OID 58698 does not exist

2011-02-09 Thread David Kerr
On Tue, Feb 08, 2011 at 10:16:02PM -0500, Tom Lane wrote: - David Kerr writes: - > I'm getting the above error in one of my dev DBs. - - Would you poke around in the system catalogs and find where the dangling - reference is located? Have you got any idea of how to reproduce this - fail

[GENERAL] pg_dump: schema with OID 58698 does not exist

2011-02-08 Thread David Kerr
howdy all, I'm getting the above error in one of my dev DBs. I've read in the archives that to stop the error from happening I can just delete entries in pg_type and pg_class, however there seemed to be some community interest in doing some debugging. (mentioned in this thread: http://archiv

Re: [GENERAL] Problems Authenticating against OpenLDAP

2010-12-06 Thread David Kerr
On Mon, Dec 06, 2010 at 07:03:59PM +0100, Rados?aw Smogura wrote: - Try with configuration parameter - conn_max_pending (number of connections waiting for processing thread) - conn_max_auth (same, but for authenticated) ok sounds good, i'll give that a shot! - If you are using anonymous auth then

[GENERAL] Problems Authenticating against OpenLDAP

2010-12-06 Thread David Kerr
I've recently configured Postgres (8.3) to authenticate against OpenLDAP this is my pg_hba.conf entry: host all all 0.0.0.0/0 ldap "ldap://ldapserver/dc=mydomain,dc=com;uid=;,ou=postgresql,dc=mydomain,dc=com"; Things are working fine most of the time. However, every once in a while i'm getting

Re: [GENERAL] pg_hba LDAP Authentication syntax

2010-11-04 Thread David Kerr
On Thu, Nov 04, 2010 at 03:35:11PM -0700, Magnus Hagander wrote: - On Thu, Nov 4, 2010 at 15:30, David Kerr wrote: - > On Thu, Nov 04, 2010 at 02:07:29PM -0700, Magnus Hagander wrote: - > - > - > - > I'm trying to translate that to the old syntax of: - > - >  

Re: [GENERAL] pg_hba LDAP Authentication syntax

2010-11-04 Thread David Kerr
On Thu, Nov 04, 2010 at 02:07:29PM -0700, Magnus Hagander wrote: - > - > I'm trying to translate that to the old syntax of: - >     ldap "ldap://w.x.y.z/ou=postgresql,dc=domain,dc=com;" - > - > basically, i don't know how to fit cn=admin and ldapbindpassword into that string. - - The search+bind

[GENERAL] pg_hba LDAP Authentication syntax

2010-11-04 Thread David Kerr
Howdy, I was hoping someone could help me with ye olde ldap authentication syntax. I'm currently using PG 8.3.9 and an upgrade is not an option. Now, that being said, since i'm very new to LDAP i decided to use PG 9 to experiment with since it looks like it has an easier syntax. So what i've g

Re: [GENERAL] Generate a dynamic sequence within a query

2010-10-21 Thread David Kerr
On Wed, Oct 20, 2010 at 09:35:11PM -0700, Darren Duncan wrote: - Josh Kupershmidt wrote: - >On Wed, Oct 20, 2010 at 6:22 PM, David Kerr wrote: - >>I know I've seen posts on how to do this, but i can't seem to find them. - >> - >>I've got a data

Re: [GENERAL] Generate a dynamic sequence within a query

2010-10-21 Thread David Kerr
On Wed, Oct 20, 2010 at 10:32:15PM -0400, Josh Kupershmidt wrote: - On Wed, Oct 20, 2010 at 6:22 PM, David Kerr wrote: - > I know I've seen posts on how to do this, but i can't seem to find them. - > - > I've got a data set - > - > A, B - > A, C - > A, D - &

Re: [GENERAL] Generate a dynamic sequence within a query

2010-10-20 Thread David Kerr
t 20, 2010 at 3:30 PM, David Kerr wrote: - - > On Wed, Oct 20, 2010 at 11:28:18PM +0100, Raymond O'Donnell wrote: - > - On 20/10/2010 23:22, David Kerr wrote: - > - >I know I've seen posts on how to do this, but i can't seem to find them. - > - > - > - >I&

Re: [GENERAL] Generate a dynamic sequence within a query

2010-10-20 Thread David Kerr
On Wed, Oct 20, 2010 at 03:47:19PM -0700, DM wrote: - select generate_series(1,(select count(*) from tax)), country from tax; - - you should use braces around the sub select. - - Thanks - Deepak - Ah, great, thanks! Dave - On Wed, Oct 20, 2010 at 3:30 PM, David Kerr wrote: - - > On

Re: [GENERAL] Generate a dynamic sequence within a query

2010-10-20 Thread David Kerr
On Wed, Oct 20, 2010 at 11:28:18PM +0100, Raymond O'Donnell wrote: - On 20/10/2010 23:22, David Kerr wrote: - >I know I've seen posts on how to do this, but i can't seem to find them. - > - >I've got a data set - > - >A, B - >A, C - >A, D - >[...] - >

[GENERAL] Generate a dynamic sequence within a query

2010-10-20 Thread David Kerr
I know I've seen posts on how to do this, but i can't seem to find them. I've got a data set A, B A, C A, D [...] and so on and i'd like to be able to wite a query that would result in 1,A,B 2,A,C 3,A,D [...] PG version is 8.3. Any ideas? Thanks Dave -- Sent via pgsql-general mailing lis

Re: [GENERAL] Centralized User Management Tool?

2010-09-29 Thread David Kerr
On Wed, Sep 29, 2010 at 02:23:14PM -0700, Joshua D. Drake wrote: - > Howdy, - > - > Does anyone know of any tools or methods to handle centralized user management within postgres? - > - > I've got about 20 DB servers (and growing) each requiring a different number and level of user access - >

[GENERAL] Centralized User Management Tool?

2010-09-29 Thread David Kerr
Howdy, Does anyone know of any tools or methods to handle centralized user management within postgres? I've got about 20 DB servers (and growing) each requiring a different number and level of user access (think dev, qa, staging, production, etc.) Corporate security guidelines state that all

Re: [GENERAL] Question about Idle in TX

2010-08-06 Thread David Kerr
On Tue, Aug 03, 2010 at 03:57:27PM -0400, Tom Lane wrote: - David Kerr writes: - > On Tue, Aug 03, 2010 at 03:49:57PM -0400, Tom Lane wrote: - > - In recent versions of PG, no. Before about 8.3 it was a Really Bad Idea, - > - because the open transaction would prevent VACUUM from r

Re: [GENERAL] Question about Idle in TX

2010-08-03 Thread David Kerr
On Tue, Aug 03, 2010 at 03:57:27PM -0400, Tom Lane wrote: - David Kerr writes: - > On Tue, Aug 03, 2010 at 03:49:57PM -0400, Tom Lane wrote: - > - In recent versions of PG, no. Before about 8.3 it was a Really Bad Idea, - > - because the open transaction would prevent VACUUM from r

Re: [GENERAL] Question about Idle in TX

2010-08-03 Thread David Kerr
On Tue, Aug 03, 2010 at 03:49:57PM -0400, Tom Lane wrote: - David Kerr writes: - > for example: If a java program connects to the DB and does "begin;" - > and then internally does a "sleep 6 days" - - > Does that cauz any issues other than eating a connection to

Re: [GENERAL] Question about Idle in TX

2010-08-03 Thread David Kerr
On Tue, Aug 03, 2010 at 03:30:46PM -0400, Greg Smith wrote: - David Kerr wrote: - >I know that "Idle in TXs" can interfere with Vaccums for example, but - >I'm not sure if that's due to them usually having some form of lock on a - >table. - > - - Locks ar

[GENERAL] Question about Idle in TX

2010-08-03 Thread David Kerr
I know that Idle in Transactions are a problem, however I'm trying to assess how much of a problem. for example: If a java program connects to the DB and does "begin;" and then internally does a "sleep 6 days" Does that cauz any issues other than eating a connection to the database? (note, not

[GENERAL] constraint/rule/trigger - insert into table X where not in table Y

2010-07-14 Thread David Kerr
Howdy, I'm trying to think of the best way to handle this situation. I've got 2 tables, X and Y Table X has a field foo varchar(20) Table Y has a field bar varchar(20) I want to enforce, that if table X.foo = 'dave' then you can't insert (or update) Y.bar = 'dave' I know this is ideally done

Re: [GENERAL] Uncable to commit: transaction marked for rollback

2010-07-01 Thread David Kerr
On 7/1/2010 11:10 AM, Tom Lane wrote: David Kerr writes: I'm intermittantly getting this error message in a java app. using Geronimo / Hibernate / Postgres 8.3.9 javax.transaction.RollbackException: Unable to commit: transaction marked for rollback You might have better luck asking

[GENERAL] Uncable to commit: transaction marked for rollback

2010-07-01 Thread David Kerr
I'm intermittantly getting this error message in a java app. using Geronimo / Hibernate / Postgres 8.3.9 javax.transaction.RollbackException: Unable to commit: transaction marked for rollback Can someone give me a scenario where this would happen? "unable to commit" makes everyone immediatly go

[GENERAL] UPDATE after Cancle

2010-06-23 Thread David Kerr
Howdy all - I just got this odd behavior in my system. This is PG 8.3.10 on RedHat 5.4 psql bla bla=# update blatab set blafield = replace(blafield,'XXX-1','XXX1-') where created_by = 'blauser'; Cancel request sent UPDATE 8231584 I checked and the updated did happen. autocommit is on, an

Re: [GENERAL] Connection Pooling

2010-04-06 Thread David Kerr
On Mon, Apr 05, 2010 at 09:46:45PM -0600, Scott Marlowe wrote: - On Mon, Apr 5, 2010 at 2:36 PM, David Kerr wrote: - > My app will have over 10k concurrent users. I have huge servers 32 cores (64bit), 64GB ram. RedHat linux. - > - > Those 10k users will all be logging in as one of 5 ap

Re: [GENERAL] Connection Pooling

2010-04-06 Thread David Kerr
On Mon, Apr 05, 2010 at 10:44:53PM -0400, Merlin Moncure wrote: - On Mon, Apr 5, 2010 at 4:36 PM, David Kerr wrote: - > On Sat, Apr 03, 2010 at 09:32:25PM -0400, Merlin Moncure wrote: - > Based on a lot of the comments i've gotten here, I'm starting to think that I've got

Re: [GENERAL] Connection Pooling

2010-04-05 Thread David Kerr
On Sat, Apr 03, 2010 at 09:32:25PM -0400, Merlin Moncure wrote: - On Fri, Mar 26, 2010 at 5:17 PM, David Kerr wrote: - > Howdy all, - > - > I have some apps that are connecting to my DB via direct JDBC and I'd like to pool their connections. - > - > I've been looking at

Re: [GENERAL] Connection Pooling

2010-03-28 Thread David Kerr
On 3/27/2010 12:46 AM, John R Pierce wrote: Allan Kamau wrote: You may also have a look at Commons DBCP from Apache software foundation, "http://commons.apache.org/dbcp/";. I have used it for a few projects and have had no problems. for that matter, JDBC has its own connection pooling in java.

[GENERAL] Connection Pooling

2010-03-26 Thread David Kerr
Howdy all, I have some apps that are connecting to my DB via direct JDBC and I'd like to pool their connections. I've been looking at poolers for a while, and pgbouncer and pgpool-ii seem to be some of the most popular, so i've started with those. I'm setting up pgbouncer, and i've hit a bit

Re: [GENERAL] ERROR: permission denied to finish prepared transaction

2010-02-17 Thread David Kerr
On Tue, Feb 16, 2010 at 12:57:28PM -0800, David Kerr wrote: - I'm seeing a bunch of these error messages: - Feb 16 12:04:51 QA-HC01-DB01 postgres-hc01[26420]: [2-1] user=xy,db=x,pid=26420 ERROR: permission denied to finish prepared transaction - Feb 16 12:04:51 QA-HC01-DB01 postgres-hc01[

[GENERAL] ERROR: permission denied to finish prepared transaction

2010-02-16 Thread David Kerr
I'm seeing a bunch of these error messages: Feb 16 12:04:51 QA-HC01-DB01 postgres-hc01[26420]: [2-1] user=xy,db=x,pid=26420 ERROR: permission denied to finish prepared transaction Feb 16 12:04:51 QA-HC01-DB01 postgres-hc01[26420]: [2-2] user=xy,db=x,pid=26420 HINT: Must be superuser or the user

Re: [GENERAL] Deadlock Detected

2010-02-11 Thread David Kerr
On Thu, Feb 11, 2010 at 05:01:37PM -0500, Vick Khera wrote: - On Thu, Feb 11, 2010 at 4:50 PM, David Kerr wrote: - > currently, i just get "Error: Deadlock Detected" but it doesn't tell me the tables involved. - > - - Where do you see this? The postgres log file su

[GENERAL] Deadlock Detected

2010-02-11 Thread David Kerr
Is there a setting that will give me a more verbose log message when a deadlock is detected? currently, i just get "Error: Deadlock Detected" but it doesn't tell me the tables involved. Thanks Dave -- Sent via pgsql-general mailing list (pgsql-general@postgresql.org) To make changes to your

Re: [GENERAL] vacuumdb ERROR: out of memory

2010-02-09 Thread David Kerr
Magnus Hagander wrote: On Tue, Feb 9, 2010 at 09:53, David Kerr wrote: Guillaume Lelarge wrote: Le 09/02/2010 09:35, David Kerr a écrit : Guillaume Lelarge wrote: Le 09/02/2010 05:49, John R Pierce a écrit : David Kerr wrote: maintenance_work_mem = 1GB So evidently, when it tries to

Re: [GENERAL] viewing large queries in pg_stat_activity

2010-02-09 Thread David Kerr
Magnus Hagander wrote: 2010/2/9 David Kerr : It seems like pg_stat_activity truncates the current_query to about 1024 characters. The field is a text, so i'm wondering if there is a way to see the full query? (I know i can turn on log_statement=all, or log_min_duration_statement) bu

Re: [GENERAL] vacuumdb ERROR: out of memory

2010-02-09 Thread David Kerr
Guillaume Lelarge wrote: Le 09/02/2010 09:35, David Kerr a écrit : Guillaume Lelarge wrote: Le 09/02/2010 05:49, John R Pierce a écrit : David Kerr wrote: maintenance_work_mem = 1GB So evidently, when it tries to actually allocate 1GB, it can't do it. Ergo, that setting is too high for

Re: [GENERAL] vacuumdb ERROR: out of memory

2010-02-09 Thread David Kerr
Guillaume Lelarge wrote: Le 09/02/2010 05:49, John R Pierce a écrit : David Kerr wrote: maintenance_work_mem = 1GB So evidently, when it tries to actually allocate 1GB, it can't do it. Ergo, that setting is too high for your machine. ... seems like i've got 2GB free. is th

Re: [GENERAL] vacuumdb ERROR: out of memory

2010-02-08 Thread David Kerr
Tom Lane wrote: David Kerr writes: Tom Lane wrote: David Kerr writes: I get: vacuumdb: vacuuming of database "assessment" failed: ERROR: out of memory DETAIL: Failed on request of size 1073741820. What have you got maintenance_work_mem set to? maintenance_work_mem = 1GB So

Re: [GENERAL] vacuumdb ERROR: out of memory

2010-02-08 Thread David Kerr
Tom Lane wrote: David Kerr writes: I'm getting error: When I try vacuumdb -z assessment or vacuumdb assessment I get: vacuumdb: vacuuming of database "assessment" failed: ERROR: out of memory DETAIL: Failed on request of size 1073741820. What have you got maintenance_

[GENERAL] viewing large queries in pg_stat_activity

2010-02-08 Thread David Kerr
It seems like pg_stat_activity truncates the current_query to about 1024 characters. The field is a text, so i'm wondering if there is a way to see the full query? (I know i can turn on log_statement=all, or log_min_duration_statement) but i'd like something that doesn't require a restart. Tha

[GENERAL] vacuumdb ERROR: out of memory

2010-02-08 Thread David Kerr
I'm getting error: When I try vacuumdb -z assessment or vacuumdb assessment I get: vacuumdb: vacuuming of database "assessment" failed: ERROR: out of memory DETAIL: Failed on request of size 1073741820. The only way i can actually analyze the DB is if i do a vacuumdb -f The database is curren

Re: [GENERAL] PostgreSQL + Hibernate, Apache Mod Security, SQL Injection and you (a love story)

2010-02-08 Thread David Kerr
On Fri, Feb 05, 2010 at 09:19:40PM +0100, Sebastian Hennebrueder wrote: - John R Pierce schrieb: - >David Kerr wrote: - >>Howdy all, - >> - >>We're using Postgres 8.3 with all of our apps connecting to the database - >>with Hibernate / JPA. - >> - >

Re: [GENERAL] PostgreSQL + Hibernate, Apache Mod Security, SQL Injection and you (a love story)

2010-02-08 Thread David Kerr
On Fri, Feb 05, 2010 at 12:09:57PM -0800, John R Pierce wrote: - that would be a function of how you use Postgresql. if you do the - typical PHP hacker style of building statements with inline values then - executing them, you're vunerable unless you totally sanitize all your - inputs. see

[GENERAL] PostgreSQL + Hibernate, Apache Mod Security, SQL Injection and you (a love story)

2010-02-05 Thread David Kerr
Howdy all, We're using Postgres 8.3 with all of our apps connecting to the database with Hibernate / JPA. Our security team is concerned about SQL Injection attacks, and would like to implement some mod_security rules to protect against it. From what I've read Postgres vanilla is pretty robus

  1   2   >