[GENERAL] Tuning read ahead continued...

2013-05-16 Thread Ramsey Gurley
Hi All, I tried bumping my read ahead up to 4096. Instead of having faster reads, it seems it actually slowed things down. In fact, most of the tuning suggestions I've tried have made little to no difference in the results I get from bonnie++. I'll include a table of values in html. I'm wonderi

Re: [GENERAL] LONG delete with LOTS of FK's

2013-05-16 Thread David Kerr
On Thu, May 16, 2013 at 06:01:51PM -0500, Larry Rosenman wrote: - On 2013-05-16 17:52, David Kerr wrote: - >On Fri, May 10, 2013 at 11:01:15AM -0500, Larry Rosenman wrote: - >- On 2013-05-10 10:57, Tom Lane wrote: - >- >Larry Rosenman writes: - >- >On 2013-05-10 09:14, Tom Lane wrote: - >- >... an

Re: [GENERAL] LONG delete with LOTS of FK's

2013-05-16 Thread Larry Rosenman
On 2013-05-16 17:52, David Kerr wrote: On Fri, May 10, 2013 at 11:01:15AM -0500, Larry Rosenman wrote: - On 2013-05-10 10:57, Tom Lane wrote: - >Larry Rosenman writes: - >On 2013-05-10 09:14, Tom Lane wrote: - >... and verify you get a cheap plan for each referencing table. - > - >We don't :( -

Re: [GENERAL] [HACKERS] PLJava for Postgres 9.2.

2013-05-16 Thread Andrew Dunstan
On 05/16/2013 05:59 PM, Paul Hammond wrote: Hi all, I've downloaded PLJava, the latest version, which doesn't seem to have a binary distribution at all for 9.2, so I'm trying to build it from the source for Postgres 9.2. I have the DB itself installed on Windows 7 64 bit as a binary install.

Re: [GENERAL] LONG delete with LOTS of FK's

2013-05-16 Thread David Kerr
On Fri, May 10, 2013 at 11:01:15AM -0500, Larry Rosenman wrote: - On 2013-05-10 10:57, Tom Lane wrote: - >Larry Rosenman writes: - >On 2013-05-10 09:14, Tom Lane wrote: - >... and verify you get a cheap plan for each referencing table. - > - >We don't :( - > - >Ugh. I bet the problem is that in s

Re: [GENERAL] pg_upgrade link mode

2013-05-16 Thread Fabio Rueda Carrascosa
As usual, you are totally right Tom. But thanks everybody for the debate. Only I have to add, is near a "must" to run with --check before actually run the command. 2013/5/16 Tom Lane > Fabio Rueda Carrascosa writes: > > Hello, I have a 9.1 cluster with 50 databases, only one table per db wi

Re: [GENERAL] PLJava for Postgres 9.2.

2013-05-16 Thread John R Pierce
On 5/16/2013 2:59 PM, Paul Hammond wrote: I've downloaded PLJava, the latest version, which doesn't seem to have a binary distribution at all for 9.2, so I'm trying to build it from the source for Postgres 9.2. I have the DB itself installed on Windows 7 64 bit as a binary install. I've had to

[GENERAL] PLJava for Postgres 9.2.

2013-05-16 Thread Paul Hammond
Hi all, I've downloaded PLJava, the latest version, which doesn't seem to have a binary distribution at all for 9.2, so I'm trying to build it from the source for Postgres 9.2. I have the DB itself installed on Windows 7 64 bit as a binary install. I've had to do a fair bit of hacking with the

Re: [GENERAL] problem with lost connection while running long PL/R query

2013-05-16 Thread Joe Conway
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 05/16/2013 08:40 AM, Tom Lane wrote: > "David M. Kaplan" writes: >> Thanks for the help. You have definitely identified the problem, >> but I am still looking for a solution that works for me. I tried >> setting vm.overcommit_memory=2, but this j

Re: [GENERAL] Running out of memory at vacuum

2013-05-16 Thread Tony Dare
On 05/16/2013 07:13 AM, Ioana Danes wrote: Hi Jeff, Yes stop/start of the application server does close all the connections to the database. Lately I did restart postgres too everytime that happened. It did happen in the past, last year sometime when I tried just to close the app and it was n

Re: [GENERAL] Regarding Postgres Plus Associate Certification

2013-05-16 Thread Zach Seaman
I would also like to know this as well. On Thu, May 16, 2013 at 2:46 PM, Oscar Calderon < ocalde...@solucionesaplicativas.com> wrote: > Hi to everybody, i just wanna ask you if somebody that already has the > certification or if is studying to get it knows if there's a book (or a set > of books)

[GENERAL] Regarding Postgres Plus Associate Certification

2013-05-16 Thread Oscar Calderon
Hi to everybody, i just wanna ask you if somebody that already has the certification or if is studying to get it knows if there's a book (or a set of books) that you recommend me that contains most of the necessary information to cover the topics of the certification, that are the next: - Postg

Re: [GENERAL] pg_upgrade link mode

2013-05-16 Thread Tom Lane
Fabio Rueda Carrascosa writes: > Hello, I have a 9.1 cluster with 50 databases, only one table per db with > 2000 rows only, but a lot of schema around each one (postgis databases) > The old cluster size is 1GB > du -chs /var/lib/postgresql/9.1/main/ > 1.1G > now I run a pg_upgrade to 9.2 with

Re: [GENERAL] pg_upgrade link mode

2013-05-16 Thread Igor Neyman
> -Original Message- > From: pgsql-general-ow...@postgresql.org [mailto:pgsql-general- > ow...@postgresql.org] On Behalf Of Lonni J Friedman > Sent: Thursday, May 16, 2013 2:23 PM > To: Igor Neyman > Cc: AI Rumman; Fabio Rueda Carrascosa; pgsql-general > Subject: Re: [GENERAL] pg_upgrade li

Re: [GENERAL] upsert functionality

2013-05-16 Thread Sajeev Mayandi
Thank you for true response will try out. Sajeev On 5/16/13 10:27 AM, "Steven Schlansker" wrote: > >On May 15, 2013, at 11:52 PM, Thomas Kellerer wrote: > >> Sajeev Mayandi, 16.05.2013 07:01: >>> Hi, >>> >>> Our company is planning to move to postreSQL. We were initially using >>> sybase wher

Re: [GENERAL] pg_upgrade link mode

2013-05-16 Thread AI Rumman
Yes Lonni. I agree with you. On Thu, May 16, 2013 at 2:23 PM, Lonni J Friedman wrote: > On Thu, May 16, 2013 at 11:03 AM, Igor Neyman > wrote: > > > > From: pgsql-general-ow...@postgresql.org [mailto: > pgsql-general-ow...@postgresql.org] On Behalf Of AI Rumman > > Sent: Thursday, May 16, 2013

Re: [GENERAL] pg_upgrade link mode

2013-05-16 Thread Lonni J Friedman
On Thu, May 16, 2013 at 11:03 AM, Igor Neyman wrote: > > From: pgsql-general-ow...@postgresql.org > [mailto:pgsql-general-ow...@postgresql.org] On Behalf Of AI Rumman > Sent: Thursday, May 16, 2013 1:56 PM > To: Fabio Rueda Carrascosa > Cc: pgsql-general > Subject: Re: [GENERAL] pg_upgrade link m

Re: [GENERAL] pg_upgrade link mode

2013-05-16 Thread AI Rumman
I told you I would never go for a production with link mode. However, I was working with pg_upgrade copy mode and in the middle I got an error saying missing some extensions. I rollback and start the operation after setting everything up. I don't know how it will behave in link mode if you fail in

[GENERAL] About replacing PostgreSQL instance

2013-05-16 Thread Oscar Calderon
Hi to all, i wanna ask you a piece of advice. The company where i work is bringing maintenance service of PostgreSQL to another company, and currently they have installed PostgreSQL 9.1.1, and they want to move to 9.3 version when it will come out. So, because the difference of versions, and becaus

Re: [GENERAL] pg_upgrade link mode

2013-05-16 Thread Igor Neyman
From: pgsql-general-ow...@postgresql.org [mailto:pgsql-general-ow...@postgresql.org] On Behalf Of AI Rumman Sent: Thursday, May 16, 2013 1:56 PM To: Fabio Rueda Carrascosa Cc: pgsql-general Subject: Re: [GENERAL] pg_upgrade link mode I always think its a bit risky to use link instead of copying.

Re: [GENERAL] pg_upgrade link mode

2013-05-16 Thread AI Rumman
I always think its a bit risky to use link instead of copying. However, I'd suggest to try the --check at first of pg_upgrade. On Thu, May 16, 2013 at 1:42 PM, Fabio Rueda Carrascosa < avances...@gmail.com> wrote: > Hello, I have a 9.1 cluster with 50 databases, only one table per db with > 200

Re: [GENERAL] 9.3 beta and materialized views

2013-05-16 Thread Kevin Grittner
Thomas Kellerer wrote: > Tom Lane wrote on 16.05.2013 19:36: >>> As the materialized view should be a "table" that can be >>> selected from, I wonder what the purpose of the rewrite rule >>> is? >> >> To store the matview's definition for use in REFRESH. >> > Ah, right. Makes sense. It will also

Re: [GENERAL] 9.3 beta and materialized views

2013-05-16 Thread Thomas Kellerer
Tom Lane wrote on 16.05.2013 19:36: As the materialized view should be a "table" that can be selected from, I wonder what the purpose of the rewrite rule is? To store the matview's definition for use in REFRESH. Ah, right. Makes sense. Thanks for the quick reply. -- Sent via pgsql-gene

Re: [GENERAL] WAL contains references to invalid pages

2013-05-16 Thread JotaComm
Hello, Fabrízio 2013/5/16 Fabrízio de Royes Mello > > On Thu, May 16, 2013 at 11:12 AM, JotaComm wrote: > >> >> [...] >> >> Yesterday I identified the following messages in my log file (slave): >> >> user=,db= WARNING: page 6629 of relation base/20449/24818 is >> uninitialized >> user=,db= CO

[GENERAL] pg_upgrade link mode

2013-05-16 Thread Fabio Rueda Carrascosa
Hello, I have a 9.1 cluster with 50 databases, only one table per db with 2000 rows only, but a lot of schema around each one (postgis databases) The old cluster size is 1GB du -chs /var/lib/postgresql/9.1/main/ 1.1G now I run a pg_upgrade to 9.2 with hard link mode, pg_upgrade --link \ --

Re: [GENERAL] 9.3 beta and materialized views

2013-05-16 Thread Tom Lane
Thomas Kellerer writes: > I'm currently adding the support for materialized views to my SQL tool, and > noticed that when creating a materialized view, a rewrite rule is also > created that looks just like a rewrite rule for a "normal" view. Yup. > As the materialized view should be a "table"

Re: [GENERAL] upsert functionality

2013-05-16 Thread Steven Schlansker
On May 15, 2013, at 11:52 PM, Thomas Kellerer wrote: > Sajeev Mayandi, 16.05.2013 07:01: >> Hi, >> >> Our company is planning to move to postreSQL. We were initially using >> sybase where upsert functionality was available using "insert on >> existing update" clause. I know there multiple ways

[GENERAL] 9.3 beta and materialized views

2013-05-16 Thread Thomas Kellerer
Hi, I'm currently adding the support for materialized views to my SQL tool, and noticed that when creating a materialized view, a rewrite rule is also created that looks just like a rewrite rule for a "normal" view. Using pg_get_ruledef() I see that the rule is defined like this: REATE RULE "

Re: [GENERAL] DELETE or TRUNCATE?

2013-05-16 Thread chiru r
Hi, Yes,DELETE would be better this case. The TRUNCATE operation required AccessExclusiveLock on Table before perform TRUNCATE operation. So,if you the table size is bing,it is batter to do ANALYZE after report and VACUUM non-peak(less business) hours. Regards, Chiru On Thu, May 16, 2013 at

Re: [GENERAL] Tuning read ahead

2013-05-16 Thread Ramsey Gurley
On May 16, 2013, at 6:01 AM, Shaun Thomas wrote: > On 05/15/2013 08:04 PM, Ramsey Gurley wrote: > >> My question: Is that advice just for the database drive, or should I >> increase read ahead on the OS/WAL disk as well? > > Definitely the database drive, but it doesn't hurt to do both. It does

Re: [GENERAL] problem with lost connection while running long PL/R query

2013-05-16 Thread Tom Lane
"David M. Kaplan" writes: > Thanks for the help. You have definitely identified the problem, but I > am still looking for a solution that works for me. I tried setting > vm.overcommit_memory=2, but this just made the query crash quicker than > before, though without killing the entire connect

Re: [GENERAL] Deploying PostgreSQL on CentOS with SSD and Hardware RAID

2013-05-16 Thread Matt Brock
So a week after asking our HP dealer, they've finally replied to say that they can't tell us what manufacturer and model the SSDs are because "HP treat this information as company confidential". Not particularly helpful. They have at least confirmed that the drives have "surprise power loss pro

Re: [GENERAL] WAL contains references to invalid pages

2013-05-16 Thread Fabrízio de Royes Mello
On Thu, May 16, 2013 at 11:12 AM, JotaComm wrote: > > [...] > Yesterday I identified the following messages in my log file (slave): > > user=,db= WARNING: page 6629 of relation base/20449/24818 is uninitialized > user=,db= CONTEXT: xlog redo vacuum: rel 1663/20449/24818; blk 6631, > lastBlockVa

Re: [GENERAL] problem with lost connection while running long PL/R query

2013-05-16 Thread David M. Kaplan
Hi, Thanks for the help. You have definitely identified the problem, but I am still looking for a solution that works for me. I tried setting vm.overcommit_memory=2, but this just made the query crash quicker than before, though without killing the entire connection to the database. I imag

Re: [GENERAL] FATAL: database "a/system_data" does not exist

2013-05-16 Thread Adrian Klaver
On 05/16/2013 05:31 AM, sumita wrote: I am running greps to find out what could be triggering it. in the scripts and programs. My only other concern is the same version of my product works without this log statements in the Postgres 9.1.3 version.Once upgraded to 9.2.4 , these log statements are

[GENERAL] DELETE or TRUNCATE?

2013-05-16 Thread François Beausoleil
Hi! I have a process that replaces the contents of a table. The canonical data store is somewhere else. At the moment, the import looks like this: CREATE TEMPORARY TABLE markets_import( LIKE x INCLUDING ALL ); COPY markets_import FROM STDIN; ... \. -- COPY a bunch of other tables BEGIN; TRUNCA

Re: [GENERAL] Running out of memory at vacuum

2013-05-16 Thread Ioana Danes
Hi Jeff, Yes stop/start of the application server does close all the connections to the database. Lately I did restart postgres too everytime that happened. It did happen in the past, last year sometime when I tried just to close the app and it was not enough. I might mix up different scenario

[GENERAL] WAL contains references to invalid pages

2013-05-16 Thread JotaComm
Hello, guys Yesterday I identified the following messages in my log file (slave): user=,db= WARNING: page 6629 of relation base/20449/24818 is uninitialized user=,db= CONTEXT: xlog redo vacuum: rel 1663/20449/24818; blk 6631, lastBlockVacuumed 6626 user=,db= PANIC: WAL contains references to i

Re: [GENERAL] Running out of memory at vacuum

2013-05-16 Thread Jeff Janes
On Thu, May 16, 2013 at 6:35 AM, Ioana Danes wrote: > Hi Jeff, > > On Tuesday, May 14, 2013, Ioana Danes wrote: > > > The fix is to restart postgres ... If I only close the connections the > problem is still these so I need to restart postgres. > > > How are you closing the connections? > > I res

Re: [GENERAL] Schema (Search path issue) on PostgreSQL9.2

2013-05-16 Thread Tom Lane
chiru r writes: > Is there any schema(set search_path) behaviour changes from PostgreSQL9.1 > to PostgreSQL9.2 ? Please read the release notes when updating to a new major version. The first item under "Server Settings" in the 9.2 release notes is: Silently ignore nonexistent schemas sp

Re: [GENERAL] problem with lost connection while running long PL/R query

2013-05-16 Thread Tom Lane
"David M. Kaplan" writes: > I have a query that uses a PL/R function to run a statistical model on > data in a postgresql table. The query runs the function 4 times, each > of which generates about 2 million lines of results, generating a final > table that has about 8 million lines. Each tim

Re: [GENERAL] Creating Extension pg_trgm

2013-05-16 Thread Tom Lane
itishree sukla writes: > I need to create pg_trgm extension, however in my current DB, we have some > function based on pg_trgm. when i am trying to create extension, it is > asking to drop all dependant function, is there anyway without dropping > any dependency i can create this extension. It

Re: [GENERAL] Running out of memory at vacuum

2013-05-16 Thread Ioana Danes
Hi Jeff, On Tuesday, May 14, 2013, Ioana Danes wrote: Hi all, > >I have a production database that sometimes runs out of memory at nightly >vacuum. > >The application runs typically with around 40 postgres connections but there >are times when the connections increase because of some queries

Re: [GENERAL] problem with lost connection while running long PL/R query

2013-05-16 Thread Ryan Kelly
On Thu, May 05/16/13, 2013 at 02:47:28PM +0200, David M. Kaplan wrote: > Hi, > > I have a query that uses a PL/R function to run a statistical model > on data in a postgresql table. The query runs the function 4 times, > each of which generates about 2 million lines of results, generating > a fin

Re: [GENERAL] Running out of memory on vacuum

2013-05-16 Thread Ioana Danes
Hello Scott, I will look into using pgbouncer at a point. For now I will try to increase the memory. From practice I see that 16GB it is not enough unless I lower max_connections to 200. I have another production server with 16 GB and it is stable if the connections open are less than 200, on

Re: [GENERAL] Schema (Search path issue) on PostgreSQL9.2

2013-05-16 Thread Raghavendra
> postgres=# select version(); > version > > > --- > PostgreSQL 9.2.4 on x86_64-unknown-linux-gnu, compiled by gcc (GCC) 4.1.2 > 20080704

Re: [GENERAL] Tuning read ahead

2013-05-16 Thread Shaun Thomas
On 05/15/2013 08:04 PM, Ramsey Gurley wrote: My question: Is that advice just for the database drive, or should I increase read ahead on the OS/WAL disk as well? Definitely the database drive, but it doesn't hurt to do both. It doesn't mention it in the book, but if you have a Debian or Ubunt

[GENERAL] Schema (Search path issue) on PostgreSQL9.2

2013-05-16 Thread chiru r
Hi All, I have seen strange behaviour in PostgreSQL9.2 version,it has been allowing to set search path any name,even the name is not created as a schema in database. Please find the below case between PostgreSQL9.1 and PostgreSQL9.2. *PostgreSQL9.2:* + postgres=# select version();

[GENERAL] problem with lost connection while running long PL/R query

2013-05-16 Thread David M. Kaplan
Hi, I have a query that uses a PL/R function to run a statistical model on data in a postgresql table. The query runs the function 4 times, each of which generates about 2 million lines of results, generating a final table that has about 8 million lines. Each time the function is called, it

Re: [GENERAL] FATAL: database "a/system_data" does not exist

2013-05-16 Thread sumita
I am running greps to find out what could be triggering it. in the scripts and programs. My only other concern is the same version of my product works without this log statements in the Postgres 9.1.3 version.Once upgraded to 9.2.4 , these log statements are appearing. -- View this message in co

Re: [GENERAL] Best way to reduce server rounds getting big BLOBs

2013-05-16 Thread Jorge Arévalo
Hello, First of all, sorry for the duplicate. I accidentally sent the same message twice, and I got responses in both of them. I include here the person who responded to the other messsage El miércoles 15 de mayo de 2013 a las 20:43, Merlin Moncure escribió: > On Wed, May 15, 2013 at 11:3

[GENERAL] Undefined reference with libpq on Visual Studio 2012

2013-05-16 Thread David Demelier
Hello all, I'm trying to build libpq using nmake from Visual Studio 2012. So I've tried the following : nmake -f win32.mak CPU=AMD64 while in postgresql-9.2.4/src directory. It does the following : link.exe -lib @C:\Users\markand\AppData\Local\Temp\nm2F60.tmp rc.exe /l 0x409 /fo"

Re: [GENERAL] Best way to reduce server rounds getting big BLOBs

2013-05-16 Thread Eduardo Morrás
>- Mensaje original - >De: Jorge Arévalo >Para: pgsql-general@postgresql.org >CC: >Enviado: Miércoles 15 de Mayo de 2013 17:08 >Asunto: [GENERAL] Best way to reduce server rounds getting big BLOBs > >Hello, > >I'd like to know what's the best way to reduce the number of server rounds