[GENERAL] GIN fast update technique and work_mem

2013-10-24 Thread Amit Langote
Hi, While going through the documentation for GIN fast update technique, I read - "...or if the pending list becomes too large (larger than work_mem), the entries are moved to the main GIN data structure using the same bulk insert techniques used during initial index creation." The "work_mem" li

Re: [GENERAL] GIN fast update technique and work_mem

2013-10-24 Thread Andres Freund
On 2013-10-24 18:40:46 +0900, Amit Langote wrote: > While going through the documentation for GIN fast update technique, I read - > > "...or if the pending list becomes too large (larger than work_mem), > the entries are moved to the main GIN data structure using the same > bulk insert techniques

[GENERAL] GIST index : order Hack : getting the order used by CLUSTER .. USING my_index

2013-10-24 Thread Rémi Cura
Hey List, I would like to be able to get the rows following the order of an index (*NOT* getting an order by accelerated, but only an order defined by an index). Something like this : SELECT my_row FROM my_table ORDER BY the_index ASC where the_index is a GIST index over points. I know there i

Re: [GENERAL] GIN fast update technique and work_mem

2013-10-24 Thread Amit Langote
On Thu, Oct 24, 2013 at 7:26 PM, Andres Freund wrote: > On 2013-10-24 18:40:46 +0900, Amit Langote wrote: >> While going through the documentation for GIN fast update technique, I read - >> >> "...or if the pending list becomes too large (larger than work_mem), >> the entries are moved to the main

Re: [GENERAL] Replication and fsync

2013-10-24 Thread Fujii Masao
On Thu, Oct 24, 2013 at 10:39 AM, wrote: > Newb question. > > I'm running 9.1 with a slave using streaming replication. A coworker wants > to turn off fsync on the master and insists that the slave will still be in > a usable state if there is a failure on the master. We all know that turning > o

Re: [GENERAL] GIST index : order Hack : getting the order used by CLUSTER .. USING my_index

2013-10-24 Thread Tom Lane
=?UTF-8?Q?R=C3=A9mi_Cura?= writes: > I would like to be able to get the rows following the order of an index > (*NOT* getting an order by accelerated, but only an order defined by an > index). Since a GiST index hasn't got any specific internal order, I fail to see the point of this.

Re: [GENERAL] Replication and fsync

2013-10-24 Thread Alban Hertroys
On 24 October 2013 15:04, Fujii Masao wrote: > On Thu, Oct 24, 2013 at 10:39 AM, wrote: >> Am I wrong? If I'm wrong, is there still danger to the slave >> in this kind of setup? > > No, I think. Corruption due to fsync being off on the master will be replicated to the slave, or - if corruption

Re: [GENERAL] GIST index : order Hack : getting the order used by CLUSTER .. USING my_index

2013-10-24 Thread Rémi Cura
Hello, I'm interested in the tree structure inherent to the gist indexing. I was thinking to retrieve it from order of index. Do you know how I could access it directly? My use case would be to take advantage of this gist ordering to order 2D points s1 : N1 N2 .. Nn so that for any given t > =?U

Re: [GENERAL] Wrong estimate in query plan

2013-10-24 Thread David Johnston
Eelke Klein wrote > What I noticed is that are no most common values mentioned ofcourse the > value 1 only occurs once in the column but as all other values are NULL > you > could argue it is a common value. A random sampling is unlikely to choose a record that only appears in 0.1 percent of the t

Re: [GENERAL] Replication and fsync

2013-10-24 Thread maillists0
Thank you for the answers. I'm still confused. If fsync is not replicated to the slave, then how is replication affected by a corrupt master? If the master dies and there's a commit recorded in the wal log that didn't actually happen, wouldn't the slave still be expected to be in a sane state, with

Re: [GENERAL] GIST index : order Hack : getting the order used by CLUSTER .. USING my_index

2013-10-24 Thread Tom Lane
=?UTF-8?Q?R=C3=A9mi_Cura?= writes: > I'm interested in the tree structure inherent to the gist indexing. > I was thinking to retrieve it from order of index. How? A SQL query would have no idea where the index page boundaries were in the sequence of retrieved tuples. > Do you know how I could a

Re: [GENERAL] GIST index : order Hack : getting the order used by CLUSTER .. USING my_index

2013-10-24 Thread Rémi Cura
Ok, thank you Tom for this precise answer ! I don't understand how the CLUSTER .. USING index command work then. It is supposed to rewrite on disk following index order. Does it do nothing for GIST index? Cheers, Rémi-C 2013/10/24 Tom Lane > =?UTF-8?Q?R=C3=A9mi_Cura?= writes: > > I'm interes

Re: [GENERAL] GIST index : order Hack : getting the order used by CLUSTER .. USING my_index

2013-10-24 Thread Tom Lane
=?UTF-8?Q?R=C3=A9mi_Cura?= writes: > I don't understand how the CLUSTER .. USING index command work then. > It is supposed to rewrite on disk following index order. Does it do nothing > for GIST index? Nobody has ever demonstrated that CLUSTER has any value for anything except btree indexes. It

Re: [GENERAL] PostgreSQL Point In Time Recovery

2013-10-24 Thread Jeff Janes
On Wed, Oct 23, 2013 at 9:10 PM, Jayadevan M wrote: > Hi, > I went through > http://www.postgresql.org/docs/9.3/static/continuous-archiving.html > and set up the archiving process. With this approach, if my database > crashes after a couple of weeks after the base backup is taken, recovering > wou

Re: [GENERAL] GIST index : order Hack : getting the order used by CLUSTER .. USING my_index

2013-10-24 Thread Rémi Cura
Great, thanks. Now you say that I never saw any improvement when clustering table with gist. You just saved me a lot of unnecessary queries :-) Cheers, Rémi-C 2013/10/24 Tom Lane > =?UTF-8?Q?R=C3=A9mi_Cura?= writes: > > I don't understand how the CLUSTER .. USING index command work then. >

Re: [GENERAL] PostgreSQL Point In Time Recovery

2013-10-24 Thread John R Pierce
On 10/24/2013 9:47 AM, Jeff Janes wrote: I restore from my base backup plus WAL quite often. It is how I get a fresh dev or test instance when I want one. (It is also how I have confidence that everything is working well and that I know what I'm doing should the time come to do a real resto

Re: [GENERAL] PostgreSQL Point In Time Recovery

2013-10-24 Thread Alan Hodgson
On Thursday, October 24, 2013 11:31:38 AM John R Pierce wrote: > On 10/24/2013 9:47 AM, Jeff Janes wrote: > > I restore from my base backup plus WAL quite often. It is how I get a > > fresh dev or test instance when I want one. (It is also how I have > > confidence that everything is working well

Re: [GENERAL] Replication and fsync

2013-10-24 Thread Jeff Janes
On Wed, Oct 23, 2013 at 6:39 PM, wrote: > Newb question. > > I'm running 9.1 with a slave using streaming replication. A coworker wants > to turn off fsync on the master and insists that the slave will still be in > a usable state if there is a failure on the master. > This would only be safe if

Re: [GENERAL] Count of records in a row

2013-10-24 Thread Robert James
On 10/22/13, Elliot wrote: > It looks like you already found a solution, but here's one with a CTE. I > cobbled this together from an older query I had for doing something > similar, for which I unfortunately lost the original source of this > approach. Also, this implies that there is something t

Re: [GENERAL] Replication and fsync

2013-10-24 Thread Alban Hertroys
On Oct 24, 2013, at 18:10, maillis...@gmail.com wrote: > Thank you for the answers. I'm still confused. If fsync is not replicated to > the slave, then how is replication affected by a corrupt master? If the > master dies and there's a commit recorded in the wal log that didn't actually > happe

[GENERAL] Is there something like a limited superuser to give to a db-assistant?

2013-10-24 Thread Andreas
Hi, how can I give adb-assistant the rights to create and drop schemas, tables, views ... BUT keep him out of certain existing schemas and tables? This position of an restricted db assistant is new for us and it wasn't considered in our access rights until now. Our DB got a wee bit komplex

[GENERAL] Need help how to manage a couple of daily DB copies.

2013-10-24 Thread Andreas
Hi, I'd like to set up a DB-Server that keeps copies of our productive db for an external db-assistant. He should prepare chores on the test-server and mail the sql scripts to me. I'll look over those scripts and run them against the productive db myself. So I'd like to have a daily cron job d

Re: [GENERAL] Is there something like a limited superuser to give to a db-assistant?

2013-10-24 Thread Tomas Vondra
On 25 Říjen 2013, 3:53, Andreas wrote: > Hi, > > how can I give adb-assistant the rights to create and drop schemas, > tables, views ... BUT keep him out of certain existing schemas and tables? > > This position of an restricted db assistant is new for us and it wasn't > considered in our access ri

Re: [GENERAL] Is there something like a limited superuser to give to a db-assistant?

2013-10-24 Thread Michael Paquier
On Fri, Oct 25, 2013 at 10:53 AM, Andreas wrote: > Hi, > > how can I give a db-assistant the rights to create and drop schemas, tables, > views ... BUT keep him out of certain existing schemas and tables? Depending on what you want to do, you will have to use a combination GRANT and REVOKE to auth

Re: [GENERAL] Need help how to manage a couple of daily DB copies.

2013-10-24 Thread Tomas Vondra
On 25 Říjen 2013, 3:53, Andreas wrote: > Hi, > > I'd like to set up a DB-Server that keeps copies of our productive db > for an external db-assistant. > He should prepare chores on the test-server and mail the sql scripts to > me. > I'll look over those scripts and run them against the productive d

Re: [GENERAL] Need help how to manage a couple of daily DB copies.

2013-10-24 Thread Michael Paquier
On Fri, Oct 25, 2013 at 10:53 AM, Andreas wrote: > Hi, > > I'd like to set up a DB-Server that keeps copies of our productive db for an > external db-assistant. > He should prepare chores on the test-server and mail the sql scripts to me. > I'll look over those scripts and run them against the pro

Re: [GENERAL] Need help how to manage a couple of daily DB copies.

2013-10-24 Thread Shridhar Daithankar
On Friday, October 25, 2013 03:53:14 AM Andreas wrote: > Hi, > > I'd like to set up a DB-Server that keeps copies of our productive db > for an external db-assistant. > He should prepare chores on the test-server and mail the sql scripts to me. > I'll look over those scripts and run them against t

Re: [GENERAL] Need help how to manage a couple of daily DB copies.

2013-10-24 Thread Andreas
Am 25.10.2013 04:15, schrieb Shridhar Daithankar: On Friday, October 25, 2013 03:53:14 AM Andreas wrote: Hi, I'd like to set up a DB-Server that keeps copies of our productive db for an external db-assistant. He should prepare chores on the test-server and mail the sql scripts to me. I'll look

Re: [GENERAL] Need help how to manage a couple of daily DB copies.

2013-10-24 Thread DDT
Dear, Append following command to crontab: D=`date -d -5day +"%Y%m%d"`;echo "DROP DATABASE test_db_$D" | psql Maybe you should change the "psql" to your psql path. -- Original -- From: "Andreas";; Date: Fri, Oct 25, 2013 09:53 AM To: "pgsql-general";

Re: [GENERAL] Replication and fsync

2013-10-24 Thread DDT
Dear According to manual, when you set "synchronous_commit" to on, the transaction commits will wait until master and slave flush the commit record of transaction to the physical storage, so I think even if turn off the fsync on master is safe for data consistency and data will not be lost if s

Re: [GENERAL] PostgreSQL Point In Time Recovery

2013-10-24 Thread Jayadevan
Jeff Janes wrote > I restore from my base backup plus WAL quite often. It is how I get a > fresh dev or test instance when I want one. (It is also how I have > confidence that everything is working well and that I know what I'm doing > should the time come to do a real restore). When that starts

Re: [GENERAL] PostgreSQL Point In Time Recovery

2013-10-24 Thread Jayadevan
Alan Hodgson wrote > That's basically what warm standby's do, isn't it? As long as they keep > recovery open it should work. A warn standby will be almost in sync with the primary, right? So recovery to point-in-time (like 10 AM this morning) won't be possible. We need a base, but it shouldn't be