Re: [HACKERS] ToDo: no blocking (waiting) DDL

2016-12-13 13:03 GMT+01:00 Alvaro Herrera : > Pavel Stehule wrote: > > Hi > > > > I don't remember well, there was maybe similar ToDo. > > > > Yesterday I got a incident on high load system when I executed DROP INDEX > > cmd. > > > > This statement waited on a finish of long transaction, but it st

Re: [HACKERS] ToDo: no blocking (waiting) DDL

Pavel Stehule wrote: > Hi > > I don't remember well, there was maybe similar ToDo. > > Yesterday I got a incident on high load system when I executed DROP INDEX > cmd. > > This statement waited on a finish of long transaction, but it stopped any > other statements. > > Can be nice when waiting

Re: [HACKERS] TODO item: Implement Boyer-Moore searching in LIKE queries

Karan Sikka writes: >> Having said that, I've had a bee in my bonnet for a long time about >> removing per-row setup cost for repetitive regex matches, and >> whatever infrastructure that needs would work for this too. > What are the per-row setup costs for regex matches? Well, they're pretty da

Re: [HACKERS] TODO item: Implement Boyer-Moore searching in LIKE queries

> Having said that, I've had a bee in my bonnet for a long time about > removing per-row setup cost for repetitive regex matches, and > whatever infrastructure that needs would work for this too. What are the per-row setup costs for regex matches? I looked at `regexp.c` and saw: ``` /* * We cach

Re: [HACKERS] TODO item: Implement Boyer-Moore searching in LIKE queries

Karan Sikka writes: > Just for the record, I'm leaning to the side of "not worth it". My > thoughts are, if you are at a scale where you care about this 20% > speedup, you would want to go all the way to an indexed structure, > because the gains you would realize would exceed 20%, and 20% may not

Re: [HACKERS] TODO item: Implement Boyer-Moore searching in LIKE queries

On Tue, Aug 2, 2016 at 1:56 PM, Alvaro Herrera wrote: >> > How desirable is this feature? To begin answering that question, >> > I did some initial benchmarking with an English text corpus to find how >> > much >> > faster BMH (Boyer-Moore-Horspool) would be compared to LIKE, and the >> > result

Re: [HACKERS] TODO item: Implement Boyer-Moore searching in LIKE queries

Yeah, you make a fair point. I was hoping more people would chime in with strong opinions to make the decision easier, but perhaps the lack of noise indicates this is a low-pri feature. I will ask around in my coworker circles to see what they think, could you do the same? Just for the record, I

Re: [HACKERS] TODO item: Implement Boyer-Moore searching in LIKE queries

Robert Haas wrote: > On Mon, Aug 1, 2016 at 1:19 PM, Karan Sikka wrote: > > Following the patch to implement strpos with Boyer-Moore-Horspool, > > it was proposed we bring BMH to LIKE as well. > > > > Original thread: > > https://www.postgresql.org/message-id/flat/27645.1220635769%40sss.pgh.pa.us#

Re: [HACKERS] TODO item: Implement Boyer-Moore searching in LIKE queries

On Mon, Aug 1, 2016 at 7:47 PM, Karan Sikka wrote: > I agree, should we remove it from the TODO list? If nobody objects, sure. -- Robert Haas EnterpriseDB: http://www.enterprisedb.com The Enterprise PostgreSQL Company -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To m

Re: [HACKERS] TODO item: Implement Boyer-Moore searching in LIKE queries

I agree, should we remove it from the TODO list? On Mon, Aug 1, 2016 at 6:13 PM, Robert Haas wrote: > On Mon, Aug 1, 2016 at 1:19 PM, Karan Sikka wrote: > > Following the patch to implement strpos with Boyer-Moore-Horspool, > > it was proposed we bring BMH to LIKE as well. > > > > Original thre

Re: [HACKERS] TODO item: Implement Boyer-Moore searching in LIKE queries

On Mon, Aug 1, 2016 at 1:19 PM, Karan Sikka wrote: > Following the patch to implement strpos with Boyer-Moore-Horspool, > it was proposed we bring BMH to LIKE as well. > > Original thread: > https://www.postgresql.org/message-id/flat/27645.1220635769%40sss.pgh.pa.us#27645.1220635...@sss.pgh.pa.us

Re: [HACKERS] ToDo: API for SQL statement execution other than SPI

> On Jul 1, 2016, at 11:39 AM, Robert Haas wrote: > > On Wed, Jun 29, 2016 at 1:55 AM, Pavel Stehule > wrote: >> I am writing two background workers - autoreindex and scheduler. In Both I >> need to execute queries from top level. I had to wrote redundant code >>

Re: [HACKERS] ToDo: API for SQL statement execution other than SPI

On Wed, Jun 29, 2016 at 1:55 AM, Pavel Stehule wrote: > I am writing two background workers - autoreindex and scheduler. In Both I > need to execute queries from top level. I had to wrote redundant code > https://github.com/okbob/autoreindex/blob/master/utils.c > autoreindex_execute_sql_command .S

Re: [HACKERS] ToDo: API for SQL statement execution other than SPI

On 29 June 2016 at 13:55, Pavel Stehule wrote: > Hi > > I am writing two background workers - autoreindex and scheduler. In Both I > need to execute queries from top level. I had to wrote redundant code > https://github.com/okbob/autoreindex/blob/master/utils.c > autoreindex_execute_sql_command

Re: [HACKERS] ToDo list update for BRIN indexes

On Mon, Dec 21, 2015 at 07:54:43AM -0500, Robert Haas wrote: > On Thu, Jul 9, 2015 at 4:49 PM, Jeff Janes wrote: > > Is there anything in the below section which has been been implemented or > > rendered irrelevant by BRIN indexes? > > > > https://wiki.postgresql.org/wiki/Todo#Indexes > > > > "Con

Re: [HACKERS] ToDo list update for BRIN indexes

On 21 December 2015 at 14:38, Robert Haas wrote: > On Mon, Dec 21, 2015 at 8:06 AM, Simon Riggs > wrote: > > On 21 December 2015 at 12:54, Robert Haas wrote: > >> > >> On Thu, Jul 9, 2015 at 4:49 PM, Jeff Janes > wrote: > >> > Is there anything in the below section which has been been implemen

Re: [HACKERS] ToDo list update for BRIN indexes

On Mon, Dec 21, 2015 at 8:06 AM, Simon Riggs wrote: > On 21 December 2015 at 12:54, Robert Haas wrote: >> >> On Thu, Jul 9, 2015 at 4:49 PM, Jeff Janes wrote: >> > Is there anything in the below section which has been been implemented >> > or >> > rendered irrelevant by BRIN indexes? >> > >> > h

Re: [HACKERS] ToDo list update for BRIN indexes

On 21 December 2015 at 12:54, Robert Haas wrote: > On Thu, Jul 9, 2015 at 4:49 PM, Jeff Janes wrote: > > Is there anything in the below section which has been been implemented or > > rendered irrelevant by BRIN indexes? > > > > https://wiki.postgresql.org/wiki/Todo#Indexes > > > > "Consider smal

Re: [HACKERS] ToDo list update for BRIN indexes

On Thu, Jul 9, 2015 at 4:49 PM, Jeff Janes wrote: > Is there anything in the below section which has been been implemented or > rendered irrelevant by BRIN indexes? > > https://wiki.postgresql.org/wiki/Todo#Indexes > > "Consider smaller indexes that record a range of values per heap page, > rather

Re: [HACKERS] TODO list updates

On 16 October 2015 18:20:59 EEST, Bruce Momjian wrote: >I think on-disk bitmap indexes would only beat GIN indexes in a >read-only database on low-cardinality columns. For example, if you had >a purchase_log table and wanted to know all the "blue" and "large" >items >sold at a specific store, I

Re: [HACKERS] TODO list updates

On Fri, Oct 16, 2015 at 12:02:03PM -0300, Alvaro Herrera wrote: > Bruce Momjian wrote: > > > Are you suggesting I remove those links? It is kind of odd to have > > links to patches for features we don't want, or just keep it? > > No, quite the contrary -- I think the links allow some other perso

Re: [HACKERS] TODO list updates

On Fri, Oct 16, 2015 at 12:00:11PM -0300, Alvaro Herrera wrote: > Bruce Momjian wrote: > > I have spend the past few days updating our TODO list, removing > > completed and now-unnecessary items: > > > > https://wiki.postgresql.org/wiki/Todo > > Thanks. We have "TodoDone" pages for items tha

Re: [HACKERS] TODO list updates

Bruce Momjian wrote: > Are you suggesting I remove those links? It is kind of odd to have > links to patches for features we don't want, or just keep it? No, quite the contrary -- I think the links allow some other person research the issue including the history of patches and discussion, and de

Re: [HACKERS] TODO list updates

Bruce Momjian wrote: > I have spend the past few days updating our TODO list, removing > completed and now-unnecessary items: > > https://wiki.postgresql.org/wiki/Todo Thanks. We have "TodoDone" pages for items that were done in specific releases, but only for 8.4, 9.0 and 9.1. I guess it

Re: [HACKERS] TODO list updates

On Fri, Oct 16, 2015 at 11:43:10AM -0300, Alvaro Herrera wrote: > Bruce Momjian wrote: > > > Probably the most controvertial change was to move on-disk bitmap > > indexes to the "not wanted" section, though I kept the links in case we > > change our minds. I just can't see how they would be a win

Re: [HACKERS] TODO list updates

Bruce Momjian wrote: > Probably the most controvertial change was to move on-disk bitmap > indexes to the "not wanted" section, though I kept the links in case we > change our minds. I just can't see how they would be a win with GIN and > in-memory bitmaps. Yeah, I recall we discussed bitmap ind

Re: [HACKERS] TODO list updates

On Fri, Oct 16, 2015 at 02:50:04PM +0530, Amit Kapila wrote: > On Fri, Oct 16, 2015 at 8:34 AM, Bruce Momjian wrote: > > I have spend the past few days updating our TODO list, removing > completed and now-unnecessary items: > >         https://wiki.postgresql.org/wiki/Todo > > > >

Re: [HACKERS] TODO list updates

On Fri, Oct 16, 2015 at 8:34 AM, Bruce Momjian wrote: > I have spend the past few days updating our TODO list, removing > completed and now-unnecessary items: > > https://wiki.postgresql.org/wiki/Todo > > Thanks. It can help encourage many new entrants to community. With Regards, Amit

Re: [HACKERS] TODO: replica information functions

On 28-07-2015 15:35, Josh Berkus wrote: pg_standby_is_streaming() returns true if the standby is configured for streaming and is currently connected with the master. returns false if the connection to the master is broken, of if there is no primary_conninfo +1.

Re: [HACKERS] TODO: replica information functions

> On 28 Jul 2015, at 21:35, Josh Berkus wrote: > > Hackers, > > Since merging recovery.conf with postgresql.conf is apparently off the > table indefinitely, we could really use some additional information > functions which work on the replica. Here's my list of what I need for > failover autom

Re: [HACKERS] TODO: replica information functions

On 2015-07-28 11:35:52 -0700, Josh Berkus wrote: > Since merging recovery.conf with postgresql.conf is apparently off the > table indefinitely Off the table as in "somebody needs to actually work on it instead of just talking about it". -- Sent via pgsql-hackers mailing list (pgsql-hackers@pos

Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

Le 23 juil. 2015 19:27, "Alvaro Herrera" a écrit : > > Laurent Laborde wrote: > > > Friendly greetings ! > > > > What's the status of parallel clusterdb please ? > > I'm having fun (and troubles) applying the vacuumdb patch to clusterdb. > > > > This thread also talk about unifying code for parall

Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

Laurent Laborde wrote: > Friendly greetings ! > > What's the status of parallel clusterdb please ? > I'm having fun (and troubles) applying the vacuumdb patch to clusterdb. > > This thread also talk about unifying code for parallelizing clusterdb and > reindex. > Was anything done about it ? Bec

Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

On Fri, Jan 2, 2015 at 3:18 PM, Amit Kapila wrote: > > > Okay, I have marked this patch as "Ready For Committer" > > Notes for Committer - > There is one behavioural difference in the handling of --analyze-in-stages > switch, when individual tables (by using -t option) are analyzed by > using thi

Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

Pavel Stehule wrote: > should not be used a pessimist controlled locking instead? > Patches welcome. -- Álvaro Herrerahttp://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services -- Sent via pgsql-hackers mailing list (pgsql-hackers@postg

Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

2015-01-29 10:28 GMT+01:00 Fabrízio de Royes Mello : > > > Em quinta-feira, 29 de janeiro de 2015, Pavel Stehule < > pavel.steh...@gmail.com> escreveu: > > Hi >> >> I am testing this feature on relative complex schema (38619 tables in db) >> and I got deadlock >> >> [pavel@localhost bin]$ /usr/lo

Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

Em quinta-feira, 29 de janeiro de 2015, Pavel Stehule < pavel.steh...@gmail.com> escreveu: > Hi > > I am testing this feature on relative complex schema (38619 tables in db) > and I got deadlock > > [pavel@localhost bin]$ /usr/local/pgsql/bin/vacuumdb test2 -fz -j 4 > vacuumdb: vacuuming database

Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

Hi I am testing this feature on relative complex schema (38619 tables in db) and I got deadlock [pavel@localhost bin]$ /usr/local/pgsql/bin/vacuumdb test2 -fz -j 4 vacuumdb: vacuuming database "test2" vacuumdb: vacuuming of database "test2" failed: ERROR: deadlock detected DETAIL: Process 24689

Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

On 23 January 2015 23:55, Alvaro Herrera, > -j1 is now the same as not specifying anything, and vacuum_one_database > uses more common code in the parallel and not-parallel cases: the not- > parallel case is just the parallel case with a single connection, so > the setup and shutdown is mostly the

Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

On 23 January 2015 21:10, Alvaro Herrera Wrote, > In case you're up for doing some more work later on, there are two > ideas > here: move the backend's TranslateSocketError to src/common, and try to > merge pg_dump's select_loop function with the one in this new code. > But that's for another pat

Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

Alvaro Herrera wrote: > I'm tweaking your v24 a bit more now, thanks -- main change is to make > vacuum_one_database be called only to run one analyze stage, so it never > iterates for each stage; callers must iterate calling it multiple times > in those cases. (There's only one callsite that nee

Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

Andres Freund wrote: > On 2014-12-31 18:35:38 +0530, Amit Kapila wrote: > > + PQsetnonblocking(connSlot[0].connection, 1); > > + > > + for (i = 1; i < concurrentCons; i++) > > + { > > + connSlot[i].connection = connectDatabase(dbname, host, port, > > username, > > +

Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

Dilip kumar wrote: > Changes: > 1. In current patch vacuum_one_database (for table list), have the table loop > outside and analyze_stage loop inside, so it will finish > All three stage for one table first and then pick the next table. But > vacuum_one_database_parallel will do the stage loop o

Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

On 22 January 2015 23:16, Alvaro Herrera Wrote, > Here's v23. > > There are two things that continue to bother me and I would like you, > dear patch author, to change them before committing this patch: > > 1. I don't like having vacuum_one_database() and a separate > vacuum_one_database_parallel()

Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

Here's v23. I reworked a number of things. First, I changed trivial stuff like grouping all the vacuuming options in a struct, to avoid passing an excessive number of arguments to functions. full, freeze, analyze_only, and_analyze and verbose are all in a single struct now. Also, the stage_comm

Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

On Thu, Jan 22, 2015 at 8:22 AM, Alvaro Herrera wrote: > > Amit Kapila wrote: > > On Wed, Jan 21, 2015 at 8:51 PM, Alvaro Herrera < alvhe...@2ndquadrant.com> > > wrote: > > > > > > I didn't understand the coding in GetQueryResult(); why do we check the > > > result status of the last returned resu

Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

Amit Kapila wrote: > On Wed, Jan 21, 2015 at 8:51 PM, Alvaro Herrera > wrote: > > > > I didn't understand the coding in GetQueryResult(); why do we check the > > result status of the last returned result only? It seems simpler to me > > to check it inside the loop, but maybe there's a reason you

Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

On Wed, Jan 21, 2015 at 8:51 PM, Alvaro Herrera wrote: > > I didn't understand the coding in GetQueryResult(); why do we check the > result status of the last returned result only? It seems simpler to me > to check it inside the loop, but maybe there's a reason you didn't do it > like that? > > A

Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

I didn't understand the coding in GetQueryResult(); why do we check the result status of the last returned result only? It seems simpler to me to check it inside the loop, but maybe there's a reason you didn't do it like that? Also, what is the reason we were ignoring those errors only in "comple

Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

On 04 January 2015 07:27, Andres Freund Wrote, > On 2014-12-31 18:35:38 +0530, Amit Kapila wrote: > > + -j class="parameter">jobs > > + --jobs= class="parameter">njobs > > + > > + > > +Number of concurrent connections to perform the operation. > > +This optio

Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

On Fri, Jan 16, 2015 at 12:53 AM, Alvaro Herrera wrote: > Michael Paquier wrote: > >> Andres, this patch needs more effort from the author, right? So >> marking it as returned with feedback. > > I will give this patch a look in the current commitfest, if you can > please set as 'needs review' inst

Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

Michael Paquier wrote: > Andres, this patch needs more effort from the author, right? So > marking it as returned with feedback. I will give this patch a look in the current commitfest, if you can please set as 'needs review' instead with me as reviewer, so that I don't forget, I would appreciate

Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

On Sun, Jan 4, 2015 at 10:57 AM, Andres Freund wrote: > On 2014-12-31 18:35:38 +0530, Amit Kapila wrote: >> + -j > class="parameter">jobs >> + --jobs=> class="parameter">njobs >> + >> + >> +Number of concurrent connections to perform the operation. >> +This o

Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

On 2014-12-31 18:35:38 +0530, Amit Kapila wrote: > + -j class="parameter">jobs > + --jobs= class="parameter">njobs > + > + > +Number of concurrent connections to perform the operation. > +This option will enable the vacuum operation to run on asynchronous > +

Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

On Fri, Jan 2, 2015 at 8:34 PM, Kevin Grittner wrote: > > Amit Kapila wrote: > > > Notes for Committer - > > There is one behavioural difference in the handling of --analyze-in-stages > > switch, when individual tables (by using -t option) are analyzed by > > using this switch, patch will process

Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

Amit Kapila wrote: > Notes for Committer - > There is one behavioural difference in the handling of --analyze-in-stages > switch, when individual tables (by using -t option) are analyzed by > using this switch, patch will process (in case of concurrent jobs) all the > given tables for stage-1 and

Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

On Fri, Jan 2, 2015 at 11:47 AM, Dilip kumar wrote: > > On 31 December 2014 18:36, Amit Kapila Wrote, > > >The patch looks good to me. I have done couple of > > >cosmetic changes (spelling mistakes, improve some comments, > > >etc.), check the same once and if you are okay, we can move > > >ahead

Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

On 31 December 2014 18:36, Amit Kapila Wrote, >The patch looks good to me. I have done couple of >cosmetic changes (spelling mistakes, improve some comments, >etc.), check the same once and if you are okay, we can move >ahead. Thanks for review and changes, changes looks fine to me.. Regards, D

Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

On Mon, Dec 29, 2014 at 11:10 AM, Dilip kumar wrote: > > On 29 December 2014 10:22 Amit Kapila Wrote, > > > I think nothing more to be handled from my side, you can go ahead with review.. > The patch looks good to me. I have done couple of cosmetic changes (spelling mistakes, improve some commen

Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

On 29 December 2014 10:22 Amit Kapila Wrote, >> Case1:In Case for CompleteDB: >> >> In base code first it will process all the tables in stage 1 then in stage2 >> and so on, so that at some time all the tables are analyzed at least up to >> certain stage. >> >> But If we process all the stages f

Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

On Wed, Dec 24, 2014 at 4:00 PM, Dilip kumar wrote: > > Case1:In Case for CompleteDB: > > In base code first it will process all the tables in stage 1 then in stage2 and so on, so that at some time all the tables are analyzed at least up to certain stage. > > But If we process all the stages for o

Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

On 19 December 2014 16:41, Amit Kapila Wrote, >One idea is to send all the stages and corresponding Analyze commands >to server in one go which means something like >"BEGIN; SET default_statistics_target=1; SET vacuum_cost_delay=0; > Analyze t1; COMMIT;" >"BEGIN; SET default_statistics_target=10;

Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

On Mon, Dec 15, 2014 at 4:18 PM, Dilip kumar wrote: > > On December 2014 17:31 Amit Kapila Wrote, > > > >Hmm, theoretically I think new behaviour could lead to more I/O in > > >certain cases as compare to existing behaviour. The reason for more I/O > > >is that in the new behaviour, while doing A

Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

On December 2014 17:31 Amit Kapila Wrote, >I suggest rather than removing, edit the comment to indicate >the idea behind code at that place. Done >Okay, I think this part of code is somewhat similar to what >is done in pg_dump/parallel.c with some differences related >to handling of inAbort. On

Re: [HACKERS] [TODO] Track number of files ready to be archived in pg_stat_archiver

On Sat, Dec 13, 2014 at 11:53 PM, Julien Rouhaud wrote: > I agree with you about the problems of the v2 patch I originally sent. > I think this v3 is the right way of keeping track of .ready files, so > it's ok for me. The v3 also still applies well on current head. Simon got a good point mentioni

Re: [HACKERS] [TODO] Track number of files ready to be archived in pg_stat_archiver

-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Le 18/11/2014 08:36, Michael Paquier a écrit : > On Wed, Oct 22, 2014 at 12:50 AM, Brightwell, Adam > wrote: >> Though, I would think that the general desire would be to keep >> the patch relevant ONLY to the necessary changes. I would not >> qualif

Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

On Mon, Dec 8, 2014 at 7:33 AM, Dilip kumar wrote: > On 06 December 2014 20:01 Amit Kapila Wrote > > >I wanted to understand what exactly the above loop is doing. > > > > >a. > > >first of all the comment on top of it says "Some of the slot > > >are free, ...", if some slot is free, then why do yo

Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

will break the do..while loop. From: Amit Kapila [mailto:amit.kapil...@gmail.com] Sent: 06 December 2014 20:01 To: Dilip kumar Cc: Magnus Hagander; Alvaro Herrera; Jan Lentfer; Tom Lane; PostgreSQL-development; Sawada Masahiko; Euler Taveira Subject: Re: [HACKERS] TODO : Allow parallel core

Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

On Sat, Dec 6, 2014 at 9:01 PM, Amit Kapila wrote: > If you agree, then we should try to avoid this change in new behaviour. Still seeing many concerns about this patch, so marking it as returned with feedback. If possible, switching it to the next CF would be fine I guess as this work is still be

Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

On Mon, Dec 1, 2014 at 12:18 PM, Dilip kumar wrote: > > On 24 November 2014 11:29, Amit Kapila Wrote, > I have verified that all previous comments are addressed and the new version is much better than previous version. > > here we are setting each target once and doing for all the tables.. > Hm

Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

On 24 November 2014 11:29, Amit Kapila Wrote, >I think still some of the comments given upthread are not handled: > >a. About cancel handling Your Actual comment was --> >One other related point is that I think still cancel handling mechanism >is not completely right, code is doing that when th

Re: [HACKERS] TODO item: Accept aliases for values in ROW(...) constructor

Hi Craig Is there agreement on proposed syntax ROW(x AS something, y AS somethingelse) ? I can start work on this topic this week. Regards Pavel 2014-11-25 2:33 GMT+01:00 Craig Ringer : > > > > ROW(x AS something, y AS somethingelse) > > Apologies, it looks like Pavel already bought this

Re: [HACKERS] TODO item: Accept aliases for values in ROW(...) constructor

> ROW(x AS something, y AS somethingelse) Apologies, it looks like Pavel already bought this up: http://www.postgresql.org/message-id/cafj8prb1t1w6g0sppn-jetyzjpluuz_fxtnbme5okd3xxvf...@mail.gmail.com and I missed it. -- Craig Ringer http://www.2ndQuadrant.com/ Postgre

Re: [HACKERS] [TODO] Track number of files ready to be archived in pg_stat_archiver

Hearing nothing from the original author, this patch that was in state "Waiting on Author" for a couple of days is switched to "returned with feedback". Regards, -- Michael

Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

On Mon, Nov 24, 2014 at 7:34 AM, Dilip kumar wrote: > > On 23 November 2014 14:45, Amit Kapila Wrote > > > > Thanks a lot for debugging and fixing the issue.. > > > > Latest patch is attached, please have a look. > I think still some of the comments given upthread are not handled: a. About canc

Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

On 23 November 2014 14:45, Amit Kapila Wrote Thanks a lot for debugging and fixing the issue.. >The stacktrace of crash is as below: >#0 0x0080108cf3a4 in .strlen () from /lib64/libc.so.6 >#1 0x0080108925bc in ._IO_vfprintf () from /lib64/libc.so.6 >#2 0x0080108bc1e0 in .__GL__IO_

Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

On Mon, Nov 17, 2014 at 8:55 AM, Dilip kumar wrote: > > On 13 November 2014 15:35 Amit Kapila Wrote, > >As mentioned by you offlist that you are not able reproduce this > > >issue, I have tried again and what I observe is that I am able to > > >reproduce it only on *release* build and some cases

Re: [HACKERS] [TODO] Track number of files ready to be archived in pg_stat_archiver

On 18 November 2014 06:20, Michael Paquier wrote: > the DBA may want to know how long is the queue of WAL files > waiting to be archived. Agreed > That's IMO something we simply forgot in the > first implementation of pg_stat_archiver That's not how it appears to me. ISTM that the information

Re: [HACKERS] [TODO] Track number of files ready to be archived in pg_stat_archiver

On Wed, Oct 22, 2014 at 12:50 AM, Brightwell, Adam wrote: > Though, I would think that the general desire would be to keep the patch > relevant ONLY to the necessary changes. I would not qualify making those > types of changes as relevant, IMHO. I do think this is potential for > cleanup, howeve

Re: [HACKERS] [TODO] Track number of files ready to be archived in pg_stat_archiver

On Tue, Nov 18, 2014 at 5:47 AM, Simon Riggs wrote: > On 21 August 2014 09:17, Julien Rouhaud wrote: > >> Track number of WAL files ready to be archived in pg_stat_archiver > > Would it be OK to ask what the purpose of this TODO item is? > > pg_stat_archiver already has a column for last_archived

Re: [HACKERS] [TODO] Track number of files ready to be archived in pg_stat_archiver

On 21 August 2014 09:17, Julien Rouhaud wrote: > Track number of WAL files ready to be archived in pg_stat_archiver Would it be OK to ask what the purpose of this TODO item is? pg_stat_archiver already has a column for last_archived_wal and last_failed_wal, so you can already work out how many

Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

On 13 November 2014 15:35 Amit Kapila Wrote, >As mentioned by you offlist that you are not able reproduce this >issue, I have tried again and what I observe is that I am able to >reproduce it only on *release* build and some cases work without >this issue as well, >example: >./vacuumdb --analyze-i

Re: [HACKERS] TODO request: log_long_transaction

On 11/7/14, 1:19 PM, Michael Banck wrote: Am Montag, den 27.10.2014, 19:29 + schrieb Thom Brown: >On 27 October 2014 19:21, Josh Berkus wrote: > >I just realized that there is one thing we can't log currently: > >transactions which last more than #ms. This is valuable diagnostic > >inform

Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

On Mon, Oct 27, 2014 at 5:26 PM, Amit Kapila wrote: > > > Going further with verification of this patch, I found below issue: > Run the testcase.sql file at below link: > http://www.postgresql.org/message-id/4205e661176a124faf891e0a6ba9135266347...@szxeml509-mbs.china.huawei.com > ./vacuumdb --ana

Re: [HACKERS] TODO request: log_long_transaction

Robert Haas wrote: >> 3. Should long transactions which are rolled back be logged as well? > > Yes. +1 >> 4. We log the statement when exceeding log_min_duration_statement, but >> for transactions, that does not make a lot of sense, or should the last >> statement be logged? I don't think that

Re: [HACKERS] TODO request: log_long_transaction

You should add this patch here, so it doesn't get forgotten: https://commitfest.postgresql.org/action/commitfest_view/open On Fri, Nov 7, 2014 at 2:19 PM, Michael Banck wrote: > 1. Should this log when the duration is exceeded (like log_lock_waits), > or on commit? I guess the latter, cause log_

Re: [HACKERS] TODO request: log_long_transaction

Hi, Am Montag, den 27.10.2014, 19:29 + schrieb Thom Brown: > On 27 October 2014 19:21, Josh Berkus wrote: > > I just realized that there is one thing we can't log currently: > > transactions which last more than #ms. This is valuable diagnostic > > information when looking for issues like ca

Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

On Tue, Oct 28, 2014 at 9:27 AM, Dilip kumar wrote: > On 28 October 2014 09:18, Amit Kapila Wrote, > > >I am worried about the case if after setting the inAbort flag, > > >PQCancel() fails (returns error). > > > > > >> If select(maxFd + 1, workerset, NULL, NULL, &tv); come out, we can know whether

Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

On 28 October 2014 09:18, Amit Kapila Wrote, >I am worried about the case if after setting the inAbort flag, >PQCancel() fails (returns error). > >> If select(maxFd + 1, workerset, NULL, NULL, &tv); come out, we can know >> whether it came out because of cancel query and handle it accordingly. >>

Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

On Tue, Oct 28, 2014 at 9:03 AM, Dilip kumar wrote: > > On 25 October 2014 17:52, Amit Kapila Wrote, > > >*** > > >*** 358,363 handle_sigint(SIGNAL_ARGS) > > >--- 358,364 > > > > > > /* Send QueryCancel if we are processing a database query */ > > > if (cancelConn != NULL)

Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

On 25 October 2014 17:52, Amit Kapila Wrote, >*** >*** 358,363 handle_sigint(SIGNAL_ARGS) >--- 358,364 > > /* Send QueryCancel if we are processing a database query */ > if (cancelConn != NULL) > { >+ inAbort = true; > if (PQcancel(cancelConn, errbuf, sizeof(errbuf))) > f

Re: [HACKERS] TODO request: log_long_transaction

On 27 October 2014 19:21, Josh Berkus wrote: > Hackers, > > I just realized that there is one thing we can't log currently: > transactions which last more than #ms. This is valuable diagnostic > information when looking for issues like causes of bloat and deadlocks. > > I'd like it to be on the

Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

On Sat, Oct 25, 2014 at 5:52 PM, Amit Kapila wrote: > > > *** > *** 358,363 handle_sigint(SIGNAL_ARGS) > --- 358,364 > > /* Send QueryCancel if we are processing a database query */ > if (cancelConn != NULL) > { > + inAbort = true; > if (PQcancel(cancelConn, errbuf, s

Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

On Tue, Oct 7, 2014 at 11:10 AM, Dilip kumar wrote: > > On 26 September 2014 12:24, Amit Kapila Wrote, > > >I don't think this can handle cancel requests properly because > > >you are just setting it in GetIdleSlot() what if the cancel > > >request came during GetQueryResult() after sending sql fo

Re: [HACKERS] [TODO] Track number of files ready to be archived in pg_stat_archiver

Julien, > Actually, I used the same loop as the archiver one (see > backend/postmaster/pgarch.c, function pgarch_readyXlog) to get the exact > same number of files. > Ah, I see. > If we change it in this patch, it would be better to change it everywhere. > What do you think ? > Hmm... I'd hav

Re: [HACKERS] [TODO] Track number of files ready to be archived in pg_stat_archiver

On Tue, Oct 21, 2014 at 7:35 AM, Brightwell, Adam < adam.brightw...@crunchydatasolutions.com> wrote: > Julien, > > The following is an initial review: > > Thanks for the review. > * Applies cleanly to master (f330a6d). > * Regression tests updated and pass, including 'check-world'. > * Documenta

Re: [HACKERS] [TODO] Track number of files ready to be archived in pg_stat_archiver

Julien, The following is an initial review: * Applies cleanly to master (f330a6d). * Regression tests updated and pass, including 'check-world'. * Documentation updated and builds successfully. * Might want to consider replacing the following magic number with a constant or perhaps calculated val

Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

On 17 October 2014 14:05, Alvaro Herrera wrote: > Of course, this is a task that requires much more thinking, design, and > discussion than just adding multi-process capability to vacuumdb ... Yes, please proceed with this patch as originally envisaged. No more comments from me. -- Simon Rigg

Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

Amit Kapila wrote: > On Fri, Oct 17, 2014 at 1:31 AM, Simon Riggs wrote: > > > > On 16 October 2014 15:09, Amit Kapila wrote: > > c) seems like the only issue that needs any thought. I don't think its > > going to be that hard. > > > > I don't see any problems with the other points. You can make

Re: [HACKERS] TODO : Allow parallel cores to be used by vacuumdb [ WIP ]

On 17 October 2014 12:52, Amit Kapila wrote: > It is quite possible, but still I think to accomplish such a function, > we need to have some mechanism where it can inform auto vacuum > and then some changes in auto vacuum to receive/read that information > and reply back. I don't think any such

  1   2   3   4   5   6   7   8   9   10   >