On 31.03.2017 13:48, Robert Haas wrote:
On Fri, Mar 31, 2017 at 3:33 AM, Konstantin Knizhnik
wrote:
It is possible to execute query concurrently using SPI?
If so, how it can be enforced?
I tried to open cursor with CURSOR_OPT_PARALLEL_OK flag but it doesn't help:
query is executed by single b
On Fri, Mar 31, 2017 at 4:18 PM, Robert Haas wrote:
> On Fri, Mar 31, 2017 at 3:33 AM, Konstantin Knizhnik
> wrote:
>> It is possible to execute query concurrently using SPI?
>> If so, how it can be enforced?
>> I tried to open cursor with CURSOR_OPT_PARALLEL_OK flag but it doesn't help:
>> query
On Fri, Mar 31, 2017 at 3:33 AM, Konstantin Knizhnik
wrote:
> It is possible to execute query concurrently using SPI?
> If so, how it can be enforced?
> I tried to open cursor with CURSOR_OPT_PARALLEL_OK flag but it doesn't help:
> query is executed by single backend while the same query been laun
Hi hackers,
It is possible to execute query concurrently using SPI?
If so, how it can be enforced?
I tried to open cursor with CURSOR_OPT_PARALLEL_OK flag but it doesn't
help: query is executed by single backend while the same query been
launched at top level uses parallel plan:
fsstate->
On Tue, Aug 16, 2016 at 1:05 AM, Rushabh Lathia
wrote:
> I agree, this make sense.
>
> Here is the patch to allocate worker instrumentation into same context
> as the regular instrumentation which is per-query context.
Looks good, committed. I am not sure it was a very good idea for
af33039317dd
On Mon, Aug 15, 2016 at 6:02 PM, Robert Haas wrote:
> On Sat, Aug 13, 2016 at 4:36 AM, Amit Kapila
> wrote:
> > AFAICS, your patch seems to be the right fix for this issue, unless we
> > need the instrumentation information during execution (other than for
> > explain) for some purpose.
>
> Hmm,
On Sat, Aug 13, 2016 at 4:36 AM, Amit Kapila wrote:
> AFAICS, your patch seems to be the right fix for this issue, unless we
> need the instrumentation information during execution (other than for
> explain) for some purpose.
Hmm, I disagree. It should be the job of
ExecParallelRetrieveInstrumen
On Sat, Aug 13, 2016 at 11:10 AM, Rushabh Lathia
wrote:
> Hi All,
>
> Recently while running tpc-h queries on postgresql master branch, I am
> noticed
> random server crash. Most of the time server crash coming while turn tpch
> query
> number 3 - (but its very random).
>
>
> Here its clear that w
Hi All,
Recently while running tpc-h queries on postgresql master branch, I am
noticed
random server crash. Most of the time server crash coming while turn tpch
query
number 3 - (but its very random).
Call Stack of server crash:
(gdb) bt
#0 0x102aa9ac in ExplainNode (planstate=0x1001a74
On Tue, Jul 5, 2016 at 3:59 PM, Peter Geoghegan wrote:
> On Tue, Jul 5, 2016 at 12:58 PM, Tom Lane wrote:
>> Perhaps we could change the wording of temp_file_limit's description
>> from "space that a session can use" to "space that a process can use"
>> to help clarify this?
>
> That's all that I
Peter Geoghegan writes:
> On Tue, Jul 5, 2016 at 12:00 PM, Robert Haas wrote:
>> I think that it is not worth mentioning specifically for
>> temp_file_limit; to me that seems to be a hole with no bottom. We'll
>> end up arguing about which GUCs should mention it specifically and
>> there will be
On Tue, Jul 5, 2016 at 12:58 PM, Tom Lane wrote:
> Perhaps we could change the wording of temp_file_limit's description
> from "space that a session can use" to "space that a process can use"
> to help clarify this?
That's all that I was looking for, really.
--
Peter Geoghegan
--
Sent via pg
On Tue, Jul 5, 2016 at 12:00 PM, Robert Haas wrote:
> I think that it is not worth mentioning specifically for
> temp_file_limit; to me that seems to be a hole with no bottom. We'll
> end up arguing about which GUCs should mention it specifically and
> there will be no end to it.
I don't think t
On Tue, Jul 5, 2016 at 1:58 PM, Peter Geoghegan wrote:
> On Tue, Jul 5, 2016 at 7:45 AM, Robert Haas wrote:
>> Since Peter doesn't seem in a hurry to produce a patch for this issue,
>> I wrote one. It is attached. I'll commit this in a day or two if
>> nobody objects.
>
> Sorry about the delay.
On Tue, Jul 5, 2016 at 7:45 AM, Robert Haas wrote:
> Since Peter doesn't seem in a hurry to produce a patch for this issue,
> I wrote one. It is attached. I'll commit this in a day or two if
> nobody objects.
Sorry about the delay.
Your patch seems reasonable, but I thought we'd also want to c
On Tue, Jun 21, 2016 at 8:15 AM, Robert Haas wrote:
> On Mon, Jun 20, 2016 at 11:01 PM, Tom Lane wrote:
>> Peter Geoghegan writes:
>>> On Wed, May 18, 2016 at 3:40 AM, Robert Haas wrote:
What I'm tempted to do is trying to document that, as a point of
policy, parallel query in 9.6 use
On Mon, Jun 20, 2016 at 11:01 PM, Tom Lane wrote:
> Peter Geoghegan writes:
>> On Wed, May 18, 2016 at 3:40 AM, Robert Haas wrote:
>>> What I'm tempted to do is trying to document that, as a point of
>>> policy, parallel query in 9.6 uses up to (workers + 1) times the
>>> resources that a single
Peter Geoghegan writes:
> On Wed, May 18, 2016 at 3:40 AM, Robert Haas wrote:
>> What I'm tempted to do is trying to document that, as a point of
>> policy, parallel query in 9.6 uses up to (workers + 1) times the
>> resources that a single session might use. That includes not only CPU
>> but al
On Wed, May 18, 2016 at 3:40 AM, Robert Haas wrote:
> What I'm tempted to do is trying to document that, as a point of
> policy, parallel query in 9.6 uses up to (workers + 1) times the
> resources that a single session might use. That includes not only CPU
> but also things like work_mem and tem
On Tue, Jun 7, 2016 at 8:32 AM, Robert Haas wrote:
> You previously offered to write a patch for this. Are you still
> planning to do that?
OK, I'll get to that in the next few days.
I'm slightly concerned that I might have missed a real problem in the
code. I'll need to examine the issue more
On Sun, Jun 5, 2016 at 4:32 PM, Peter Geoghegan wrote:
> On Wed, May 18, 2016 at 12:01 PM, Peter Geoghegan wrote:
>>> I think for 9.6 we just have to document this issue. In the next
>>> release, we could (and might well want to) try to do something more
>>> clever.
>>
>> Works for me. You may w
On Wed, May 18, 2016 at 12:01 PM, Peter Geoghegan wrote:
>> I think for 9.6 we just have to document this issue. In the next
>> release, we could (and might well want to) try to do something more
>> clever.
>
> Works for me. You may wish to update comments within fd.c at the same time.
I've crea
On Sun, May 22, 2016 at 10:36 AM, Tatsuo Ishii wrote:
>> The brief introudce of MPI(Message Passing Interface) as following URL,
>> which is a message protocol used for parallel computinng, just like DSM
>> does in parallel query. The DSM play a message passing role(in fact, it's.
>> by passing th
> The brief introudce of MPI(Message Passing Interface) as following URL,
> which is a message protocol used for parallel computinng, just like DSM
> does in parallel query. The DSM play a message passing role(in fact, it's.
> by passing the query plan/raw node tree to anthor worker) in parallel
>
What's MPI?
Best regards,
--
Tatsuo Ishii
SRA OSS, Inc. Japan
English: http://www.sraoss.co.jp/index_en.php
Japanese:http://www.sraoss.co.jp
> Maybe we can refere to the MPI test cases.
>
> On Sun, May 22, 2016 at 3:19 PM, Hao Lee wrote:
>
>> What kind of cases do you want to run? beside the m
Thank you for the suggesion. Currently no particular test cases are
in my mind. That's the reason why I need input from
community. Regarding the test schedule, PGECons starts the planning
from next month or so. So I guess test starts no earlier than July.
Best regards,
--
Tatsuo Ishii
SRA OSS, In
Robert,
(and others who are involved in parallel query of PostgreSQL)
PostgreSQL Enterprise Consortium (one of the PostgreSQL communities in
Japan, in short "PGECons") is planning to test the parallel query
performance of PostgreSQL 9.6. Besides TPC-H (I know you have already
tested on an IBM box)
On Wed, May 18, 2016 at 3:40 AM, Robert Haas wrote:
>> I'll write a patch to fix the issue, if there is a consensus on a solution.
>
> I think for 9.6 we just have to document this issue. In the next
> release, we could (and might well want to) try to do something more
> clever.
Works for me. Yo
On 18 May 2016 at 22:40, Robert Haas wrote:
> On Tue, May 17, 2016 at 6:40 PM, Peter Geoghegan wrote:
> > On Tue, May 17, 2016 at 3:33 PM, Peter Geoghegan wrote:
> >> Fundamentally, since temporary_files_size enforcement simply
> >> piggy-backs on low-level fd.c file management, without any
> >
On Tue, May 17, 2016 at 6:40 PM, Peter Geoghegan wrote:
> On Tue, May 17, 2016 at 3:33 PM, Peter Geoghegan wrote:
>> Fundamentally, since temporary_files_size enforcement simply
>> piggy-backs on low-level fd.c file management, without any
>> consideration of what the temp files contain, it'll be
On Tue, May 17, 2016 at 3:33 PM, Peter Geoghegan wrote:
> Fundamentally, since temporary_files_size enforcement simply
> piggy-backs on low-level fd.c file management, without any
> consideration of what the temp files contain, it'll be hard to be sure
> that parallel workers will not have issues.
On Tue, May 17, 2016 at 1:53 PM, Amit Kapila wrote:
> What kind of special treatment are you expecting for temporary_files_size,
> also why do you think it is required? Currently neither we build hash in
> parallel nor there is any form of parallel sort work.
I expect only that temporary_files_s
On Wed, May 18, 2016 at 12:55 AM, Peter Geoghegan wrote:
>
> temp_file_limit "specifies the maximum amount of disk space that a
> session can use for temporary files, such as sort and hash temporary
> files", according to the documentation. That's not true when parallel
> query is in use, since th
temp_file_limit "specifies the maximum amount of disk space that a
session can use for temporary files, such as sort and hash temporary
files", according to the documentation. That's not true when parallel
query is in use, since the global variable temporary_files_size
receives no special treatment
On Mon, Apr 18, 2016 at 09:56:28AM -0400, Robert Haas wrote:
> On Fri, Apr 15, 2016 at 12:45 AM, Jeff Janes wrote:
> > Should every relevant contrib extension get a version bump with a
> > transition file which is nothing but a list of "alter function blah
> > blah blah parallel safe" ?
>
> Yes,
On Fri, Apr 15, 2016 at 12:45 AM, Jeff Janes wrote:
> Should every relevant contrib extension get a version bump with a
> transition file which is nothing but a list of "alter function blah
> blah blah parallel safe" ?
Yes, I think that's what we would need to do. It's a lot of work,
albeit most
On 15 April 2016 at 12:45, Jeff Janes wrote:
> I think there are a lot of extensions which create functions which
> could benefit from being declared parallel safe. But how does one go
> about doing that?
>
> create extension xml2;
> select xml_valid(filler),count(*) from pgbench_accounts group
I think there are a lot of extensions which create functions which
could benefit from being declared parallel safe. But how does one go
about doing that?
create extension xml2;
select xml_valid(filler),count(*) from pgbench_accounts group by 1;
Time: 3205.830 ms
alter function xml_valid (text)
On Wed, Mar 9, 2016 at 12:34 AM, Robert Haas wrote:
> On Tue, Mar 8, 2016 at 8:23 AM, Michael Paquier
> wrote:
>> On Tue, Mar 8, 2016 at 9:51 PM, Craig Ringer wrote:
>>> On 8 March 2016 at 20:30, Ashutosh Sharma wrote:
While testing a parallel scan feature on standby server, it is fou
On Tue, Mar 8, 2016 at 8:23 AM, Michael Paquier
wrote:
> On Tue, Mar 8, 2016 at 9:51 PM, Craig Ringer wrote:
>> On 8 March 2016 at 20:30, Ashutosh Sharma wrote:
>>>
>>> While testing a parallel scan feature on standby server, it is found that
>>> the parallel query fails with an error "ERROR: f
On Tue, Mar 8, 2016 at 9:51 PM, Craig Ringer wrote:
> On 8 March 2016 at 20:30, Ashutosh Sharma wrote:
>>
>>
>> While testing a parallel scan feature on standby server, it is found that
>> the parallel query fails with an error "ERROR: failed to initialize
>> transaction_read_only to 0".
>>
>
>
On 8 March 2016 at 20:30, Ashutosh Sharma wrote:
>
> While testing a parallel scan feature on standby server, it is found that
> the parallel query fails with an error "*ERROR: failed to initialize
> transaction_read_only to 0*".
>
>
Looks like it might be a good idea to add some tests to src/te
Hi All,
While testing a parallel scan feature on standby server, it is found that
the parallel query fails with an error "*ERROR: failed to initialize
transaction_read_only to 0*".
Following are the steps used to reproduce the issue:
*Master :-*
edb=# create table ert(n int);
edb=# insert into
On Thu, Jan 24, 2013 at 02:34:49PM -0800, Paul Ramsey wrote:
> On Tuesday, January 15, 2013 at 2:14 PM, Bruce Momjian wrote:
>
> I mentioned last year that I wanted to start working on parallelism:
>
> https://wiki.postgresql.org/wiki/Parallel_Query_Execution
>
> I believe it is time
On Tuesday, January 15, 2013 at 2:14 PM, Bruce Momjian wrote:
> I mentioned last year that I wanted to start working on parallelism:
>
> https://wiki.postgresql.org/wiki/Parallel_Query_Execution
>
> I believe it is time to start adding parallel execution to the backend.
> We already have some pa
On Wednesday, January 16, 2013, Stephen Frost wrote:
> * Bruce Momjian (br...@momjian.us ) wrote:
> > I am not sure how a COPY could be easily parallelized, but I supposed it
> > could be done as part of the 1GB segment feature. People have
> > complained that COPY is CPU-bound, so it might be ve
On Wed, Jan 16, 2013 at 11:56:21PM -0300, Claudio Freire wrote:
> On Wed, Jan 16, 2013 at 11:44 PM, Bruce Momjian wrote:
> > On Wed, Jan 16, 2013 at 05:04:05PM -0800, Jeff Janes wrote:
> >> On Tuesday, January 15, 2013, Stephen Frost wrote:
> >>
> >> * Gavin Flower (gavinflo...@archidevsys.co.
On Wed, Jan 16, 2013 at 11:44 PM, Bruce Momjian wrote:
> On Wed, Jan 16, 2013 at 05:04:05PM -0800, Jeff Janes wrote:
>> On Tuesday, January 15, 2013, Stephen Frost wrote:
>>
>> * Gavin Flower (gavinflo...@archidevsys.co.nz) wrote:
>> > How about being aware of multiple spindles - so if the
On Wed, Jan 16, 2013 at 05:04:05PM -0800, Jeff Janes wrote:
> On Tuesday, January 15, 2013, Stephen Frost wrote:
>
> * Gavin Flower (gavinflo...@archidevsys.co.nz) wrote:
> > How about being aware of multiple spindles - so if the requested
> > data covers multiple spindles, then data c
On Wed, Jan 16, 2013 at 10:04 PM, Jeff Janes wrote:
>> Hmm...
>>
>> How about being aware of multiple spindles - so if the requested data
>> covers multiple spindles, then data could be extracted in parallel. This
>> may, or may not, involve multiple I/O channels?
>
>
>
> effective_io_concurrency
On Tuesday, January 15, 2013, Gavin Flower wrote:
> On 16/01/13 11:14, Bruce Momjian wrote:
>
> I mentioned last year that I wanted to start working on parallelism:
>
> https://wiki.postgresql.org/wiki/Parallel_Query_Execution
>
> Years ago I added thread-safety to libpq. Recently I added
On Tuesday, January 15, 2013, Stephen Frost wrote:
> * Gavin Flower (gavinflo...@archidevsys.co.nz ) wrote:
> > How about being aware of multiple spindles - so if the requested
> > data covers multiple spindles, then data could be extracted in
> > parallel. This may, or may not, involve multiple I
On Wed, Jan 16, 2013 at 07:57:01PM -0200, Dickson S. Guedes wrote:
> 2013/1/16 Bruce Momjian :
> > Wiki updated:
> >
> > https://wiki.postgresql.org/wiki/Parallel_Query_Execution
>
> Could we add CTE to that opportunities list? I think that some kind of
> queries in CTE queries could be ea
2013/1/16 Bruce Momjian :
> Wiki updated:
>
> https://wiki.postgresql.org/wiki/Parallel_Query_Execution
Could we add CTE to that opportunities list? I think that some kind of
queries in CTE queries could be easilly parallelized.
[]s
--
Dickson S. Guedes
mail/xmpp: gue...@guedesoft.net -
On Wed, Jan 16, 2013 at 10:06:51PM +0100, Pavel Stehule wrote:
> 2013/1/16 Stephen Frost :
> > * Bruce Momjian (br...@momjian.us) wrote:
> >> I am not sure how a COPY could be easily parallelized, but I supposed it
> >> could be done as part of the 1GB segment feature. People have
> >> complained
2013/1/16 Stephen Frost :
> * Bruce Momjian (br...@momjian.us) wrote:
>> I am not sure how a COPY could be easily parallelized, but I supposed it
>> could be done as part of the 1GB segment feature. People have
>> complained that COPY is CPU-bound, so it might be very interesting to
>> see if we c
* Bruce Momjian (br...@momjian.us) wrote:
> I am not sure how a COPY could be easily parallelized, but I supposed it
> could be done as part of the 1GB segment feature. People have
> complained that COPY is CPU-bound, so it might be very interesting to
> see if we could offload some of that parsin
On 01/16/2013 12:20 PM, Bruce Momjian wrote:
On Wed, Jan 16, 2013 at 09:05:39AM -0500, Andrew Dunstan wrote:
On 01/15/2013 11:32 PM, Bruce Momjian wrote:
On Wed, Jan 16, 2013 at 01:28:18PM +0900, Michael Paquier wrote:
On Wed, Jan 16, 2013 at 1:22 PM, Josh Berkus wrote:
Claudio, Stephe
On Wed, Jan 16, 2013 at 09:05:39AM -0500, Andrew Dunstan wrote:
>
> On 01/15/2013 11:32 PM, Bruce Momjian wrote:
> >On Wed, Jan 16, 2013 at 01:28:18PM +0900, Michael Paquier wrote:
> >>
> >>On Wed, Jan 16, 2013 at 1:22 PM, Josh Berkus wrote:
> >>
> >> Claudio, Stephen,
> >>
> >> It really
On Wed, Jan 16, 2013 at 08:11:06AM -0500, Robert Haas wrote:
> > We kind of do - when in a CF we should do reviewing of existing
> > patches, when outside a CF we should do discussions and work on new
> > features. It's on http://wiki.postgresql.org/wiki/CommitFest. It
> > doesn't specifically say
On Wed, Jan 16, 2013 at 01:48:29AM -0300, Alvaro Herrera wrote:
> Bruce Momjian escribió:
> > On Wed, Jan 16, 2013 at 01:28:18PM +0900, Michael Paquier wrote:
> > >
> > >
> > > On Wed, Jan 16, 2013 at 1:22 PM, Josh Berkus wrote:
> > >
> > > Claudio, Stephen,
> > >
> > > It really seems
On Wed, Jan 16, 2013 at 01:37:28PM +0900, Michael Paquier wrote:
>
>
> On Wed, Jan 16, 2013 at 1:32 PM, Bruce Momjian wrote:
>
> On Wed, Jan 16, 2013 at 01:28:18PM +0900, Michael Paquier wrote:
> >
> >
> > On Wed, Jan 16, 2013 at 1:22 PM, Josh Berkus wrote:
> >
> >
On Wed, Jan 16, 2013 at 08:42:29AM -0500, Stephen Frost wrote:
> * Daniel Farina (dan...@heroku.com) wrote:
> > I have been skimming the commitfest application, and unlike some of
> > the previous commitfests a huge number of patches have had review at
> > some point in time, but probably need more
On Wed, Jan 16, 2013 at 10:33 AM, Stephen Frost wrote:
> * Claudio Freire (klaussfre...@gmail.com) wrote:
>> Well, there's the fault in your logic. It won't be as linear.
>
> I really don't see how this has become so difficult to communicate.
>
> It doesn't have to be linear.
>
> We're currently d
On 01/15/2013 11:32 PM, Bruce Momjian wrote:
On Wed, Jan 16, 2013 at 01:28:18PM +0900, Michael Paquier wrote:
On Wed, Jan 16, 2013 at 1:22 PM, Josh Berkus wrote:
Claudio, Stephen,
It really seems like the areas where we could get the most "bang for the
buck" in parallelism wo
* Daniel Farina (dan...@heroku.com) wrote:
> I have been skimming the commitfest application, and unlike some of
> the previous commitfests a huge number of patches have had review at
> some point in time, but probably need more...so looking for the red
> "Nobody" in the 'reviewers' column probably
* Tom Lane (t...@sss.pgh.pa.us) wrote:
> In case you hadn't noticed, we've totally lost control of the CF
> process.
I concur.
> Quite aside from the lack of progress on closing CF3, major
> hackers who should know better are submitting significant new feature
> patches now, despite our agreeme
* Claudio Freire (klaussfre...@gmail.com) wrote:
> Well, there's the fault in your logic. It won't be as linear.
I really don't see how this has become so difficult to communicate.
It doesn't have to be linear.
We're currently doing massive amounts of parallel processing by hand
using partitioni
On Wed, Jan 16, 2013 at 6:52 AM, Magnus Hagander wrote:
> On Wed, Jan 16, 2013 at 12:03 AM, Bruce Momjian wrote:
>> On Tue, Jan 15, 2013 at 11:01:04PM +, Simon Riggs wrote:
>>> On 15 January 2013 22:55, Bruce Momjian wrote:
>>>
>>> >> Why is this being discussed now?
>>> >
>>> > It is for 9.
On Wed, Jan 16, 2013 at 2:07 AM, Tom Lane wrote:
> Alvaro Herrera writes:
>> There are still 34 items needing attention in CF3. I suggest that, if
>> you have some spare time, your help would be very much appreciated
>> there. The commitfest that started on Jan 15th has 65 extra items.
>> Anyth
On Wed, Jan 16, 2013 at 12:03 AM, Bruce Momjian wrote:
> On Tue, Jan 15, 2013 at 11:01:04PM +, Simon Riggs wrote:
>> On 15 January 2013 22:55, Bruce Momjian wrote:
>>
>> >> Why is this being discussed now?
>> >
>> > It is for 9.4 and will take months. I didn't think there was a better
>> > t
On Tue, Jan 15, 2013 at 11:07 PM, Tom Lane wrote:
> Alvaro Herrera writes:
>> There are still 34 items needing attention in CF3. I suggest that, if
>> you have some spare time, your help would be very much appreciated
>> there. The commitfest that started on Jan 15th has 65 extra items.
>> Anyt
Alvaro Herrera writes:
> There are still 34 items needing attention in CF3. I suggest that, if
> you have some spare time, your help would be very much appreciated
> there. The commitfest that started on Jan 15th has 65 extra items.
> Anything currently listed in CF3 can rightfully be considered
On Tuesday, January 15, 2013, Simon Riggs wrote:
> On 15 January 2013 22:55, Bruce Momjian >
> wrote:
>
> >> Why is this being discussed now?
> >
> > It is for 9.4 and will take months. I didn't think there was a better
> > time. We don't usually discuss features during beta testing.
>
> Bruce,
Bruce Momjian escribió:
> On Wed, Jan 16, 2013 at 01:28:18PM +0900, Michael Paquier wrote:
> >
> >
> > On Wed, Jan 16, 2013 at 1:22 PM, Josh Berkus wrote:
> >
> > Claudio, Stephen,
> >
> > It really seems like the areas where we could get the most "bang for the
> > buck" in paralle
On Wed, Jan 16, 2013 at 12:55 AM, Stephen Frost wrote:
>> If memory serves me correctly (and it does, I suffered it a lot), the
>> performance hit is quite considerable. Enough to make it "a lot worse"
>> rather than "not as good".
>
> I feel like we must not be communicating very well.
>
> If the
On Wed, Jan 16, 2013 at 1:32 PM, Bruce Momjian wrote:
> On Wed, Jan 16, 2013 at 01:28:18PM +0900, Michael Paquier wrote:
> >
> >
> > On Wed, Jan 16, 2013 at 1:22 PM, Josh Berkus wrote:
> >
> > Claudio, Stephen,
> >
> > It really seems like the areas where we could get the most "bang for
On Wed, Jan 16, 2013 at 01:28:18PM +0900, Michael Paquier wrote:
>
>
> On Wed, Jan 16, 2013 at 1:22 PM, Josh Berkus wrote:
>
> Claudio, Stephen,
>
> It really seems like the areas where we could get the most "bang for the
> buck" in parallelism would be:
>
> 1. Parallel sort
>
On Wed, Jan 16, 2013 at 1:22 PM, Josh Berkus wrote:
> Claudio, Stephen,
>
> It really seems like the areas where we could get the most "bang for the
> buck" in parallelism would be:
>
> 1. Parallel sort
> 2. Parallel aggregation (for commutative aggregates)
> 3. Parallel nested loop join (especia
Claudio, Stephen,
It really seems like the areas where we could get the most "bang for the
buck" in parallelism would be:
1. Parallel sort
2. Parallel aggregation (for commutative aggregates)
3. Parallel nested loop join (especially for expression joins, like GIS)
--
Josh Berkus
PostgreSQL Exp
* Josh Berkus (j...@agliodbs.com) wrote:
> Actually, thanks to much faster storage (think SSD, SAN), it's easily
> possible for PostgreSQL to become CPU-limited on a seq scan query, even
> when reading from disk.
Particularly with a complex filter being applied or if it's feeding into
something ab
>> but there will be
>> quite a few cases where it's much, much better.
>
> Just cached segments.
Actually, thanks to much faster storage (think SSD, SAN), it's easily
possible for PostgreSQL to become CPU-limited on a seq scan query, even
when reading from disk.
--
Josh Berkus
PostgreSQL Expe
* Claudio Freire (klaussfre...@gmail.com) wrote:
> On Wed, Jan 16, 2013 at 12:13 AM, Stephen Frost wrote:
> > Sequentially scanning the *same* data over and over is certainly
> > counterprouctive. Synchroscans fixed that, yes. That's not what we're
> > talking about though- we're talking about s
On Wed, Jan 16, 2013 at 12:13 AM, Stephen Frost wrote:
> * Claudio Freire (klaussfre...@gmail.com) wrote:
>> On Tue, Jan 15, 2013 at 8:19 PM, Bruce Momjian wrote:
>> > The 1GB idea is interesting. I found in pg_upgrade that file copy would
>> > just overwhelm the I/O channel, and that doing mult
* Claudio Freire (klaussfre...@gmail.com) wrote:
> On Tue, Jan 15, 2013 at 8:19 PM, Bruce Momjian wrote:
> > The 1GB idea is interesting. I found in pg_upgrade that file copy would
> > just overwhelm the I/O channel, and that doing multiple copies on the
> > same device had no win, but those were
On Tue, Jan 15, 2013 at 8:19 PM, Bruce Momjian wrote:
>> Given our row-based storage architecture, I can't imagine we'd do
>> anything other than take a row-based approach to this.. I would think
>> we'd do two things: parallelize based on partitioning, and parallelize
>> seqscan's across the ind
On Wed, Jan 16, 2013 at 09:11:20AM +0900, Michael Paquier wrote:
>
>
> On Wed, Jan 16, 2013 at 7:14 AM, Bruce Momjian wrote:
>
> I mentioned last year that I wanted to start working on parallelism:
>
> https://wiki.postgresql.org/wiki/Parallel_Query_Execution
>
> Years ago
On Wed, Jan 16, 2013 at 7:14 AM, Bruce Momjian wrote:
> I mentioned last year that I wanted to start working on parallelism:
>
> https://wiki.postgresql.org/wiki/Parallel_Query_Execution
>
> Years ago I added thread-safety to libpq. Recently I added two parallel
> execution paths to pg_u
On Tue, Jan 15, 2013 at 06:15:57PM -0500, Stephen Frost wrote:
> * Gavin Flower (gavinflo...@archidevsys.co.nz) wrote:
> > How about being aware of multiple spindles - so if the requested
> > data covers multiple spindles, then data could be extracted in
> > parallel. This may, or may not, involve
* Gavin Flower (gavinflo...@archidevsys.co.nz) wrote:
> How about being aware of multiple spindles - so if the requested
> data covers multiple spindles, then data could be extracted in
> parallel. This may, or may not, involve multiple I/O channels?
Yes, this should dovetail with partitioning and
On Wed, Jan 16, 2013 at 12:03:50PM +1300, Gavin Flower wrote:
> On 16/01/13 11:14, Bruce Momjian wrote:
>
> I mentioned last year that I wanted to start working on parallelism:
>
> https://wiki.postgresql.org/wiki/Parallel_Query_Execution
>
> Years ago I added thread-safety t
On Tue, Jan 15, 2013 at 11:01:04PM +, Simon Riggs wrote:
> On 15 January 2013 22:55, Bruce Momjian wrote:
>
> >> Why is this being discussed now?
> >
> > It is for 9.4 and will take months. I didn't think there was a better
> > time. We don't usually discuss features during beta testing.
>
On 16/01/13 11:14, Bruce Momjian wrote:
I mentioned last year that I wanted to start working on parallelism:
https://wiki.postgresql.org/wiki/Parallel_Query_Execution
Years ago I added thread-safety to libpq. Recently I added two parallel
execution paths to pg_upgrade. The first paral
On 15 January 2013 22:55, Bruce Momjian wrote:
>> Why is this being discussed now?
>
> It is for 9.4 and will take months. I didn't think there was a better
> time. We don't usually discuss features during beta testing.
Bruce, there are many, many patches on the queue. How will we ever get
to
On 15 January 2013 22:14, Bruce Momjian wrote:
> I mentioned last year that I wanted to start working on parallelism:
We don't normally begin discussing topics for next release just as a
CF is starting.
Why is this being discussed now?
--
Simon Riggs http://www.2ndQuadrant.
On Tue, Jan 15, 2013 at 10:53:29PM +, Simon Riggs wrote:
> On 15 January 2013 22:14, Bruce Momjian wrote:
>
> > I mentioned last year that I wanted to start working on parallelism:
>
> We don't normally begin discussing topics for next release just as a
> CF is starting.
>
> Why is this bei
On Tue, Jan 15, 2013 at 10:39:10PM +, Peter Geoghegan wrote:
> On 15 January 2013 22:14, Bruce Momjian wrote:
> > I believe it is time to start adding parallel execution to the backend.
> > We already have some parallelism in the backend:
> > effective_io_concurrency and helper processes. I t
On 15 January 2013 22:14, Bruce Momjian wrote:
> I believe it is time to start adding parallel execution to the backend.
> We already have some parallelism in the backend:
> effective_io_concurrency and helper processes. I think it is time we
> start to consider additional options.
A few months
* Bruce Momjian (br...@momjian.us) wrote:
> Parallelism isn't going to help all queries, in fact it might be just a
> small subset, but it will be the larger queries. The pg_upgrade
> parallelism only helps clusters with multiple databases or tablespaces,
> but the improvements are significant.
T
I mentioned last year that I wanted to start working on parallelism:
https://wiki.postgresql.org/wiki/Parallel_Query_Execution
Years ago I added thread-safety to libpq. Recently I added two parallel
execution paths to pg_upgrade. The first parallel path allows execution
of external bina
1 - 100 of 104 matches
Mail list logo
