2 |1.2
3 |1.2
4 |1.2
5 |1.2
6 |1.2
7 |1.2
...
9435 |1.2
9619 |1.2
9620 |1.2
( rows)
INSERT 0
Regards,
Greg Nancarrow
Fujitsu Australia
ther idea?
>
Note that the planner code updated by the patch does avoid creating a
Parallel INSERT plan in the case of inserting into a table with a
foreign key (so commandIds won't be created in the parallel-worker
code).
I'm not sure how to distinguish the "invoked from inside
Id(bool isSubXact)
>> TransactionId xid;
>>
>> /*
>> - * Workers synchronize transaction state at the beginning of each
>> parallel
>> - * operation, so we can't account for new XIDs after that point.
>> - */
>> - if (IsInParallelMode())
>> - elog(ERROR, "cannot assign TransactionIds during a parallel
>> operation");
>> -
>> - /*
>>* During bootstrap initialization, we return the special bootstrap
>>* transaction id.
>>*/
>
>Same thing, this code cannot just be allowed to be reachable. What
>prevents you from assigning two different xids from different workers
>etc?
At least in the case of Parallel INSERT, the leader for the Parallel
INSERT gets a new xid (GetCurrentFullTransactionId) and it is passed
through and assigned to each of the workers during their
initialization (so they are assigned the same xid).
Regards,
Greg Nancarrow
Fujitsu Australia
ave a RETURNING clause with INSERT INTO SELECT?
>
Hi Bharath,
Thanks for pointing out more cases I need to exclude and things I need
to investigate further.
I have taken note of them, and will do more testing and improvement.
At least RETURNING clause with INSERT INTO SELECT is working!
Regards,
Greg Nancarrow
Fujitsu Australia
n place, it doesn't allow
a xid to be acquired for the leader, without removing/changing that
parallel-mode check in GetNewTransactionId().
Regards,
Greg Nancarrow
Fujitsu Australia
esh, you are right, I need to test this and add it to the
list of further exclusions that the patch needs to check for.
Hopefully I can provide an updated patch soon that caters for these
additional identified cases.
Regards,
Greg Nancarrow
Fujitsu Australia
ble ugly
hack, in that where it needs to call GetCurrentFullTransactionId(), it
must temporarily escape parallel-mode (recalling that parallel-mode is
set true right at the top of ExectePlan() in the cases of Parallel
INSERT/SELECT).
Regards,
Greg Nancarrow
Fujitsu Australia
0003-ParallelInsertSelect.patch
Description: Binary data
Hi Vignesh and Bharath,
Seems like the Parallel Copy patch is regarding RI_TRIGGER_PK as
parallel-unsafe.
Can you explain why this is?
Regards,
Greg Nancarrow
Fujitsu Australia
not done and
finished, I'm still working on the costing, and haven't posted an
updated patch for it yet.
As far as cost_gather() method is concerned, for Parallel INSERT, it
can probably use the same costing approach as the CTAS patch except in
the case of a specified RETURNING clause.
Regards,
Greg Nancarrow
Fujitsu Australia
ct is an
enum, so accepts any int value).
Regards,
Greg Nancarrow
Fujitsu Australia
ed by Andres Freund)
that just allows the underlying SELECT part of "INSERT INTO ... SELECT
..." to be parallel.
Regards,
Greg Nancarrow
Fujitsu Australia
0004-ParallelInsertSelect.patch
Description: Binary data
0001-InsertParallelSelect.patch
Description: Binary data
new table type triggers?
>
My attached patch does not allow parallel INSERT if there are any
row-level triggers (as the trigger functions could see a different and
unpredictable table state compared to non-parallel INSERT, even if
otherwise parallel-safe).
Regards,
Greg Nancarrow
Fujitsu Australia
A4eK1%2BANNEaMJCCXm4naweP5PLY6LhJMvGo_V7-Pnfbh6GsOA%40mail.gmail.com
>
The patch I last posted for parallel INSERT does detect use of
transition tables in this case (trigdesc->trig_insert_new_table) and
disables INSERT parallelism (I tested it against Amit's example), yet
still otherwise allows AFTER STATEMENT triggers for parallel INSERT.
Regards,
Greg Nancarrow
Fujitsu Australia
t;nparts; i++)
{
if (get_rel_relkind(pd->oids[i]) == RELKIND_FOREIGN_TABLE)
{
table_close(rel, NoLock);
return false;
}
}
}
Thoughts?
Regards,
Greg Nancarrow
Fujitsu Australia
fer
anyway, so no need to ask for additional MemSet() of all buffer bytes
to 0 prior to memcpy().
Regards,
Greg Nancarrow
Fujitsu Australia
On Wed, Oct 7, 2020 at 7:25 PM Greg Nancarrow wrote:
>
> On Wed, Oct 7, 2020 at 12:40 AM Bharath Rupireddy
> wrote:
> > In parallel, we are not doing anything(due to the same reason
> > explained in above comment) to find whether there is a foreign
> > partition or no
for your prototype code for coordination of combo command IDs
with the workers.
It does give me the incentive to look beyond that issue and see
whether parallel Update and parallel Delete are indeed possible. I'll
be sure to give it a go!
> PS Why not use git format-patch to create patches?
Guess I was being a bit lazy - will use git format-patch in future.
Regards,
Greg Nancarrow
Fujitsu Australia
nks the data is not aligned, so maybe not a big deal. Oh well,
without alignment, that means use of memcpy() cannot really be avoided
here for serializing/de-serializing ints etc., let's hope the compiler
optimizes it as best it can.
Regards,
Greg Nancarrow
Fujitsu Australia
lues, the planner doesn't want to generate a
parallel plan. Lowering parallel-related GUCs like parallel_tuple_cost
(which I normally do for testing) influences it of course, but the
linear relationship still seems wrong.
Regards,
Greg Nancarrow
Fujitsu Australia
g purposes, provided this is
> * actually a safe thing to do.
> */
> - if (force_parallel_mode != FORCE_PARALLEL_OFF && top_plan->parallel_safe)
> + if (force_parallel_mode != FORCE_PARALLEL_OFF && parse->commandType
> == CMD_SELECT && top_plan->parallel_safe)
> {
> Gather*gather = makeNode(Gather);
>
> [1] -
> https://www.postgresql.org/message-id/CAA4eK1%2BE-pM0U6qw7EOF0yO0giTxdErxoJV9xTqN%2BLo9zdotFQ%40mail.gmail.com
>
Regards,
Greg Nancarrow
Fujitsu Australia
On Fri, Oct 9, 2020 at 8:41 PM Amit Kapila wrote:
>
> On Fri, Oct 9, 2020 at 2:37 PM Greg Nancarrow wrote:
> >
> > Speaking of costing, I'm not sure I really agree with the current
> > costing of a Gather node. Just considering a simple Parallel Se
GetCurrentTransactionId())
and the ERROR "cannot assign XIDs in a parallel worker" results.
GetCurrentFullTransactionId() must be called in the leader, somewhere
(and will be included in the transaction state that is serialized in
the parallel DSM).
If not done here, then where?
Regards,
Greg Nancarrow
Fujitsu Australia
llowing INSERT with parallel SELECT,
you're right, neither of those additional "commandType == CMD_SELECT"
checks are needed, so I'll remove them. (In the main patch, I think
the first check can be removed, once the XID handling is fixed; the
second check is definitely needed though).
Regards,
Greg Nancarrow
Fujitsu Australia
Let me know your thoughts on how to deal with these issues.
Can you see a problem with only having parallel-mode set for scope of
Gather/GatherMerge, or do you have some other idea?
Regards,
Greg Nancarrow
Fujitsu Australia
d as a leader backend does the final phase
> (merge the results by workers). So, we can only push the Select part
> of the statement.
>
Precisely, that's why I didn't make those changes for GatherMerge.
Regards,
Greg Nancarrow
Fujitsu Australia
* of the RETURNING tlist widths. But it's what happened
> historically,
> -* and improving it is a task for another day.
> -*/
> if (pathnode->path.rows > 0)
> total_size /= pathnode->path.rows;
> pathnode->path.pathtarget->width = rint(total_size);
Agree, thanks (bug in existing Postgres code, right?)
Regards,
Greg Nancarrow
Fujitsu Australia
On Mon, Oct 12, 2020 at 2:11 PM Thomas Munro wrote:
>
> On Mon, Oct 12, 2020 at 3:42 PM Greg Nancarrow wrote:
> > On Sun, Oct 11, 2020 at 1:39 PM Thomas Munro wrote:
> > > pathnode->path.total_cost += subpath->total_cost;
> > > -
ng just the
Select part of Insert parallel, I need to add checks (along the same
lines as for Parallel Insert) to avoid the parallel Select in certain
potentially-unsafe cases.
Regards,
Greg Nancarrow
Fujitsu Australia
ose need to be
addressed (extra checks required to avoid parallel SELECT in certain
cases).
Also, I'm seeing a partition-related error when running
installcheck-world - I'm investigating that.
Regards,
Greg Nancarrow
Fujitsu Australia
v2-0001-Enable-parallel-SELECT-for-INSERT-INTO-.-SELECT.patch
Description: Binary data
On Fri, Oct 16, 2020 at 3:43 PM Amit Kapila wrote:
>
> On Thu, Oct 15, 2020 at 6:13 PM Amit Kapila wrote:
> >
> > On Thu, Oct 15, 2020 at 9:56 AM Greg Nancarrow wrote:
> > >
> > > Also, I'm seeing a partition-related error when running
> >
k on the relation, rather than block until
potentially end of the other concurrent transaction in which the
exclusive lock is held.
Examples of when a relation is exclusively locked
(AccessExclusiveLock) until end-of-transaction include:
ALTER TABLE DROP COLUMN
ALTER TABLE ... RENAME
ALTE
401 - 431 of 431 matches
Mail list logo
