Hi Dilip, Amit,
> > 5. Can you please once repeat the performance test done by Keisuke-San
> > to see if you have similar observations? Additionally, see if you are
> > also seeing the inconsistency related to the Truncate message reported
> > by him and if so why?
> >
>
> Okay, I will set up and
At Fri, 2 Oct 2020 09:00:53 +0900, Etsuro Fujita
wrote in
> On Tue, Sep 29, 2020 at 4:45 AM Etsuro Fujita wrote:
> > BTW: I noticed that you changed the ExecProcNode() API so that an
> > Append calling FDWs can know wether they return tuples immediately or
> > not:
>
> > That is, 1) in postgre
At Fri, 02 Oct 2020 13:51:35 +0900 (JST), Kyotaro Horiguchi
wrote in
> Sorry for the slippery brain... ^2
> At Fri, 02 Oct 2020 13:38:22 +0900 (JST), Kyotaro Horiguchi
> wrote in
> > At Fri, 2 Oct 2020 10:56:21 +0900, Fujii Masao
> > wrote in
> > >
> > > On 2020/10/02 10:06, Kyotaro Hor
From: Fujii Masao
> > The speedup has already been achieved with higher durability by
> > wal_level=minimal in that case.
>
> I was thinking the same, i.e., wal_level=minimal + wal_skip_threshold would
> speed up that initial data loading.
First of all, thank you Horiguchi-san for trying to impr
Sorry for the slippery brain...
At Fri, 02 Oct 2020 13:38:22 +0900 (JST), Kyotaro Horiguchi
wrote in
> At Fri, 2 Oct 2020 10:56:21 +0900, Fujii Masao
> wrote in
> >
> > On 2020/10/02 10:06, Kyotaro Horiguchi wrote:
> > > 1. Flip BM_PERMANENT of active buffers
> > > 2. adding/removing init f
At Fri, 02 Oct 2020 11:44:46 +0900 (JST), Kyotaro Horiguchi
wrote in
> At Thu, 1 Oct 2020 12:55:34 +, "k.jami...@fujitsu.com"
> wrote in
> - * XXX currently it sequentially searches the buffer pool, should
> be
> - * changed to more clever ways of searching. However,
At Fri, 2 Oct 2020 10:56:21 +0900, Fujii Masao
wrote in
>
> On 2020/10/02 10:06, Kyotaro Horiguchi wrote:
> > At Thu, 1 Oct 2020 08:14:42 +, "osumi.takami...@fujitsu.com"
> > wrote in
> >> When I compare the 2 ideas,
> >> one of the benefits of this ALTER TABLE 's improvement
> >> is that
On 2020-10-02 10:21, Fujii Masao wrote:
On 2020/10/01 13:35, Fujii Masao wrote:
On 2020/10/01 12:56, Masahiro Ikeda wrote:
On 2020-10-01 11:33, Amit Kapila wrote:
On Thu, Oct 1, 2020 at 6:53 AM Kyotaro Horiguchi
wrote:
At Thu, 1 Oct 2020 09:05:19 +0900, Fujii Masao
wrote in
>
>
> On 20
On Tue, Sep 29, 2020 at 9:12 PM Thomas Munro wrote:
> On Tue, Sep 29, 2020 at 7:11 PM Michael Paquier wrote:
> > #2 0x009027d2 in ExceptionalCondition
> > (conditionName=conditionName@entry=0xa80846 "!barrier->static_party",
>
> > #4 0x00682ebf in ExecParallelHashJoinNewBatch
>
At Wed, 30 Sep 2020 22:38:59 -0700, Noah Misch wrote in
noah> Perhaps wal_level=minimal should stop its pedantic call for
max_wal_senders=0.
noah> As long as the relevant error messages are clear, it would be fine for
noah> wal_level=minimal to ignore max_wal_senders and size resources as thou
At Thu, 1 Oct 2020 12:55:34 +, "k.jami...@fujitsu.com"
wrote in
> On Thursday, October 1, 2020 4:52 PM, Tsunakawa-san wrote:
>
> > From: Kyotaro Horiguchi
> > > I thought that the advantage of this optimization is that we don't
> > > need to visit all buffers? If we need to run a full-sc
On 2020/10/02 3:26, Andres Freund wrote:
Hi Ian, Andrew, All,
On 2020-09-30 15:43:17 -0700, Andres Freund wrote:
Attached is an updated version of the test (better utility function,
stricter regexes, bailing out instead of failing just the current when
psql times out). I'm leaving it in this t
On 2020/10/02 10:06, Kyotaro Horiguchi wrote:
At Thu, 1 Oct 2020 08:14:42 +, "osumi.takami...@fujitsu.com"
wrote in
Hi, Horiguchi-San and Fujii-San.
Thank you so much both of you.
the table needs to be rewriitten. One idea for that is to improve that
command so that it skips the tabl
On 2020/10/01 13:35, Fujii Masao wrote:
On 2020/10/01 12:56, Masahiro Ikeda wrote:
On 2020-10-01 11:33, Amit Kapila wrote:
On Thu, Oct 1, 2020 at 6:53 AM Kyotaro Horiguchi
wrote:
At Thu, 1 Oct 2020 09:05:19 +0900, Fujii Masao
wrote in
>
>
> On 2020/09/30 20:21, Amit Kapila wrote:
> >
Mark Wong writes:
> I'm getting Tom set up with access too, in case he has time before me to
> get a stack trace to see what's happening...
tl;dr: it's hard to conclude that this is anything but a compiler bug.
I was able to reproduce this on shoveler's host, but only when using
the compiler sho
On Thu, Oct 1, 2020 at 6:08 PM Tomas Vondra
wrote:
>
> On Thu, Oct 01, 2020 at 09:02:57AM -0400, James Coleman wrote:
> >On Thu, Oct 1, 2020 at 3:09 AM Jaime Casanova
> > wrote:
> >>
> >> On Wed, 30 Sep 2020 at 21:21, James Coleman wrote:
> >> >
> >> > On Sat, Sep 26, 2020 at 2:49 PM Jaime Casano
At Thu, 1 Oct 2020 08:14:42 +, "osumi.takami...@fujitsu.com"
wrote in
> Hi, Horiguchi-San and Fujii-San.
>
>
> Thank you so much both of you.
> > > the table needs to be rewriitten. One idea for that is to improve that
> > > command so that it skips the table rewrite if wal_level=minimal.
From: Fujii Masao
> I think that we can improve that, for example, by storing backend id
> into WalSndCtl and making pg_stat_get_wal_senders() directly
> get the walsender's LocalPgBackendStatus with the backend id,
> rather than joining pg_stat_get_activity() and pg_stat_get_wal_senders().
Yeah,
From: Michael Paquier
> CRC calculation would unlikely be the bottleneck here, no? I would assume
> that the extra lseek() calls needed to look after the record data to be more
> harmful.
Maybe, although I'm not sure lseek() is necessary. I simply thought walsender
was designed to just read an
On Thu, Oct 01, 2020 at 11:18:30AM +0200, Drouvot, Bertrand wrote:
> Had a look at it and did a few tests: looks all good to me.
>
> No objections at all, thanks!
Thanks for double-checking. Applied, then.
--
Michael
signature.asc
Description: PGP signature
On 2020/10/01 10:50, tsunakawa.ta...@fujitsu.com wrote:
From: Kyotaro Horiguchi
Another reason that I mildly want to object to subdivided functions is
I was annoyed that a stats view makes many individual calls to
functions that internally share the same statistics entry. That
behavior requ
On Fri, Oct 02, 2020 at 12:16:25AM +, tsunakawa.ta...@fujitsu.com wrote:
> IIUC, walsender tries hard to send WAL as fast as possible to reduce
> replication lag and transaction response time, so it doesn't try to
> peek each WAL record. I think it's good.
CRC calculation would unlikely be th
From: Konstantin Knizhnik
> Investigating one of customer's support cases I found out that walsender
> is not calculating WAL records CRC and send them to replicas without any
> checks.
> As a result damaged WAL record causes errors on all replicas:
IIUC, walsender tries hard to send WAL as fast
On Tue, Sep 29, 2020 at 4:45 AM Etsuro Fujita wrote:
> BTW: I noticed that you changed the ExecProcNode() API so that an
> Append calling FDWs can know wether they return tuples immediately or
> not:
> That is, 1) in postgresIterateForeignScan() postgres_fdw sets the new
> PlanState’s flag asyncs
On Tue, Sep 29, 2020 at 8:38 PM David G. Johnston <
david.g.johns...@gmail.com> wrote:
> On Tue, Sep 29, 2020 at 7:58 PM Tom Lane wrote:
>
>> Maybe we could use terminology along the lines of "added to the
>> queue" and "removed from the queue"?
>>
>
> Quickly looking it over with this in mind th
On Thu, Oct 01, 2020 at 02:58:53PM -0400, Tom Lane wrote:
> Hmm ... I think that that is pretty standard behavior for a lot of
> our utility commands. Trying something at random,
The behavior handling is a bit inconsistent. For example EXPLAIN and
VACUUM don't do that, because their parenthesize
On 10/1/20 4:22 PM, Andres Freund wrote:
> Hi,
>
> On 2020-10-01 16:00:20 -0400, Andrew Dunstan wrote:
>> My strong suspicion is that we're getting unwanted CRs. Note the
>> presence of numerous instances of this in PostgresNode.pm:
>
>> $stdout =~ s/\r\n/\n/g if $Config{osname} eq 'msys';
>>
On Thu, Oct 01, 2020 at 12:12:44PM -0700, Andres Freund wrote:
> Hi Mark,
>
> shoveler has been failing for a while with an odd error. E.g.
> https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=shoveler&dt=2020-09-18%2014%3A01%3A48
>
> Illegal instruction
> pg_dumpall: error: pg_dump failed o
On 2020-Oct-01, Tomas Vondra wrote:
> OK, so this seems like a data corruption bug in BRIN, actually.
Oh crap. You're right -- the data needs to be detoasted before being
put in the index.
I'll have a look at how this can be fixed.
On Thu, Oct 01, 2020 at 09:02:57AM -0400, James Coleman wrote:
On Thu, Oct 1, 2020 at 3:09 AM Jaime Casanova
wrote:
On Wed, 30 Sep 2020 at 21:21, James Coleman wrote:
>
> On Sat, Sep 26, 2020 at 2:49 PM Jaime Casanova
> wrote:
> >
> > Hi,
> >
> > With sqlsmith I found a query that gives this
Hi,
On 2020-10-01 16:44:03 -0400, Andrew Dunstan wrote:
> > I assume it's not, as the comments says
> > # Note: on Windows, IPC::Run seems to convert \r\n to \n in program
> > output
> > # if we're using native Perl, but not if we're using MSys Perl. So do
> > it
> > # by hand in th
On 10/1/20 4:22 PM, Andres Freund wrote:
> Hi,
>
> On 2020-10-01 16:00:20 -0400, Andrew Dunstan wrote:
>> My strong suspicion is that we're getting unwanted CRs. Note the
>> presence of numerous instances of this in PostgresNode.pm:
>
>> $stdout =~ s/\r\n/\n/g if $Config{osname} eq 'msys';
>>
Hi,
On 2020-10-01 16:00:20 -0400, Andrew Dunstan wrote:
> My strong suspicion is that we're getting unwanted CRs. Note the
> presence of numerous instances of this in PostgresNode.pm:
> $stdout =~ s/\r\n/\n/g if $Config{osname} eq 'msys';
>
> So you probably want something along those lines
On 10/1/20 2:26 PM, Andres Freund wrote:
> Hi Ian, Andrew, All,
>
> On 2020-09-30 15:43:17 -0700, Andres Freund wrote:
>> Attached is an updated version of the test (better utility function,
>> stricter regexes, bailing out instead of failing just the current when
>> psql times out). I'm leaving
Hi Mark,
shoveler has been failing for a while with an odd error. E.g.
https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=shoveler&dt=2020-09-18%2014%3A01%3A48
Illegal instruction
pg_dumpall: error: pg_dump failed on database "template1", exiting
waiting for server to shut down done
Non
"Daniel Verite" writes:
> Currently, it's not an error for CREATE COLLATION to be invoked
> with options repeated several times. The last (rightmost) value is kept
> and the others are lost.
> ...
> I suggest the attached simple patch to raise an error when any of
> these options is specified mult
Hi,
Currently, it's not an error for CREATE COLLATION to be invoked
with options repeated several times. The last (rightmost) value is kept
and the others are lost.
For instance CREATE COLLATION x (lc_ctype='en_US.UTF8',
lc_collate='en_US.UTF8', lc_ctype='C')
silently ignores lc_ctype='en_US.UTF
On Wed, Sep 30, 2020 at 07:57:19AM -0400, John Naylor wrote:
On Mon, Sep 28, 2020 at 10:12 PM Tomas Vondra
wrote:
Is it actually all that different from the existing BRIN indexes?
Consider this example:
create table x (a text, b text, c text);
create index on x using brin (a,b,c);
create or
Hi Ian, Andrew, All,
On 2020-09-30 15:43:17 -0700, Andres Freund wrote:
> Attached is an updated version of the test (better utility function,
> stricter regexes, bailing out instead of failing just the current when
> psql times out). I'm leaving it in this test for now, but it's fairly
> easy to
On Thu, Oct 1, 2020 at 11:19 AM Tom Lane wrote:
> I'm unsure this topic is worth messing with. But if we do so, I'd kind
> of like to move scanstr() out of parser/scansup.c. Since it's not used
> by the core scanner, only bootstrap, it seems out of place there; and
> it's confusing because someb
Given the plan example:
CREATE TABLE measurement (
city_id int not null,
logdate date not null,
peaktempint,
unitsales int
) PARTITION BY RANGE (logdate);
CREATE TABLE measurement_y2006m02 PARTITION OF measurement
FOR VALUES FROM ('2006-02-01') TO
On 30.09.2020 22:58, Rahila Syed wrote:
Hi Anastasia,
I tested the syntax with some basic commands and it works fine,
regression tests also pass.
Thank you for your review.
Couple of comments:
1. The syntax used omits the { IMMEDIATE | DEFERRED} keywords
suggested in
the earlier discussion
On Thu, Oct 1, 2020 at 8:10 PM Fujii Masao wrote:
>
> pg_stat_clear_snapshot() can be used to reset the entry.
>
Thanks. I wasn't knowing it.
>
> + EXIT WHEN proccnt = 0;
> +END LOOP;
>
> Isn't it better to sleep here, to avoid th busy loop?
>
+1.
>
> So what I thought was so
Hi hackers,
Investigating one of customer's support cases I found out that walsender
is not calculating WAL records CRC and send them to replicas without any
checks.
As a result damaged WAL record causes errors on all replicas:
LOG: incorrect resource manager data checksum in record a
John Naylor writes:
> In the attached, the GUC_scanstr() function body is moved to scansup.c
> to replace scanstr(), but with a different order of if-statements to
> make the diff smaller. Since we have control over what goes in the BKI
> file, we can use single-quoted escape strings there, allowi
On 2020/10/01 21:14, Bharath Rupireddy wrote:
On Wed, Sep 30, 2020 at 11:32 PM Fujii Masao
wrote:
And another way, if we don't want to use wait_pid() is to have a
plpgsql stored procedure, that in a loop keeps on checking for the
backed pid from pg_stat_activity, exit when pid is 0. and th
On Thu, Oct 1, 2020 at 4:44 PM Amit Kapila wrote:
>
> On Thu, Oct 1, 2020 at 10:12 AM Dilip Kumar wrote:
> >
> > On Wed, Sep 30, 2020 at 3:00 PM Amit Kapila wrote:
> > >
> > > On Wed, Sep 30, 2020 at 12:16 PM Dilip Kumar
> > > wrote:
> >
> > > And you might need to update the below comment as
Michael Paquier writes:
> And one month later, here we are with the following results and a CF
> now closed:
> Committed: 48
> Moved to next CF: 147
> Withdrawn: 4
> Rejected: 3
> Returned with Feedback: 33
> Total: 235
Once again, thanks for doing all that tedious work!
On Wed, 30 Sep 2020 at 16:02, tsunakawa.ta...@fujitsu.com
wrote:
>
> From: Masahiko Sawada
> > To avoid misunderstanding, I didn't mean to disregard the performance.
> > I mean especially for the transaction management feature it's
> > essential to work fine even in failure cases. So I hope we ha
Michael Paquier writes:
> On Wed, Sep 30, 2020 at 10:38:59PM -0700, Noah Misch wrote:
>> In favor of minimal values, we've had semaphore-starved buildfarm members in
>> the past. Perhaps those days are over, seeing that this commit has not yet
>> broken a buildfarm member in that particular way.
On Thu, Oct 1, 2020 at 3:09 AM Jaime Casanova
wrote:
>
> On Wed, 30 Sep 2020 at 21:21, James Coleman wrote:
> >
> > On Sat, Sep 26, 2020 at 2:49 PM Jaime Casanova
> > wrote:
> > >
> > > Hi,
> > >
> > > With sqlsmith I found a query that gives this error:
> > > ERROR: ORDER/GROUP BY expression n
On Thursday, October 1, 2020 4:52 PM, Tsunakawa-san wrote:
> From: Kyotaro Horiguchi
> > I thought that the advantage of this optimization is that we don't
> > need to visit all buffers? If we need to run a full-scan for any
> > reason, there's no point in looking-up already-visited buffers aga
On Wed, Sep 30, 2020 at 11:32 PM Fujii Masao
wrote:
>
> > And another way, if we don't want to use wait_pid() is to have a
> > plpgsql stored procedure, that in a loop keeps on checking for the
> > backed pid from pg_stat_activity, exit when pid is 0. and then proceed
> > to issue the next foreign
Hi,
Looking at the guc file code, GUC_scanstr() is almost the same as the
exported function scanstr(), except the latter requires finicky extra
coding around double quotes both in its callers and while creating the
input.
In the attached, the GUC_scanstr() function body is moved to scansup.c
to r
> On 1 Oct 2020, at 12:54, Heikki Linnakangas wrote:
> Most checks when converting between SQL and Python types use the PLy_elog()
> function, which uses the genericc ERRCODE_EXTERNAL_ROUTINE_EXCEPTION error
> code, but I think ERRCODE_ARRAY_SUBSCRIPT_ERROR is better.
On that note, wouldn't th
On Thu, Oct 1, 2020 at 10:12 AM Dilip Kumar wrote:
>
> On Wed, Sep 30, 2020 at 3:00 PM Amit Kapila wrote:
> >
> > On Wed, Sep 30, 2020 at 12:16 PM Dilip Kumar wrote:
>
> > And you might need to update the below comment as well:
> > typedef struct ReorderBufferTXN
> > {
> > ..
> > /*
> > * List o
In PLySequence_ToArray_recurse(), there's this check:
if (PySequence_Length(list) != dims[dim])
ereport(ERROR,
(errmsg("wrong length of inner sequence: has length
%d, but %d was expected",
(i
On Thu, Oct 1, 2020 at 2:43 PM Keisuke Kuroda
wrote:
>
> > > I test the patch on the master HEAD(9796f455) and it works fine.
> > > * make installcheck-world: passed
> > > * walsender process continues to use 100% of the CPU 1core when
> > > TRUNCATE 1000 partition: 1s or less
> > > ** before patc
On Tue, 15 Sep 2020 at 07:17, Andres Freund wrote:
>
> Hi,
>
> On 2020-09-09 17:02:58 +0900, Ian Barwick wrote:
> > Attached, though bear in mind I'm not very familiar with parts of this,
> > particularly 2PC stuff, so consider it educated guesswork.
>
> Thanks for this, and the test case!
>
> You
> > I test the patch on the master HEAD(9796f455) and it works fine.
> > * make installcheck-world: passed
> > * walsender process continues to use 100% of the CPU 1core when
> > TRUNCATE 1000 partition: 1s or less
> > ** before patch : 230s
>
> Does this result indicate that it is still CPU bound
Hi, Horiguchi-San and Fujii-San.
Thank you so much both of you.
> > the table needs to be rewriitten. One idea for that is to improve that
> > command so that it skips the table rewrite if wal_level=minimal.
> > Of course, also you can change wal_level after marking the table as
> > unlogged.
>
On Wed, Sep 30, 2020 at 10:38:59PM -0700, Noah Misch wrote:
> In favor of minimal values, we've had semaphore-starved buildfarm members in
> the past. Perhaps those days are over, seeing that this commit has not yet
> broken a buildfarm member in that particular way. Keeping max_wal_senders=10
>
This patch fixes compilation on windows and compilation of the documentation.
Matthieu Garrigues
On Thu, Oct 1, 2020 at 8:41 AM Matthieu Garrigues
wrote:
>
> On Thu, Oct 1, 2020 at 6:35 AM Michael Paquier wrote:
>
> > The documentation is failing to build, and the patch does not build
> > corre
From: Kyotaro Horiguchi
> I thought that the advantage of this optimization is that we don't
> need to visit all buffers? If we need to run a full-scan for any
> reason, there's no point in looking-up already-visited buffers
> again. That's just wastefull cycles. Am I missing somethig?
>
> I do
At Thu, 1 Oct 2020 13:37:29 +0900, Michael Paquier wrote
in
> On Wed, Jan 22, 2020 at 02:38:19PM +0900, Kyotaro Horiguchi wrote:
> > I changed my mind to attach the benchmark patch as .txt file,
> > expecting the checkers not picking up it as a part of the patchset.
> >
> > I have in the precis
On Thu, Sep 24, 2020 at 03:03:46PM +0900, Michael Paquier wrote:
> Hmm. I still think that knowing at least about a FPW could be an
> interesting piece of information even here. Anyway, instead of
> copying a logic that exists already in xlog_outrec(), why not moving
> the block information print
On Wed, 30 Sep 2020 at 21:21, James Coleman wrote:
>
> On Sat, Sep 26, 2020 at 2:49 PM Jaime Casanova
> wrote:
> >
> > Hi,
> >
> > With sqlsmith I found a query that gives this error:
> > ERROR: ORDER/GROUP BY expression not found in targetlist
> >
[...]
> >
> > But if I set enable_incremental_s
Hi,
On Fri, Sep 25, 2020 at 4:28 PM torikoshia wrote:
> Thanks for all your comments, I updated the patch.
Thanks for updating the patch.
I did a brief test and code review.
> I added a shared hash table consisted of minimal members
> mainly for managing whether the file is dumped or not.
> Some
68 matches
Mail list logo
