Ora2Pg v19.1 has been released

2018-09-26 Thread Gilles Darold
Noumea, New Caledonia - September 27th, 2018 **Ora2Pg 19.1 released** Version 19.1 of Ora2Pg, a free and reliable tool used to migrate an Oracle database to PostgreSQL, has been officially released and is publicly available for download. This release fix several issues reported during the last m

Re: Problem while setting the fpw with SIGHUP

2018-09-26 Thread Amit Kapila
On Thu, Sep 27, 2018 at 10:34 AM Michael Paquier wrote: > > On Thu, Sep 27, 2018 at 10:03:59AM +0530, Amit Kapila wrote: > > I think, in this case, it might be advisable to just fix the problem > > (a) which is what has been reported originally in the thread and > > AFAICS, the fix for that is cle

Re: BUG #15307: Low numerical precision of (Co-) Variance

2018-09-26 Thread Madeleine Thompson
By the way, I did not see the discussion thread or notice that an author of the paper I mentioned and the reporter of the bug were the same person until after I wrote the review. Oops.

Re: BUG #15307: Low numerical precision of (Co-) Variance

2018-09-26 Thread Madeleine Thompson
The following review has been posted through the commitfest application: make installcheck-world: tested, passed Implements feature: tested, passed Spec compliant: not tested Documentation:not tested This is my first PostgreSQL review. Hopefully I'm getting it right. I

Re: Let's stop with the retail rebuilds of src/port/ files already

2018-09-26 Thread Michael Paquier
On Wed, Sep 26, 2018 at 09:24:48PM -0700, Andres Freund wrote: > On September 26, 2018 9:03:05 PM PDT, Tom Lane wrote: >> What I think would make sense is to push this and see what the >> buildfarm thinks of it. If there are unfixable problems then >> we won't have wasted time fleshing out the co

Re: Problem while setting the fpw with SIGHUP

2018-09-26 Thread Michael Paquier
On Thu, Sep 27, 2018 at 10:03:59AM +0530, Amit Kapila wrote: > I think, in this case, it might be advisable to just fix the problem > (a) which is what has been reported originally in the thread and > AFAICS, the fix for that is clear as compared to the problem (b). If > you agree, then we can dis

Re: Add RESPECT/IGNORE NULLS and FROM FIRST/LAST options

2018-09-26 Thread Andrew Gierth
> "Krasiyan" == Krasiyan Andreev writes: Krasiyan> I am using last version from more than two months ago in Krasiyan> production environment with real data and I didn't find any Krasiyan> bugs, so I'm marking this patch as ready for committer in the Krasiyan> commitfest app. Oliver (or a

Re: Add RESPECT/IGNORE NULLS and FROM FIRST/LAST options

2018-09-26 Thread Andrew Gierth
> "Tom" == Tom Lane writes: >> So I've tried to rough out a decision tree for the various options >> on how this might be implemented (discarding the "use precedence >> hacks" option). Opinions? Additions? Tom> I think it'd be worth at least drafting an implementation for the Tom> lexic

Re: Problem while setting the fpw with SIGHUP

2018-09-26 Thread Amit Kapila
On Wed, Sep 19, 2018 at 10:50 AM Michael Paquier wrote: > > Agreed. "If we need to do that in the start process," we need to > > change the shared flag and issue FPW_CHANGE always when the > > database state is different from configuration file, regardless > > of what StartXLOG() did until the poi

Re: Let's stop with the retail rebuilds of src/port/ files already

2018-09-26 Thread Andres Freund
Hi, On September 26, 2018 9:03:05 PM PDT, Tom Lane wrote: >Andres Freund writes: >> On 2018-09-26 19:10:40 -0400, Tom Lane wrote: >>> I'm getting tired of having to make fixes like ce4887bd0. I think >>> we should rearrange things so that src/port/ and src/common/ compile >>> all their files

Re: Performance improvements for src/port/snprintf.c

2018-09-26 Thread Andres Freund
On September 26, 2018 9:04:08 PM PDT, Thomas Munro wrote: >On Thu, Sep 27, 2018 at 3:55 PM Andrew Gierth > wrote: >> > "Andres" == Andres Freund writes: >> Andres> Hm, stb's results just for floating point isn't bad. The >above >> Andres> numbers were for %f %f. But as the minimal usage

Re: Performance improvements for src/port/snprintf.c

2018-09-26 Thread Andres Freund
On September 26, 2018 8:53:27 PM PDT, Andrew Gierth wrote: >> "Andres" == Andres Freund writes: > > Andres> Hm, stb's results just for floating point isn't bad. The above >Andres> numbers were for %f %f. But as the minimal usage would be about > Andres> the internal usage of dopr(), here'

Re: Let's stop with the retail rebuilds of src/port/ files already

2018-09-26 Thread Tom Lane
Andres Freund writes: > On 2018-09-26 19:10:40 -0400, Tom Lane wrote: >> I'm getting tired of having to make fixes like ce4887bd0. I think >> we should rearrange things so that src/port/ and src/common/ compile >> all their files a third time using shared-library-friendly switches, >> put them in

Re: Performance improvements for src/port/snprintf.c

2018-09-26 Thread Thomas Munro
On Thu, Sep 27, 2018 at 3:55 PM Andrew Gierth wrote: > > "Andres" == Andres Freund writes: > Andres> Hm, stb's results just for floating point isn't bad. The above > Andres> numbers were for %f %f. But as the minimal usage would be about > Andres> the internal usage of dopr(), here's compa

Re: Performance improvements for src/port/snprintf.c

2018-09-26 Thread Andrew Gierth
> "Andres" == Andres Freund writes: Andres> Hm, stb's results just for floating point isn't bad. The above Andres> numbers were for %f %f. But as the minimal usage would be about Andres> the internal usage of dopr(), here's comparing %.*f: Andres> snprintf time = 1324.87 ms total, 0.0002

Re: Performance improvements for src/port/snprintf.c

2018-09-26 Thread Tom Lane
Andres Freund writes: > On 2018-09-26 21:44:41 -0400, Tom Lane wrote: >> BTW, were you thinking of plugging in strfromd() inside snprintf.c, >> or just invoking it directly from float[48]out? The latter would >> presumably be cheaper, and it'd solve the most pressing performance >> problem, if no

Performance of the partitioning in the large scale

2018-09-26 Thread Kato, Sho
Hi, Thanks to your efforts the performance of the partitioned table has improved greatly. Since I evaluated the performance by combining the performance improvement patches proposed in PG12, I share it. The purpose of this evaluation is to organize performance issues of the table which is part

Re: Performance improvements for src/port/snprintf.c

2018-09-26 Thread Andres Freund
On 2018-09-26 21:44:41 -0400, Tom Lane wrote: > Andres Freund writes: > > Reading around the interwebz lead me to look at ryu > > > https://dl.acm.org/citation.cfm?id=3192369 > > https://github.com/ulfjack/ryu/tree/46f4c5572121a6f1428749fe3e24132c3626c946 > > > That's an algorithm that always ge

Re: Performance improvements for src/port/snprintf.c

2018-09-26 Thread Thomas Munro
On Thu, Sep 27, 2018 at 1:18 PM Andres Freund wrote: > On 2018-09-26 17:57:05 -0700, Andres Freund wrote: > > snprintf time = 1324.87 ms total, 0.000264975 ms per iteration > > pg time = 1434.57 ms total, 0.000286915 ms per iteration > > stbsp time = 552.14 ms total, 0.000110428 ms per iteration >

Re: Performance improvements for src/port/snprintf.c

2018-09-26 Thread Andres Freund
Hi, On 2018-09-26 21:30:25 -0400, Tom Lane wrote: > Here's a rebased version of <15785.1536776...@sss.pgh.pa.us>. > > I think we should try to get this reviewed and committed before > we worry more about the float business. It would be silly to > not be benchmarking any bigger changes against th

Re: Performance improvements for src/port/snprintf.c

2018-09-26 Thread Tom Lane
Andres Freund writes: > Reading around the interwebz lead me to look at ryu > https://dl.acm.org/citation.cfm?id=3192369 > https://github.com/ulfjack/ryu/tree/46f4c5572121a6f1428749fe3e24132c3626c946 > That's an algorithm that always generates the minimally sized > roundtrip-safe string output f

Re: Performance improvements for src/port/snprintf.c

2018-09-26 Thread Tom Lane
Here's a rebased version of <15785.1536776...@sss.pgh.pa.us>. I think we should try to get this reviewed and committed before we worry more about the float business. It would be silly to not be benchmarking any bigger changes against the low-hanging fruit here. regards, t

Re: Performance improvements for src/port/snprintf.c

2018-09-26 Thread Andres Freund
Hi, On 2018-09-26 17:57:05 -0700, Andres Freund wrote: > snprintf time = 1324.87 ms total, 0.000264975 ms per iteration > pg time = 1434.57 ms total, 0.000286915 ms per iteration > stbsp time = 552.14 ms total, 0.000110428 ms per iteration Reading around the interwebz lead me to look at ryu http

Re: Performance improvements for src/port/snprintf.c

2018-09-26 Thread Andres Freund
On 2018-09-26 17:40:22 -0700, Andres Freund wrote: > On 2018-09-26 20:25:44 -0400, Tom Lane wrote: > > Andres Freund writes: > > > On 2018-09-26 19:45:07 -0400, Tom Lane wrote: > > >> No, there are no additional layers that weren't there before --- > > >> snprintf.c's snprintf() slots in directly

Re: Performance improvements for src/port/snprintf.c

2018-09-26 Thread Andres Freund
On 2018-09-26 20:25:44 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2018-09-26 19:45:07 -0400, Tom Lane wrote: > >> No, there are no additional layers that weren't there before --- > >> snprintf.c's snprintf() slots in directly where the platform's did before. > > > Hm? What I mean is th

Re: Performance improvements for src/port/snprintf.c

2018-09-26 Thread Tom Lane
Andres Freund writes: > On 2018-09-26 19:45:07 -0400, Tom Lane wrote: >> No, there are no additional layers that weren't there before --- >> snprintf.c's snprintf() slots in directly where the platform's did before. > Hm? What I mean is that we can't realistically be faster with the > current arc

Re: Performance improvements for src/port/snprintf.c

2018-09-26 Thread Andres Freund
On 2018-09-26 19:45:07 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2018-09-26 15:04:20 -0700, Andres Freund wrote: > >> I assume this partially is just the additional layers of function calls > >> (psprintf, pvsnprintf, pg_vsnprintf, dopr) that are now done, in > >> addition to pretty mu

Re: Performance improvements for src/port/snprintf.c

2018-09-26 Thread Tom Lane
Andres Freund writes: > I kinda wonder if we shouldn't replace the non pg_* functions in > snprintf.c with a more modern copy of a compatibly licensed libc. Looks > to me like our implementation has forked off some BSD a fair while ago. Maybe, but the benchmarking I was doing last month didn't co

Re: Performance improvements for src/port/snprintf.c

2018-09-26 Thread Tom Lane
Andres Freund writes: > On 2018-09-26 15:04:20 -0700, Andres Freund wrote: >> I assume this partially is just the additional layers of function calls >> (psprintf, pvsnprintf, pg_vsnprintf, dopr) that are now done, in >> addition to pretty much the same work as before (i.e. sprintf("%.*f")). No,

Re: Performance improvements for src/port/snprintf.c

2018-09-26 Thread Andres Freund
On 2018-08-17 14:32:59 -0400, Tom Lane wrote: > I've been looking into the possible performance consequences of that, > in particular comparing snprintf.c to the library versions on macOS, > FreeBSD, OpenBSD, and NetBSD. While it held up well in simpler cases, > I noted that it was significantly s

Re: Let's stop with the retail rebuilds of src/port/ files already

2018-09-26 Thread Andres Freund
Hi, On 2018-09-26 19:10:40 -0400, Tom Lane wrote: > I'm getting tired of having to make fixes like ce4887bd0. I think > we should rearrange things so that src/port/ and src/common/ compile > all their files a third time using shared-library-friendly switches, > put them into new .a files, and hav

Re: Allowing printf("%m") only where it actually works

2018-09-26 Thread Tom Lane
Andres Freund writes: > Oh, come on. One can be disabled with a GUC, has (although not good > enough) intelligence when it switches on, the other has ... none of > that. Obviously performance is always a balancing act, but you'd be > pretty pissed at anybody else regressing performance in a non-f

Let's stop with the retail rebuilds of src/port/ files already

2018-09-26 Thread Tom Lane
I'm getting tired of having to make fixes like ce4887bd0. I think we should rearrange things so that src/port/ and src/common/ compile all their files a third time using shared-library-friendly switches, put them into new .a files, and have libpq and the ecpg libraries just include those libraries

Re: [PATCH] Improve geometric types

2018-09-26 Thread Tom Lane
Tomas Vondra writes: > On 09/26/2018 06:45 PM, Tom Lane wrote: >> gaur's not happy, but rather surprisingly, it looks like we're >> mostly OK elsewhere. Do you need me to trace down exactly what's >> going wrong on gaur? > Or you could just try doing > select '(0,0)'::point * '(-3,4)'::point

Re: Proposal for Signal Detection Refactoring

2018-09-26 Thread Michael Paquier
On Wed, Sep 26, 2018 at 09:50:00AM +0200, Chris Travers wrote: > On Wed, Sep 26, 2018 at 12:41 AM Michael Paquier > wrote: >> Agreed. Are there any objections with the proposal of changing the >> interruption pending flags in miscadmin.h to use sig_atomic_t? >> ClientConnectionLost would gain PGD

Re: Allowing printf("%m") only where it actually works

2018-09-26 Thread Andres Freund
On 2018-09-26 18:31:07 -0400, Tom Lane wrote: > Andres Freund writes: > > On 2018-09-26 17:41:36 -0400, Tom Lane wrote: > >> Well, if you're unhappy about snprintf.c's performance, you could review > >> https://commitfest.postgresql.org/19/1763/ > >> so I can push that. In my tests, that got us d

Re: Allowing printf("%m") only where it actually works

2018-09-26 Thread Tom Lane
Andres Freund writes: > On 2018-09-26 17:41:36 -0400, Tom Lane wrote: >> Well, if you're unhappy about snprintf.c's performance, you could review >> https://commitfest.postgresql.org/19/1763/ >> so I can push that. In my tests, that got us down to circa 10% penalty >> for float conversions. > Uh

Re: Performance improvements for src/port/snprintf.c

2018-09-26 Thread Andres Freund
On 2018-09-26 15:04:20 -0700, Andres Freund wrote: > On 2018-09-12 14:14:15 -0400, Tom Lane wrote: > > Alexander Kuzmenkov writes: > > > I benchmarked this, using your testbed and comparing to libc sprintf > > > (Ubuntu GLIBC 2.27-0ubuntu3) and another implementation I know [1], all > > > compiled

Re: Performance improvements for src/port/snprintf.c

2018-09-26 Thread Andres Freund
On 2018-09-12 14:14:15 -0400, Tom Lane wrote: > Alexander Kuzmenkov writes: > > I benchmarked this, using your testbed and comparing to libc sprintf > > (Ubuntu GLIBC 2.27-0ubuntu3) and another implementation I know [1], all > > compiled with gcc 5. > > Thanks for reviewing! > > The cfbot noticed

Re: Allowing printf("%m") only where it actually works

2018-09-26 Thread Andres Freund
Hi, On 2018-09-26 17:41:36 -0400, Tom Lane wrote: > Andres Freund writes: > > I'm not saying we shouldn't default to our printf - in fact I think we > > probably past due to use a faster float->string conversion than we > > portably get from the OS - but I don't think we can default to our > > sp

Re: transction_timestamp() inside of procedures

2018-09-26 Thread Peter Eisentraut
On 26/09/2018 17:54, Alvaro Herrera wrote: > What could be the use for the transaction timestamp? I think one of the > most important uses (at least in pg_stat_activity) is to verify that > transactions are not taking excessively long time to complete; that's > known to cause all sorts of trouble

Re: Allowing printf("%m") only where it actually works

2018-09-26 Thread Tom Lane
Andres Freund writes: > I'm not saying we shouldn't default to our printf - in fact I think we > probably past due to use a faster float->string conversion than we > portably get from the OS - but I don't think we can default to our > sprintf without doing something about the float conversion perf

Re: [PATCH] Improve geometric types

2018-09-26 Thread Tomas Vondra
On 09/26/2018 10:58 PM, Tom Lane wrote: > Tomas Vondra writes: >> Hmmm, interesting. It seems both failures happen in the chunk that >> multiplies paths with points, i.e. essentially point_mul_point. So it >> seems most platforms end up with > >> (0,0) * (-3,4) = (-0, 0) > >> while gaur a

Re: [PATCH] Improve geometric types

2018-09-26 Thread Tom Lane
Tomas Vondra writes: > Hmmm, interesting. It seems both failures happen in the chunk that > multiplies paths with points, i.e. essentially point_mul_point. So it > seems most platforms end up with > (0,0) * (-3,4) = (-0, 0) > while gaur apparently thinks it's (0,0). And indeed, that's what t

Re: Regression tests fail once XID counter exceeds 2 billion

2018-09-26 Thread James Coleman
Added to TODO: Add function to allow easier transaction id comparisons http://archives.postgresql.org/pgsql-hackers/2011-11/msg00786.php Did this ever happen? Or did it fall through the cracks?

Re: pgbench's expression parsing & negative numbers

2018-09-26 Thread Fabien COELHO
Hello Andres, +"9223372036854775808" { + /* yuk: special handling for minint */ + return MAXINT_PLUS_ONE_CONST; + } Yikes, do we really need this? If we dealt with - in the lexer, we sh

Re: pg_ls_tmpdir()

2018-09-26 Thread Laurenz Albe
Bossart, Nathan wrote: > Attached is a patch to add a pg_ls_tmpdir() function that lists the > contents of a specified tablespace's pgsql_tmp directory. This is > very similar to the existing pg_ls_logdir() and pg_ls_waldir() > functions. > > By providing more visibility into the temporary file d

Re: Allowing printf("%m") only where it actually works

2018-09-26 Thread Tom Lane
Andres Freund writes: > The strerror push, I assume it's that at least, broke something on icc: > https://buildfarm.postgresql.org/cgi-bin/show_log.pl?nm=fulmar&dt=2018-09-26%2018%3A00%3A16 Yeah. It looks like the problem is that configure's test for strerror_r's return type does not work on icc

Re: [PATCH] Improve geometric types

2018-09-26 Thread Tomas Vondra
On 09/26/2018 06:45 PM, Tom Lane wrote: > Tomas Vondra writes: >> Pushed. Now let's wait for the buildfarm to complain ... > > gaur's not happy, but rather surprisingly, it looks like we're > mostly OK elsewhere. Do you need me to trace down exactly what's > going wrong on gaur? > Hmmm, intere

Re: pgbench's expression parsing & negative numbers

2018-09-26 Thread Andres Freund
Hi, On 2018-08-10 10:24:29 +0200, Fabien COELHO wrote: > diff --git a/src/bin/pgbench/exprscan.l b/src/bin/pgbench/exprscan.l > index 5c1bd88128..e8faea3857 100644 > --- a/src/bin/pgbench/exprscan.l > +++ b/src/bin/pgbench/exprscan.l > @@ -191,16 +191,26 @@ notnull [Nn][Oo][Tt][Nn]

Re: pgsql: Remove absolete function TupleDescGetSlot().

2018-09-26 Thread Andres Freund
Hi, On 2018-09-26 12:41:38 +0900, Michael Paquier wrote: > On Tue, Sep 25, 2018 at 06:42:51PM -0700, Andres Freund wrote: > > My point is that FuncCallContext->slot isn't actually all that related > > to TupleDescGetSlot() and could be used entirely independently. > > That's fair. Why not just r

Re: transction_timestamp() inside of procedures

2018-09-26 Thread Daniel Verite
Tom Lane wrote: > I agree that it would be surprising for transaction timestamp to be newer > than statement timestamp. To me it's more surprising to start a new transaction and having transaction_timestamp() still pointing at the start of a previous transaction. This feels like a side-

Re: Allowing printf("%m") only where it actually works

2018-09-26 Thread Andres Freund
On 2018-09-26 12:09:34 -0700, Andres Freund wrote: > and then fails with: > xlc_r -qmaxmem=33554432 -D_LARGE_FILES=1 -DRANDOMIZE_ALLOCATED_MEMORY > -qnoansialias -g -O2 -qmaxmem=16384 -qsrcmsg -D_REENTRANT -D_THREAD_SAFE > -D_POSIX_PTHREAD_SEMANTICS -o libpq.so.5 libpq.a -Wl,-bE:libpq.exp > -L.

Re: Allowing printf("%m") only where it actually works

2018-09-26 Thread Andres Freund
On 2018-09-26 11:57:34 -0700, Andres Freund wrote: > On 2018-09-26 11:09:59 -0400, Tom Lane wrote: > > Michael Paquier writes: > > > On Tue, Sep 25, 2018 at 12:05:42PM -0400, Tom Lane wrote: > > >> Alvaro Herrera writes: > > >>> Actually I think it *is* useful to do it like this, because then the

Re: Allowing printf("%m") only where it actually works

2018-09-26 Thread Andres Freund
On 2018-09-26 11:09:59 -0400, Tom Lane wrote: > Michael Paquier writes: > > On Tue, Sep 25, 2018 at 12:05:42PM -0400, Tom Lane wrote: > >> Alvaro Herrera writes: > >>> Actually I think it *is* useful to do it like this, because then the > >>> user knows to fix the netmsg.dll problem so that they

Re: Implementing SQL ASSERTION

2018-09-26 Thread David Fetter
On Tue, Sep 25, 2018 at 12:04:12AM +0100, Joe Wildish wrote: > Hi Peter, > > > My feeling is that if we want to move forward on this topic, we need to > > solve the concurrency question first. All these optimizations for when > > we don't need to check the assertion are cool, but they are just >

Re: Query is over 2x slower with jit=on

2018-09-26 Thread Andres Freund
On 2018-09-26 23:45:35 +0530, Amit Khandekar wrote: > On Tue, 25 Sep 2018 at 14:17, Andres Freund wrote: > > > > On 2018-09-18 10:03:02 +0530, Amit Khandekar wrote: > > > Attached v3 patch that does the above change. > > > > Attached is a revised version of that patch. I've changed quite a few >

Re: Query is over 2x slower with jit=on

2018-09-26 Thread Amit Khandekar
On Tue, 25 Sep 2018 at 14:17, Andres Freund wrote: > > On 2018-09-18 10:03:02 +0530, Amit Khandekar wrote: > > Attached v3 patch that does the above change. > > Attached is a revised version of that patch. I've changed quite a few > things: > - I've reverted the split of "base" and "provider spec

Re: Allowing printf("%m") only where it actually works

2018-09-26 Thread Andres Freund
Hi, On 2018-09-24 13:18:47 -0400, Tom Lane wrote: > 0002 changes things so that we always use our snprintf, removing all the > configure logic associated with that. In the commit message you wrote: > Preliminary performance testing suggests that as it stands, snprintf.c is > faster than the nati

Re: Participate in GCI as a Mentor

2018-09-26 Thread Sarah Conway Schnurr
Tahir, Thank you for spending time thinking of task ideas! We are ideally attempting to think of tasks that could benefit PostgreSQL as a whole either through code, outreach/advocacy, quality assurance, or design based contributions, as that seems to be the trend for other tasks submitted by open

Re: transction_timestamp() inside of procedures

2018-09-26 Thread Andres Freund
On 2018-09-26 12:54:43 -0300, Alvaro Herrera wrote: > On 2018-Sep-26, Tom Lane wrote: > > > Alvaro Herrera writes: > > > On 2018-Sep-26, Tom Lane wrote: > > >> I agree that it would be surprising for transaction timestamp to be newer > > >> than statement timestamp. So for now at least, I'd be s

Re: [PATCH] Improve geometric types

2018-09-26 Thread Tom Lane
Tomas Vondra writes: > Pushed. Now let's wait for the buildfarm to complain ... gaur's not happy, but rather surprisingly, it looks like we're mostly OK elsewhere. Do you need me to trace down exactly what's going wrong on gaur? regards, tom lane

Re: Allowing printf("%m") only where it actually works

2018-09-26 Thread Tom Lane
I wrote: > While looking over the thread, I remembered I wanted to convert > strerror_r into a wrapper as well. Think I'll go do that next, > because really it'd be better for snprintf.c to be calling strerror_r > not strerror. So while chasing that, I realized that libpq contains its own version

Re: transction_timestamp() inside of procedures

2018-09-26 Thread Alvaro Herrera
On 2018-Sep-26, Tom Lane wrote: > Alvaro Herrera writes: > > On 2018-Sep-26, Tom Lane wrote: > >> I agree that it would be surprising for transaction timestamp to be newer > >> than statement timestamp. So for now at least, I'd be satisfied with > >> documenting the behavior. > > > Really? I t

Re: Online verification of checksums

2018-09-26 Thread Stephen Frost
Greetings, * Fabien COELHO (coe...@cri.ensmp.fr) wrote: > >Note that a short read isn't an error and falls under the 'new' blocks > >discussion above. > > I'm really unsure that a short read should really be coldly skipped: > > If the check is offline, then one file is in a very bad state, this

Re: Online verification of checksums

2018-09-26 Thread Fabien COELHO
Hello Stephen, I certainly don't see a lot of point in doing much more than what was discussed previously for 'new' blocks (counting them as skipped and moving on). Sure. An actual read() error (that is, a failure on a read() call such as getting back EIO), on the other hand, is something

Re: Online verification of checksums

2018-09-26 Thread Michael Banck
Hi, Am Mittwoch, den 26.09.2018, 10:54 -0400 schrieb Stephen Frost: > * Michael Banck (michael.ba...@credativ.de) wrote: > > Am Mittwoch, den 26.09.2018, 13:23 +0200 schrieb Fabien COELHO: > > > There are debatable changes of behavior: > > > > > > if (errno == ENOENT) return / continue... > >

Re: Online verification of checksums

2018-09-26 Thread Fabien COELHO
The patch is missing a documentation update. I've added that now. I think the only change needed was removing the "server needs to be offline" part? Yes, and also checking that the described behavior correspond to the new version. There are debatable changes of behavior: if (errno

Re: Allowing printf("%m") only where it actually works

2018-09-26 Thread Tom Lane
Michael Paquier writes: > On Tue, Sep 25, 2018 at 12:05:42PM -0400, Tom Lane wrote: >> Alvaro Herrera writes: >>> Actually I think it *is* useful to do it like this, because then the >>> user knows to fix the netmsg.dll problem so that they can continue to >>> investigate the winsock problem. If

Re: Global snapshots

2018-09-26 Thread Arseny Sher
Hello, Andrey Borodin writes: > I like the idea that with this patch set universally all postgres > instances are bound into single distributed DB, even if they never > heard about each other before :) This is just amazing. Or do I get > something wrong? Yeah, in a sense of xact visibility we c

Re: Online verification of checksums

2018-09-26 Thread Stephen Frost
Greetings, * Michael Banck (michael.ba...@credativ.de) wrote: > Am Mittwoch, den 26.09.2018, 13:23 +0200 schrieb Fabien COELHO: > > There are debatable changes of behavior: > > > > if (errno == ENOENT) return / continue... > > > > For instance, a file disappearing is ok online, but not so if

Re: [HACKERS] Bug in to_timestamp().

2018-09-26 Thread Liudmila Mantrova
On 09/22/2018 10:00 PM, Alexander Korotkov wrote: On Thu, Sep 20, 2018 at 3:52 PM Alexander Korotkov wrote: On Thu, Sep 20, 2018 at 6:09 AM amul sul wrote: Agreed, thanks for working on this. Pushed, thanks. Please, find attached patch improving documentation about letters/digits in to_dat

Re: Online verification of checksums

2018-09-26 Thread Michael Banck
Hi, Am Mittwoch, den 26.09.2018, 13:23 +0200 schrieb Fabien COELHO: > Patch v3 applies cleanly, code compiles and make check is ok, but the > command is probably not tested anywhere, as already mentioned on other > threads. Right. > The patch is missing a documentation update. I've added that

Re: transction_timestamp() inside of procedures

2018-09-26 Thread Tom Lane
Alvaro Herrera writes: > On 2018-Sep-26, Tom Lane wrote: >> I agree that it would be surprising for transaction timestamp to be newer >> than statement timestamp. So for now at least, I'd be satisfied with >> documenting the behavior. > Really? I thought it was practically obvious that for tran

Re: transction_timestamp() inside of procedures

2018-09-26 Thread Alvaro Herrera
On 2018-Sep-26, Tom Lane wrote: > Bruce Momjian writes: > > On Wed, Sep 26, 2018 at 02:38:25PM +0200, Peter Eisentraut wrote: > >> We could certainly address this by adding three or four or five new > >> timestamps that cover all these varieties. But perhaps it's worth > >> asking what these tim

Re: transction_timestamp() inside of procedures

2018-09-26 Thread Alvaro Herrera
On 2018-Sep-25, Bruce Momjian wrote: > Well, it is an entire paragraph, so it might be too much. If you > download the zip file here: > > http://www.wiscorp.com/sql200n.zip > > and open 5CD2-02-Foundation-2006-01.pdf, at the top of page 289 under > General Rules, paragraph label 3 has the

Re: transction_timestamp() inside of procedures

2018-09-26 Thread Tom Lane
Bruce Momjian writes: > On Wed, Sep 26, 2018 at 02:38:25PM +0200, Peter Eisentraut wrote: >> We could certainly address this by adding three or four or five new >> timestamps that cover all these varieties. But perhaps it's worth >> asking what these timestamps are useful for and which ones we re

Re: txid_status returns NULL for recently commited transactions

2018-09-26 Thread Michail Nikolaev
Hello. Got some new information. There are 6 replicas and master in cluster. I rebooted two replicas... And they started to work as expected! So, on master and 4 untouched replicas: txid_current() -> 4459388265 txid_status(BIGINT '4459388265') -> NULL On two rebooted replicas: txid_status(BIGIN

Re: transction_timestamp() inside of procedures

2018-09-26 Thread Bruce Momjian
On Wed, Sep 26, 2018 at 02:38:25PM +0200, Peter Eisentraut wrote: > On 22/09/2018 00:35, Bruce Momjian wrote: > > I have always thought of clock/statement/transation as decreasing levels > > of time precision, and it might be odd to change that. I don't think we > > want to change the behavior of

Re: transction_timestamp() inside of procedures

2018-09-26 Thread Peter Eisentraut
On 22/09/2018 00:35, Bruce Momjian wrote: > I have always thought of clock/statement/transation as decreasing levels > of time precision, and it might be odd to change that. I don't think we > want to change the behavior of statement_timestamp() in procedures, so > that kind of requires us not to

Re: [patch]overallocate memory for curly braces in array_out

2018-09-26 Thread Keiichi Hirobe
Hi, Thanks for the reply. I am not sure whether to fix another bug, but I fixed and I attached a new patch, please check it. Please note that I inserted a line for updating "overall_length" at line 1185. I checked below case. select ARRAY[ARRAY[1,2,3],ARRAY[4,5,6]]; (regular case) select '[3:4

Re: Implementing SQL ASSERTION

2018-09-26 Thread Peter Eisentraut
On 25/09/2018 01:04, Joe Wildish wrote: > Having said all that: there are obviously going to be some expressions > that cannot be proven to have no potential for invalidating the assertion > truth. I guess this is the prime concern from a concurrency PoV? Before we spend more time on this, I think

Re: Online verification of checksums

2018-09-26 Thread Fabien COELHO
Hallo Michael, Patch v3 applies cleanly, code compiles and make check is ok, but the command is probably not tested anywhere, as already mentioned on other threads. The patch is missing a documentation update. There are debatable changes of behavior: if (errno == ENOENT) return / conti

Re: pgbench - add pseudo-random permutation function

2018-09-26 Thread Fabien COELHO
Hello, That is necessary because most people consume PostgreSQL through packages from distributions that have to work on an Athlon II or whatever, so we can't just use -msse4.2 for every translation unit. So it becomes our job to isolate small bits of code that use newer instructions, if it's

Re: pgbench - add pseudo-random permutation function

2018-09-26 Thread Thomas Munro
On Wed, Sep 26, 2018 at 8:26 PM Fabien COELHO wrote: > > modular_multiply() is an interesting device. I will leave this to > > committers with a stronger mathematical background than me, but I have > > a small comment in passing: > > For testing this function, I have manually commented out the va

64-bit hash function for hstore and citext data type

2018-09-26 Thread amul sul
Hi all, Commit[1] has added 64-bit hash functions for core data types and in the same discussion thread[2] Robert Haas suggested to have the similar extended hash function for hstore and citext data type. Attaching patch proposes the same. Regards, Amul 1] https://git.postgresql.org/gitweb/?p=p

Re: [PATCH] Improve geometric types

2018-09-26 Thread Tomas Vondra
On 09/23/2018 11:55 PM, Tomas Vondra wrote: > On 09/03/2018 04:08 AM, Tom Lane wrote: >> Tomas Vondra writes: >>> On 08/17/2018 11:23 PM, Tom Lane wrote: Yeah, we've definitely hit such problems before. The geometric logic seems particularly prone to it because it's doing more and subtl

Re: Progress reporting for pg_verify_checksums

2018-09-26 Thread Fabien COELHO
Hallo Michael, About patch v3: applies cleanly and compiles. The xml documentation should be updated! (It is kind of hard to notice what is not there:-) See "doc/src/sgml/ref/pg_verify_checksums.sgml". The time() granularity to the second makes the result awkward on small tests: 8/15545

Re: pgbench - add pseudo-random permutation function

2018-09-26 Thread Fabien COELHO
Hello Thomas, modular_multiply() is an interesting device. I will leave this to committers with a stronger mathematical background than me, but I have a small comment in passing: For testing this function, I have manually commented out the various shortcuts so that the manual multiplication

Re: Proposal for Signal Detection Refactoring

2018-09-26 Thread Chris Travers
On Tue, Sep 25, 2018 at 3:23 PM Tom Lane wrote: > Chris Travers writes: > > However, what I think one could do is use a struct of volatile > > sig_atomic_t members and macros for checking/setting. Simply writing a > > value is safe in C89 and higher. > > Yeah, we could group those flags in a s

Re: Proposal for Signal Detection Refactoring

2018-09-26 Thread Chris Travers
On Wed, Sep 26, 2018 at 12:41 AM Michael Paquier wrote: > On Mon, Sep 24, 2018 at 09:27:35PM -0700, Andres Freund wrote: > > On 2018-09-25 11:50:47 +0900, Michael Paquier wrote: > >> PGDLLIMPORT changes don't get back-patched as well... > > > > We've been more aggressive with that lately, and I t

Re: Participate in GCI as a Mentor

2018-09-26 Thread Tahir Ramzan
Honorable Concern, I am planning to mentor these ideas IN Google Code-in as these are learning, interesting, informative and doable for young students: Compare PostgreSQL with another RDBMS: • DB2 • MySQL • Oracle • SQLite • MS SQL Server • Sybase Compare PostgreSQL with a NoSQL Column Database:

Re: Implement predicate propagation for non-equivalence clauses

2018-09-26 Thread Richard Guo
Hi, Thanks everyone for reviewing. We updated the patch and added more strict checks for the non-equivalence clauses before deducing new quals, including: 1) The non-equivalence clause for deduction can only be type of OpExpr or ScalarArrayOpExpr, with two arguments. 2) The operator of the non-e