Re: [RFC] Removing "magic" oids

2019-07-20 Thread Andres Freund
Hi, On 2019-07-20 12:58:55 -0700, Noah Misch wrote: > On Sat, Jul 20, 2019 at 12:56:34PM -0700, Andres Freund wrote: > > Fair enough. Are you planning to commit your changes? > > https://git.postgresql.org/gitweb/?p=postgresql.git;a=commitdiff;h=8a0cbb88524e8b6121597285b811640ee793b3e8 Oh, sorry

Re: [RFC] Removing "magic" oids

2019-07-20 Thread Noah Misch
On Sat, Jul 20, 2019 at 12:56:34PM -0700, Andres Freund wrote: > On 2019-07-20 11:21:52 -0700, Noah Misch wrote: > > On Fri, Jul 19, 2019 at 10:12:57AM -0700, Andres Freund wrote: > > > On 2019-07-07 10:00:35 -0700, Noah Misch wrote: > > > > +# Test concurrent OID generation via pg_enum_oid_index.

Re: [RFC] Removing "magic" oids

2019-07-20 Thread Andres Freund
Hi, On 2019-07-20 11:21:52 -0700, Noah Misch wrote: > On Fri, Jul 19, 2019 at 10:12:57AM -0700, Andres Freund wrote: > > On 2019-07-07 10:00:35 -0700, Noah Misch wrote: > > > +# Test concurrent OID generation via pg_enum_oid_index. This indirectly > > > +# exercises LWLock and spinlock concurrenc

Re: [RFC] Removing "magic" oids

2019-07-20 Thread Noah Misch
On Fri, Jul 19, 2019 at 10:12:57AM -0700, Andres Freund wrote: > On 2019-07-07 10:00:35 -0700, Noah Misch wrote: > > +# Test concurrent OID generation via pg_enum_oid_index. This indirectly > > +# exercises LWLock and spinlock concurrency. > > +my $labels = join ',', map { "'l$_'" } 1 .. 1000; > >

Re: [RFC] Removing "magic" oids

2019-07-19 Thread Andres Freund
Hi, On 2019-07-07 10:00:35 -0700, Noah Misch wrote: > On Tue, Nov 20, 2018 at 01:20:04AM -0800, Andres Freund wrote: > > On 2018-11-14 21:02:41 -0800, Andres Freund wrote: > > > > The point of the test is to exercise OidGenLock by issuing many parallel > > > > GetNewOidWithIndex() and verifying ab

Re: [RFC] Removing "magic" oids

2019-07-07 Thread Noah Misch
On Tue, Nov 20, 2018 at 01:20:04AM -0800, Andres Freund wrote: > On 2018-11-14 21:02:41 -0800, Andres Freund wrote: > > On 2018-11-15 04:57:28 +, Noah Misch wrote: > > > On Wed, Nov 14, 2018 at 12:01:52AM -0800, Andres Freund wrote: > > > > - one pgbench test tested concurrent insertions into a

Re: [RFC] Removing "magic" oids

2018-11-27 Thread Andrew Dunstan
On 11/26/18 5:50 PM, Andres Freund wrote: With some adjustments to the tests to remove problematic cases (e.g. postgres_fdw's ft_pg_type) the tests pass. The exception is HEAD->HEAD. The change is that the LOs are not dumped in the same order pre and post upgrade. I can change the tests to al

Re: [RFC] Removing "magic" oids

2018-11-26 Thread Andres Freund
Hi, On 2018-11-21 16:30:07 +0100, Andreas Karlsson wrote: > On 11/21/18 1:07 AM, Andres Freund wrote: > > Let's see what I broke :/ > > There is a small typo in the old release notes. Thanks, pushed! Greetings, Andres Freund

Re: [RFC] Removing "magic" oids

2018-11-26 Thread Andres Freund
Hi, On 2018-11-22 17:12:31 -0500, Andrew Dunstan wrote: > > On 11/22/18 4:14 PM, Andres Freund wrote: > > Hi, > > > > On 2018-11-21 23:32:07 -0500, Andrew Dunstan wrote: > > > On 11/21/18 7:14 PM, Andres Freund wrote: > > > >Could you check whether you > > > > still encounter the issue after

Re: [RFC] Removing "magic" oids

2018-11-22 Thread Andrew Dunstan
On 11/22/18 4:14 PM, Andres Freund wrote: Hi, On 2018-11-21 23:32:07 -0500, Andrew Dunstan wrote: On 11/21/18 7:14 PM, Andres Freund wrote: Could you check whether you still encounter the issue after applying the attached fix? This has largely fixed the problem, so I think this should

Re: [RFC] Removing "magic" oids

2018-11-22 Thread Andres Freund
Hi, On 2018-11-21 23:32:07 -0500, Andrew Dunstan wrote: > On 11/21/18 7:14 PM, Andres Freund wrote: > > Could you check whether you > > still encounter the issue after applying the attached fix? > > > > > This has largely fixed the problem, so I think this should be applied. Cool, will do so

Re: [RFC] Removing "magic" oids

2018-11-21 Thread Andrew Dunstan
On 11/21/18 7:14 PM, Andres Freund wrote: Could you check whether you still encounter the issue after applying the attached fix? This has largely fixed the problem, so I think this should be applied. With some adjustments to the tests to remove problematic cases (e.g. postgres_fdw's ft_

Re: [RFC] Removing "magic" oids

2018-11-21 Thread Andres Freund
Hi, On 2018-11-21 10:36:42 -0800, Andres Freund wrote: > >pg_upgrade against old versions, by the look of it. Even after I drop > >oids from user tables, I get errors like this when running pg_dumpall > >against a pg_upgraded REL_9_4_STABLE datadir: Not pg_upgrade in general, I did test that. "

Re: [RFC] Removing "magic" oids

2018-11-21 Thread Andres Freund
On November 21, 2018 10:17:57 AM PST, Andrew Dunstan wrote: > >On 11/20/18 7:07 PM, Andres Freund wrote: >> >> >> Let's see what I broke :/ >> > > >pg_upgrade against old versions, by the look of it. Even after I drop >oids from user tables, I get errors like this when running pg_dumpall >ag

Re: [RFC] Removing "magic" oids

2018-11-21 Thread Andrew Dunstan
On 11/20/18 7:07 PM, Andres Freund wrote: Let's see what I broke :/ pg_upgrade against old versions, by the look of it. Even after I drop oids from user tables, I get errors like this when running pg_dumpall against a pg_upgraded REL_9_4_STABLE datadir: 2018-11-21 13:01:58.582 EST [1

Re: [RFC] Removing "magic" oids

2018-11-21 Thread Andreas Karlsson
On 11/21/18 1:07 AM, Andres Freund wrote: Let's see what I broke :/ There is a small typo in the old release notes. Andreas diff --git a/doc/src/sgml/release-9.1.sgml b/doc/src/sgml/release-9.1.sgml index 3407d8ad739..a522e7e0225 100644 --- a/doc/src/sgml/release-9.1.sgml +++ b/doc/src/sgml/re

Re: [RFC] Removing "magic" oids

2018-11-20 Thread Andres Freund
Hi, On 2018-11-21 14:32:23 +1300, Thomas Munro wrote: > On Wed, Nov 21, 2018 at 1:08 PM Andres Freund wrote: > > Let's see what I broke :/ > > Per rhinoceros, contrib/sepgsql. This patch fixes the build for me > (on Debian with libselinux1-dev installed) Thanks. I pushed your fix. > though I

Re: [RFC] Removing "magic" oids

2018-11-20 Thread Thomas Munro
On Wed, Nov 21, 2018 at 1:08 PM Andres Freund wrote: > Let's see what I broke :/ Per rhinoceros, contrib/sepgsql. This patch fixes the build for me (on Debian with libselinux1-dev installed), though I don't know how to test that it works and make check does nothing useful in there. -- Thomas M

Re: [RFC] Removing "magic" oids

2018-11-20 Thread Andres Freund
Hi, On 2018-11-20 01:52:27 -0800, Andres Freund wrote: > I'm pretty happy with the new state. Unless somebody announces they want > to do a review soon-ish, I'm planning to commit this soon. It's a > painful set to keep up2date, and it's blocking a few other patches. I'm > sure we'll find some th

Re: [RFC] Removing "magic" oids

2018-11-20 Thread Andres Freund
Hi, On 2018-11-20 21:49:27 +0700, John Naylor wrote: > On 11/20/18, Andres Freund wrote: > > I'm > > sure we'll find some things to adapt around the margins, but imo the > > patch as a whole looks pretty reasonable. > > bki.sgml is now a bit out of date. I've attached a draft fix. Thanks! Gree

Re: [RFC] Removing "magic" oids

2018-11-20 Thread Andres Freund
Hi, On 2018-11-20 11:25:22 -0500, Stephen Frost wrote: > Greetings, > > * Robert Haas (robertmh...@gmail.com) wrote: > > On Sun, Oct 14, 2018 at 6:35 PM Tom Lane wrote: > > > Andres Freund writes: > > > > Does anybody have engineering / architecture level comments about this > > > > proposal? >

Re: [RFC] Removing "magic" oids

2018-11-20 Thread Stephen Frost
Greetings, * Robert Haas (robertmh...@gmail.com) wrote: > On Sun, Oct 14, 2018 at 6:35 PM Tom Lane wrote: > > Andres Freund writes: > > > Does anybody have engineering / architecture level comments about this > > > proposal? > > > > FWIW, I'm -1 on making OIDs be not-magic for SELECT purposes.

Re: [RFC] Removing "magic" oids

2018-11-20 Thread Robert Haas
On Sun, Oct 14, 2018 at 6:35 PM Tom Lane wrote: > Andres Freund writes: > > Does anybody have engineering / architecture level comments about this > > proposal? > > FWIW, I'm -1 on making OIDs be not-magic for SELECT purposes. Yeah, it's > a wart we wouldn't have if we designed the system today,

Re: [RFC] Removing "magic" oids

2018-11-20 Thread John Naylor
On 11/20/18, Andres Freund wrote: > I'm > sure we'll find some things to adapt around the margins, but imo the > patch as a whole looks pretty reasonable. bki.sgml is now a bit out of date. I've attached a draft fix. -John Naylor diff --git a/doc/src/sgml/bki.sgml b/doc/src/sgml/bki.sgml index 0

Re: [RFC] Removing "magic" oids

2018-11-20 Thread Andres Freund
On 2018-11-14 21:02:41 -0800, Andres Freund wrote: > Hi, > > On 2018-11-15 04:57:28 +, Noah Misch wrote: > > On Wed, Nov 14, 2018 at 12:01:52AM -0800, Andres Freund wrote: > > > - one pgbench test tested concurrent insertions into a table with > > > oids, as some sort of stress test for lwlo

Re: [RFC] Removing "magic" oids

2018-11-15 Thread John Naylor
On 11/16/18, Andres Freund wrote: > On 2018-11-15 17:25:21 +0700, John Naylor wrote: >> I don't see an advantage to having a different range, but maybe it >> should error out if $maxoid reaches FirstBootstrapObjectId. > > Hm. Not sure I really see the point. Note we didn't have that check > before

Re: [RFC] Removing "magic" oids

2018-11-15 Thread Andres Freund
Hi, On 2018-11-15 17:25:21 +0700, John Naylor wrote: > On 11/15/18, Andres Freund wrote: > > I've now revised this slightly. genbki.pl now computes the maximum oid > > explicitly assigned in .dat files, and assignes oids to all 'oid' > > columns without a value. It does so starting directly at t

Re: [RFC] Removing "magic" oids

2018-11-15 Thread John Naylor
On 11/15/18, Andres Freund wrote: > I've now revised this slightly. genbki.pl now computes the maximum oid > explicitly assigned in .dat files, and assignes oids to all 'oid' > columns without a value. It does so starting directly at the maximum > value. I personally don't see need to have impli

Re: [RFC] Removing "magic" oids

2018-11-14 Thread Andres Freund
Hi, On 2018-11-15 04:57:28 +, Noah Misch wrote: > On Wed, Nov 14, 2018 at 12:01:52AM -0800, Andres Freund wrote: > > - one pgbench test tested concurrent insertions into a table with > > oids, as some sort of stress test for lwlocks and spinlocks. I *think* > > this doesn't really have to

Re: [RFC] Removing "magic" oids

2018-11-14 Thread Noah Misch
On Wed, Nov 14, 2018 at 12:01:52AM -0800, Andres Freund wrote: > - one pgbench test tested concurrent insertions into a table with > oids, as some sort of stress test for lwlocks and spinlocks. I *think* > this doesn't really have to be a system oid column, and this was just > because that's

Re: [RFC] Removing "magic" oids

2018-10-27 Thread Amit Kapila
On Sun, Oct 28, 2018 at 12:26 AM Andres Freund wrote: > > Hi, > > On 2018-10-28 00:21:23 +0530, Amit Kapila wrote: > > On Sun, Sep 30, 2018 at 9:18 AM Andres Freund wrote: > > > I think we should drop WITH OIDs support. pg_dump should convert WITH > > > OIDs tables into tables that have an expli

Re: [RFC] Removing "magic" oids

2018-10-27 Thread Andres Freund
Hi, On 2018-10-28 00:21:23 +0530, Amit Kapila wrote: > On Sun, Sep 30, 2018 at 9:18 AM Andres Freund wrote: > > I think we should drop WITH OIDs support. pg_dump should convert WITH > > OIDs tables into tables that have an explicit oid column (with an > > appropriate default function), pg_upgrad

Re: [RFC] Removing "magic" oids

2018-10-27 Thread Amit Kapila
On Sun, Sep 30, 2018 at 9:18 AM Andres Freund wrote: > > Hi, > > In my opinion the current WITH OIDs system has numerous weaknesses: > > 1) The fact that oids are so magic means that if we get pluggable >storage, the design of the potential pluggable systems is constrained >and similar mag

Re: [RFC] Removing "magic" oids

2018-10-27 Thread Amit Kapila
On Wed, Oct 17, 2018 at 9:22 PM Stephen Frost wrote: > > Greetings, > > * Robert Haas (robertmh...@gmail.com) wrote: > > On Sun, Oct 14, 2018 at 6:43 PM Andres Freund wrote: > > > I'm not sure we want that however - yes, the short time pain will be > > > lower, but do we really want to inflict th

Re: [RFC] Removing "magic" oids

2018-10-17 Thread Stephen Frost
Greetings, * Robert Haas (robertmh...@gmail.com) wrote: > On Sun, Oct 14, 2018 at 6:43 PM Andres Freund wrote: > > I'm not sure we want that however - yes, the short time pain will be > > lower, but do we really want to inflict the confusion about invisible > > oids on our users for the next 20 y

Re: [RFC] Removing "magic" oids

2018-10-17 Thread Stephen Frost
Greetings, * Thomas Munro (thomas.mu...@enterprisedb.com) wrote: > On Mon, Oct 15, 2018 at 11:35 AM Tom Lane wrote: > > Andres Freund writes: > > > Does anybody have engineering / architecture level comments about this > > > proposal? > > > > FWIW, I'm -1 on making OIDs be not-magic for SELECT p

Re: [RFC] Removing "magic" oids

2018-10-16 Thread Robert Haas
On Sun, Oct 14, 2018 at 6:43 PM Andres Freund wrote: > I'm not sure we want that however - yes, the short time pain will be > lower, but do we really want to inflict the confusion about invisible > oids on our users for the next 20 years? I think there's a fair argument > to be made that we should

Re: [RFC] Removing "magic" oids

2018-10-14 Thread Thomas Munro
On Mon, Oct 15, 2018 at 11:35 AM Tom Lane wrote: > Andres Freund writes: > > Does anybody have engineering / architecture level comments about this > > proposal? > > FWIW, I'm -1 on making OIDs be not-magic for SELECT purposes. Yeah, it's > a wart we wouldn't have if we designed the system today

Re: [RFC] Removing "magic" oids

2018-10-14 Thread Andres Freund
Hi, On 2018-10-14 18:34:50 -0400, Tom Lane wrote: > Andres Freund writes: > > Does anybody have engineering / architecture level comments about this > > proposal? > > FWIW, I'm -1 on making OIDs be not-magic for SELECT purposes. Yeah, it's > a wart we wouldn't have if we designed the system tod

Re: [RFC] Removing "magic" oids

2018-10-14 Thread Tom Lane
Andres Freund writes: > Does anybody have engineering / architecture level comments about this > proposal? FWIW, I'm -1 on making OIDs be not-magic for SELECT purposes. Yeah, it's a wart we wouldn't have if we designed the system today, but the wart is thirty years old. I think changing that wi

Re: [RFC] Removing "magic" oids

2018-10-14 Thread Andres Freund
Hi, On 2018-09-29 20:48:10 -0700, Andres Freund wrote: > In my opinion the current WITH OIDs system has numerous weaknesses: > > 1) The fact that oids are so magic means that if we get pluggable >storage, the design of the potential pluggable systems is constrained >and similar magic has

Re: [RFC] Removing "magic" oids

2018-10-03 Thread Vik Fearing
On 30/09/18 05:48, Andres Freund wrote: > 5a) The fact that system table oids don't show up in selects by default >makes it more work than necessary to look at catalogs. As a user, this is a big pain point for me. +1 for getting rid of it. -- Vik Fearing

Re: [RFC] Removing "magic" oids

2018-10-03 Thread Andreas Karlsson
On 09/30/2018 05:48 AM, Andres Freund wrote:> I think we should drop WITH OIDs support. +1 2) We need to be able to manually insert into catalog tables. Both initdb and emergency surgery. My current hack is doing so directly in nodeModifyTable.c but that's beyond ugly. I think we sho

Re: [RFC] Removing "magic" oids

2018-09-30 Thread David Fetter
On Sat, Sep 29, 2018 at 08:48:10PM -0700, Andres Freund wrote: > Hi, > > In my opinion the current WITH OIDs system has numerous weaknesses: > > 1) The fact that oids are so magic means that if we get pluggable >storage, the design of the potential pluggable systems is constrained >and si