Re: unused_oids script is broken with bsd sed

2018-05-04 Thread John Naylor
On 5/4/18, Tom Lane wrote: > Well, the work's already done, and personally I think the code is > cleaner after 9bf28f96c and bad51a49a regardless of whether there's any > performance benefit. You could call 1f1cd9b5d a wart if you wanted. > But we've done largely the same thing to pgindent, for o

Re: unused_oids script is broken with bsd sed

2018-05-03 Thread Tom Lane
Peter Eisentraut writes: > On 5/3/18 15:37, Tom Lane wrote: >> I took a quick look into this. It's very easy to do so far as the Perl >> code is concerned: > I think in order to introduce warts like that, we have to have really > great savings. I haven't seen any actual analysis what the curren

Re: unused_oids script is broken with bsd sed

2018-05-03 Thread Peter Eisentraut
On 5/3/18 15:37, Tom Lane wrote: > I took a quick look into this. It's very easy to do so far as the Perl > code is concerned: I think in order to introduce warts like that, we have to have really great savings. I haven't seen any actual analysis what the current problem is, other than one perso

Re: unused_oids script is broken with bsd sed

2018-05-03 Thread Tom Lane
Alvaro Herrera writes: > Tom Lane wrote: >> However, RenameTempFile is also used by Gen_fmgrtab.pl, and having the >> same sort of no-touch semantics for fmgroids.h and friends would have some >> additional fallout. The makefiles would think they have to keep >> re-running Gen_fmgrtab.pl if fmgro

Re: unused_oids script is broken with bsd sed

2018-05-03 Thread Andrew Dunstan
On Thu, May 3, 2018 at 3:37 PM, Tom Lane wrote: > > Anyway, I'm happy to go make this happen; it looks like only another hour > or so's worth of work to fix the makefiles. But I wonder if anyone will > say this is too much churn for post-feature-freeze and we should shelve > it till v12. > I thi

Re: unused_oids script is broken with bsd sed

2018-05-03 Thread Alvaro Herrera
Tom Lane wrote: > However, RenameTempFile is also used by Gen_fmgrtab.pl, and having the > same sort of no-touch semantics for fmgroids.h and friends would have some > additional fallout. The makefiles would think they have to keep > re-running Gen_fmgrtab.pl if fmgroids.h is older than the mod t

Re: unused_oids script is broken with bsd sed

2018-05-03 Thread Tom Lane
Robert Haas writes: > On Thu, Apr 26, 2018 at 11:29 AM, Tom Lane wrote: >> Personally, I use ccache which doesn't seem to care too much, but I agree >> than in some usages, extra touches of headers would be costly. Perhaps >> it's worth adding logic to avoid overwriting an existing output file >

Re: unused_oids script is broken with bsd sed

2018-05-03 Thread Robert Haas
On Thu, Apr 26, 2018 at 11:29 AM, Tom Lane wrote: > Personally, I use ccache which doesn't seem to care too much, but I agree > than in some usages, extra touches of headers would be costly. Perhaps > it's worth adding logic to avoid overwriting an existing output file > unless it changed? I'm n

Re: unused_oids script is broken with bsd sed

2018-04-26 Thread Tom Lane
John Naylor writes: > On 4/26/18, Tom Lane wrote: >> I notice that duplicate_oids is now a good factor of 10 slower than it was >> before (~0.04 sec to ~0.4 sec, on my machine). While this doesn't seem >> like a problem for manual use, it seems annoying as part of the standard >> build process,

Re: unused_oids script is broken with bsd sed

2018-04-26 Thread Tom Lane
I wrote: > Andreas Karlsson writes: >> What about using FindBin (http://perldoc.perl.org/FindBin.html)? > A quick check suggests that that's present in the core perl > distribution pretty far back, so I think it'll work. > Question is, is it any better than John's "dirname(__FILE__)" > solution?

Re: unused_oids script is broken with bsd sed

2018-04-26 Thread Tom Lane
Andreas Karlsson writes: > What about using FindBin (http://perldoc.perl.org/FindBin.html)? A quick check suggests that that's present in the core perl distribution pretty far back, so I think it'll work. Question is, is it any better than John's "dirname(__FILE__)" solution?

Re: unused_oids script is broken with bsd sed

2018-04-26 Thread Andreas Karlsson
On 04/26/2018 01:37 PM, John Naylor wrote:> Patch 0002 is my attempt at this. Not sure how portable this is. It's also clunkier than before in that you need to tell it where to find Catalog.pm, since it seems Perl won't compile unless "use lib" points to a string and not an expression. Suggestion

Re: unused_oids script is broken with bsd sed

2018-04-26 Thread John Naylor
On 4/26/18, Tom Lane wrote: > John Naylor writes: >> For those following along, these scripts still assume we're in the >> catalog directory. I can hack on that part tomorrow if no one else >> has. > > I didn't touch this point. Patch 0002 is my attempt at this. Not sure how portable this is. It

Re: unused_oids script is broken with bsd sed

2018-04-25 Thread Tom Lane
John Naylor writes: > Just after you posted, I sent a patch that tweaks the API of > Catalog.pm for toast and index oids. If you use that API in your > patch, you can get rid of the extra bookkeeping you added for those > oids. I've adjusted these two patches to play together, and pushed them. >

Re: unused_oids script is broken with bsd sed

2018-04-25 Thread John Naylor
On 4/26/18, Stas Kelvich wrote: > New version is attached. I've put iterator into Catalog.pm to avoid > copy-pasting > it over two scripts. Also instead of greping through *.dat and *.h files > I've > used parsers provided in Catalog module. That way should be more > sustainable > to format change

Re: unused_oids script is broken with bsd sed

2018-04-25 Thread Tom Lane
John Naylor writes: > On 4/25/18, Tom Lane wrote: >> I think we should rewrite >> both of them to use the Catalog.pm infrastructure. > If we're going to use Catalog.pm for that, it seems more convenient to > expose toast and index oids directly rather than in strings formatted > specifically for

Re: unused_oids script is broken with bsd sed

2018-04-25 Thread John Naylor
On 4/25/18, Tom Lane wrote: > Andrew Dunstan writes: >> +many for rewriting in perl. Do you want to have a go at that? If not I >> will. > > +1 ... I was mildly astonished that this didn't already have to happen > as part of the bootstrap data conversion effort. It's certainly not > hard to imag

Re: unused_oids script is broken with bsd sed

2018-04-25 Thread Stas Kelvich
> On 25 Apr 2018, at 17:55, John Naylor wrote: > > On 4/25/18, Stas Kelvich wrote: >>> On 25 Apr 2018, at 17:18, Tom Lane wrote: >>> I think we should rewrite >>> both of them to use the Catalog.pm infrastructure. >> >> Okay, seems reasonable. I'll put shared code in Catalog.pm and >> update

Re: unused_oids script is broken with bsd sed

2018-04-25 Thread Andrew Dunstan
On 04/25/2018 10:55 AM, John Naylor wrote: > On 4/25/18, Stas Kelvich wrote: >>> On 25 Apr 2018, at 17:18, Tom Lane wrote: >>> I think we should rewrite >>> both of them to use the Catalog.pm infrastructure. >> Okay, seems reasonable. I'll put shared code in Catalog.pm and >> update patch. > I

Re: unused_oids script is broken with bsd sed

2018-04-25 Thread Andreas Karlsson
On 04/25/2018 04:59 PM, David Fetter wrote: While we're improving things, there's not really a reason this program should need to be run from any particular spot. It can detect where it is and act on that information. +1 I would appreciate this change, and if Stats does not feel like adding i

Re: unused_oids script is broken with bsd sed

2018-04-25 Thread David Fetter
On Wed, Apr 25, 2018 at 09:55:54PM +0700, John Naylor wrote: > On 4/25/18, Stas Kelvich wrote: > >> On 25 Apr 2018, at 17:18, Tom Lane wrote: > >> I think we should rewrite > >> both of them to use the Catalog.pm infrastructure. > > > > Okay, seems reasonable. I'll put shared code in Catalog.pm

Re: unused_oids script is broken with bsd sed

2018-04-25 Thread John Naylor
On 4/25/18, Stas Kelvich wrote: >> On 25 Apr 2018, at 17:18, Tom Lane wrote: >> I think we should rewrite >> both of them to use the Catalog.pm infrastructure. > > Okay, seems reasonable. I'll put shared code in Catalog.pm and > update patch. I don't think you need any new code in Catalog.pm, I

Re: unused_oids script is broken with bsd sed

2018-04-25 Thread Stas Kelvich
> On 25 Apr 2018, at 17:18, Tom Lane wrote: > I think we should rewrite > both of them to use the Catalog.pm infrastructure. Okay, seems reasonable. I'll put shared code in Catalog.pm and update patch. -- Stas Kelvich Postgres Professional: http://www.postgrespro.com The Russian Postgres Comp

Re: unused_oids script is broken with bsd sed

2018-04-25 Thread Stas Kelvich
> On 25 Apr 2018, at 17:18, Tom Lane wrote: > > Andrew Dunstan writes: >> +many for rewriting in perl. Do you want to have a go at that? If not I >> will. > > +1 ... I was mildly astonished that this didn't already have to happen > as part of the bootstrap data conversion effort. It's certai

Re: unused_oids script is broken with bsd sed

2018-04-25 Thread Tom Lane
Andrew Dunstan writes: > +many for rewriting in perl. Do you want to have a go at that? If not I > will. +1 ... I was mildly astonished that this didn't already have to happen as part of the bootstrap data conversion effort. It's certainly not hard to imagine future extensions to the .dat file f

Re: unused_oids script is broken with bsd sed

2018-04-25 Thread Andrew Dunstan
On 04/25/2018 06:22 AM, Stas Kelvich wrote: > Hi. > > BSD sed in macOS doesn't understand word boundary operator "\b". So after > 372728b0d49 unused_oids doesn’t match all oids in new *.dat files marking > all of them as unused. > > It is possible to write more portable regexps, e.g. change "\b"