Re: speed up text_position() for utf-8

2021-12-17 Thread John Naylor
- PG11 | 1220 | 1220 | 1150 master | 385 | 2420 | 1980 0001 | 390 | 2180 | 1670 0002 | 389 | 1330 | 1100 0003 | 391 | 2100 | 1360 -- John Naylor EDB: http://www.enterprisedb.com v2-0002-Refactor-text_position_get_m

Re: speed up verifying UTF-8

2021-12-20 Thread John Naylor
On Fri, Dec 17, 2021 at 9:29 AM John Naylor wrote: > > I plan to push v25 early next week, unless there are further comments. Pushed, thanks everyone! -- John Naylor EDB: http://www.enterprisedb.com

Re: do only critical work during single-user vacuum?

2021-12-21 Thread John Naylor
pages or WAL". There are some more sophisticated things we can do in this regard, but something like the above could also be useful in normal operation. In fact, that "normal" could be just after we restarted after doing the bare-minimum in single-user mode, and want to continue freezing and keep things under control. -- John Naylor EDB: http://www.enterprisedb.com

Re: do only critical work during single-user vacuum?

2021-12-21 Thread John Naylor
ecify vacuum_failsafe_age/vacuum_multixact_failsafe_age in a session, including in single-user mode. Perhaps a new boolean called FAILSAFE_ONLY. If no table is specified, then when generating the list of tables, include only those with relfrozenxid/relminmxid greater than their failsafe thresholds. -- John Naylor EDB: http://www.enterprisedb.com

Re: do only critical work during single-user vacuum?

2021-12-21 Thread John Naylor
On Tue, Dec 21, 2021 at 5:56 PM Peter Geoghegan wrote: > > On Tue, Dec 21, 2021 at 1:31 PM John Naylor > wrote: > > On second thought, we don't really need another number here. We could > > simply go by the existing failsafe parameter, and if the admin wants a > &g

Re: do only critical work during single-user vacuum?

2021-12-22 Thread John Naylor
on autovacuums. --min-xid-age > option and --min-mxid-age option of vacuumdb command would be good > examples. So I think this new command/facility might not necessarily > need to be specific to single-user mode. If we want to leave open the possibility to specify these parameters, a SQL-call

Re: Introducing PgVA aka PostgresVectorAcceleration using SIMD vector instructions starting with hex_encode

2022-01-03 Thread John Naylor
ric datatype, merging, generate_series, integer and floating > point output …) which could be addressed later on. Float output has already been pretty well optimized. CRC checksums already have a hardware implementation on x86 and Arm. I don't know of any practical workload where generate_series() is too slow. Aggregation is an interesting case, but I'm not sure what the current bottlenecks are. -- John Naylor EDB: http://www.enterprisedb.com

Re: Should we improve "PID XXXX is not a PostgreSQL server process" warning for pg_terminate_backend(<>)?

2022-01-11 Thread John Naylor
ctly. Also, v5 was a big enough change from v4 that I put Nathan as the first author. -- John Naylor EDB: http://www.enterprisedb.com

Re: do only critical work during single-user vacuum?

2022-01-11 Thread John Naylor
be worth overriding the min ages as well, but haven't done so here. It can be executed in normal mode (although it's not expected to be), which makes testing easier and allows for a future possibility of not requiring shutdown at all, by e.g. terminating non-superuser connections. -- John Nay

Re: do only critical work during single-user vacuum?

2022-01-12 Thread John Naylor
On Tue, Jan 11, 2022 at 8:57 PM Peter Geoghegan wrote: > On Tue, Jan 11, 2022 at 4:59 PM John Naylor > > For the PoC I wanted to try re-using existing keywords. I went with > > "VACUUM LIMIT" since LIMIT is already a keyword that cannot be used as > > a table n

Re: do only critical work during single-user vacuum?

2022-01-12 Thread John Naylor
On Tue, Jan 11, 2022 at 9:20 PM Justin Pryzby wrote: > > On Tue, Jan 11, 2022 at 07:58:56PM -0500, John Naylor wrote: > > + // FIXME: also check reloption > > + // WIP: 95% is a starting point for discussion > > + if ((table_xid_age <

Re: do only critical work during single-user vacuum?

2022-01-12 Thread John Naylor
case, the admin would still need to add INDEX_CLEANUP = off for minimum downtime, and it should be really simple. - For the general case, we would now have the ability to vacuum a table, and possibly have no effect at all. That seems out of place with the other options. -- John Naylor EDB: http://www.enterprisedb.com

Re: A qsort template

2022-01-12 Thread John Naylor
tgresql.org/message-id/CAD21AoAfOZvmfR0j8VmZorZjL7RhTiQdVttNuC4W-Shdc2a-AA%40mail.gmail.com [2] https://www.postgresql.org/message-id/CAFBsxsG_c24CHKA3cWrOP1HynWGLOkLb8hyZfsD9db5g-ZPagA%40mail.gmail.com -- John Naylor EDB: http://www.enterprisedb.com

Re: do only critical work during single-user vacuum?

2022-01-13 Thread John Naylor
just wants to manually vacuum everything (including > indexes) with an age above 500M on the weekends. There is already vaccumdb for that, and I think it's method of selecting tables is sound -- I'm not convinced that pushing table selection to the server command as "options" is an improvement. -- John Naylor EDB: http://www.enterprisedb.com

Re: do only critical work during single-user vacuum?

2022-01-14 Thread John Naylor
le other things. -- John Naylor EDB: http://www.enterprisedb.com

Re: WIP: BRIN multi-range indexes

2021-03-17 Thread John Naylor
850 rows=1128038 loops=1) Recheck Cond: ((create_dt >= '2020-02-01 00:00:00-04'::timestamp with time zone) AND (create_dt < '2020-03-01 00:00:00-04'::timestamp with time zone)) Rows Removed by Index Recheck: 140698 Heap Blocks: lossy=15104 -> Bitmap Index Scan on cd_multi (cost=0.00..143.36 rows=1128092 width=0) (actual time=1.609..1.609 rows=151040 loops=1) Index Cond: ((create_dt >= '2020-02-01 00:00:00-04'::timestamp with time zone) AND (create_dt < '2020-03-01 00:00:00-04'::timestamp with time zone)) Planning Time: 1.421 ms Execution Time: 319.891 ms (8 rows) So, 0007 doesn't make a difference in this case, but 0008 does. -- John Naylor EDB: http://www.enterprisedb.com

Re: non-HOT update not looking at FSM for large tuple update

2021-03-17 Thread John Naylor
uum, since in parallel tests there could still be tuples visible to other transactions. It's still possible to test almost-all-free by inserting a small tuple. - Draft commit message -- John Naylor EDB: http://www.enterprisedb.com v4-0001-Fix-bug-in-heap-space-management-that-was-overly-.patch Description: Binary data

Re: WIP: BRIN multi-range indexes

2021-03-17 Thread John Naylor
On Wed, Mar 17, 2021 at 3:16 PM Tomas Vondra wrote: > Ummm, no attachment ;-) Oops, here it is. -- John Naylor EDB: http://www.enterprisedb.com jcn-costing-test.sql Description: Binary data

Re: popcount

2021-03-18 Thread John Naylor
Hi David, Just a nitpick: +SET bytea_output TO hex; Since we don't see the string in the output, I don't immediately see the reason to change the output format here? Aside from that, this patch works as expected, and is ready for committer. -- John Naylor EDB: http://www.enterprisedb.com

Re: Force lookahead in COPY FROM parsing

2021-03-18 Thread John Naylor
er/blackhole_am -- John Naylor EDB: http://www.enterprisedb.com

Re: Perform COPY FROM encoding conversions in larger chunks

2021-03-18 Thread John Naylor
st-copy.txt ; done; On Linux x86-64, gcc 8.4, I get these numbers (minimum of five runs): master: 109ms v6 do encoding in larger chunks: 109ms v7 utf8 SIMD: 98ms -- John Naylor EDB: http://www.enterprisedb.com

Re: Perform COPY FROM encoding conversions in larger chunks

2021-03-18 Thread John Naylor
On Thu, Mar 18, 2021 at 2:05 PM John Naylor wrote: > > I wrote: > > > I went ahead and rebased these. > > It looks like FreeBSD doesn't like this for some reason. On closer examination, make check was "terminated", not that the tests failed... -- John Naylor EDB: http://www.enterprisedb.com

Re: non-HOT update not looking at FSM for large tuple update

2021-03-19 Thread John Naylor
; > This is slightly more verbose, but I think this clarifies the > reasoning why we need this a bit better. Feel free to reject or adapt > as needed. I like this in general, but still has some rough edges. I've made another attempt in v5 incorporating your suggestions. Let me know what you think. -- John Naylor EDB: http://www.enterprisedb.com v5-0001-Fix-bug-in-heap-space-management-that-was-overly-.patch Description: Binary data

Re: Speeding up GIST index creation for tsvectors

2021-03-19 Thread John Naylor
On Fri, Mar 19, 2021 at 8:57 AM Amit Khandekar wrote: > > On Thu, 11 Mar 2021 at 04:25, John Naylor wrote: > > Okay, so it's hard-coded in various functions in contrib modules. In that > > case, let's just keep the existing coding for those. In fact, the comments

Re: Tying an object's ownership to datdba

2021-03-24 Thread John Naylor
, note that its behavior > doesn't match its documentation > (https://postgr.es/m/flat/20060728170615.gy20...@kenobi.snowman.net). Is this something that needs fixing separately? -- John Naylor EDB: http://www.enterprisedb.com

Re: truncating timestamps on arbitrary intervals

2021-03-24 Thread John Naylor
er to read, depending on taste. I think it's quite possible some users will prefer that. All we need is to add something like proargnames => '{bin_width,input,origin}' to the catalog, right? Also, I noticed that I put in double semicolons in the new functions somehow.

Re: truncating timestamps on arbitrary intervals

2021-03-24 Thread John Naylor
cannot contain units of a month or larger. The original seems fine to me here. -- John Naylor EDB: http://www.enterprisedb.com

Re: Tying an object's ownership to datdba

2021-03-25 Thread John Naylor
for another review? I've marked it Ready for Committer. -- John Naylor EDB: http://www.enterprisedb.com

Re: non-HOT update not looking at FSM for large tuple update

2021-03-27 Thread John Naylor
page" logic does so. An > FSM-using test would contain a VACUUM.) I plan to commit the attached > version; compared to v5, it updates comments and renames this variable. Looks good to me, thanks! -- John Naylor EDB: http://www.enterprisedb.com

Re: Add missing function abs (interval)

2021-03-29 Thread John Naylor
seems like it might be reasonable to implement a built-in function as an SQL function they tend to be internal. By default it should be internal. -- John Naylor EDB: http://www.enterprisedb.com

Re: truncating timestamps on arbitrary intervals

2021-03-30 Thread John Naylor
Currently, when the origin is after the input, the result is the timestamp at the end of the bin, rather than the beginning as expected. The attached puts the result consistently at the beginning of the bin. -- John Naylor EDB: http://www.enterprisedb.com rationalize-future-origin.patch

Re: truncating timestamps on arbitrary intervals

2021-03-30 Thread John Naylor
also treat negative strides differently somehow, but I don't immediately see a useful and/or intuitive change in behavior to come of that. -- John Naylor EDB: http://www.enterprisedb.com synopsis-and-add-to-description.patch Description: Binary data

Re: [POC] verifying UTF-8 using SIMD instructions

2021-04-01 Thread John Naylor
v9 is just a rebase. -- John Naylor EDB: http://www.enterprisedb.com From e876049ad3b153e8725ab23f65ae8f021a970470 Mon Sep 17 00:00:00 2001 From: John Naylor Date: Thu, 1 Apr 2021 08:24:05 -0400 Subject: [PATCH v9] Replace pg_utf8_verifystr() with two faster implementations: On x86-64, we use

Re: truncating timestamps on arbitrary intervals

2021-04-01 Thread John Naylor
ions. Yes, thanks I'm aware of it. It's a bit more feature-rich, and I wanted to have something basic that users can have available without installing an extension. -- John Naylor EDB: http://www.enterprisedb.com

Re: Force lookahead in COPY FROM parsing

2021-04-02 Thread John Naylor
ave nothing further so it's RFC. The patch is pretty simple compared to the earlier ones, but is worth running the fuzzer again as added insurance? As an aside, I noticed the URL near the top of copyfromparse.c that explains a detail of macros has moved from http://www.cit.gu.ed

Re: truncating timestamps on arbitrary intervals

2021-04-10 Thread John Naylor
On Sat, Apr 10, 2021 at 7:43 AM Peter Eisentraut < peter.eisentr...@enterprisedb.com> wrote: > > On 30.03.21 18:06, John Naylor wrote: > > Currently, when the origin is after the input, the result is the > > timestamp at the end of the bin, rather than the beginning as ex

Re: factorial function/phase out postfix operators?

2020-07-29 Thread John Naylor
On Wed, Jul 22, 2020 at 8:47 AM Mark Dilger wrote: > > > > > On Jul 18, 2020, at 1:00 AM, John Naylor > > wrote: > > > > pg_get_keywords() should probably have a column to display ability to > > act as a bare col label. Perhaps a boolean? If so, what do yo

Re: Problems with the FSM, heap fillfactor, and temporal locality

2020-08-21 Thread John Naylor
pdating related tuples would concentrate dead tuples on fewer pages, and vacuums would more quickly free up space that can actually be used to store multiple new tuples. [1] https://www.slideshare.net/masahikosawada98/vacuum-more-efficient-than-ever-99916671 -- John Naylorhttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: factorial function/phase out postfix operators?

2020-08-24 Thread John Naylor
v14. I imagine the release candidates are not too far away by now, and if we are confident enough in the direction the patches in this thread are going, we should probably consider a deprecation notice soon. -- John Naylorhttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: Problems with the FSM, heap fillfactor, and temporal locality

2020-08-24 Thread John Naylor
would make the FSM much bigger, which doesn't sound attractive at all. Any change of the FSM file would require pg_upgrade to rewrite the FSM, but it still doesn't seem like a lot of code. Other ideas? -- John Naylorhttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: Problems with the FSM, heap fillfactor, and temporal locality

2020-08-26 Thread John Naylor
stored with the relcache target block doesn't match the current one. Also num-new-blocks above could be scaled down from the actual number of blocks added, just to make sure writes aren't happening all over the place. There might be holes in this idea, but it may be worth trying to be bet

Re: factorial function/phase out postfix operators?

2020-08-26 Thread John Naylor
ic for this patch. Not sure it matters much, either. > > Well, I did touch that function a bit, adding a new column, and the number of > rows returned is exactly 450, so if I'm not going to update it, who will? > The count may increase over time if other keywords are added, but I doubt

Re: factorial function/phase out postfix operators?

2020-08-27 Thread John Naylor
On Wed, Aug 26, 2020 at 6:57 PM Mark Dilger wrote: > > > > > On Aug 26, 2020, at 6:33 AM, John Naylor > > wrote: > > > and since POSTFIXOP is gone > > it doesn't seem to matter how low it is. In fact, I found that the > > lines with INDENT

Re: factorial function/phase out postfix operators?

2020-08-27 Thread John Naylor
a tag to make it more visible, like in the attached, compressed to avoid confusing the cfbot. -- John Naylorhttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services postfix-depr.patch.gz Description: GNU Zip compressed data

Re: Deprecating postfix and factorial operators in PostgreSQL 13

2020-08-28 Thread John Naylor
, it looks like in v14 we'll just have a different operator entirely for the example, so ideally we would backpatch that change for v13. What you have is good enough in a pinch, though. -- John Naylorhttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: factorial function/phase out postfix operators?

2020-08-28 Thread John Naylor
On Thu, Aug 27, 2020 at 5:04 PM Tom Lane wrote: > I'm a bit inclined to kill them both off and standardize on factorial() > (not numeric_fac). +1 -- John Naylorhttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: factorial function/phase out postfix operators?

2020-08-28 Thread John Naylor
don't think we don't need to do any precedence refactoring in order to apply the functional change of these patches. We could leave that for follow-on patches once we figure out the best way forward, which could take some time. -- John Naylorhttps://www.2ndQuadrant.com/ Pos

Re: factorial function/phase out postfix operators?

2020-09-02 Thread John Naylor
this file: -SELECT 4!; -SELECT !!3; +SELECT factorial(4); +SELECT factorial(3); -- John Naylorhttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: Problems with the FSM, heap fillfactor, and temporal locality

2020-09-02 Thread John Naylor
On Wed, Sep 2, 2020 at 1:57 AM Peter Geoghegan wrote: > > On Wed, Aug 26, 2020 at 1:46 AM John Naylor > wrote: > > The fact that that logic extends by 20 * numwaiters to get optimal > > performance is a red flag that resources aren't being allocated > > effi

Re: [doc] minor wording improvement in a couple of places

2020-09-07 Thread John Naylor
le for additional details." >> > > I agree this sounds very weird. However, this phrasing is also fairly common > in comments around the code. Seems to be a literal translation of Latin "quod vide" (q.v.), so intentional, just sounds a bit old fashioned. I'd

Re: factorial function/phase out postfix operators?

2020-09-07 Thread John Naylor
On Thu, Sep 3, 2020 at 11:50 AM Mark Dilger wrote: > [v7] Ok, I've marked it ready for committer. -- John Naylorhttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: PATCH: Attempt to make dbsize a bit more consistent

2020-09-08 Thread John Naylor
data * is in the main fork rather than some other), it can use this implementation * of the relation_size callback rather than implementing its own. -- John Naylorhttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: WIP: BRIN multi-range indexes

2020-09-09 Thread John Naylor
st to compare size: BRIN 32kB BRIN multi 136kB Btree 188MB [1] https://www.postgresql.org/message-id/459eef3e-48c7-0f5a-8356-992442a78bb6%402ndquadrant.com -- John Naylorhttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: Minor fixes for upcoming version 13

2020-09-09 Thread John Naylor
n postgres.po and > ecpg.po. So I think it should at least puzzle translators. Maybe instead Assert(0) with a comment or something like that? -- John Naylorhttps://www.2ndQuadrant.com/ PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services

Re: WIP: BRIN multi-range indexes

2020-09-10 Thread John Naylor
Recheck Cond: ((create_dt >= '2020-02-01 00:00:00-04'::timestamp with time zone) AND (create_dt < '2020-03-01 00:00:00-04'::timestamp with time zone)) Rows Removed by Index Recheck: 141619 Heap Blocks: lossy=15104 -> Bitmap Index Scan

Re: [RFC] building postgres with meson

2021-10-12 Thread John Naylor
s: ninja, gdbm, ca-certificates, openssl@1.1, readline, sqlite and python@3.9 Also, could utility makefile targets be made to work? I'm thinking in particular of update-unicode and reformat-dat-files, for example. -- John Naylor EDB: http://www.enterprisedb.com

Re: [RFC] building postgres with meson

2021-10-13 Thread John Naylor
On Tue, Oct 12, 2021 at 4:59 PM Andres Freund wrote: > On 2021-10-12 15:55:22 -0400, John Naylor wrote: > > (I couldn't get a build working but I'll leave that aside for now). > > If you want to do that separately, I'll try to fix it. Okay, I pulled the latest com

Re: [RFC] building postgres with meson

2021-10-13 Thread John Naylor
uilddir with > meson configure -dldap=disabled or when setting up a new builddir by passing > -dldap=disabled at that time. Somehow our emails got lower-cased down here, but I tried it with capital D: meson configure -Dldap=disabled inside the build dir and got this: ../meson.build:278:2:

Re: [RFC] building postgres with meson

2021-10-13 Thread John Naylor
n/pgdev/meson/build/testrun/main/pg_regress/log/initdb.log dyld: Library not loaded: /usr/local/lib/libpq.5.dylib Referenced from: /Users/john/pgdev/meson/build/tmp_install/usr/local/bin/initdb Reason: image not found -- John Naylor EDB: http://www.enterprisedb.com

Re: [RFC] building postgres with meson

2021-10-14 Thread John Naylor
art the server just fine, so that much works. -- John Naylor EDB: http://www.enterprisedb.com

Re: [RFC] building postgres with meson

2021-10-14 Thread John Naylor
System Information says it's disabled. Running "csrutil status" complains of an unsupported configuration, which doesn't sound good, so I should probably go fix that independent of anything else. :-/ -- John Naylor EDB: http://www.enterprisedb.com

Re: [RFC] building postgres with meson

2021-10-14 Thread John Naylor
urning something off to allow a debugger to run. Here are all the settings, in case it matters: Apple Internal: disabled Kext Signing: enabled Filesystem Protections: enabled Debugging Restrictions: disabled DTrace Restrictions: enabled NVRAM Protections: enabled BaseSystem Verification: enabled -- John Naylor EDB: http://www.enterprisedb.com

Re: [RFC] building postgres with meson

2021-10-14 Thread John Naylor
ct like installcheck? [checks] Yep, "meson test" ran fine (*). It still ran the temp install first, but in any case it worked. The full "configure step" was meson setup build --buildtype debug -Dldap=disabled -Dcassert=true -Dprefix=$(pwd)/inst * (all passed but skipped subs

Re: speed up verifying UTF-8

2021-10-19 Thread John Naylor
h > https://github.com/BobSteagall/utf_utils/blob/6b7a465265de2f5fa6133d653df0c9bdd73bbcf8/src/utf_utils.cpp > in the header of src/port/pg_utf8_fallback.c? > > It would make the URL more stable in case the file gets renamed. > > Vladimir > Makes sense, so done that way. -- John Naylor EDB: htt

Re: [RFC] building postgres with meson

2021-10-19 Thread John Naylor
Hi, I know this is still in the evaluation stage, but I did notice some discrepencies in the Flex flags. With the attached patch, the read-only data segment seems to match up pretty well now. -- John Naylor EDB: http://www.enterprisedb.com sync-flex-flags-with-autoconf-build.patch Description

[RFC] speed up count(*)

2021-10-20 Thread John Naylor
ffort? -- John Naylor EDB: http://www.enterprisedb.com

Re: [RFC] speed up count(*)

2021-10-20 Thread John Naylor
on is that people don't have quite as high an expectation for queries with more complex predicates and/or grouping. It would be interesting to see what the balance is. -- John Naylor EDB: http://www.enterprisedb.com

Re: [RFC] speed up count(*)

2021-10-20 Thread John Naylor
indexes for min/max is not possible? I just gathered it was not implemented in the planner, going by the explain plan in the toy query I tried, and then I got the lightbulb in my head about count(*). -- John Naylor EDB: http://www.enterprisedb.com

Re: [RFC] building postgres with meson

2021-10-22 Thread John Naylor
le `unaccent.rules' ...so in this case it seems not to know to use CWD here. Anyway, this can be put off until the very end, since it's not run often. You've demonstrated how these targets would work, and that's good enough for now. -- John Naylor EDB: http://www.enterprisedb.com

Re: [Proposal] Fully WAL logged CREATE DATABASE - No Checkpoints

2021-11-23 Thread John Naylor
, because of the possibility of failure + * during transaction commit, but it should handle most scenarios. And do we need additional tests? Maybe we don't, but it seems good to make sure. I haven't looked at 0007, and I have no opinion on which approach is better. -- John Naylor EDB: http://www.enterprisedb.com

Re: Parallel vacuum workers prevent the oldest xmin from advancing

2021-11-24 Thread John Naylor
, let's wait for a day or so to see if anybody else has any ideas to > write a test for this case, otherwise, I'll check these once again and > push. I set this "committed" in the CF app. -- John Naylor EDB: http://www.enterprisedb.com

Re: Non-decimal integer literals

2021-11-25 Thread John Naylor
ing junk after numeric literal at or near "1b" +LINE 1: SELECT 1b; + ^ We might consider some tests for ECPG since lack of coverage has been a problem. Also, I'm curious: how does the spec work as far as deciding the year of release, or feature-freezing of new items? -- John Naylor EDB: http://www.enterprisedb.com

Re: WIP: a way forward on bootstrap data

2018-03-22 Thread John Naylor
of pg_proc.h with both formats as I understand them. I turned a couple comments into multi-line comments to demonstrate. I think without spaces it's just as hard to read as with multiple annotations. I'd vote for

Re: WIP: a way forward on bootstrap data

2018-03-22 Thread John Naylor
e this octal in pg_type.h chartypdelim BKI_DEFAULT(\054); Which I hope is fine. Were you thinking of this comment in pg_attribute.h? We use the double-quoted empty string for postgres.bki and change it to '\0' for schemapg.h. /* One of the ATTRIBUTE_IDENTITY_* constants below, or '\0' */ charattidentity BKI_DEFAULT(""); -John Naylor

Re: WIP: a way forward on bootstrap data

2018-03-22 Thread John Naylor
ght "only a machine could be so precisely awkward" -John Naylor

Re: WIP: a way forward on bootstrap data

2018-03-22 Thread John Naylor
d change it to '\0' for schemapg.h. > >> /* One of the ATTRIBUTE_IDENTITY_* constants below, or '\0' */ >> char attidentity BKI_DEFAULT(""); > > That definitely seems like a hack --- why not BKI_DEFAULT('\0') ? Hmm, yes, the way I had it, the comment is a mystery. I'll switch it around. -John Naylor

Re: WIP: a way forward on bootstrap data

2018-03-25 Thread John Naylor
0007). -A couple minor cosmetic rearrangements and comment/commit message edits. Open items: -Test MSVC. -Arrange for rewrite_dat.pl to run when perltidy does. -I was a bit cavalier about when to use =/:= in the Makefiles. Not sure if there's a preferred project style for when the choice doesn&

Re: WIP: a way forward on bootstrap data

2018-03-26 Thread John Naylor
it'd be good to split off the data-format section of the README into a new file, maybe README.data, which will contain code snippets and some example scenarios. I'll include the example pg_proc.prokind merger among those. -John Naylor

Re: WIP: a way forward on bootstrap data

2018-03-26 Thread John Naylor
With the attachments this time. -John Naylor #-- # # pg_aggregate.dat #Initial contents of the pg_aggregate system relation. # # Portions Copyright (c) 1996-2018, PostgreSQL Global Development Group # Portions Copyright (c

Re: WIP: a way forward on bootstrap data

2018-03-26 Thread John Naylor
rather silly to have SGML documentation for the > .BKI file format, which now will be an internal matter that hardly > any developers need worry about, but not for the .DAT file format. > But I understand if that seems a bridge too far for today --- certainly > a README file is way better than nothing. Makes sense on all points. I’m not optimistic about creating a new sgml doc on time, but I’ll keep it in mind. -John Naylor

Re: WIP: a way forward on bootstrap data

2018-03-27 Thread John Naylor
ikely built already, running duplicate_oids in the process. I think I'll consolidate all documentation patches into one, at the end of the series for maximum flexibility. I liked the idea of spreading the doc changes over the patches, but there is not a huge amount of time left. -John Naylor --- /ho

Re: JIT compiling with LLVM v12.2

2018-03-29 Thread John Naylor
our intent, maybe that's also useful information. :-) -John Naylor diff --git a/src/backend/jit/README b/src/backend/jit/README index bfed319..7924127 100644 --- a/src/backend/jit/README +++ b/src/backend/jit/README @@ -13,12 +13,12 @@ the CPU that just handles that expression, yielding a speedup

Re: pgsql: Add documentation for the JIT feature.

2018-03-29 Thread John Naylor
ed, JIT'd, and jitted. No one agrees on that, but it seems very common to use "JIT" as a verb. See the LLVM docs: https://llvm.org/docs/DebuggingJITedCode.html -John Naylor

Re: WIP: a way forward on bootstrap data

2018-03-31 Thread John Naylor
hich to put convenience targets for formatting. On the other hand, it kind of makes sense to have the files describing the schema (.h) and the contents (.dat) in the same directory. I'm inclined to leave things as they are for that reason. -John Naylor v13-bootstrap-data-conversion.tar.gz Des

Re: WIP: a way forward on bootstrap data

2018-04-05 Thread John Naylor
t. I'll put out a new one in a couple days, to give a chance for further review and discussion of the defaults. I didn't feel the need to respond to the other messages, but yours and Andres' points are well taken. -John Naylor diff --git a/src/backend/catalog/genbki.pl b/src/backe

Re: WIP: a way forward on bootstrap data

2018-04-05 Thread John Naylor
-CATALOG(pg_am,2601) +CATALOG(pg_am,2601) BKI_REL_LABEL(AccessMethod) I haven't thought this through yet. I imagine it will add as well as remove a bit of complexity, code-wise. The upside is most CATALOG lines will remain unchanged, and those that do won't end up quite as long. I can try a draft tomorrow to see how it looks, unless you see an obvious downside. -John Naylor

Re: WIP: a way forward on bootstrap data

2018-04-05 Thread John Naylor
ues{aggfnoid} = lookup_procname($values{aggfnoid}); + } aggfnoid is in pg_aggregate, and pg_am already had a regproc lookup. Do you remember the intent here? -John Naylor

Re: WIP: a way forward on bootstrap data

2018-04-05 Thread John Naylor
ble to figure out where to route the descriptions. Fairly trivial (attached), and shouldn't be too hard to integrate into the series. -John Naylor diff --git a/src/backend/catalog/README.data b/src/backend/catalog/README.data index 22ad0f2..2c05fab 100644 --- a/src/backend/catalog/

Re: WIP: a way forward on bootstrap data

2018-04-05 Thread John Naylor
ould "float" more, and the scope is not as clear. I'm okay with whatever the community thinks, but at this point I'm inclined to leave things as they are and focus on the other points of review for the next patchset. While on the subject of viewing, I do have a badly outdated patch that would create a postgres.sql file which would load into a development schema so one could query the bootstrap data in a database without running initdb. I could update it at a future point. -John Naylor

Re: WIP: a way forward on bootstrap data

2018-04-06 Thread John Naylor
wner => PGUID, opfname => abstime_ops, at ../../../src/backend/catalog/Catalog.pm line 259, <$ifd> line 16. Makefile:23: recipe for target 'reformat-dat-files' failed make: *** [reformat-dat-files] Error 25 I think the context is good for pg_attribute, because there is no file to read from. I'll think about the lookup code. -John Naylor

Re: WIP: a way forward on bootstrap data

2018-04-06 Thread John Naylor
aultValues, is to be able to finger the location > of the bad tuple by filename and line number, but I have no idea whether > it's practical to annotate the tuples with that while reading the .dat > files. Any thoughts? Done (patch 0007). So far only lookup_oids() uses it. -John Nayl

Re: WIP: a way forward on bootstrap data

2018-04-06 Thread John Naylor
with an EXPOSE_TO_CLIENT_CODE macro. All your additional patches apply still apply over it. Your SGML patch can only apply if my doc patch is not applied, but I've included it anyway for the sake of no surprises. I'll check back in 24 hours to see if everything still applies. -John

Re: WIP: a way forward on bootstrap data

2018-04-08 Thread John Naylor
I wrote: > I'll check back in 24 hours to see if everything still applies. There were a couple more catalog changes that broke patch context, so attached is version 16. -John Naylor v16-bootstrap-data-conversion.tar.gz Description: GNU Zip compressed data

Re: WIP: a way forward on bootstrap data

2018-04-08 Thread John Naylor
mits, and to all who participated in previous discussion. -John Naylor

Re: Verbosity of genbki.pl

2018-04-09 Thread John Naylor
uess a second one for fmgroids.h and related files. >> >> I don't have a strong preference. Opinions? > > Second point, pls. I'd like to see some stage done The attached patch does #2. -John Naylor diff --git a/src/backend/catalog/Catalog.pm b/src/backend/catalog/Catal

Re: Documentation for bootstrap data conversion

2018-04-09 Thread John Naylor
to "bootstrap" relations rather than "bootstrapped" relations. The attached patch makes code comments more like the docs in this regard. -John Naylor diff --git a/src/backend/bootstrap/bootstrap.c b/src/backend/bootstrap/bootstrap.c index 59cd4b1..3da7861 100644 --- a/src/backe

Re: Documentation for bootstrap data conversion

2018-04-09 Thread John Naylor
n is done, it'd be better > to remove that special case. Would you send a patch for that? Sure, attached. -John Naylor diff --git a/src/include/catalog/reformat_dat_file.pl b/src/include/catalog/reformat_dat_file.pl index bbceb16..038ba7b 100644 --- a/src/include/catalog/reformat_dat_file

Re: Documentation for bootstrap data conversion

2018-04-16 Thread John Naylor
of the system "access method" catalog (pg_am) so we may as well go the extra step and do: definition of the "access method" system catalog (pg_am) Thoughts? Beyond that, there are many cases where the language might be debatable, or at least it's not obvious to the casual observer whether it could also be referring to an index or toast table, so it's probably best to leave well enough alone for most cases. -John Naylor

remove quoting hacks and simplify bootscanner.l

2018-04-17 Thread John Naylor
be quoted. Currently, Catalog.pm, genbki.pl, and bootscanner.l all have different ideas on how to parse and format array types. Patch 0003 rips all that out and does it once and for all in Catalog.pm. Change in postgres.bki: Array types now look like '_foo'

Re: WIP: a way forward on bootstrap data

2018-04-17 Thread John Naylor
edSecLabel) BKI_SHARED_RELATION BKI_ROWTYPE_OID(4066) BKI_WITHOUT_OIDS BKI_SCHEMA_MACRO -- with Patch 0002: 5 lines > 80 chars 102 CATALOG(pg_shseclabel,3592,SharedSecLabel) SHARED_RELATION ROWTYPE_OID(4066) WITHOUT_OIDS SCHEMA_MACRO -John Naylor From ff1384643e47ef423ab13cfd847f12c0969029f1 Mo

<    1   2   3   4   5   6   7   8   9   10   >