Re: \describe*

2019-02-23 Thread Corey Huinker
> > Given that this patch has been added to the last commitfest for v12, I > think we should mark it as targeting 13, so it can be skipped over by > people looking to get things into v12. Even leaving fairness aside, I > don't think it's likely to be ready quickly enough... > Obviously this patch

Re: some ri_triggers.c cleanup

2019-02-25 Thread Corey Huinker
> > Right, this makes a lot of sense, similar to how ri_restrict() combines > RESTRICT and NO ACTION. > I'm pretty sure that's where I got the idea, yes.

Re: Referential Integrity Checks with Statement-level Triggers

2019-02-25 Thread Corey Huinker
> > > In order to avoid per-row calls of the constraint trigger functions, we > could > try to "aggregate" the constraint-specific events somehow, but I think a > separate queue would be needed for the constraint-specific events. > > In general, the (after) triggers and constraints have too much in

Referential Integrity Checks with Statement-level Triggers

2018-12-17 Thread Corey Huinker
Back when Pg added statement-level triggers, I was interested in the potential promise of moving referential integrity checks to statement-level triggers. The initial conversation, along with Kevin Grittner's POC script (in SQL) that showed a potential for a 98% reduction in time spent doing RI ch

Statement-level Triggers For Uniqueness Checks

2018-12-17 Thread Corey Huinker
In digging around the codebase (see thread: Referential Integrity Checks with Statement-level Triggers), I noticed that unique constraints are similarly enforced with a per-row trigger. The situation with unique indexes is fairly similar to the situation with RI checks: there is some overhead to u

Re: Statement-level Triggers For Uniqueness Checks

2018-12-24 Thread Corey Huinker
| t | 0 || \x || | pg_inserted_transition_table (2 rows) Any idea where I went wrong? On Mon, Dec 17, 2018 at 9:56 AM Corey Huinker wrote: > In digging around the codebase (see thread: Referential Integrity Checks > with Statement-level Triggers),

Re: Statement-level Triggers For Uniqueness Checks

2019-01-08 Thread Corey Huinker
On Fri, Jan 4, 2019 at 7:49 AM Peter Eisentraut wrote: > > On 25/12/2018 00:56, Corey Huinker wrote: > > The regression diff (attached) seems to imply that the triggers simply > > are not firing, though. > > The reason for this was explained by Dean. If you take out the c

Re: Add A Glossary

2020-02-11 Thread Corey Huinker
ually found the time to take a second pass at this. Attached is a revised patch. From f087e44fe4db7996880cf4df982297018d444363 Mon Sep 17 00:00:00 2001 From: Corey Huinker Date: Wed, 12 Feb 2020 04:17:59 + Subject: [PATCH] add glossary page with initial definitions --- doc/src/sgml/filelist.sg

Re: Resolving the python 2 -> python 3 mess

2020-02-17 Thread Corey Huinker
> > A possible gotcha in this approach is if there are any python 2/3 > incompatibilities that would not manifest as syntax errors or > obvious runtime errors, but would allow old code to execute and > silently do the wrong thing. One would hope that the Python crowd > weren't dumb enough to do th

Re: Resolving the python 2 -> python 3 mess

2020-02-18 Thread Corey Huinker
> > So, as with Jesse's example, what I'm wondering is whether or not 2to3 > will fix that for you (or even flag it). The basic difference between > the two alternatives I suggested is whether we force people to put their > python function through that converter before we'll even try to run it. >

Re: Extracting only the columns needed for a query

2019-06-16 Thread Corey Huinker
> > The thing that most approaches to this have fallen down on is triggers --- > that is, a trigger function might access columns mentioned nowhere in the > SQL text. (See 8b6da83d1 for a recent example :-() If you have a plan > for dealing with that, then ... > Well, if we had a trigger languag

Re: \describe*

2019-06-22 Thread Corey Huinker
> > > So what is the uptake on implementing this at the server side, ie. > > DESCRIBE? > > I'm pretty skeptical of this idea, unless you are willing to throw > away at least one and possibly both of the following goals: > > 1. Compatibility with psql's existing \d behavior. > I don't think *compat

Re: SHOW CREATE

2019-07-05 Thread Corey Huinker
On Fri, Jul 5, 2019 at 12:32 PM David Fetter wrote: > Folks, > > Corey Huinker put together the documentation for this proposed > feature. Does this seem like a reasonable way to do it? > > In doing that work, it became clear that the command was serving two masters: 1. A

Re: Catching missing Datum conversions

2019-07-20 Thread Corey Huinker
> > I should probably split this into "actionable" (categories 3 and 4) > and "noise and scaffolding" patches. > Breaking down the noise-and-scaffolding into some subgroups might make the rather long patches more palatable/exceedingly-obvious: * (Datum) 0 ---> NullDatum * 0 > NullDatum * The D

Re: Desirability of client-side expressions in psql?

2018-07-10 Thread Corey Huinker
> > >psql> \if :i >= 5 > > I think we're ok with that so long as none of the operators or values has a \ in it. What barriers do you see to re-using the pgbench grammar?

Re: Alter index rename concurrently to

2018-07-25 Thread Corey Huinker
> > You appear to be saying that you think that renaming an index > concurrently is not safe. In that case, this patch should be rejected. > However, I don't think it necessarily is unsafe. What we need is some > reasoning about the impact, not a bunch of different options that we > don't underst

Re: Feature proposal for psql

2020-09-19 Thread Corey Huinker
>> One limitation of this approach is that \set can't span lines, so >> writing complex queries would be kinda painful. But that would >> be a good limitation to address separately; \set isn't the only >> metacommand where can't-span-lines is a problem sometimes. >> If you seriously want to pursue

Re: [RFC] Add an until-0 loop in psql

2018-04-27 Thread Corey Huinker
As of v11, DO blocks can do transactions. I think this will meet your needs. A loop that starts at point X in the code and terminates at point Y has to know how to jump back in the file (if there even is a file!) to point X and re-interpret commands as it makes it's way back through the "file" tow

Re: [RFC] Add an until-0 loop in psql

2018-04-27 Thread Corey Huinker
>> What you don't see here is that you're using your psql process's >> available open file handles as a stack, and when you hit that limit psql >> will fail. If you remove that limit, then you get a bit further before psql >> segfaults on you. I think I got ~2700 files deep before that happened. Yo

Re: [RFC] Add an until-0 loop in psql

2018-04-28 Thread Corey Huinker
> > > I did it too. It is not too hard - there was not a agreement on syntax. > > can be nice some like CTE > > WITH PROCEDURE x(a int, b int) AS ... $$ SELECT x(10); > I've seen "WITH function" syntax in Oracle (here's an example: https://oracle-base.com/articles/12c/with-clause-enhancements-

Re: [RFC] Add an until-0 loop in psql

2018-04-30 Thread Corey Huinker
On Mon, Apr 30, 2018 at 7:05 AM Pierre Ducroquet < pierre.ducroq...@people-doc.com> wrote: > On Monday, April 30, 2018 1:01:25 PM CEST Daniel Verite wrote: > > Corey Huinker wrote: > > > As of v11, DO blocks can do transactions. I think this will meet your > >

Re: Query Rewrite for Materialized Views (Postgres Extension)

2018-06-18 Thread Corey Huinker
> > Hope it is useful or interesting for someone! Questions or comments are >> very welcome. >> > > good idea. > > Regards > > Pavel > In a recent PgConf NYC presentation [1] I was talking about the technical hurdles to implementing materialized views that could be kept up to date at all times, an

Re: [HACKERS] SQL procedures

2017-11-22 Thread Corey Huinker
> > > T-SQL procedures returns data or OUT variables. > > I remember, it was very frustrating > > Maybe first result can be reserved for OUT variables, others for multi > result set > > It's been many years, but if I recall correctly, T-SQL returns a series of result sets, with no description of th

Re: CREATE ROUTINE MAPPING

2018-09-10 Thread Corey Huinker
On Mon, Sep 10, 2018 at 3:28 AM Hannu Krosing wrote: > Hi Corey > > Have you looked at pl/proxy ? > I have, a long while ago. > It does this and then some (sharding) > PL/proxy isn't a part of the SQL Standard. PL/proxy only connects to other libpq-speaking databases. The hope with routine ma

Re: Statistics Import and Export

2024-10-14 Thread Corey Huinker
> > However, this function seems to accept -1 for the relpages parameter. > Below is an example of execution: > --- > postgres=> CREATE TABLE data1(c1 INT PRIMARY KEY, c2 VARCHAR(10)); > CREATE TABLE > postgres=> SELECT pg_set_relation_stats('data1', relpages=>-1); > pg_set_relation_stats > --

Re: Statistics Import and Export

2024-10-14 Thread Corey Huinker
On Mon, Oct 14, 2024 at 3:40 PM Corey Huinker wrote: > However, this function seems to accept -1 for the relpages parameter. >> Below is an example of execution: >> --- >> postgres=> CREATE TABLE data1(c1 INT PRIMARY KEY, c2 VARCHAR(10)); >> CRE

Re: Statistics Import and Export

2024-10-15 Thread Corey Huinker
> Oh, I see. It appears that there's a special case for partitioned > tables that sets relpages=-1 in do_analyze_rel() around line 680. It's > a bit inconsistent, though, because even partitioned indexes have > relpages=0. Furthermore, the parameter is of type BlockNumber, so > InvalidBlockNumber w

Re: Statistics Import and Export

2024-10-18 Thread Corey Huinker
> > What is the benefit of a return value from the pg_set_*_stats variants? > As far as I can tell, there is none because they throw an ERROR if > anything goes wrong, so they should just return VOID. What am I > missing? > Probably nothing. The nuances of "this stat didn't get set but these other

Re: Statistics Import and Export

2024-10-18 Thread Corey Huinker
> > > > Patch that allows relation_statistics_update to continue after one failed > stat (0001) attached, along with bool->void change (0002). > Once more, with feeling: From 0cf8a90c37ec95bbbefb274bc3d48a077b4868b8 Mon Sep 17 00:00:00 2001 From: Corey Huinker Date: Wed,

Re: Statistics Import and Export

2024-10-16 Thread Corey Huinker
ing false. From c0efe3fb2adac102e186e086bf94fb29fabba28b Mon Sep 17 00:00:00 2001 From: Corey Huinker Date: Tue, 15 Oct 2024 19:09:59 -0400 Subject: [PATCH v2 1/3] Allow pg_set_relation_stats() to set relpages to -1. While the default value for relpages is 0, if a partitioned table with at least one child has been analyzed, then t

Re: Soft errors and ThrowErrorData() comment updates

2024-10-17 Thread Corey Huinker
On Wed, Oct 16, 2024 at 4:20 PM Jeff Davis wrote: > The comments around ThrowErrorData() and how it might apply to soft > errors is slightly confusing. Attached a patch which hopefully > clarifies things. > > From a distance, ThrowErrorData() is somewhat like ReThrowError(), but > it's actually q

Re: Statistics Import and Export

2024-10-17 Thread Corey Huinker
On Wed, Oct 16, 2024 at 7:20 PM Corey Huinker wrote: > Code fix with comment on why nobody expects a relpages -1. Test case to >> demonstrate that relpages -1 can happen, and updated doc to reflect the new >> lower bound. >> > > Additional fixes, now in a patch-set:

Re: Statistics Import and Export

2024-10-10 Thread Corey Huinker
> > This seems like a reasonable refactoring exercise that we could take care > of before the rest of the patch set goes in. I added one new reference to > dopt.schemaOnly in commit bd15b7d, so that should probably be revised to > !dumpData, too. I also noticed a few references to dataOnly/schema

Re: Statistics Import and Export

2024-10-22 Thread Corey Huinker
> > > If the relpages option contains -1 only for partitioned tables, shouldn't > pg_set_relation_stats restrict the values that can be > > specified by table type? The attached patch limits the value to -1 or more > if the target > > is a partition table, and 0 or more otherwise. > > Changing relp

Re: Statistics Import and Export

2024-11-04 Thread Corey Huinker
ats. Users might not see the error, existing tooling wouldn't be able to act on the error, and there are legitimate non-upgrade uses of --analyze-in-stages. MAIN CONCERN GOING FORWARD This change to vacuumdb will require some reworking of the vacuum_one_database() function so that the list of

Re: Statistics Import and Export

2024-11-12 Thread Corey Huinker
> > We should probably not allow that, because you cannot ANALYZE system > columns: > Makes sense, and the fix is changing a single character (unless we think it warrants a test case).

Re: split func.sgml to separated individual sgml files

2024-11-13 Thread Corey Huinker
> > The following is step-by-step logic. > > The end result (one file per section) seems good to me. I suspect that reviewer burden may be the biggest barrier to going forward. Perhaps breaking up the changes so that each new sect1 file gets its own commit, allowing the reviewer to more easily (if

Re: Statistics Import and Export

2024-10-31 Thread Corey Huinker
> > > (c) we are considering whether to use an in-place heap update for the > relation stats, so that a large restore doesn't bloat pg_class -- I'd > like feedback on this idea > I'd also like feedback, though I feel very strongly that we should do what ANALYZE does. In an upgrade situation, nea

Re: Statistics Import and Export

2024-10-15 Thread Corey Huinker
On Tue, Oct 15, 2024 at 2:50 PM Corey Huinker wrote: > > Oh, I see. It appears that there's a special case for partitioned >> tables that sets relpages=-1 in do_analyze_rel() around line 680. It's >> a bit inconsistent, though, because even partitioned indexes hav

Re: Statistics Import and Export

2024-11-27 Thread Corey Huinker
> > > For people that want to use the old behavior of recomputing _all_ > statistics not just the missing ones, we could add a different switch, > or an (optional) option to --analyze-in-stages. > > The current patchset provides that in the form of the parameter "--force-analyze", which is a modifi

Re: More CppAsString2() in psql's describe.c

2024-11-27 Thread Corey Huinker
> > Any objections against applying it? > +1. No objections. It's a great help in communicating meaning.

Re: More CppAsString2() in psql's describe.c

2024-11-28 Thread Corey Huinker
> > > LGTM, I didn't scan for omissions but the ones in the patch look right to > me. > I sort of wish we had a shorter macro as CppAsString2() get's pretty > verbose > when used frequently like this. > I don't quite understand the etymology of the name (it's some variation on C++'s std::to_string

Re: Statistics Import and Export

2024-11-18 Thread Corey Huinker
On Mon, Nov 18, 2024 at 3:32 PM Bruce Momjian wrote: > > How would/could this effect tables that have the vacuum and analyze > > scale_factors different from the rest of db via the ALTE RTABLE > statement? > > > > (I do this a lot) > > I don't think it would affect it since those control autovacu

Re: Statistics Import and Export

2024-11-19 Thread Corey Huinker
> > I don't have a strong opinion here, but I suspect that if I was creating > vacuumdb from scratch, I'd have suggested a --missing-only flag that would > only work for --analyze-only/--analyze-in-stages. That way, folks can > still regenerate statistics if they want, but we also have an answer f

Re: PGSERVICEFILE as part of a normal connection string

2024-11-19 Thread Corey Huinker
> > > > Interesting. We've never had tests for that even for "service". > Perhaps it would be the time to add some tests for the existing case > and the one you are adding? Your test suite should make that easy to > add. > Currently, a lot of our utility scripts (anything that uses connectDatab

Re: Statistics Import and Export

2024-11-26 Thread Corey Huinker
> > > Comments on 0003: > > * If we commit 0003, is it a useful feature by itself or does it > require that we commit some or all of 0004-0014? Which of these need to > be in v18? > Useful by itself. 0004 seems needed to me, unless we're fine with ~50% bloat in pg_class on a new-upgraded system,

Re: Statistics Import and Export

2024-11-26 Thread Corey Huinker
ault for relpages becomes -1 on inherited tables (which it hasn't, yet, and might never). From 0f616d4d44f7e29517ac348ded09da1fef1bcbd9 Mon Sep 17 00:00:00 2001 From: Corey Huinker Date: Thu, 14 Nov 2024 04:58:17 -0500 Subject: [PATCH v33 02/11] Add --no-data option. This option is useful fo

Re: Statistics Import and Export

2024-12-07 Thread Corey Huinker
On Sat, Dec 7, 2024 at 2:27 PM Jeff Davis wrote: > On Tue, 2024-11-26 at 17:11 -0500, Corey Huinker wrote: > > > > > * Why does binary upgrade cause statistics to be dumped? Can you > > > just > > > make pg_upgrade specify the appropriate set of flags? > &g

Re: Statistics Import and Export

2025-02-06 Thread Corey Huinker
rding has been done, some of which I'm still not totally satisfied with, so I'm going to give it another look tomorrow, but am putting this out for reviewers in the mean time. From fb095b7ea75ef366eaae5eb7b5322b9869a760ea Mon Sep 17 00:00:00 2001 From: Corey Huinker Date: Tue, 21 Jan 2025 11:

Re: Statistics Import and Export

2025-02-09 Thread Corey Huinker
actually meant for any dump that has all schema excluded. From 2de404f8187466061469abfaf3a773290cce6af8 Mon Sep 17 00:00:00 2001 From: Corey Huinker Date: Tue, 21 Jan 2025 11:52:58 -0500 Subject: [PATCH v47 1/2] Lock table first when setting index relation statistics. Jian He reported [1] a missing lock relation bug

Re: should we have a fast-path planning for OLTP starjoins?

2025-02-05 Thread Corey Huinker
> > > Hmmm, yeah. But that's only for the INNER JOIN case. But I've seen many > of these star join queries with LEFT JOIN too, and then the FKs are not > needed. All you need is a PK / unique index on the other side. Indeed, many installations specifically _remove_ foreign keys because of the dre

Re: Statistics Import and Export

2025-02-05 Thread Corey Huinker
On Sat, Jan 25, 2025 at 10:02 AM Corey Huinker wrote: > Fixed. Holding off on posting updated patch pending decision on what's the >>> best thing to do with partitioned indexes. >> >> > Though I was able to get it to work multiple ways, the one that seems to >

Re: Statistics Import and Export

2025-02-05 Thread Corey Huinker
On Mon, Jan 27, 2025 at 11:09 AM Corey Huinker wrote: > On Mon, Jan 27, 2025 at 9:05 AM jian he > wrote: > >> On Tue, Jan 21, 2025 at 7:31 AM Jeff Davis wrote: >> > >> > On Mon, 2025-01-20 at 16:45 -0500, Corey Huinker wrote: >> > > >> &g

Re: Statistics Import and Export

2024-12-11 Thread Corey Huinker
relpages looks increasingly unlikely. From c10e1c1f87bc8039dfae38077d6b37446bcf408d Mon Sep 17 00:00:00 2001 From: Corey Huinker Date: Thu, 14 Nov 2024 04:58:17 -0500 Subject: [PATCH v34 02/11] Add --no-data option. This option is useful for situations where someone wishes to test query plans from a prod

Re: Statistics Import and Export

2024-12-12 Thread Corey Huinker
pg_dump/pg_upgrade 0006 - attribute stats optimization 0007-0012 - vacuumdb On Wed, Dec 11, 2024 at 10:49 PM Corey Huinker wrote: > +1, assuming such an option is wanted at all. I suppose it should be >> there for the unlikely (and hopefully impossible) case that statistics >>

Additional comments around need_escapes in pg_parse_json()

2024-12-20 Thread Corey Huinker
o help the next person come to that conclusion sooner than I did. From 84a8990622a4b83729600d9db43e8678502548f5 Mon Sep 17 00:00:00 2001 From: Corey Huinker Date: Fri, 20 Dec 2024 05:08:50 -0500 Subject: [PATCH v1] Explain impact of need_escapes in JSON parsing. Adding some additional comments t

Re: Statistics Import and Export

2024-12-19 Thread Corey Huinker
> > The biggest functional change is the way dependencies are handled for > matview stats. Materialized views ordinarily end up in > SECITON_PRE_DATA, but in some cases they can be postponed to > SECTION_POST_DATA. You solved that by always putting the matview stats > in SECTION_POST_DATA. > Accur

Re: Additional comments around need_escapes in pg_parse_json()

2024-12-22 Thread Corey Huinker
> > > Comments are not quite accurate, but I'll look at cleaning them up and > adding them. > Much appreciated. Also, I don't think we need all of the comments, but I figured it was easier for me to comment at every possible good place, and let you pick your favorites.

Re: Statistics Import and Export

2025-01-20 Thread Corey Huinker
> > Are you saying that there is a path for a partitioned index to have > stats today? If so, we can just follow that locking protocol. If not, > I'm concerned about trying to define the locking protocol for doing so > as a side-effect of this work. > A quick test on v17 indicates that, no, there

Re: Statistics Import and Export

2025-01-20 Thread Corey Huinker
> > > Let's just block updating stats for partitioned indexes > until there's a defined way to do so. > -1. That will break pg_upgrade tests if there's any statistics out there for any partitioned index. > Also, we should check privileges on the right object, consistent with > how ANALYZE does

Re: Statistics Import and Export

2025-01-20 Thread Corey Huinker
On Mon, Jan 20, 2025 at 10:59 PM Corey Huinker wrote: > Are you saying that there is a path for a partitioned index to have >> stats today? If so, we can just follow that locking protocol. If not, >> I'm concerned about trying to define the locking protocol for doing so &g

Re: Statistics Import and Export

2025-01-20 Thread Corey Huinker
Updated attached, burning numbers in the stats-sequence even though the pg_dump patches have been left out for the time being. From 03a15471b0f5a6776331d430cca2df619047271b Mon Sep 17 00:00:00 2001 From: Corey Huinker Date: Mon, 20 Jan 2025 16:18:51 -0500 Subject: [PATCH v40] Lock table first whe

Re: Statistics Import and Export

2025-01-21 Thread Corey Huinker
> > > > > /* > * For indexes, we follow what do_analyze_rel() does so as to avoid any > * deadlocks with analyze/vacuum, which is to take out a > * ShareUpdateExclusive on table/matview first and only after that take > * a a AccessShareLock on the index itself. > */ > t

Re: Statistics Import and Export

2025-01-21 Thread Corey Huinker
> > >> >> > Sorry about that, my nvim config is auto-including stuff and it's annoying. > > Now with less includes and fewer typos: From 93e67e5304132e680b628b3e92f9fd242d998abd Mon Sep 17 00:00:00 2001 From: Corey Huinker Date: Tue, 21 Jan 2025 11:52:58 -0500

Re: Extended Statistics set/restore/clear functions.

2025-01-23 Thread Corey Huinker
On Wed, Jan 22, 2025 at 5:50 PM Tomas Vondra wrote: > Hi, > > Thanks for continuing to work on this. > > On 1/22/25 19:17, Corey Huinker wrote: > > This is a separate thread for work started in [1] but focused purely on > > getting the following functions working: >

Re: Extended Statistics set/restore/clear functions.

2025-01-23 Thread Corey Huinker
> > I see there's a couple MCV-specific functions in the extended_stats.c. >> Shouldn't those go into mvc.c instead? >> > > I wanted to put it there, but there was a reason I didn't and I've now > forgotten what it was. I'll make an effort to relocate it to mcv.c. > Looking at it now, I see that c

Re: vacuumdb changes for stats import/export

2025-01-24 Thread Corey Huinker
> > Thoughts? > I don't have anything to add to what Nathan said, but thought I should say so since this thread was broken off from my earlier thread. Eagerly awaiting feedback.

Re: Extended Statistics set/restore/clear functions.

2025-01-23 Thread Corey Huinker
> > > > > * no negative attnums in key list > Disregard this suggestion - negative attnums mean the Nth expression in the extended stats object, though it boggles the mind how we could have 222 expressions... > > * no duplicate attnums in key list > This one is still live, am considering. At

Re: Statistics Import and Export

2025-01-20 Thread Corey Huinker
On Mon, Jan 20, 2025 at 3:22 AM jian he wrote: > On Thu, Jan 16, 2025 at 4:56 AM Corey Huinker > wrote: > >> > >> > >> > >> I do like the idea of a "Statistics for ..." prefix, and I think it's > >> doable. > > > >

Re: Statistics Import and Export

2025-01-20 Thread Corey Huinker
hat Attached is the patch, along with the regression test output prior to the change to stat_utils.c. Other pg_dump work has been left out of v39 to focus on this. regression.diffs Description: Binary data From a86d16c6413e624d7785c8347fe4e9fbf9a8ce66 Mon Sep 17 00:00:00 2001 From: Corey Hu

Re: Statistics Import and Export

2025-01-20 Thread Corey Huinker
> > I believe you are referring to Tom's statement that "it'll be a > serious, serious error for [stats] not to be SECTION_DATA". The > statement is somewhat softened by the sentence that follows, and > slightly more by [2]. But it's pretty clear that SECTION_POST_DATA is, > at best, an implementat

Re: Statistics Import and Export

2025-01-27 Thread Corey Huinker
On Mon, Jan 27, 2025 at 9:05 AM jian he wrote: > On Tue, Jan 21, 2025 at 7:31 AM Jeff Davis wrote: > > > > On Mon, 2025-01-20 at 16:45 -0500, Corey Huinker wrote: > > > > > > What I struggle to understand is how that purpose isn't served better > >

Re: Extended Statistics set/restore/clear functions.

2025-01-27 Thread Corey Huinker
> > I'd like to merge these down to 3 patches again, but I'm keeping them > separate for this patchset to isolate the attnum-checking code for this > go-round. > These are mock-ups of the to/from JSON functions, but building from/to text rather than the not-yet-committed pg_ndistinct and pg_depend

Re: Statistics Import and Export

2025-01-21 Thread Corey Huinker
to break as many things as they fix, so I'm still working on them. From 40ec06c622c4be1de0e87509b2655e1d3ff62b4b Mon Sep 17 00:00:00 2001 From: Corey Huinker Date: Tue, 21 Jan 2025 11:52:58 -0500 Subject: [PATCH v43 1/2] Lock table first when setting index relation statistics. Jian He rep

Re: Extended Statistics set/restore/clear functions.

2025-01-29 Thread Corey Huinker
On Tue, Jan 28, 2025 at 11:25 AM jian he wrote: > hi. > I reviewed 0001 only. > > in src/backend/statistics/mvdistinct.c > > no need #include "nodes/pg_list.h" since > src/include/statistics/statistics.h sub level include "nodes/pg_list.h" > > no need #include "utils/palloc.h" > sicne #include "p

Re: Extended Statistics set/restore/clear functions.

2025-01-29 Thread Corey Huinker
On Wed, Jan 29, 2025 at 2:50 AM jian he wrote: > hi. > > select '{"1, 0B100101":"NaN"}'::pg_ndistinct; > pg_ndistinct > > {"1, 37": -2147483648} > (1 row) > I think my initial reaction is to just refuse those special values, but I'll look into the parsing code to

Re: Statistics Import and Export

2024-12-21 Thread Corey Huinker
> > > I like the description "STATISTICS DATA" because it differentiates from > You have Tomas to thank for that: https://www.postgresql.org/message-id/bf724b21-914a-4497-84e3-49944f9776f6%40enterprisedb.com the extended stats definitions. It might be worth differentiating > between "RELATION ST

Re: Statistics Import and Export

2025-01-16 Thread Corey Huinker
> > pg_dump --dbname=src2 --table=tenk1 --statistics-only --verbose > x.sql > there no pg_restore_attribute_stats, pg_restore_relation_stats there > for table tenk1. > There aren't any statistics, should there be? > pg_restore_relation_stats: yes. looking into that. pg_restore_attribute_stats: yes

Re: Statistics Import and Export

2025-01-15 Thread Corey Huinker
will set me straight. Attached is just the pg_dump stuff, and only for relation/attribute stats. The extended stats and vacuumdb work will be in their own threads going forward. From fdb53409458bc9aeed3496f355173ac97062afd1 Mon Sep 17 00:00:00 2001 From: Corey Huinker Date: Sat, 16 Mar 2024 1

Re: Statistics Import and Export

2025-02-11 Thread Corey Huinker
en yes, we could trim it down, but as it is I think it's a limitation of the testing structure. But aside from creating a whole extra XYZ_pg_dump.pl file, I don't think there's a way to do that. The previous 0001 is now committed (thanks!) so only one remains. From afb3d0fd81fb

Re: Statistics Import and Export

2025-03-15 Thread Corey Huinker
> > > https://www.postgresql.org/docs/current/ddl-priv.html > > The above text indicates that we should do the check, but also that > it's not terribly important for actual security. > Ok, I'm convinced. > > > If we do, we'll want to change downgrade the following errors to > > warn+return fals

Re: Statistics Import and Export

2025-03-15 Thread Corey Huinker
On Thu, Mar 6, 2025 at 3:48 AM Jeff Davis wrote: > On Wed, 2025-03-05 at 23:04 -0500, Corey Huinker wrote: > > > > Anyway, here's a rebased set of the existing up-for-consideration > > patches, plus the optimization of avoiding querying on non-expression > &g

Re: Statistics Import and Export

2025-03-19 Thread Corey Huinker
> > This replaces regclassin with custom lookups of the namespace and > relname, but misses some of the complexities that regclassin is > handling. For instance, it calls RangeVarGetRelid(), which calls > LookupExplicitNamespace(), which handles temp tables and > InvokeNamespaceSearchHook(). > > At

Re: Statistics Import and Export

2025-03-16 Thread Corey Huinker
> > * The custom format actually does two WriteToc() calls, and since these > patches move the queries to this part of pg_dump, it means we'll run all > the queries twice. The comments around this code suggest that the second > pass isn't strictly necessary and that it is really only useful

Re: vacuumdb changes for stats import/export

2025-03-17 Thread Corey Huinker
> > > While preparing this for commit, I noticed that the expression index part > of the query was disregarding attstattarget. To fix, I've modified that > part to look at the index's pg_attribute entries. > +1, should have been there all along.

Re: Statistics Import and Export

2025-03-17 Thread Corey Huinker
On Mon, Mar 17, 2025 at 10:24 AM Nathan Bossart wrote: > On Sun, Mar 16, 2025 at 05:32:15PM -0400, Corey Huinker wrote: > >> > >> * The custom format actually does two WriteToc() calls, and since these > >> patches move the queries to this part of pg_dump, it me

Re: Statistics Import and Export

2025-04-05 Thread Corey Huinker
> > > Also, why do we need the clause "WHERE s.tablename = ANY($2)"? Isn't > > that already implied by "JOIN unnest($1, $2) ... s.tablename = > > u.tablename"? > > Good question. Corey, do you recall why this was needed? > In my patch, that SQL statement came with the comment: + /* + * The resul

Re: Statistics Import and Export

2025-04-05 Thread Corey Huinker
> > * Changed to use LookupExplicitNamespace() > Seems good. > * Added test for temp tables > +1 > * Doc fixes So this patch swings the pendulum a bit back towards accepting some things as errors. That's understandable, as we're never going to have a situation where we can guarantee that th

Re: Statistics Import and Export

2025-04-05 Thread Corey Huinker
> > The first is that i_relallfrozen is undefined in versions earlier than > 18. That's trivial to fix, we just add "0 AS relallfrozen," in the > earlier versions, but still refrain from outputting it. > Ok, so long as we refrain from outputting it, I'm cool with whatever we store internally. >

Re: Statistics Import and Export

2025-04-04 Thread Corey Huinker
until v9.4. Looking into it... > > > This patch shrinks the array size to 1 for versions < 9.4, which keeps the modern code fairly elegant. From fe551ab55622f95d84ac4c4d79fba898c6b60057 Mon Sep 17 00:00:00 2001 From: Corey Huinker Date: Fri, 4 Apr 2025 19:30:00 -0400 Subject: [PATC

Re: Statistics Import and Export

2025-03-25 Thread Corey Huinker
And here's the rebase (after bde2fb797aaebcbe06bf60f330ba5a068f17dda7). The order of the patches is different, but the purpose of each is the same as before. From 1f9b2578f55fa1233121bcf5949a6f69d6cf8cee Mon Sep 17 00:00:00 2001 From: Corey Huinker Date: Fri, 14 Mar 2025 03:54:26 -0400 Subject: [PATCH v1

Re: Statistics Import and Export

2025-03-25 Thread Corey Huinker
> > The original reason we wanted to issue warnings was to allow ourselves > a chance to change the meaning of parameters, add new parameters, or > even remove parameters without causing restore failures. If there are > any ERRORs that might limit our flexibility I think we should downgrade > those

Re: Add partial :-variable expansion to psql \copy

2025-04-01 Thread Corey Huinker
> > I'm hesitating about the right syntax, though, for an input backslash > command which in effect would really only apply to COPY? ISTM that \g* is > used for "go", i.e. a semi-colon replacement which executes the SQL, and we > should want the same thing, which suggests: > making it a \g-variant

Re: Disallow redundant indexes

2025-04-28 Thread Corey Huinker
> > > I've had this idea before, and even wrote a quick POC at one point, but > I had it simply throw a warning rather than an > > error. That avoids the need for any GUC, which I agree is not a good > idea. And it still allows people to create a > > duplicate index if they really want to. > > > >

Re: Can we use Statistics Import and Export feature to perforamance testing?

2025-04-12 Thread Corey Huinker
> > * Question > > By using Statistics Import and Export feature, is it possible to achieve > the above request by following procedure? > > > > (1) Export the statistics from production environment by using pg_dump > --statistics-only. > > (2) On the staging environment, set the autovacuum related

Re: Can we use Statistics Import and Export feature to perforamance testing?

2025-04-12 Thread Corey Huinker
> > You might be getting confused because the code does look at the > pg_class fields, but that's only to estimate the tuple density. When > pg_class has those estimates, they're used to calculate the estimated > density by doing reltuples / relpages, but that average rows per page > > Thanks for t

Re: someone else to do the list of acknowledgments

2025-04-12 Thread Corey Huinker
> > The whole thing might take about 20 to 30 hours wall-clock time. > After this dev cycle, things with a defined end to them hold a greater attraction than they did previously. > So, there is some time to think about this. Please discuss here if > you're interested or have questions. > I am

Re: Can we use Statistics Import and Export feature to perforamance testing?

2025-04-12 Thread Corey Huinker
> > at the *actual size* of the relation and takes that into account when > scaling the statistics (see table_block_relation_estimate_size() in > tableam.c). If the table sizes don't match between the two servers > then there's no guarantees the planner will produce the same plan. > Sorry that I d

Re: Statistics Import and Export

2025-02-28 Thread Corey Huinker
On Fri, Feb 28, 2025 at 4:25 PM Jeff Davis wrote: > On Fri, 2025-02-28 at 14:56 -0600, Nathan Bossart wrote: > > On Fri, Feb 28, 2025 at 12:54:03PM -0800, Jeff Davis wrote: > > > (Aside: I assume everyone here agrees that pg_upgrade should > > > transfer > > > the stats by default.) > > > > That

Re: Statistics Import and Export

2025-02-27 Thread Corey Huinker
> > +--- error: relation is wrong type > +SELECT pg_catalog.pg_restore_relation_stats( > +'relation', 0::oid, > +'relpages', 17::integer, > +'reltuples', 400.0::real, > +'relallvisible', 4::integer); > > Why do you need to specify all the stats (relpages, reltuples,

<    1   2   3   4   5   >