Proposal: Deferred Replica Filtering for PostgreSQL Logical Replication

2025-03-15 Thread Dean
replica filtering allows for session-specific, per-row, and per-column filtering - features currently not supported by existing replication filters, enhancing security and data privacy. I look forward to hearing your thoughts! Best, Dean S

Re: Proposal: Deferred Replica Filtering for PostgreSQL Logical Replication

2025-03-18 Thread Dean
DRF is to allow per-subscriber variations in change broadcasts, enabling granular control over what data is sent to each subscriber based on their session context. Best, Dean S On Mon, Mar 17, 2025 at 4:32 AM Amit Kapila wrote: > On Sun, Mar 16, 2025 at 12:59 AM Dean wrote: > &g

Some optimisations for numeric division

2022-02-23 Thread Dean Rasheed
always be one or two digits. Testing this with a bunch of random exp() and ln() computations I saw a speed-up of 15-20%, and it reduced the run time of the numeric-big regression test by around 10%, which seems worth having. Regards, Dean From 41732ad9a44dcd12e52d823fb59cb23cce4fe217 Mon Sep 17 0

Re: Some optimisations for numeric division

2022-02-23 Thread Dean Rasheed
_var() and div_var_fast() could hand off to it for one and two digit divisors. Regards, Dean

Re: Some optimisations for numeric division

2022-02-25 Thread Dean Rasheed
On Wed, 23 Feb 2022 at 22:55, Tom Lane wrote: > > Dean Rasheed writes: > > > One thought that occurred to me was that it's a bit silly that > > exp_var() and ln_var() have to use a NumericVar for what could just be > > an int, if we had a div_var_int() function

Re: Some optimisations for numeric division

2022-02-25 Thread Dean Rasheed
On Fri, 25 Feb 2022 at 10:45, Dean Rasheed wrote: > > Attached is the updated patch series (0001 and 0002 unchanged). > And another update following feedback from the cfbot. Regards, Dean From 41732ad9a44dcd12e52d823fb59cb23cce4fe217 Mon Sep 17 00:00:00 2001 From: Dean Rasheed Date

Re: [PATCH] Add reloption for views to enable RLS

2022-02-25 Thread Dean Rasheed
ding a comment to explain why it's now important to extract the reloptions *before* fetching the relation's rules and triggers. 6). The second set of tests added to rowsecurity.sql seem to have nothing to do with RLS, and probably belong in updatable_views.sql, and I think it would be worth adding a few more tests for things like views on top of views. Regards, Dean

Re: Some optimisations for numeric division

2022-02-27 Thread Dean Rasheed
On Fri, 25 Feb 2022 at 18:30, Tom Lane wrote: > > Dean Rasheed writes: > > And another update following feedback from the cfbot. > > This patchset LGTM. On my machine there doesn't seem to be any > measurable performance change for the numeric regression test, >

Re: [PATCH] Add reloption for views to enable RLS

2022-03-02 Thread Dean Rasheed
aying that this version of PG now supports security invoker views is going to mean a lot more to people who already use that feature in other databases. What are other people's opinions? Regards, Dean

Re: [PATCH] Add reloption for views to enable RLS

2022-03-18 Thread Dean Rasheed
. The logical/consistent thing to do for security invoker views is to do the permission checks as the invoking user, so I've done that. Barring any other comments or objections, I'll push this in a couple of days or so, after a bit more proof-reading. Regards, Dean diff --git a/doc/src/

Re: [PATCH] Add reloption for views to enable RLS

2022-03-22 Thread Dean Rasheed
On Mon, 21 Mar 2022 at 09:47, Laurenz Albe wrote: > > Thanks for your diligent work on this, and the patch looks good to me. Thanks for looking again. Pushed. Regards, Dean

Re: Additional improvements to extended statistics

2020-12-01 Thread Dean Rasheed
in the next day or so. Regards, Dean

Re: Additional improvements to extended statistics

2020-12-02 Thread Dean Rasheed
t would be possible to construct queries that would benefit from this, so it would be good to get that committed too. Barring any further comments, I'll push this sometime soon. Regards, Dean diff --git a/src/backend/optimizer/path/clausesel.c b/src/backend/optimizer/path/clausesel.c n

Re: Additional improvements to extended statistics

2020-12-03 Thread Dean Rasheed
On Wed, 2 Dec 2020 at 16:34, Tomas Vondra wrote: > > On 12/2/20 4:51 PM, Dean Rasheed wrote: > > > > Barring any further comments, I'll push this sometime soon. > > +1 > Pushed. Regards, Dean

Re: PoC/WIP: Extended statistics on expressions

2020-12-07 Thread Dean Rasheed
; and "extended expression statistics require at least one expression" would go away, and the error "extended statistics require at least 2 columns" might become more specific, depending on the stats kind. Regards, Dean [1] https://www.postgresql.org/message-id/flat/1009.1579038764%40sss.pgh.pa.us#8624792a20ae595683b574f5933dae53

Re: PoC/WIP: Extended statistics on expressions

2020-12-07 Thread Dean Rasheed
On Mon, 7 Dec 2020 at 14:15, Tomas Vondra wrote: > > On 12/7/20 10:56 AM, Dean Rasheed wrote: > > it might actually be > > neater to have separate documented syntaxes for single- and > > multi-column statistics: > > > > CREATE STATISTICS [ IF NOT EXISTS ] st

Re: Additional improvements to extended statistics

2020-12-07 Thread Dean Rasheed
On Wed, 2 Dec 2020 at 15:51, Dean Rasheed wrote: > > The sort of queries I had in mind were things like this: > > WHERE (a = 1 AND b = 1) OR (a = 2 AND b = 2) > > However, the new code doesn't apply the extended stats directly using > clauselist_selectivity_or() for

Re: PoC/WIP: Extended statistics on expressions

2020-12-11 Thread Dean Rasheed
same column. The only case where duplication won't be avoidable is where there are multiple MCV stats sharing the same expression, but that's probably quite unlikely in practice, and it seems acceptable to leave improving that as a possible future optimisation. Regards, Dean

Re: PoC/WIP: Extended statistics on expressions

2021-01-04 Thread Dean Rasheed
t with the CREATE STATISTICS syntax. * The pg_dump output for a stats object whose only kind is "expressions" is broken -- it includes a spurious "()" for the kinds list. That's it for now. I'll look at the optimiser changes next, and try to post more comments later this week. Regards, Dean

Bug in numeric_power() if exponent is INT_MIN

2021-01-04 Thread Dean Rasheed
, as in the attached patch. This code was added in 7d9a4737c2, which first appeared in PG 9.6, so barring objections, I'll push and back-patch this fix that far. Regards, Dean diff --git a/src/backend/utils/adt/numeric.c b/src/backend/utils/adt/numeric.c new file mode 100644 index 68d8791..7cf565

Re: PoC/WIP: Extended statistics on expressions

2021-01-05 Thread Dean Rasheed
On Tue, 5 Jan 2021 at 00:45, Tomas Vondra wrote: > > On 1/4/21 4:34 PM, Dean Rasheed wrote: > > > > * In src/bin/psql/describe.c, I think the \d output should also > > exclude the "expressions" stats kind and just list the other kinds (or > > have no kinds

Re: PoC/WIP: Extended statistics on expressions

2021-01-06 Thread Dean Rasheed
ntentionally crazy examples (which are still possible even with this check), then there are probably many legitimate cases where someone might want to use non-immutable functions in stats, and this check just forces them to create an immutable wrapper function. Regards, Dean

Re: PoC/WIP: Extended statistics on expressions

2021-01-07 Thread Dean Rasheed
= 0; EXPLAIN SELECT * FROM (SELECT 1) t, foo WHERE x%10 = 0; (in the second query, the stats don't get applied). Regards, Dean

Re: PoC/WIP: Extended statistics on expressions

2021-03-05 Thread Dean Rasheed
uot;merge" multiple statistics together. But even then it'd require > many statistics objects to make a difference. Possibly. There's a danger in trying to change too much at once though. As it stands, I think it's fairly close to being committable, with just a little more tidying up. Regards, Dean

Re: pgbench - add pseudo-random permutation function

2021-03-11 Thread Dean Rasheed
On Thu, 11 Mar 2021 at 00:58, Bruce Momjian wrote: > > Maybe Dean Rasheed can help because of his math background --- CC'ing him. > Reading the thread I can see how such a function might be useful to scatter non-uniformly random values. The implementation looks plausible too,

Re: pgbench - add pseudo-random permutation function

2021-03-11 Thread Dean Rasheed
i (see Knuth, Section 3.4.2 > Algorithm P) > True, but n can be very large, so that might use a lot of memory and involve a lot of pre-processing. Regards, Dean

Re: Additional improvements to extended statistics

2020-11-12 Thread Dean Rasheed
On Thu, 12 Nov 2020 at 14:18, Tomas Vondra wrote: > > Here is an improved WIP version of the patch series, modified to address > the issue with repeatedly applying the extended statistics, as discussed > with Dean in this thread. It's a bit rough and not committable, but I > n

Re: Additional improvements to extended statistics

2020-11-17 Thread Dean Rasheed
On Thu, 12 Nov 2020 at 14:18, Tomas Vondra wrote: > > Here is an improved WIP version of the patch series, modified to address > the issue with repeatedly applying the extended statistics, as discussed > with Dean in this thread. It's a bit rough and not committable, but I > n

Re: Additional improvements to extended statistics

2020-11-19 Thread Dean Rasheed
tion to existing functions, but I agree that in this case it's a bit of a mouthful. I don't think "_internal" is appropriate though, since the clauselist function isn't internal. Perhaps using just "_ext" would be OK. Regards, Dean

Re: [bug+patch] Inserting DEFAULT into generated columns from VALUES RTE

2020-11-20 Thread Dean Rasheed
ty-check error triggered by INSERT ... OVERRIDING USER VALUES. I also added a few additional comments where I found the existing code a little non-obvious. I haven't touched the existing error messages. I think that's a subject for a separate patch. Regards, Dean diff --git a/src/backen

Re: [bug+patch] Inserting DEFAULT into generated columns from VALUES RTE

2020-11-23 Thread Dean Rasheed
that just adjusting the error string would be > helpful, as attached. +1 > (I'm also wondering why the second case is generic ERRCODE_SYNTAX_ERROR > and not ERRCODE_GENERATED_ALWAYS. Didn't change it here, though.) I can't see any reason for it to be different, and ERRCODE_GENERATED_ALWAYS seems like the right code to use for both cases. Regards, Dean

Re: proposal: possibility to read dumped table's name from file

2020-11-25 Thread Dean Rasheed
ect types in different files), and for a config file to contain its own "--option-file" arguments, to allow config files to include other config files. The current design feels far too limited to me, and requires new code and new syntax to be added each time we extend it, so I'm -1 on this patch as it stands. Regards, Dean

Re: proposal: possibility to read dumped table's name from file

2020-11-26 Thread Dean Rasheed
eparate external tool to convert YAML or JSON to our plain format. I don't know. But supporting all pg_dump options makes more things possible. >> I wasn't very excited about multiple switch files either, though >> depending on how the implementation is done, that could be simple >> enough to be in the might-as-well category. >> That's what I was hoping. Regards, Dean

Re: Additional improvements to extended statistics

2020-11-29 Thread Dean Rasheed
. One estimate is made worse, but in that case we only have 2 sets of partial stats: SELECT * FROM mcv_lists_multi WHERE a = 0 OR b = 0 OR c = 0 OR d = 0 with stats on (a,b) and (c,d) so it's not surprising that combining (a = 0 OR b = 0) with (c = 0 OR d = 0) mis-estimates a bit. I

Re: PoC/WIP: Extended statistics on expressions

2021-01-18 Thread Dean Rasheed
LATERAL ( + SELECT + * + FROM ( + SELECT + unnest(pg_get_statisticsobjdef_expressions(s.oid)) AS expr, + unnest(sd.stxdexpr)::pg_statistic AS a + ) x + ) stat ON true; Regards, Dean

Re: PoC/WIP: Extended statistics on expressions

2021-01-21 Thread Dean Rasheed
cleaner if no code assumed any particular order, but I can believe that it might be convenient in some circumstances. > But I wonder if it'd be easier to just calculate the actual max attnum > and then use it instead of MaxHeapAttributeNumber ... Hmm, I'm not sure how that would work. There still needs to be an attnum that gets stored in the database, and it has to continue to work if the user adds columns to the table. That's why I was advocating storing negative values, though I haven't actually tried it to see what might go wrong. Regards, Dean

Re: PoC/WIP: Extended statistics on expressions

2021-01-22 Thread Dean Rasheed
nct, dependencies, mcv) ON i FROM t > I guess that's not surprising, given that old psql knows nothing about expressions in stats. In general, I think connecting old versions of psql to newer servers is not supported. You're lucky if \d works at all. So it shouldn't be this patch's responsibility to make that output nicer. Regards, Dean

Re: Schema variables - new implementation for Postgres 15

2022-01-13 Thread Dean Rasheed
nk it's acceptable to make it so easy for a user to break the system in this way. Those are examples that a malicious user might use, but even without such examples, I think it would be far too easy to inadvertently break a large application by defining a variable that conflicted with a column name you didn't know about. Regards, Dean

Re: Schema variables - new implementation for Postgres 15

2022-01-13 Thread Dean Rasheed
long as it's clearly documented. I don't think a warning is necessary. (FWIW, testing with dbfiddle, that appears to match Db2's behaviour). Regards, Dean

Re: PoC/WIP: Extended statistics on expressions

2021-03-17 Thread Dean Rasheed
e there's a better way to pick the statistics, but I > think our conclusion so far was that people should just create > statistics covering all the columns in the query, to not have to match > multiple statistics like this. Yes, I think that's always likely to work better, especially for ndistinct stats, where all possible permutations of subsets of the columns are included, so a single ndistinct stat can work well for a range of different queries. Regards, Dean

Re: PoC/WIP: Extended statistics on expressions

2021-03-17 Thread Dean Rasheed
hose stats aren't really applicable to (c+d), which probably ought to be treated as independent. IOW, it might have been better to estimate the first case as ndistinct((a+b)) * ndistinct(c) * ndistinct(d) and the second case as ndistinct((a+b)) * ndistinct((c+d)) Regards, Dean

Re: PoC/WIP: Extended statistics on expressions

2021-03-17 Thread Dean Rasheed
On Wed, 17 Mar 2021 at 19:07, Tomas Vondra wrote: > > On 3/17/21 7:54 PM, Dean Rasheed wrote: > > > > it might have been better to estimate the first case as > > > > ndistinct((a+b)) * ndistinct(c) * ndistinct(d) > > > > and the second case as >

Re: PoC/WIP: Extended statistics on expressions

2021-03-17 Thread Dean Rasheed
On Wed, 17 Mar 2021 at 20:48, Dean Rasheed wrote: > > For reference, here is the test case I was using (which isn't really very > good for > catching dependence between columns): > And here's a test case with much more dependence between the columns: DROP TABLE IF EXI

Re: PoC/WIP: Extended statistics on expressions

2021-03-18 Thread Dean Rasheed
g > them sometime early next week. I don't want to leave it until the very > last moment in the CF. +1. I think they're in good enough shape for that process to start. Regards, Dean

Re: pgbench - add pseudo-random permutation function

2021-03-22 Thread Dean Rasheed
se algorithms cause adjacent inputs to always end up separated by an odd number), so a judgement call will be required to decide what is random enough. Regards, Dean #include #include #include typedef unsigned char uint8; typedef long int64; typedef unsigned long uint64; typedef unsigned __

Re: PoC/WIP: Extended statistics on expressions

2021-03-24 Thread Dean Rasheed
eviewing the docs, I noticed a couple of omissions, and had a few other suggestions (attached). Regards, Dean diff --git a/doc/src/sgml/catalogs.sgml b/doc/src/sgml/catalogs.sgml new file mode 100644 index dadca67..382cbd7 --- a/doc/src/sgml/catalogs.sgml +++ b/doc/src/sgml/catalogs.sgml @@ -7377,6 +

Re: PoC/WIP: Extended statistics on expressions

2021-03-24 Thread Dean Rasheed
ink they'd both end up being treated the same by estimate_multivariate_ndistinct(), since there wouldn't be any stats matching the expression, only the individual Var's. Maybe changing the first place would be the more bulletproof fix though. Regards, Dean

Re: PoC/WIP: Extended statistics on expressions

2021-03-24 Thread Dean Rasheed
tand them. > Hmm, I think "univariate" and "multivariate" are pretty ubiquitous, when used to describe statistics. You could use "single-column" and "multi-column", but then "column" isn't really right anymore, since it might be a column or an expression. I can't think of any other terms that fit. Regards, Dean

Re: PoC/WIP: Extended statistics on expressions

2021-03-25 Thread Dean Rasheed
ecific to extended statistics - indexes have exactly the > same issue. Not sure how common this is in practice. Hmm, that's unfortunate. Maybe it's not that common in practice though. I'm not sure if there is any practical way to fix it, but if there is, I guess we'd want to apply the same fix to both stats and indexes, and that certainly seems out of scope for this patch. Regards, Dean

Re: PoC/WIP: Extended statistics on expressions

2021-03-25 Thread Dean Rasheed
nk it's probably safer to revert to the previous code, which matched what index_create() did. Regards, Dean

Re: PoC/WIP: Extended statistics on expressions

2021-03-26 Thread Dean Rasheed
nded_stats.c:2532 - s/statitics/statistics/ 7). I don't think that the big XXX comment near the start of estimate_multivariate_ndistinct() is really relevant anymore, now that the code has been simplified and we no longer extract Vars from expressions, so perhaps it can just be deleted. Regards, Dean

Re: pgbench - add pseudo-random permutation function

2021-03-30 Thread Dean Rasheed
On Mon, 22 Mar 2021 at 13:43, Dean Rasheed wrote: > > On Sun, 14 Mar 2021 at 16:08, Fabien COELHO wrote: > > > > > My main question on this now is, do you have a scholar reference for > > > this algorithm? > > > > Nope, otherwise I would have put a

Re: pgbench - add pseudo-random permutation function

2021-03-30 Thread Dean Rasheed
he rotate code can be simplified, in particular the > ?: should be avoided because it may induce branches quite damaging to > processor performance. Yeah, I wondered about that. Perhaps there's a "trick" that can be used to simplify it. Pre-computing the number of bits in the mask would probably help. I'll give it some thought. Regards, Dean

Re: pgbench - add pseudo-random permutation function

2021-03-30 Thread Dean Rasheed
On Tue, 30 Mar 2021 at 20:31, Dean Rasheed wrote: > > Yeah, that's probably a fair point. However, all the existing pgbench > random functions are using it, so I think it's fair enough for > permute() to do the same (and actually 2^48 is pretty huge). Switching > to a 6

Re: pgbench - add pseudo-random permutation function

2021-03-31 Thread Dean Rasheed
ber of bits in the mask > > would probably help. > > See pg_popcount64(). > Actually, I used pg_leftmost_one_pos64() to calculate the mask length, allowing the mask to be computed from that, so there is no longer a need for compute_mask(), which seems like a neat little simplificati

Re: pgbench - add pseudo-random permutation function

2021-04-01 Thread Dean Rasheed
of the tests, but it closes that potential loophole of ignoring part of the seed. In all my tests, the biggest improvement was between v23 and v24 of the patch. By comparison, the later versions have been relatively small improvements, and it's probably now "random enough" for the in

Re: pgbench - add pseudo-random permutation function

2021-04-04 Thread Dean Rasheed
lows more rounds for the same cost. Additionally, following the same pattern as existing code reduces the risk of bugs, and builds on proven, tested code. You may wish to submit a separate patch to replace pgbench's use of *rand48() with something else, and that would be discussed on its own merits, but I don't see why that should hold up adding permute(). Regards, Dean

Re: pgbench - add pseudo-random permutation function

2021-04-06 Thread Dean Rasheed
t least removes any *direct* use of doubles in permute() (though of course they're still used indirectly), and means that if getrand() is improved in the future, permute() will benefit too. Regards, Dean

Re: Wrong results from in_range() tests with infinite offset

2020-07-21 Thread Dean Rasheed
ties. So, it's not quite an open-and-shut bug fix, so > I just put it in HEAD (for now anyway). > Yeah, that seems fair enough, and it's quite an obscure corner-case that has gone unnoticed for quite some time. Regards, Dean

Re: Infinities in type numeric

2020-07-22 Thread Dean Rasheed
On Tue, 21 Jul 2020 at 23:18, Tom Lane wrote: > > Here's a v4 that syncs numeric in_range() with the new behavior of > float in_range(), and addresses your other comments too. > LGTM. Regards, Dean

Re: [HACKERS] PATCH: multivariate histograms and MCV lists

2018-03-26 Thread Dean Rasheed
changed from real to int -- all the data values will be coerced to integers, losing precision, and any ndistinct and dependency stats would likely be completely wrong afterwards. IMO that's a bug, and should be back-patched independently of these new types of extended stats. Thoughts? Regards, Dean

Re: [HACKERS] PATCH: multivariate histograms and MCV lists

2018-03-26 Thread Dean Rasheed
On 26 March 2018 at 14:08, Tomas Vondra wrote: > On 03/26/2018 12:31 PM, Dean Rasheed wrote: >> A wider concern I have is that I think this function is trying to be >> too clever by only resetting selected stats. IMO it should just reset >> all stats unconditionally when th

Re: [HACKERS] PATCH: multivariate histograms and MCV lists

2018-03-26 Thread Dean Rasheed
ceding comment): /* AND clauses assume nothing matches, initially */ memset(bool_matches, STATS_MATCH_FULL, sizeof(char) * mcvlist->nitems); Still reading... Regards, Dean

Re: [HACKERS] PATCH: multivariate histograms and MCV lists

2018-03-26 Thread Dean Rasheed
On 26 March 2018 at 20:17, Tomas Vondra wrote: > On 03/26/2018 09:01 PM, Dean Rasheed wrote: >> Also, just above that, in statext_clauselist_selectivity(), it >> computes the list stat_clauses, then doesn't appear to use it >> anywhere. I think that would have been the

Re: [HACKERS] PATCH: multivariate histograms and MCV lists

2018-03-27 Thread Dean Rasheed
but the second fails because a NOT clause always only has one argument. Regards, Dean

Re: [HACKERS] PATCH: multivariate histograms and MCV lists

2018-03-27 Thread Dean Rasheed
every sub-clause matched with equality against the MCV list, and in an outer AND clause that can be added to the outer eqmatches list, which is the list of attributes for which any sub-clause matched with equality. Regards, Dean

Re: [HACKERS] PATCH: multivariate histograms and MCV lists

2018-03-27 Thread Dean Rasheed
On 27 March 2018 at 14:58, Dean Rasheed wrote: > On 27 March 2018 at 01:36, Tomas Vondra wrote: >> 4) handling of NOT clauses in MCV lists (and in histograms) >> >> The query you posted does not fail anymore... >> > Ah, it turns out the previous query wasn't

Re: [HACKERS] PATCH: multivariate histograms and MCV lists

2018-03-28 Thread Dean Rasheed
-- c.f. scalarineqsel()). For an OR clause, you might want to count the number of non-MCV matches, because logically each one adds another "(1 - mcv_totalsel)) / otherdistinct" to the total selectivity. It's not immediately obvious how that can be made to fit into the current code structure. Perhaps it could be made to work by tracking the overall selectivity as it goes along. Or perhaps it could track the count/proportion of non-MCV matches. Regards, Dean

Re: [HACKERS] PATCH: multivariate histograms and MCV lists

2018-03-28 Thread Dean Rasheed
the overall selectivity would seem to be mcv_matchsel + (1-mcv_totalsel) * nonmcv_sel and there would be no need for mcv_update_bitmap() to track eqmatches or return fullmatch, and it wouldn't actually matter whether or not we had equality clauses or if all the MCV columns were used. Regards, Dean

Re: [HACKERS] PATCH: multivariate histograms and MCV lists

2018-04-07 Thread Dean Rasheed
h hasn't really had a fair chance to-date, despite Tomas' efforts and quick responses to all review comments. At this stage though, all I can say is that I'll make every effort to keep reviewing it, and I hope Tomas will persist, so that it has a better chance in PG12. Regards, Dean

Re: BF failure: could not open relation with OID XXXX while querying pg_views

2019-09-15 Thread Dean Rasheed
incautiously > >>> changed one of the other tests that run concurrently with "rules"? > > >> Looks like stats_ext.sql could be the problem. It creates and drops > >> priv_test_view, not in a schema. Adding Dean, author of commit > >> d7f8d2

Re: BF failure: could not open relation with OID XXXX while querying pg_views

2019-09-15 Thread Dean Rasheed
On Sun, 15 Sep 2019 at 11:11, Tomas Vondra wrote: > > On Sun, Sep 15, 2019 at 10:16:30AM +0100, Dean Rasheed wrote: > > > >Ah sorry, I missed this thread before. As author of that commit, it's > >really on me to fix it, and the cause seems pretty clear-cut, so I

Re: BF failure: could not open relation with OID XXXX while querying pg_views

2019-09-15 Thread Dean Rasheed
On Sun, 15 Sep 2019 at 12:20, Tomas Vondra wrote: > > On Sun, Sep 15, 2019 at 11:27:19AM +0100, Dean Rasheed wrote: > >On Sun, 15 Sep 2019 at 11:11, Tomas Vondra > >wrote: > >> > >> On Sun, Sep 15, 2019 at 10:16:30AM +0100, Dean Rasheed wrote: > >>

Re: BUG #15293: Stored Procedure Triggered by Logical Replication is Unable to use Notification Events

2019-02-27 Thread Marc Dean
If you are trying to get around the issue for now, what my team did was cron an insert statement on the database server. We have a queue table that has some of these triggers setup so it was easy to write a no-op row to the queue. This had the side effect of flushing the notification queue. We hav

Re: BUG #15623: Inconsistent use of default for updatable view

2019-02-28 Thread Dean Rasheed
ve come via a previous invocation of rewriteValuesRTE() that may have completely rebuilt the lists, it seemed best to be sure that it hadn't done something unexpected. It's about to use that to read from the attrnos array, so it might read beyond the array bounds if any of the prior logic was faulty. Thanks for looking. Regards, Dean

Re: Row Level Security − leakproof-ness and performance implications

2019-02-28 Thread Dean Rasheed
t and weak security barriers on a per-table basis, depending on how sensitive the data is in each table (I'm not a fan of using GUCs to control this). Regards, Dean

Re: [HACKERS] PATCH: multivariate histograms and MCV lists

2019-03-09 Thread Dean Rasheed
upported clause types that follows (e) is the same a (c), but with a more general description. 10). It looks like most of the subsequent description of the algorithm is out of date and needs rewriting. All the stuff about full matches and the use of ndistinct is now obsolete. Regards, Dean

Re: [HACKERS] PATCH: multivariate histograms and MCV lists

2019-03-10 Thread Dean Rasheed
On Sat, 9 Mar 2019 at 18:33, Dean Rasheed wrote: > > On Thu, 28 Feb 2019 at 19:56, Tomas Vondra > wrote: > > Attached is an updated version of this patch series. > > Here are some random review comments. I'll add more later, but I'm out > of energy for today. &

Re: [HACKERS] PATCH: multivariate histograms and MCV lists

2019-03-10 Thread Dean Rasheed
On Sun, 10 Mar 2019 at 13:09, Dean Rasheed wrote: > Here are some more comments: > One more thing --- the comment for statext_clauselist_selectivity() says: * So (simple_selectivity - base_selectivity) may be seen as a correction for * the part not covered by the MCV list. That'

Re: [HACKERS] PATCH: multivariate histograms and MCV lists

2019-03-11 Thread Dean Rasheed
On Sun, 10 Mar 2019 at 17:36, Tomas Vondra wrote: > On 3/10/19 2:09 PM, Dean Rasheed wrote: > > 14). The attnums Bitmapset passed to > > statext_is_compatible_clause_internal() is an input/output argument > > that it updates. That should probably be documented. Whe

Re: [HACKERS] PATCH: multivariate histograms and MCV lists

2019-03-11 Thread Dean Rasheed
On Sun, 10 Mar 2019 at 22:28, David Rowley wrote: > > On Mon, 11 Mar 2019 at 06:36, Tomas Vondra > wrote: > > > > On 3/9/19 7:33 PM, Dean Rasheed wrote: > > > I wonder if it's possible to write smaller, more targeted tests. > > > Currently "s

Re: [Patch] Log10 and hyperbolic functions for SQL:2016 compliance

2019-03-11 Thread Dean Rasheed
was interesting to play around with them anyway. Regards, Dean #include #include #include /* * A commonly used algorithm, due to Kahan (citation needed). * * Max error: 1 ulp. * Avg error: 0.158 ulp. * * It's not obvious, but this does appear to be monotonic at the cutover point *

Re: pgsql: Add support for hyperbolic functions, as well as log10().

2019-03-13 Thread Dean Rasheed
xplicitly handle special cases like Inf and NaN to ensure POSIX compatibility on all platforms. Regards, Dean >

Re: pgsql: Add support for hyperbolic functions, as well as log10().

2019-03-14 Thread Dean Rasheed
On Thu, 14 Mar 2019 at 04:41, Tom Lane wrote: > > Dean Rasheed writes: > > I'm amazed that jacana's asinh() returned -0 for an input of +0. > > Even more amusingly, it returns NaN for acosh('infinity'), cf > https://buildfarm.postgresql.org/cgi-bin/show

Re: [HACKERS] PATCH: multivariate histograms and MCV lists

2019-03-16 Thread Dean Rasheed
just using the column dependencies, so in this case UpdateStatisticsForTypeChange() never gets called anyway? Regards, Dean

Re: [HACKERS] PATCH: multivariate histograms and MCV lists

2019-03-16 Thread Dean Rasheed
lyse.c). Over-wide values should be excluded early on, and it will need to track whether or not any such values were excluded, because then it wouldn't be appropriate to treat the stats as complete and keep the entire list, without calling get_mincount_for_mcv_list(). That's it for now. Regards, Dean

Re: [HACKERS] PATCH: multivariate histograms and MCV lists

2019-03-17 Thread Dean Rasheed
on the 20190315 patch yesterday. I've just updated to the 20190317 patch. It looks like you forgot to update the argument name in the header file though. Regards, Dean

Re: [HACKERS] PATCH: multivariate histograms and MCV lists

2019-03-17 Thread Dean Rasheed
t the remaining items are common enough to give accurate estimates). * In the serialisation code, remove the size limit entirely. We know that each value is now at most 1024 bytes, and there are at most 1 items, and at most 8 columns, so the total size is already reasonably well bounded. In the worst case, it might be around 80MB, but in practice, it's always likely to be much much smaller than that. Regards, Dean

Re: [HACKERS] PATCH: multivariate histograms and MCV lists

2019-03-17 Thread Dean Rasheed
WHERE stxrelid = 'mcv_lists'::regclass; -- check change of column type resets the MCV statistics ALTER TABLE mcv_lists ALTER COLUMN c TYPE numeric; SELECT stxmcv IS NOT NULL AS has_mcv FROM pg_statistic_ext WHERE stxrelid = 'mcv_lists'::regclass; Regards, Dean

Re: [HACKERS] PATCH: multivariate histograms and MCV lists

2019-03-24 Thread Dean Rasheed
really was an improvement. If so, then the test result can be updated and perhaps whatever was done ought to be factored into the MV-stats' calculation of base frequencies. If not, the test is providing valuable feedback that perhaps it wasn't such a good improvement after all. Regards, Dean

Re: BUG #15708: RLS 'using' running as wrong user when called from a view

2019-03-24 Thread Dean Rasheed
owner -- i.e., we need to propagate the checkAsUser for the RTE with RLS to any subqueries in its RLS policies. It looks like the best place to fix it is in get_policies_for_relation(), since that's where all the policies to be applied for a given RTE are pulled together. Patch attached. Reg

Re: [HACKERS] PATCH: multivariate histograms and MCV lists

2019-03-26 Thread Dean Rasheed
herwise there is a risk of keeping too many MCV items, with the ones at the tail end of the list producing poor estimates. Regards, Dean

Re: [HACKERS] PATCH: multivariate histograms and MCV lists

2019-03-26 Thread Dean Rasheed
On Tue, 26 Mar 2019 at 11:59, Dean Rasheed wrote: > > On Mon, 25 Mar 2019 at 23:36, Tomas Vondra > wrote: > > > > Attached is an updated patch... > > I just looked through the latest set of changes and I have a couple of > additional review comments: > I

Re: BUG #15708: RLS 'using' running as wrong user when called from a view

2019-03-27 Thread Dean Rasheed
On Mon, 25 Mar 2019 at 20:27, Stephen Frost wrote: > > * Dean Rasheed (dean.a.rash...@gmail.com) wrote: > > > It looks like the best place to fix it is in > > get_policies_for_relation(), since that's where all the policies to be > > applied for a given RTE are

Re: ANALYZE: ERROR: tuple already updated by self

2019-07-29 Thread Dean Rasheed
right object, with > the flag set to either true/false. Not trivial, but doable. > Isn't it just a matter of passing the inh flag to get_relation_statistics() from get_relation_info(), so then the optimiser would get the right kind of stats data, depending on whether or not inheritance was requested in the query. Regards, Dean

Re: Multivariate MCV list vs. statistics target

2019-08-01 Thread Dean Rasheed
he default when adding this, otherwise users who don't use this new feature might be surprised by the change in behaviour. Regards, Dean

Re: MCV lists for highly skewed distributions

2018-01-21 Thread Dean Rasheed
On 21 January 2018 at 07:26, John Naylor wrote: > I spent a few hours hacking on this, and it turns out calculating the > right number of MCVs taking into account both uniform and highly > non-uniform distributions is too delicate a problem for me to solve > right now. The logic sugge

Re: stricter MCV tests for uniform distributions (was Re: MCV lists for highly skewed distributions)

2018-01-22 Thread Dean Rasheed
On 22 January 2018 at 08:07, John Naylor wrote: > On 1/21/18, Dean Rasheed wrote: >> It occurs to me that maybe a better test to exclude a value from the >> MCV list would be to demand that its relative standard error not be >> too high. Such a test, in addition to the exi

Re: WINDOW RANGE patch versus leakproofness

2018-01-31 Thread Dean Rasheed
s somehow see a window function (or any other unexpected node type) it will return true and the resulting qual/restrictinfo will be marked leaky, and not pushed through security barriers. Regards, Dean

Re: WINDOW RANGE patch versus leakproofness

2018-01-31 Thread Dean Rasheed
On 31 January 2018 at 21:51, Robert Haas wrote: > On Wed, Jan 31, 2018 at 5:52 AM, Dean Rasheed > wrote: >> On 30 January 2018 at 16:42, Tom Lane wrote: >>> So I'm thinking that (a) we do not need to check for leaky functions used >>> in window support, a

  1   2   3   4   5   6   7   >