u can take the comment out of the loop and just describe the
algorithm we're using, and make a note that we have to byteswap first.
"Indeterminate" could be explained briefly as well.
These are minor comments.
Regards,
Jeff Davis
On Sat, 2024-01-20 at 07:40 +0530, vignesh C wrote:
> This thread has been idle for a year now, It has stalled after a lot
> of discussion.
> @Jeff Davis: Do you want to try to restart the discussion by posting
> an updated version and see what happens?
Thank you for following up. Y
> shouldn't
> be a second, botton-up way, based on the other specified CREATE
> COLLATION parameters. That's just too much logical zig-zag, IMO.
Also a good point. I am withdrawing this patch from the CF, and we can
reconsider the idea later perhaps in some other form.
Regards,
Jeff Davis
Let me know what you think.
0004 looks good to me. No urgency so feel free to hold it until a
convenient time.
Regards,
Jeff Davis
> subscription. I wouldn't object to it, but TAP tests are heavy. It
> should be possible to write the same test as a SQL test by creating
> two databases and switching between them. Do you think it's worth
> trying that way?
I'm not entirely sure what you mean here, bu
for anyone who wants human-friendly
collation or tailoring, and is willing to put up with some collation
stability risk and lower collation performance.
Both have their place and the user is free to mix and match as needed,
thanks to the COLLATE clause for columns and queries.
Regards,
Jeff Davis
LSNs range
over buffers 4, 5, and 6, and 4 has already been evicted, should we try
to return LSN data from 5 and 6 at the proper offset in the dest
buffer? If so, we'd need to adjust the API so the caller knows what
parts of the dest buffer were filled in.
Regards,
Jeff Davis
From 34d
ted? Also, do you think a test module is required for
the basic functionality in 0001, or only when we start doing more
complex things like reading past the Flush pointer?
0003: can you explain why this is useful for wal summarizer to read
from the buffers?
Regards,
Jeff Davis
r read the WAL on current TLI? I'm not so
> sure about it, I haven't explored it in detail.
Let's just not call XLogReadFromBuffers from there.
Regards,
Jeff Davis
ver, and user mapping. A FDW wouldn't be able to use,
for example, some kind of configuration table or GUC to control how the
connection string gets created. That's easy enough to solve with
documentation.
I'll work up a new patch for this.
Regards,
Jeff Davis
using this if you ran into a collation versioning/migration problem?
Regards,
Jeff Davis
ned functions, it is. But basically, the
problems with SECURITY DEFINER all amount to "the author of the code
needs to be careful", which is a lot more intuitive than the problems
with SECURITY INVOKER.
Another option is having some kind SECURITY NONE that would run the
code as a very
uts "YES".
From 8fd8d42a6ef9a4589776260e539f10d730e1c3f1 Mon Sep 17 00:00:00 2001
From: Jeff Davis
Date: Wed, 22 Feb 2023 10:24:52 -0800
Subject: [PATCH v1] Allow meson to find ICU in non-standard locations.
---
meson.build | 7 +--
1 file changed, 5 insertions(+), 2 deletions(-)
d
ECURITY NONE, while still preventing
> either party from hijacking privileges of the other.
Interesting idea, I haven't heard of something like that being done
before. Is there some precedent for that or a use case where it's
helpful?
Regards,
Jeff Davis
On Wed, 2023-02-22 at 20:49 +0100, Peter Eisentraut wrote:
> > On 14.02.23 00:45, Jeff Davis wrote:
> > > > Now the patches are:
> > > >
> > > > 0001: pg_strcoll/pg_strxfrm
> > > > 0002: pg_locale_deterministic()
> > > >
On Fri, 2023-02-17 at 15:07 -0800, Jeff Davis wrote:
> 2. Update the pg_database entry for template0. This has less
> potential
> for surprise in case people are actually using template0 for a
> template.
New patches attached.
0001: default autoconf to build with ICU (meson alread
orking.
I think we need some doc updates to clarify which features are affected
by -Dextra_lib_dirs/-Dpkg_config_path.
Regards,
Jeff Davis
only
be able to use the clever SRF with tables they have privileges on. But
that's not what most functions do, and it's certainly not what most
default expressions do.
Regards,
Jeff Davis
On Fri, 2023-02-24 at 15:54 -0800, Jeff Davis wrote:
> 0001: default autoconf to build with ICU (meson already uses
> 'auto')
What's the best way to prepare for the impact of this on the buildfarm?
How should we migrate to using --without-icu for those animals not
curren
just trying to get some
agreement that SECURITY INVOKER is central to a lot of our security
woes, and that we should be treating it with skepticism on a
fundamental level.
Individual proposals for how to get away from SECURITY INVOKER should
be evaluated on their merits (i.e. don't break a bunch of stuff).
Regards,
Jeff Davis
On Mon, 2023-02-27 at 16:13 -0500, Robert Haas wrote:
> On Mon, Feb 27, 2023 at 1:25 PM Jeff Davis wrote:
> > I think you are saying that we should still run Alice's code with
> > the
> > privileges of Bob, but somehow make that safe(r) for Bob. Is that
> > right?
On Mon, 2023-02-20 at 15:23 -0800, Jeff Davis wrote:
>
> New patch attached. The new patch also includes a GUC that (when
> enabled) validates that the collator is actually found.
New patch attached.
Now it always preserves the exact locale string during pg_upgrade, and
does not a
essions) first; but once we have
that, it opens up the design space to better solutions or at least
mitigations. Is that right?
Regards,
Jeff Davis
are some
best practices for writing secure code". And we have some different
documentation for writing a cool SECURITY INVOKER function and how to
get other users to trust you enough to run it. That sounds a LOT more
understandable for users.
Regards,
Jeff Davis
on the right one.
Not sure if that's a good idea though.
[1]
https://www.postgresql.org/message-id/20230106034036.2m4qnn7ep7b5i...@awork3.anarazel.de
--
Jeff Davis
PostgreSQL Contributor Team - AWS
t I assume that your more
general point is that if we take some responsibility away from the
invoker and place it on the definer, then it creates room for new kinds
of problems. And I agree.
The point of moving responsibility to the definer is that the definer
can actually do something to protect themse
so that others don't get
confused, but it's not really a problem for me any more.
Also there's the issue that libxml2.so pulls in the system's ICU
regardless. I don't think that causes a major problem, but I thought
I'd mention it.
Regards,
Jeff Davis
ed much to be both useful and
safe. We may not need the exact same solution for both default
expressions and triggers. Some details to work through, though.
Regards,
Jeff Davis
etWait instead of WaitLatch.
There are a few details to sort out, like how to enable callers to
easily create the right WaitEventSet (it obviously needs to include
MyLatch, for instance) and update it with the right socket events.
--
Jeff Davis
PostgreSQL Contributor Team - AWS
rticular
> case that might not be necessary to build this new API.
In this case it looks easier to add the right API than to be sure about
whether it's needed or not.
Regards,
Jeff Davis
On Thu, 2023-03-02 at 11:45 -0800, Jeff Davis wrote:
> In this case it looks easier to add the right API than to be sure
> about
> whether it's needed or not.
I attached a sketch of one approach. I'm not very confident that it's
the right API or even that it works as I in
ranscode between LATIN1 and WIN1252. So I changed the ecpg test to
just use SQL_ASCII for the client_encoding (not the server encoding).
Michael Meskes added the client_encoding parameter test in 5e7710e725,
so he might have a comment about that as well.
Since I removed the code, I didn't se
ensions, where the types are not
guaranteed to have stable OIDs. Did you consider any proposals that
work with type names?
Regards,
Jeff Davis
tandard
collation, I think "not built with ICU" (feature not supported) is a
better error than "collation doesn't exist". It also effectively
reserves the name "unicode".
--
Jeff Davis
PostgreSQL Contributor Team - AWS
uldn't someone using another encoding also be
able to use ucs_basic? I'm not sure if that's a practical problem or
not; I'm just curious. Does ICU provide a locale for sorting by code
point?
Regards,
Jeff Davis
ge of a non-builtin
type.
I don't have a great solution for that, though. Maybe it's only
practical for builtin types.
Regards,
Jeff Davis
On Fri, 2023-03-03 at 21:45 -0800, Jeff Davis wrote:
> > > 0002: update template0 in new cluster (as described above)
I think 0002 is about ready and I plan to commit it soon unless someone
has more comments.
I'm holding off on 0001 for now, because you objected. But I still
t
certain way
during comparisons?
Regards,
Jeff Davis
On Wed, 2023-03-08 at 17:30 +0100, Peter Eisentraut wrote:
> So should we withdraw the patch from the commit fest?
Withdrawn. If someone else is interested we can still pursue some
improvements.
Regards,
Jeff Davis
On Wed, 2023-03-08 at 07:21 +0100, Peter Eisentraut wrote:
> On 04.03.23 19:29, Jeff Davis wrote:
> > It looks like the way you've handled this is by inserting the
> > collation
> > with collprovider=icu even if built without ICU support. I think
> > that's
&g
On Thu, 2023-03-09 at 10:36 +0100, Peter Eisentraut wrote:
> 0002 seems fine to me.
Committed 0002 with some test improvements.
>
> Let's come back to that after dealing with the other two.
Leaving 0001 open for now.
0003 committed after addressing your comments.
--
Jeff Da
On Thu, 2023-03-09 at 11:21 +0100, Peter Eisentraut wrote:
> How about this patch version?
Looks good to me.
Regards,
Jeff Davis
to the "new" spellings, but the old ones aren't wrong.
I see what you mean; I'm not sure the best thing to do here. We are
adjusting the string passed by the user, and it feels like some users
might want to know that. It's a NOTICE, not a WARNING, so it's not
meant to imply that it's wrong.
But at the same time I can see it being annoying or confusing. If it's
confusing, perhaps a wording change and documentation would improve it?
If it's annoying, we might need to have an option and/or a different
log level?
> It also doesn't appear to address
> how to handle ICU before version 54.
Do you have a specific concern here?
Regards,
Jeff Davis
ing a pretty good job of that. Regardless, we
need to remove the dependence on LC_CTYPE and LC_COLLATE when the
provider is ICU first (we're close to that point but not quite there).
Regards,
Jeff Davis
could be more buildfarm
failures that I'll need to look at afterward, so I'll count this as a
"trial fix".
Regards,
Jeff Davis
On Fri, 2023-03-10 at 07:48 -0800, Jeff Davis wrote:
> On Fri, 2023-03-10 at 15:48 +0100, Peter Eisentraut wrote:
> > Yes, of course. So we can't really do what I was thinking of.
>
> OK, I plan to commit something like the patch in this thread soon. I
> just need to add
ively, we could canonicalize to the ICU format locale IDs. Or
add something to parse out the attributes from a language tag.
Regards,
Jeff Davis
he issue in v16 and
later, but those using old versions of ICU and old versions of postgres
would still be vulnerable to these kinds of typos.
Regards,
Jeff Davis
[1] https://icu.unicode.org/download/55m1
[2]
https://unicode-org.github.io/icu-docs/apidoc/dev/icu4c/utypes_8h.html#a3343c1c8a8377277046774691c98d78c
gt; the issue tracker was migrated at some point or such...
The dates are misleading in both git (migrated from SVN circa 2016) and
JIRA (migrated circa 2018, see
https://unicode-org.atlassian.net/browse/ICU-1 ). It seems 55.1 was
released in either 2014 or 2015.
Regards,
Jeff Davis
ached.
One loose end is that we really should support language tags like "und"
in those older versions (54 and earlier). Your commit d72900bded
avoided the problem, but perhaps we should fix it by looking for "und"
and replacing it with "root" while opening, or someth
On Tue, 2023-03-14 at 10:10 -0700, Jeff Davis wrote:
> One loose end is that we really should support language tags like
> "und"
> in those older versions (54 and earlier). Your commit d72900bded
> avoided the problem, but perhaps we should fix it by looking for
> &quo
On Tue, 2023-03-14 at 23:47 -0700, Jeff Davis wrote:
> On Tue, 2023-03-14 at 10:10 -0700, Jeff Davis wrote:
> > One loose end is that we really should support language tags like
> > "und"
> > in those older versions (54 and earlier). Your commit d72900bded
> >
On Wed, 2023-03-15 at 15:18 -0700, Jeff Davis wrote:
> I left out the validation patch for now, and I'm evaluating a
> different
> approach that will attempt to match to the locales retrieved with
> uloc_countAvailable()/uloc_getAvailable().
I like this approach, attached new
On Tue, 2023-03-14 at 08:48 -0700, Jeff Davis wrote:
> Actually, now that I think about it, we could just search all known
> locales using either ucol_getAvailable() or uloc_getAvailable(), and
> see if there's a match. Not very clean, but it should catch most
> problems. I
to make the binary format more practical to use,
can we document the expectations better? It seems the expecatation is
that the binary format just never changes, and that if it does, that's
a new type name.
Regards,
Jeff Davis
On Mon, 2023-03-20 at 10:04 -0700, Jeff Davis wrote:
> CREATE TABLE a(d date, t timestamptz);
> SET format_binary='25,1082,1184';
> SELECT * FROM a;
> d | t
> ---+---
> ! |
> (1 row)
Oops, missing the following statement after the CREATE TABLE:
GIS and he doesn't see PostGIS using this.
One of the things I like about Postgres is that the features all work
together, and that user-defined objects are generally as good as built-
in ones. Sometimes there's a reason to make a special case (e.g. syntax
support or something), but in this case it seems like we could support
user-defined types just fine, right? It's also just more friendly and
readable to use type names, especially if it's a GUC.
Regards,
Jeff Davis
On Mon, 2023-03-20 at 20:18 -0400, Dave Cramer wrote:
> For JAVA pools it's probably OK, but for pools like pgbouncer we have
> no control of who is going to get the connection next.
Can pgbouncer use different pools for different settings of
format_binary?
Regards,
Jeff Davis
y practical case I can think of.
Another solution would be to have some extra catalog field in pg_type
that would be a "binary format identifier" and use that rather than the
type name to match up binary parsers with the proper type.
Am I over-thinking this?
Regards,
Jeff Davis
gt; change the type name.
That sounds reasonable to me. It could also be useful for other
extension objects (or the extension itself) to avoid other kinds of
weirdness from name collisions or major version updates or extensions
that depend on other extensions.
Regards,
Jeff Davis
g an extra round-trip. I'm not sure if that's a
reasonable thing to expect the client to do, so I agree that we should
offer a better way.
Regards,
Jeff Davis
where.
I was wondering about that -- thank you, I changed it back to use "if"
rather than "#ifdef".
New series attached (starting at 0002 to better correspond to the
previous series).
--
Jeff Davis
PostgreSQL Contributor Team - AWS
From fbe03dc596b5e12f4dda60269e044caa58f8b
we
> don't want them to be unduly complicated to process.
That's a good point. I don't know if that is a major design issue or
not; but it certainly adds complexity to the proposal and/or clients
implementing it.
Regards,
Jeff Davis
ion.
You seemed to generally think it was a reasonable idea, but wanted to
wait for the other patch. I think it's the right breakdown of
privileges even now, and I don't see a reason to give ourselves a
headache later trying to split up the privileges later.
Regards,
Jeff Davis
[
or even exercise) the former without the latter.
I could get rid of the SQL-callable function and move the rest of the
changes into 0006. I'll see if that arrangement works better, and that
way we can add the SQL-callable function later (or perhaps not at all
if it's not desired).
Regards,
Jeff Davis
, the connection string and the mechanics of the
subscription are really different concepts.
Regards,
Jeff Davis
On Thu, 2023-03-09 at 11:23 -0800, Jeff Davis wrote:
> Looks good to me.
Another thought: for ICU, do we want the default collation to be
UNICODE (root collation)? What we have now gets the default from the
environment, which is consistent with the libc provider.
But now that we have the UNIC
On Thu, 2023-03-23 at 10:16 -0700, Jeff Davis wrote:
> I could get rid of the SQL-callable function and move the rest of the
> changes into 0006. I'll see if that arrangement works better, and
> that
> way we can add the SQL-callable function later (or perhaps not at all
>
ke a bad idea.
The other patch you posted seems like it makes a lot of progress in
that direction, and I think that should go in first. That was one of
the items I suggested previously[2], so thank you for working on that.
Regards,
Jeff Davis
[1]
https://www.postgresql.org/me
lication can target.
We check for ordinary access privileges, which I think is your next
point, so the above paragraph confuses me a bit.
> This fix interacts in an interesting way with Mark Dilger's work,
> committed by Jeff Davis, to make logical replication respect table
> permissions. I i
ut it seems like some kind
of escape hatch is wise, at least for now.
> Or maybe it should be a log-level type option, so you can set it to
> error, warning, and also completely off?
As the validation patch seems closer to acceptance, I changed it to be
before the canonicalization patch. New
On Fri, 2023-03-24 at 10:00 -0400, Robert Haas wrote:
> On Fri, Mar 24, 2023 at 3:59 AM Jeff Davis wrote:
> > That's a little confusing, why not just always use the
> > SECURITY_RESTRICTED_OPERATION? Is there a use case I'm missing?
>
> Some concern was expresse
in the subscription process, and if so, why?
The doc here:
https://www.postgresql.org/docs/devel/sql-createfunction.html#SQL-CREATEFUNCTION-SECURITY
mentions search_path, but other hazards don't really seem applicable.
(Is the trigger creating new roles?)
Regards,
Jeff Davis
On Fri, 2023-03-24 at 15:45 -0400, Jeff Janes wrote:
> Ever since 27b62377b47f9e7bf58613, I have been getting "ERROR:
> mergejoin input data is out of order" for the attached reproducer.
Thank you for the report! And the simple repro.
Fixed in 81a6d57e33.
Regards,
Jeff Davis
get that
one in first. If we can't, it's probably because we learned something
important.
Regards,
Jeff Davis
On Thu, 2024-02-29 at 17:02 -0800, Jeff Davis wrote:
> Attached is an implementation of a per-database option STRICT_UNICODE
> which enforces the use of assigned code points only.
The CF app doesn't seem to point at the latest patch:
https://www.postgresql.org
On Thu, 2024-03-14 at 15:38 +0100, Peter Eisentraut wrote:
> On 14.03.24 09:08, Jeff Davis wrote:
> > 0001 (the C.UTF-8 locale) is also close...
>
> If have tested this against the libc locale C.utf8 that was available
> on
> the OS, and the behavior is consistent.
That wa
lternatives discussed? Do we have
consensus that this is the right thing to do?
And if we use this approach, is there extra validation or testing that
can be done?
Regards,
Jeff Davis
h has been around for a while, and I didn't
see any objection, either.
I like this feature from a user perspective. So +1 from me.
Regards,
Jeff Davis
[1]
https://www.postgresql.org/message-id/7db39b45-821f-4894-ada9-c19570b11...@postgresfriends.org
elationStatsStmt(), which returns NULL, and
then the caller segfaults.
> I'm looking at those, but in the mean time I'm seeking feedback on
> the progress so far.
Still looking, but one quick comment is that the third argument of
dumpRelationStats() should be const, which eliminates a warning.
Regards,
Jeff Davis
On Fri, 2024-03-15 at 15:30 -0700, Jeff Davis wrote:
> Still looking, but one quick comment is that the third argument of
> dumpRelationStats() should be const, which eliminates a warning.
A few other comments:
* pg_set_relation_stats() needs to do an ACL check so you can't set t
uld make
0001 essentially a complete feature itself.
I think it would also make the changes in pg_dump simpler, and the
tests in 0001 a lot simpler.
Regards,
Jeff Davis
On Sun, 2024-03-17 at 17:46 -0400, Tom Lane wrote:
> Jeff Davis writes:
> > New series attached.
>
> Coverity thinks there's something wrong with builtin_validate_locale,
> and as far as I can tell it's right: the last ereport is unreachable,
> because required_enc
n't have to choose a default.
Regards,
Jeff Davis
why are the callers also checking?
I haven't looked in a lot of detail.
Regards,
Jeff Davis
nitize_address is deprecated in
clang[1] in favor of 'no_sanitize("address")'. It doesn't appear to be
deprecated in gcc[2].
Aside from that, +1.
Regards,
Jeff Davis
[1]
https://clang.llvm.org/docs/AddressSanitizer.html#disabling-instrumentation-with-attribute-no-sanitize-address
[2] https://gcc.gnu.org/onlinedocs/gcc/Common-Function-Attributes.html
a brief test.
One thing we can do is some extra validation for consistency, like
checking that the arrays are properly sorted, check for negative
numbers in the wrong place, or fractions larger than 1.0, etc.
Regards,
Jeff Davis
| {0.4,0.3,0.2}
(1 row)
Can you show an example where it's not?
>
> Maybe we could have the functions restricted to a role or roles:
>
> 1. pg_write_all_stats (can modify stats on ANY table)
> 2. pg_write_own_stats (can modify stats on tables owned by user)
If we go that route, we are giving up on the ability for users to
restore stats on their own tables. Let's just be careful about
validating data to mitigate this risk.
Regards,
Jeff Davis
ow it works for at least one thing, and we can always improve it
later. For instance, we might call the hook several times and pass it a
"phase" argument.
Regards,
Jeff Davis
If the answer is
yes, should they be differentiated somehow so that you can know where
the slow queries are running?
Regards,
Jeff Davis
dump and reload stats on your own
tables, and then not worry about it.
> Not off hand, no.
To me it seems like inconsistent data to have most_common_freqs in
anything but descending order, and we should prevent it.
> >
Regards,
Jeff Davis
8a7744e
> Or we should create a function separate from initcap.
If we create a new function, that also gives us the opportunity to
accept optional arguments to control the behavior rather than relying
on collation for every decision.
Regards,
Jeff Davis
ples during a flush. The
only way I could imagine solving all of these problems is declaring
somehow that your transaction won't do any of these complicated things,
and that you don't mind getting constraint violations at the wrong
time. So I recommend that you punt on this problem.
Regar
does it mean "must not" or something else), and it still
exists in master. Do we care about the calculation being wrong if
there's an unfinished write? If not, I'll just clarify that the
calculation doesn't take into account still-buffered data. If we do
care, then something might need to be fixed.
Regards,
Jeff Davis
tect the problem in my setup, so I added a unit test
in case_test.c where it's easier to see the valgrind problem.
Regards,
Jeff Davis
...
That makes sense for a function in the target list, because different
customers might be from different locales and therefore want different
treatment of the dotted-vs-dotless-i.
Thoughts? Should we use the collation by default but then allow
parameters to override? Or should we just consider this a new set of
functions?
(All of this is v18 material, of course.)
Regards,
Jeff Davis
CT.
Andres, is this patch overall closer to what you had in mind in the
email here:
https://www.postgresql.org/message-id/20230603223824.o7iyochli2dww...@alap3.anarazel.de
?
Regards,
Jeff Davis
s? It seems the primary sources of
complexity came from rules, partitioning, and updatable views, is that
right?
Regards,
Jeff Davis
arly need that to support index insertions for COPY/IIS, so as-
is the API feels incomplete. Thoughts?
Regards,
Jeff Davis
y how we'd find those other areas (if they
exist) aside from just having more eyes on the code.
>
> So on reflection, rather than trying to rush to get this into v17, I
> think it would be better to leave it to v18.
Thank you for letting me know. That allows some time for others to have
a look.
Regards,
Jeff Davis
201 - 300 of 1501 matches
Mail list logo
