ver1','u1',true);
create subscription sub1 server server1 publication pub1;
I don't think we need to add an end-to-end test for each FDW, because
it's just using the assembled connection string. To see if it's
assembling the connection string properly, we can unit test with
pg_conninfo_from_server().
Regards,
Jeff Davis
27;t offer any other mechanism to, e.g., get case mapping according
to the "tr_TR" locale.
I think what Postgres does here, re-purposing the COLLATE clause to
allow tailoring of case mapping, is imperfect but reasonable. My
proposal doesn't change that.
Regards,
Jeff Davis
> Make sure to click the link to show the counterexamples and
> discussion,
> that's the best part.
Yes, it can be hard to reason about this stuff but I believe Unicode
provides a lot of good data and guidance to work from. If you think my
proposal relies on one of those assumptions let me know.
To the extent that I do rely on any of those assumptions, it's mostly
to match libc's "C.UTF-8" behavior.
Regards,
Jeff Davis
. CONNECTION makes
little effort, only checking whether the password is specified or not.
I'd like to understand why they are different and what we can unify.
Also, right now dblink has it's own dblink_fdw, and perhaps a server
FOR CONNECTION ONLY should become the preferred method instead.
ection string out of the options itself and do another
round of validation, is that what you had in mind?
> We can push down a join
> between a replicated table and foreign table down to the foreign
> server.
Interesting idea.
Regards,
Jeff Davis
s in
pg_dump and elsewhere? Also I didn't see a great place to document it.
Admittedly, I didn't complete the dummy-FDW approach, so perhaps it
works out better overall. I can give it a try.
Regards,
Jeff Davis
On Wed, 2023-08-30 at 09:09 -0700, Jeff Davis wrote:
> Admittedly, I didn't complete the dummy-FDW approach, so perhaps it
> works out better overall. I can give it a try.
We need to hide the dummy FDW from pg_dump. And we need to hide it from
psql's \dew, because that's use
nd it seems to violate the idea of an "extension" entirely.
Can you explain more concretely how we might resolve that?
Regards,
Jeff Davis
about revoking it, is kind of strange. I don't think we really want
> to
> encourage users to tinker with predefined roles in this kind of way.
> I
> think there are better ways of achieving the goals here.
Such as?
Regards,
Jeff Davis
t effectively, only Bob
can test it.
In other words, there are some consistency aspects to how we run code
that go beyond pure security. A function author typically has
assumptions about the execution context of a function (the user, the
sandbox restrictions, the search_path, etc.) and guiding users towards
a consistent execution context in typical cases is a good thing.
Regards,
Jeff Davis
ly
> (as in logical vs physical view of data).
I still don't understand how a core dependency on an extension would
work.
Regards,
Jeff Davis
an't be inside pg_catalog. So we'd
need some special logic somewhere to make pg_dump and psql \dew work as
expected, and I'm not quite sure what to do there.
Regards,
Jeff Davis
n it does. Labeling is
> all-or-nothing.
Here I'm getting a little lost in what you mean by "prohibited
operation". Most languages mostly use SPI, and whatever sandboxing
checks you do should work there, too. Are you talking about completely
separate side effects like writing files or opening sockets?
Regards,
Jeff Davis
lso allow other FDWs but the
subscription would have to do its own validation.
Regards,
Jeff Davis
h of a practical difference between having privileges but being
in a context where they won't be used, and dropping the privileges
entirely during that time. I suppose the answer is that the EXECUTE
privilege will still be used, but as I said above, that doesn't
entirely make sense to me, either.
Regards,
Jeff Davis
eff (added now in CC), the refactoring done in d87d548c seems to be
> at the origin of this confusion, because, before this commit, we
> never
> generated this specific error for all these APIs where the locale is
> undefined. What is your take here?
Agreed.
Regards,
Jeff Davis
rovider" is fine for me across the board as these should not be
> user-visible.
That's fine with me.
Regards,
Jeff Davis
On Tue, 2023-09-12 at 09:03 +0900, Michael Paquier wrote:
> On Mon, Sep 11, 2023 at 12:15:49PM -0700, Jeff Davis wrote:
> > That's fine with me.
>
> Okay. Then, please find attached a v4 for HEAD and REL_16_STABLE.
One question: would it make sense to use __func__?
Other t
On Mon, 2023-08-07 at 15:39 -0700, Nathan Bossart wrote:
> 0003 is looking pretty good, too, but I think we
> should get some more eyes on it, given the complexity.
Attached rebased version of 0003.
--
Jeff Davis
PostgreSQL Contributor Team - AWS
From f5b055aea1bf08928de1bffcfd7b202e28
s be useful enough to justify a new data type? Would it be
confusing about when to choose one versus the other?
* Would cross-type comparisons between TEXT and UTEXT become a major
problem that would reduce the utility?
* Should "some_utext_value = some_text_value" coerce the LHS
On Tue, 2023-09-05 at 12:08 -0700, Jeff Davis wrote:
> OK, so we could have a built-in FDW called pg_connection that would
> do
> the right kinds of validation; and then also allow other FDWs but the
> subscription would have to do its own validation.
While working on this, I found
ses __func__ as Jeff has suggested.
>
> What do you think?
Looks good to me, thank you.
Regards,
Jeff Davis
On Tue, 2023-09-12 at 13:55 -0700, Jeff Davis wrote:
> On Mon, 2023-08-07 at 15:39 -0700, Nathan Bossart wrote:
> > 0003 is looking pretty good, too, but I think we
> > should get some more eyes on it, given the complexity.
>
> Attached rebased version of 0003.
Is someone
the
operation is part of a special carve out (logical replication or a
maintenance command). And don't use the predefined role
pg_write_all_data, because that's unsafe for most imaginable use cases.
Regards,
Jeff Davis
y see an
> option other than (1) "change something in the system design so that
> people use search_path wrongly less often" or (2) "make it so that it
> doesn't matter as much if people using the wrong search_path" but
> maybe I'm missing a clever idea.
Perhaps there are some clever ideas about maintaining compatibility
within the approaches (1) or (2), which might make one of them more
appealing.
Regards,
Jeff Davis
On Tue, 2023-09-19 at 20:23 -0700, Maciek Sakrejda wrote:
> On Tue, Sep 19, 2023 at 5:56 PM Jeff Davis wrote:
> > ...
> > On Tue, 2023-09-19 at 11:41 -0400, Robert Haas wrote:
> > > That leads to a second idea, which is having it continue
> > > to be a GUC but
h these nuances. Even though the SET clause
has been there for a long time, our documentation on the subject is
insufficient and misleading. And on top of that, it's extra typing and
noise for every schema file. Until we make some changes I don't think
we can expect users to do as you suggest.
Regards,
Jeff Davis
MarkBufferDirty(pageData->buffer);
}
END_CRIT_SECTION();
Am I missing something or is that a problem?
Regards,
Jeff Davis
out being able to
touch the connection string itself). I'd just document that in its own
section, and mention a few caveats / mistakes to avoid. For instance,
when the superuser is defining the connection, don't forget to set
password_required=false, so that when you reassign to a non-superuser
then the connection doesn't break.
Regards,
Jeff Davis
On Mon, 2023-09-25 at 11:30 -0400, Robert Haas wrote:
> On Fri, Sep 22, 2023 at 4:05 PM Jeff Davis wrote:
> > You expect
> > Bob to do something like:
> >
> > CREATE FUNCTION ... SET search_path = pg_catalog, pg_temp ...
> >
> > for all functions, not j
all, e.g., in an index expression.
* it avoids annoyances with pg_temp, because that's not searched for
functions/operators anyway
* perhaps we could force the object search_path to be empty for
IMMUTABLE functions?
I haven't thought it through in detail, but it seems like a promising
assword, make sure
to set password_required=false before transferring ownership, otherwise
it will start failing."
Documenting the behavior is good, too, but I find the behavior
difficult to document, so examples will help.
Regards,
Jeff Davis
;s not always the case, e.g. log_newpage_range().
Regards,
Jeff Davis
From c54749a7939721f0a838115abce09967216cbc0f Mon Sep 17 00:00:00 2001
From: Jeff Davis
Date: Tue, 26 Sep 2023 12:10:11 -0700
Subject: [PATCH v1] Fix GenericXLogFinish().
Mark the buffers dirty before writing WAL.
Disc
XLogRegisterBuffer(), and fix all the
> callers.
That site is pretty trivial to fix, but there are also a couple places
in hash.c and hashovfl.c that are registering a clean page and it's not
clear to me exactly what's going on.
Regards,
Jeff Davis
From 9548bb49865db3a04bbdd
passwords, we use the salt to
select the right scram password, and then proceed from there?
I'm not very excited about the idea of naming passwords, or having
passwords with default names. I can't think of anything better right
now, so it might be OK.
> - Add tests
> - Add/updat
On Wed, 2023-09-27 at 10:36 -0400, Robert Haas wrote:
> On Tue, Sep 26, 2023 at 9:36 PM Jeff Davis wrote:
> > That site is pretty trivial to fix, but there are also a couple
> > places
> > in hash.c and hashovfl.c that are registering a clean page and it's
> > no
reason.
Regards,
Jeff Davis
l we have a chance to analyze/refactor each of
the callers that need it.
Regards,
Jeff Davis
ormalized with NFC, it could be frustrating if
the retrieved string has a different binary representation than the
source data. But it could also be frustrating to look at two strings
made up of ordinary characters that look identical and for the database
to consider them unequal.
Regards,
Jeff Davis
ndexes break, and text
comparisons are slow
A stable idea of unicode normalization based on using only assigned
code points is very tempting.
Regards,
Jeff Davis
at means that indices
> need to normalize strings, but tables need to store unnormalized
> strings.
That's an interesting idea. Would the equality operator normalize
first, or are you saying that the index would need to recheck the
results?
Regards,
Jeff Davis
#x27;s not helping to reach the compromise that I'm trying to reach in
this thread.
Regards,
Jeff Davis
orance benefit as-is
(correct me if things have changed) and I don't know if a reduction in
syscalls alone is enough to justify it. But if it paves the way for
direct IO for WAL, that does seem worth it.
Regards,
Jeff Davis
certainly users who would prefer that the codepoints they send to the
database are preserved exactly, and also users who would like to be
able to use unassigned code points.
Regards,
Jeff Davis
cases even if not), then UTF-8 seems like the right solution.
Text data encoded 17 different ways requires a lot of bookkeeping in
the type system, and it also requires injecting a bunch of fallible
transcoding operators around just to compare strings.
Regards,
Jeff Davis
of unicode-based type, it would only
work with encodings that are a subset of unicode.
Regards,
Jeff Davis
would use that,
and why.
It would take an awful lot of effort to implement and make the code
more complex, so we'd really need to see some serious demand for that.
Regards,
Jeff Davis
t to whatever unanticipated things happen in the future; rather
than to make the code very complicated trying to anticipate everything,
and then being completely unable to adapt it when something
unanticipated happens anyway.
Regards,
Jeff Davis
y identical strings, but it's
also a violation of unicode. (This is another reason why normalization
might not be for everyone, but I believe it's still good in typical
cases.)
Regards,
Jeff Davis
ress them (names, numbers, validity period,
something). But by isolating the problem to deprecated passwords only,
it feels like the system is still being restored to a clean state with
at most one single current password. The awkwardness is contained to
old passwords which will hopefully go away soon anyway and not
represent permanent clutter.
Regards,
Jeff Davis
assigned according to the postgres unicode version but unassigned
according to the ICU version. But that would be tracked by the
collation version as you pointed out earlier, so upgrading ICU would be
like any other ICU upgrade (with the same risks). Right?
Regards,
Jeff Davis
to receive the data -- nobody has a
presented a reason in this thread that we need per-column encodings on
the server.
Regards,
Jeff Davis
8 to be the "one true
way". I am not suggesting that we do that, but it seems odd to go in
the opposite direction and have greater flexibility for many encodings.
Regards,
Jeff Davis
f clutter:
* old passwords not being garbage-collected
* the identifier of the current password always changing (perhaps fine
if it'a a "created at" ID?)
Regards,
Jeff Davis
uld be coupled with
a date, and that way all old passwords would always have some validity
period. That might make it a bit easier to manage if we do need more
than two passwords.
Regards,
Jeff Davis
an saying there aren't any.
> (b) that we messed up with
> client_encoding and shouldn't add any more mistakes of the same ilk
> or
> (c) that we should really be looking at redesigning the way
> client_encoding works, too.
(b) doesn't seem like a very helpful perspective without some ideas
toward (c). I think (c) is worth discussing but we don't have to block
on it.
Regards,
Jeff Davis
I committed and backported 0001 (the GenericXLogFinish() fix but not
the Assert).
Strangely I didn't see the -committers email come through yet. If
anyone notices anything wrong, please let me know before the final v11
release.
On Thu, 2023-09-28 at 12:05 -0700, Jeff Davis wrote:
> Als
t;make check" no longer picks up the locale from
the system environment by default.
What is the new way to run the regression tests with an actual locale?
If it's no longer done by default, won't that dramatically reduce the
coverage of non-C locales?
Regards,
Jeff Davis
On Wed, 2023-10-11 at 08:56 +0200, Peter Eisentraut wrote:
> On 11.10.23 03:08, Jeff Davis wrote:
> > * unicode_is_valid(text): returns true if all codepoints are
> > assigned, false otherwise
>
> We need to be careful about precise terminology. "Valid" has a
&
a very coarse version number.
In any case, a CHECK constraint to avoid unassigned code points has
utility to be forward-compatible with normalization, and also might
just be a good sanity check.
Regards,
Jeff Davis
es; this is a big improvement. I'll leave it to Robert
to commit it, so that he can ensure it matches how he expected the
feature to be used and sufficiently covers the behavioral aspects.
Regards,
Jeff Davis
On Wed, 2020-02-26 at 19:14 -0800, Jeff Davis wrote:
> Rebased on your change. This simplified the JIT and interpretation
> code
> quite a bit.
Attached another version.
* tweaked EXPLAIN output some more
* rebased and cleaned up
* Added back the enable_hashagg_disk flag (default
ouping set
in its own memory context.
Also, is there a reason you report two different memory values
(hashtable and tuples)? I don't object, but it seems like a little too
much detail.
Regards,
Jeff Davis
On Fri, 2020-03-13 at 10:27 -0700, Andres Freund wrote:
> On 2020-03-13 10:15:46 -0700, Jeff Davis wrote:
> > Also, is there a reason you report two different memory values
> > (hashtable and tuples)? I don't object, but it seems like a little
> > too
> > much detai
so.
> + if (from_tape)
> + partition_mem += HASHAGG_READ_BUFFER_SIZE;
> + partition_mem = npartitions * HASHAGG_WRITE_BUFFER_SIZE;
>
> => That looks wrong ; should say += ?
Good catch! Fixed.
Regards,
Jeff Davis
diff --git a/doc/src/sgml/config.sgm
matches the word "allocation" in spirit,
it's not technically correct, so feel free to suggest a new name for
MemoryContextMemAllocated().
Regards,
Jeff Davis
diff --git a/src/backend/utils/mmgr/aset.c b/src/backend/utils/mmgr/aset.c
index c0623f106d2..ccf78ffe0cb 100644
---
On Mon, 2020-03-16 at 11:45 -0700, Jeff Davis wrote:
> Attached is a patch that makes mem_allocated a method (rather than a
> field) of MemoryContext, and allows each memory context type to track
> the memory its own way. They all do the same thing as before
> (increment/decrement a
se there would be two
types of spilled data (tuples and partial states), but it could be
useful in some cases.
Regards,
Jeff Davis
s 4MB, and the doubling
behavior goes to 8MB, so it's a problem with default settings.
Regards,
Jeff Davis
#x27;t make a lot of
sense to count that last large mostly-empty block against work_mem.
Regards,
Jeff Davis
meone trying to figure out where a backend is leaking
> memory to erroneous conclusions.
MemoryContextStats() was unaffected by my now-reverted change.
Regards,
Jeff Davis
On Mon, 2020-03-16 at 11:45 -0700, Jeff Davis wrote:
> Although that's technically correct, the purpose of
> MemoryContextMemAllocated() is to give a "real" usage number so we
> know
> when we're out of work_mem and need to spill (in particular, the
> disk-
>
On Thu, 2020-03-19 at 15:33 -0400, Robert Haas wrote:
> On Thu, Mar 19, 2020 at 3:27 PM Jeff Davis wrote:
> > I think omitting the tail of the current block is an unqualified
> > improvement for the purpose of obeying work_mem, regardless of the
> > OS.
> > The block si
ashAgg itself, or both.
Regards,
Jeff Davis
o unused bits of the Datum. That
would work if the transtype is by-ref (low bits if pointer will be
unused), or if the type's size is less than 8, or if the particular
aggregate doesn't need either of those booleans. It would get messy,
but saving 8 bytes per group is non-trivial.
Regards,
Jeff Davis
;t
see any purpose for the argument to BuildTupleHashTableExt(), nor the
purpose of the "entrysize" field of TupleHashTableData.
> If we want to optimize memory usage, I think I'd first go for
> allocating
> the group's "firstTuple" together with all the AggStatePerGroupDatas.
That's a good idea.
Regards,
Jeff Davis
ntext, but the planner
will call this function before the hash agg memory context is created.
It seems to make more sense for this to just be an AllocSet-specific
function for now.
Regards,
Jeff Davis
diff --git a/src/backend/executor/nodeAgg.c b/src/backend/executor/nodeAgg.c
index 2d94d7
On Tue, 2020-03-24 at 18:12 -0700, Jeff Davis wrote:
> I considered making it a method of a memory context, but the planner
> will call this function before the hash agg memory context is
> created.
I implemented this approach also; attached.
It works a little better by having acce
rong too.
So do you generally favor the EstimateMemoryChunkSpace() patch (that
works for all context types)? Or do you have another suggestion? Or do
you think we should just do nothing?
Regards,
Jeff Davis
y no obviously correct design for the "chunk
> size" functions, and not having overhead is better than ~correctly
> estimating it.
Trying to actually eliminate the overhead sounds like v14 to me.
I believe the formula for AllocSet overhead can be approximated with:
16 + size/4
That would probably be better than a constant but a little hacky. I can
do that as a spot fix if this patch proves unpopular.
Regards,
Jeff Davis
On Wed, 2020-03-18 at 15:41 -0700, Jeff Davis wrote:
> In an off-list discussion, Andres suggested that MemoryContextStats
> could be refactored to achieve this purpose, perhaps with flags to
> avoid walking through the blocks and freelists when those are not
> needed.
Attached refac
;= 1. That allowed me to remove the
conditional and simplify a bit.
Regards,
Jeff Davis
that change. For instance, the ProcessUtility_hook changed, and
that's used much more widely.
As long as we generally agree that some kind of custom rmgrs are the
way to go, if we change the API or implementation around from version
to version I can easily update my table access method.
Regards
mgr
loaded anyway, so a sparse array or hashtable seemed wasteful. If
custom rmgrs become popular we probably need to have a larger ID space
anyway, but it seems like overengineering to do so now.
Regards,
Jeff Davis
That way all the stuff around StdRdOptions is
unchanged. When the table AM needs the parameter value, it can parse
pg_class.reloptions for itself and save it in rd_amcache.
Regards,
Jeff Davis
ize of the stack buffer.
That suggests that maybe TEXTBUFSIZE is too large (1024) and perhaps we
should drop it down, but I need to investigate more.
In general, I'm trying to slowly get the locale stuff more isolated.
Regards,
Jeff Davis
lengths.
On Fri, 2022-11-04 at 15:06 -0700, Jeff Davis wrote:
> But I think the complex hacks are just the transformation into UTF 16
> and calling of wcscoll(). If that's done from within pg_strcoll()
> (after my patch), then I think it just works, right?
I included a patch (0
On Thu, 2022-10-13 at 12:49 -0700, Jeff Davis wrote:
> It may violate our torn page protections for checksums, as well...
I could not reproduce a problem here, but I believe one exists when
checksums are enabled, because it bypasses the protections of
UpdateMinRecoveryPoint(). By not updat
the future, but this behavior
has been stable for a while.
The thread here:
https://postgr.es/m/ee47ee24-2928-96e3-a2b1-97cbe07b2c7b%40garret.ru
also indicates that external projects and tools are relying on our
rules for the page LSNs.
Regards,
Jeff Davis
From 6594a9698a9edde666f883fcb14c
a previous proposal[1], except
it can now be done as an extension. If it's useful, we could even put
it in contrib with a real RMGR ID.
Though I'm also fine just adding a test module to start with.
Regards,
Jeff Davis
[1]
https://www.postgresql.org/message-id/20ee0b0ae6958804a88fe9580157587720faf664.ca...@j-davis.com
guarantee that things won't change in the future, but the behavior it
describes has been true for quite a while now.
--
Jeff Davis
PostgreSQL Contributor Team - AWS
ng from
> the administrator.
How is this related to the search-by-collversion design? It seems like
it's hard no matter what.
--
Jeff Davis
PostgreSQL Contributor Team - AWS
egards,
Jeff Davis
c message, rather than a random
one, which allows more complete testing
* fixed a couple bugs
* used a static initializer for the RmgrData rather than memset,
which shows a better example
I also separately committed a patch to mark the argument of
RegisterCustomRmgr as "const".
g.pl?nm=lapwing&dt=2022-11-16%2000%3A40%3A11
Fixed, thank you. I'll be more diligent about pushing to github CI
first.
Regards,
Jeff Davis
ivilege is revoked, that it
revokes a stable set of capabilities).
[2]
https://www.postgresql.org/message-id/YzIAGCrxoXibAKOD%40tamriel.snowman.net
--
Jeff Davis
PostgreSQL Contributor Team - AWS
> Hence desire to build something that at least admits that it's
> primitive and
> just gives you some controls, in a first version.
Using either the library version or the collation version seems
reasonably simple to me. But from a documentation and usability
standpoint, the way they are currently mixed seems confusing.
--
Jeff Davis
PostgreSQL Contributor Team - AWS
ver libraries the admin can provide;
but adding some observability into the search should mitigate any
confusion.
Can you go over the advantages of approaches 2-4 again? Is it just a
concern about burdening the admin with finding the right ICU library
version for a given collation version? That's a valid concern, but I
don't think that should be an overriding design point. It seems more
important to model the collation versions properly.
--
Jeff Davis
PostgreSQL Contributor Team - AWS
Incidentally, this answers our other question about whether the
collation version can change in a minor version update. Perhaps not,
but if they fix this bug and backport it, then the version *string*
will change in a minor update. Ugh.
Regards,
Jeff Davis
On Mon, 2022-11-28 at 21:57 -0500, Robert Haas wrote:
> That is ... astonishingly bad.
https://unicode-org.atlassian.net/browse/CLDR-16175
--
Jeff Davis
PostgreSQL Contributor Team - AWS
601 - 700 of 1493 matches
Mail list logo
