> On Mar 19, 2025, at 15:06, Christophe Pettus wrote:
>
> From an (admittedly somewhat naïve) look at the code, it appears that having
> it honor the cost delay wouldn't introduce excessive blocking, as long as the
> delay wasn't implemented at a really dumb place
> On Mar 19, 2025, at 14:12, Melanie Plageman wrote:
>
> Do you know how big the FSM was?
Unfortunately, no; both installations are locked-down hosted environments.
> As others have said, it could be worth adding a phase to
> pg_stat_progress_vacuum.
+1.
> On Mar 19, 2025, at 15:12, Andres Freund wrote:
> That can be checked with pg_relation_size(), e.g.
> SELECT pg_relation_size('pg_class', 'fsm');
> nothing special required.
Thanks!
> On Mar 19, 2025, at 12:12, Álvaro Herrera wrote:
> Assuming this analysis is correct, I agree that the FSM vacuuming should
> also be throttled, as long as that can be done without blocking
> concurrent operations (insertions) on the table.
From an (admittedly somewhat naïve) look at the cod
We're tracking down an issue that we've seen in two separate installations so
far, which is that, at the very end of a vacuum, the vacuum operation starts
using *very* high levels of CPU and (sometimes) I/O, often to the point that
the system becomes unable to service other requests. We've seen
We've run into multiple situations recently where clients were effectively
unable to run CREATE INDEX or REINDEX (CONCURRENTLY or otherwise) due to the
load that they imposed on the system, mostly in I/O. I'd like to propose a
cost-based delay system for maintenance operations such as these, pa
> On May 7, 2024, at 05:02, Amit Kapila wrote:
>
>
> In PG-14, we have added a feature in logical replication to stream
> long in-progress transactions which should reduce spilling to a good
> extent. You might want to try that.
That's been my principal recommendation (since that would also
Thank you for the reply!
> On May 1, 2024, at 02:18, Ashutosh Bapat wrote:
> Is there a large transaction which is failing to be replicated repeatedly -
> timeouts, crashes on upstream or downstream?
AFAIK, no, although I am doing this somewhat by remote control (I don't have
direct access to
Hi,
I wanted to check my understanding of how control flows in a walsender doing
logical replication. My understanding is that the (single) thread in each
walsender process, in the simplest case, loops on:
1. Pull a record out of the WAL.
2. Pass it to the recorder buffer code, which,
3. Sor
Hi,
I wanted to hop in here on one particular issue:
> On Dec 12, 2023, at 02:01, Tomas Vondra wrote:
> - desirability of the feature: Random IDs (UUIDs etc.) are likely a much
> better solution for distributed (esp. active-active) systems. But there
> are important use cases that are likely to
> On Jan 12, 2023, at 12:35, Andres Freund wrote:
>
> On 2023-01-12 12:28:39 -0800, Christophe Pettus wrote:
>> What's the distinction between errdetail and errdetail_log in the ereport
>> interface?
>
> Only goes to the server log, not to the client.
Thanks!
What's the distinction between errdetail and errdetail_log in the ereport
interface?
avepoints, which would break the BEGIN/EXCEPTION/END
semantics.
It's not clear to me what the alternative semantics would be. Can you propose
specific database behavior for a COMMIT or ROLLBACK inside a
BEGIN/EXCEPTION/END block which retain the savepoint behavior of
BEGIN/EXCEPTION/END?
-
ons, of course!
--
-- Christophe Pettus
x...@thebuild.com
e need to
create a wiki page to accurately describe the failure scenarios for both
exclusive and non-exclusive backups, and the recovery actions for them. If it
exists already, my search attempts weren't successful. If it doesn't, I'm
happy to start one.
--
-- Christophe Pettus
x...@thebuild.com
re seemed to be an argument going on that all of the
cool kids will have moved off the old interface and there was essentially no
cost to removing it in v12 or v13, and that didn't correspond to my experience.
--
-- Christophe Pettus
x...@thebuild.com
lly less contentious, and the change is more or less "drop the file
into mumble/conf.d rather than mumble", which is less of a break.
--
-- Christophe Pettus
x...@thebuild.com
your backup system" is going to be.
--
-- Christophe Pettus
x...@thebuild.com
eeling that too many dependencies are dragged in, or just lack of familiarity
with them.
--
-- Christophe Pettus
x...@thebuild.com
be
> removed, and then we'll actually remove it in PG13.
That's not my position, certainly; I still object to its removal.
--
-- Christophe Pettus
x...@thebuild.com
plementable using shell scripts.
It's not complicated why: Backup has to interact with a lot more components of
the overall environment than a CSV export, and those components vary *wildly*
from one installation to another, and are often over-contrained by local policy.
--
-- Christophe Pettus
x...@thebuild.com
sed issues for monitoring, etc.
Some of the incompatible catalog changes (in particular, in pg_stat_activity) I
thought were gratuitous, but we did them, and no point in relitigating that
now. I'd say that the internal layout of PGDATA is fairly weak promise
compared to an SQL-level construct,
the change sound more routine
than it is. It's an "API" that a *lot* of installations depend on.
> I don't agree that simply documenting the issues with
> it is a sufficient solution to make us keep it.
Understood, but I think we need to document it no matter what.
--
-- Christophe Pettus
x...@thebuild.com
course, but they're not the same class of promise as a C API.
> Sure, it'd be nice if someone would fix it to list out the problems with
> the exclusive API, but that requires someone wanting to spend time on
> it.
I'll take a turn at it.
--
-- Christophe Pettus
x...@thebuild.com
use this one, and *here's why*." Saying, "Don't use this one because we
said so" when it is an API of long standing that works just as it always did
isn't going to cut it.
--
-- Christophe Pettus
x...@thebuild.com
ng to it."
Rather than deprecate the existing API, I'd rather see the documentation
updated to discuss the danger cases.
--
-- Christophe Pettus
x...@thebuild.com
". I was just bitted by that this week.
--
-- Christophe Pettus
x...@thebuild.com
dump) would be an undoubtedly
superior choice. A long depreciation window would cover a lot of those
situations.
--
-- Christophe Pettus
x...@thebuild.com
> On Jul 26, 2018, at 19:35, Peter Geoghegan wrote:
>
> Why, specifically, would it make them unhappy?
Forensic and archive backups in .tar format (which I know of users doing) would
require a two-step restore process on newer versions.
--
-- Christophe Pettus
x...@thebuild.com
> On Jul 18, 2018, at 14:33, Thomas Munro wrote:
> Here you side step those questions completely and make that the end
> user's problem. I like it.
+1. This is a clever solution, since any kind of key vault or other system
could be dropped in there.
--
-- Christop
a
checkpoint or have a --checkpoint option along the lines of pg_basebackup?
This scenario (pg_rewind being run very quickly after secondary promotion) is
not uncommon when there's scripting around the switch-over process.
--
-- Christophe Pettus
x...@thebuild.com
> On Jul 12, 2018, at 19:22, Christophe Pettus wrote:
>
>
>> On Jul 12, 2018, at 17:52, Michael Paquier wrote:
>> Wild guess: you did not issue a checkpoint on the promoted standby
>> before running pg_rewind.
>
> I don't believe a manual checkpoint wa
rtup after the timeline switch:
> 2018-07-10 19:28:38 UTC [5068]: [1-1] user=,db=,app=,client= LOG: checkpoint
> starting: force
The pg_rewind was started about 90 seconds later.
--
-- Christophe Pettus
x...@thebuild.com
max_locks_per_xact setting: 64
track_commit_timestamp setting: off
Maximum data alignment: 8
Database block size: 8192
Blocks per segment of large relation: 131072
WAL block size: 8192
Bytes per WAL segment:16777216
Maximum length of identifiers:64
Maximum columns in an index: 32
Maximum size of a TOAST chunk:1996
Size of a large-object chunk: 2048
Date/time type storage: 64-bit integers
Float4 argument passing: by value
Float8 argument passing: by value
Data page checksum version: 0
--
-- Christophe Pettus
x...@thebuild.com
p for about 10 seconds (although it had completed recovery) before being
shut down again, and pg_rewind applied to it to reconnect it with the promoted
secondary.
--
-- Christophe Pettus
x...@thebuild.com
r Chamber that is answerable only to itself. It also allows
for an appeal mechanism.
--
-- Christophe Pettus
x...@thebuild.com
ling options to handle that
in a pooling environment.
The next most common problem are prepared statements breaking, which certainly
qualifies as a session-level feature.
--
-- Christophe Pettus
x...@thebuild.com
id the limitations that a non-core pooler has.
--
-- Christophe Pettus
x...@thebuild.com
much less fussed by this in contrib/ (with the same concern you noted), at
minimum as an example of how to do logging in other formats.
--
-- Christophe Pettus
x...@thebuild.com
hem just file, for example. You have to handle newlines that
are part of a log message somehow; a newline in a PostgreSQL query, for
example, needs to be emitted to the logs.
--
-- Christophe Pettus
x...@thebuild.com
> On Apr 15, 2018, at 11:00, David Arnold wrote:
>
> CSV Logs: https://pastebin.com/uwfmRdU7
Is the issue that there are line breaks in things like lines 7-9?
--
-- Christophe Pettus
x...@thebuild.com
I have reviewed some log samples and all DO contain some kind of multi line
> logs which are very uncomfortable to parse reliably in a log streamer.
... but I don't see any actual examples of those. Can you elaborate?
--
-- Christophe Pettus
x...@thebuild.com
> On Apr 15, 2018, at 10:07, Christophe Pettus wrote:
>
>
>> On Apr 15, 2018, at 09:51, David Arnold wrote:
>>
>> 1. Throughout this vivid discussion a good portion of support has already
>> been manifested for the need of a more structured (machine readab
m afraid I don't see that. While it's true that as a standard, CSV is
relatively ill-defined, as a practical matter in PostgreSQL it is very easy to
write code that parses .csv format.
--
-- Christophe Pettus
x...@thebuild.com
unpartitioned table will always be created in the default
tablespace unless otherwise specified, but child tables are sufficiently
distinct from that case that I don't see it as a painful asymmetry.
--
-- Christophe Pettus
x...@thebuild.com
here's any
> need to panic.
No reason to panic, yes. We can assume that if this was a very big persistent
problem, it would be much more widely reported. It would, however, be good to
find a way to get the error surfaced back up to the client in a way that is not
just monitoring
t be popular with people running on file systems or OSes that
don't have this issue. (Setting aside the daunting prospect of implementing
that.)
--
-- Christophe Pettus
x...@thebuild.com
memory.
I guess in theory that they could swap them, but swapping out a file system
buffer in hopes that sometime in the future it could be properly written
doesn't seem very architecturally sound to me.
--
-- Christophe Pettus
x...@thebuild.com
storage coming along) to address this.
While the failure modes are more common, the solution (a PITR backup) is one
that an installation should have anyway against media failures.
--
-- Christophe Pettus
x...@thebuild.com
an uncorrectable error when we finally get around to
reading it.
--
-- Christophe Pettus
x...@thebuild.com
is (a) fix the error, then (b) replay from a checkpoint before the
error occurred, but it appears we can't even guarantee that a PostgreSQL
process will be the one to see the error.
--
-- Christophe Pettus
x...@thebuild.com
> On Apr 2, 2018, at 17:05, Andres Freund wrote:
>
> Don't we pretty much already have agreement in that? And Craig is the main
> proponent of it?
For sure on the second sentence; the first was not clear to me.
--
-- Christophe Pettus
x...@thebuild.com
n that, I think the PANIC option is the soundest one, as
unappetizing as it is.
--
-- Christophe Pettus
x...@thebuild.com
warehouse from an OLTP system.
--
-- Christophe Pettus
x...@thebuild.com
54 matches
Mail list logo
