ke of simplicity's, several
GUC's have been added. It would be good to think through the parallel
launch condition without them.
As always, any thoughts and opinions are very welcome!
--
Best regards,
Maxim Orlov.
WIP-Allow-autovacuum-to-process-indexes-of-single-table.patch
Description: Binary data
I moved the topic to the next commitfest.
--
Best regards,
Maxim Orlov.
"palloc" is a
relatively old one, and no one expected the number of requests to exceed 1
GB. Now we have the ability to set the shared_buffers to a huge number
(without discussing now whether this makes any real sense), thus this limit
for palloc becomes a problem.
--
Best regards,
Maxim Orlov.
Here is a rebase, v14.
--
Best regards,
Maxim Orlov.
From ee4b3b3c3ad3293460eb1f0418d87a065b9a589b Mon Sep 17 00:00:00 2001
From: Maxim Orlov
Date: Wed, 4 May 2022 15:53:36 +0300
Subject: [PATCH v14 5/7] TEST: initdb option to initialize cluster with
non-standard xid/mxid/mxoff
To date
I think I figured it out. Here is v4.
If the number of requests is less than 1 GB, the algorithm stays the same
as before. If we need to process more, we will do it incrementally with
slices of 1 GB.
Best regards,
Maxim Orlov.
v4-0001-Process-sync-requests-incrementally-in-AbsorbSync.patch
beginning and processing requests again.
I'd like to hear your opinion on the subject.
--
Best regards,
Maxim Orlov.
v3-0001-Limit-AbsorbSyncRequests-to-1Gb-at-once.patch
Description: Binary data
After done some testing, I found a bug in the patch. If more requests were
pushed while we release the lock, num_requests could not be set to zero.
Here is a fixed version.
--
Best regards,
Maxim Orlov.
v2-0001-AbsorbSyncRequests-incrementally-instead-of-doing.patch
Description: Binary data
postgres to use huge
amount of RAM. Is this really a problem if he will demand some more to
process sync request?
--
Best regards,
Maxim Orlov.
v1-0001-AbsorbSyncRequests-incrementally-instead-of-doing.patch
Description: Binary data
Great! Looks very good. Thanks a lot!
--
Best regards,
Maxim Orlov.
On Wed, 26 Feb 2025 at 11:54, Andres Freund wrote:
>
> 4) Do compaction incrementally, instead of doing it for all requests at
> once.
Yeah, good idea! I completely forgot about that. Thanks!
--
Best regards,
Maxim Orlov.
r. Just
write the name that you think is correct.
--
Best regards,
Maxim Orlov.
v3-0001-Avoid-double-spinlock-release.patch
Description: Binary data
d with MCXT_ALLOC_HUGE flag.
3. Do not use any allocation and use CheckpointerShmem->requests directly
in case of > 1G size of the required allocation.
Case (3) is not an option, in my opinion. So, we following (1) or (2).
Personally, I'm for (2), PFA v0 patch.
--
Best regards,
Max
mple: imagine a palloc failure while holding this spinlock in
> this elog().
Indeed. PFA the correct one.
--
Best regards,
Maxim Orlov.
v2-0001-Avoid-double-spinlock-release.patch
Description: Binary data
odule?
PFA patch. I don't use pg_attribute_always_inline for fastgetattr and
heap_getattr because they are relatively
large. I think it's worth leaving the possibility for debugging here.
--
Best regards,
Maxim Orlov.
v2-0001-Use-pg_attribute_always_inline-for-static-inline-.patch
Description: Binary data
this change is
needed or not.
--
Best regards,
Maxim Orlov.
So, patch looks good to me. Implements described functionality. Test case
also provided.
I think it's ready to be viewed by a committer.
--
Best regards,
Maxim Orlov.
quot; member in "ArrayCoerceExpr"
and erroneously
consider them to have the same structure. Maybe some refactoring may be
done here, but,
obviously, this is not a goal of this patch
--
Best regards,
Maxim Orlov.
ed it, but I'm now
> thinking about withdrawing it.
>
Thanks for explanation, at your will.
--
Best regards,
Maxim Orlov.
reuse" T_RelabelType case, as it made for T_OpExpr and
T_DistinctExpr?
--
Best regards,
Maxim Orlov.
>
I tend to agree with you here about the semantics of such casts. But
consistency and "predictability" of
behaviour for the casts are a bit more important in my view. At least,
based on my experience. So, I'm
for 0002 patch.
--
Best regards,
Maxim Orlov.
I have to admit I was wrong with previous v2 patch. Sorry.
Apparently, the chances of committing this very low, but here is the
correct one.
--
Best regards,
Maxim Orlov.
v3-0001-Use-pstrdup-for-remote_host-and-remote_port-save-.patch
Description: Binary data
ffset
does not exist, consider
running vacuum before pg_upgrdade" or smth. Please, correct me if I'm wrong.
--
Best regards,
Maxim Orlov.
for a second table;
3) drop first table;
4) stop pg cluster;
5) remove pg_multixact/offsets/
6) upgrade?
PFA, v10-0016-TEST-try-to-replicate-buggy-oldest-offset.patch
This test will fail now, for an obvious reason, but is this case a relevant
one?
--
Best regards,
Maxim Orlov.
<>
I'm glad you are bringing up this issue. By the way, there are two more
annoying places in postmaster.c for pg16 and older. See, strdup() also may
fail if insufficient memory available.
PFA patch for a REL_16_STABLE. It also applies to older versions.
--
Best regards,
Maxim Orlov.
v2-000
formance by removing locks for a "not
a bug, but a feature". This seems odd to me. Arguably, the number of people
who need faster temp relations is greater than the number of people who
want to have access to temp relations of other backends.
--
Best regards,
Maxim Orlov.
ate segments are generated.
- pg_upgarde tests.
Here is oldinstall ENV is for. Run pg_upgrade for old cluster with multi
and offset values just like in previous step. i.e. with various
combinations.
- Self pg_upgarde.
--
Best regards,
Maxim Orlov.
From 2642f597832cbed0ebc54202de4e0f
Shame on me! I've sent an erroneous patch set. Version 7 is defective. Here
is the proper version v8 with minor refactoring in segresize.c.
Also, I rename bump cat version patch into txt in order not to break cfbot.
--
Best regards,
Maxim Orlov.
From 73b8663093ff1c58def9a80abab142a12c993bf
resentation in
combination with switching multi to 64 bit. This seems a bit more
appropriate in my view.
As for your optimization suggestions, I like them. I don’t against them,
but I’m afraid to disrupt the clarity of thought, especially since the
algorithm is not the simplest.
--
Best regards,
Maxim Orlov.
patch useful. Overall, looks good to me.
--
Best regards,
Maxim Orlov.
./generate.sh
Then the test.sh is used to run various upgrades.
OLDBIN=/.../pgsql-old NEWBIN=/.../pgsql-new ./test.sh
I hope that helps!
--
Best regards,
Maxim Orlov.
#!/usr/bin/env python3
import sys;
import threading;
import psycopg2;
def test_multixact(tblname: str):
with psycopg2
s invalid value once.
The first options seem too weird for me. So, we have to repack members and
bypass invalid value.
All patches are for master@38c18710b37a2d
--
Best regards,
Maxim Orlov.
v7-0001-Use-64-bit-format-output-for-multixact-offsets.patch
Description: Binary data
v7-0004-
[1].
[0]
https://www.postgresql.org/message-id/flat/d4a68c6d-d6c4-d52a-56cb-babb8177b5fe%40oss.nttdata.com
[1]
https://www.postgresql.org/message-id/flat/CACG%3DezZzMbjMzshhe%2BLDd4NJ0SeRPvCH9%2BLFS7SAPbM6Qxwe5g%40mail.gmail.com
--
Best regards,
Maxim Orlov.
igher value. For now, I choose 2^32-1. In
other world, legit logic, in my view, here would be to trigger autovacuum
if the number of offsets (i.e. difference nextOffset - oldestOffset)
exceeds 2^32-1. PFA patch set.
--
Best regards,
Maxim Orlov.
v5-0002-Use-64-bit-multixact-offsets.patch
Desc
must be
holding MultiXactGenLock in order to track oldestOffset to do
"nextOff - oldestOff" calculation.
>
> I'd love to see some tests for the pg_upgrade code. Something like a
> little perl script to generate test clusters with different wraparound
> scenarios etc.
Ag
lematic for example to clean out a crashed session's temp
> tables. See the "orphan temporary tables" logic in autovacuum.c.)
>
OK, now I've realized the problem. Thanks a lot!
--
Best regards,
Maxim Orlov.
s
here
and it's worth the effort to make a POC patch?
--
Best regards,
Maxim Orlov.
On Fri, 13 Sept 2024 at 09:11, Tatsuo Ishii wrote:
> Thanks. Attached is the v4 patch. I am going push it if there's no
> objection.
>
Looks good to me. Thank you for your work.
--
Best regards,
Maxim Orlov.
wraparound by switching to 64 bits.
[0] https://commitfest.postgresql.org/49/5205/
[1]
https://www.postgresql.org/message-id/flat/CACG%3DezaWg7_nt-8ey4aKv2w9LcuLthHknwCawmBgEeTnJrJTcw%40mail.gmail.com
--
Best regards,
Maxim Orlov.
add a
PG_USED_FOR_ASSERTS_ONLY
to solve the last issue.
Again, overall patch looks good and seems useful to me. Here is the rebased
v5 version based on Heikki's patch set above.
--
Best regards,
Maxim Orlov.
v5-0004-Move-TRACE-calls-into-WaitOnLock.patch
Description: Binary data
v5-0
Here is v3. I removed CATALOG_VERSION_NO change, so this should be done by
the actual commiter.
--
Best regards,
Maxim Orlov.
v3-0002-Use-64-bit-multixact-offsets.patch
Description: Binary data
v3-0001-Use-64-bit-format-output-for-multixact-offsets.patch
Description: Binary data
v3-0003
anation, I get it.
--
Best regards,
Maxim Orlov.
27;--enable-tap-tests' '--enable-depend' ....
meson:
$ ./pg_config --configure
--
Best regards,
Maxim Orlov.
; committer to advance CATALOG_VERSION_NO when needed.
>
OK, I got it. My intention here was to help to test the patch. If someone
wants to have a
look at the patch, he won't need to make changes in the code. In the next
iteration, I'll
remove CATALOG_VERSION_NO version change.
--
Best regards,
Maxim Orlov.
On Wed, 4 Sept 2024 at 03:07, Tatsuo Ishii wrote:
>
> Agreed. Probably add to explain.sql?
>
Yeah, I think this is an appropriate place.
--
Best regards,
Maxim Orlov.
as in the
mentioned 1eff8279d494b9.
--
Best regards,
Maxim Orlov.
Here is rebase. Apparently I'll have to do it often, since the
CATALOG_VERSION_NO changed in the patch.
--
Best regards,
Maxim Orlov.
v2-0001-Use-64-bit-format-output-for-multixact-offsets.patch
Description: Binary data
v2-0003-Make-pg_upgrade-convert-multixact-offsets.patch
Descri
uccessful.
To be honest, this step is not obvious. Especially than there was no such
step before. But
docs and https://wiki.postgresql.org/wiki/Meson are completely silenced
about it.
--
Best regards,
Maxim Orlov.
t here: installcheck-world doesn't
work on the current master branch until I explicitly install
injection_points extension. In my view, it's a bit wired, since
neither test_decoding, pg_stat_statements or pg_prewarm demand it.
--
Best regards,
Maxim Orlov.
code.
>
> I suspect this needs some additional verbiage about also installing
> src/test/modules/injection_points if you've enabled injection points.
>
> (I think we haven't noticed because most people just use "make check"
> instead.)
>
OK, many thanks for a comprehensive explanation!
--
Best regards,
Maxim Orlov.
TE EXTENSION
injection_points;' at
/home/omg/proj/build/../postgres/src/test/perl/PostgreSQL/Test/Cluster.pm
line 2140.
# Postmaster PID for node "master" is 820423
...
Cleary, Postgres can't find injection_points extension.
Am I doing something wrong, or it is a problem with injection points extension
itself?
--
Best regards,
Maxim Orlov.
retty sure this is just a beginning of the conversation, so any
opinions and reviews, as always, are very welcome!
--
Best regards,
Maxim Orlov.
From 2e1f05b3b0504153e57188e968bb19cb6741c087 Mon Sep 17 00:00:00 2001
From: Maxim Orlov
Date: Wed, 6 Mar 2024 11:11:33 +0300
Subject: [PATCH v1 2/3] U
speculation.
Does anyone know if there are reasons to deliberately ignore the HEAP_XMAX
INVALID flag? Or this is just an unfortunate oversight.
PFA, my approach on this issue.
--
Best regards,
Maxim Orlov.
v2-0001-Invalidate-xmax-if-HEAP_XMAX_INVALID-is-set.patch
Description: Binary data
t/ff143b24-a093-40da-9833-d36b83726bdf%40iki.fi#61d5a0e1cf6ab94b0e8aae8559bc4cf7
--
Best regards,
Maxim Orlov.
v1-0003-Switch-from-macro-functions-to-inline-in-multixac.patch
Description: Binary data
v1-0002-Fix-using-of-MultiXactOffset-type-insted-of-Trans.patch
Description: Binary
you for a reply! Yes, it seems to be that way. Many thanks for the
clarification.
--
Best regards,
Maxim Orlov.
ch_v1_abort' follow
/usr/bin/ld: src/interfaces/libpq/libpq.so.5.17.p/fe-auth-scram.c.o: in
function `scram_exchange':
...
many many many more
...
My OS info:
$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:Ubuntu 22.04.4 LTS
Release:22.04
Codename: jammy
Previously, I've got the same troubles on my old trusty 32-bit laptop on
Debian. But, there was no time to dig deeper in the problem
at the time. And now same for 64-bit Ubuntu. The most common reason for
such errors are not passing appropriate sanitizer
flags to LDFLAGS, but this is not the case. What could be the reason for
this? Am I doin' something wrong?
Exact the same sequence, but for GCC, works splendidly.
--
Best regards,
Maxim Orlov.
t find an exact link
for that discussion. On the other hand, such a casting is already used
throughout the code. So, just for the
sake of the consistency, I would like to stay with these casts.
On Tue, 23 Apr 2024 at 16:03, wenhui qiu wrote:
> Hi Maxim Orlov
>Thank you so much for you
.postgresql.org/message-id/flat/CACG%3DezY7msw%2Bjip%3Drtfvnfz051dRqz4s-diuO46v3rAoAE0T0g%40mail.gmail.com
[3]
https://postgr.es/m/CAJ7c6TPDOYBYrnCAeyndkBktO0WG2xSdYduTF0nxq%2BvfkmTF5Q%40mail.gmail.com
--
Best regards,
Maxim Orlov.
0001-WIP-mxidoff-to-64bit.patch
Description: Binary data
ct the reality.
--
Best regards,
Maxim Orlov.
v1-0002-Use-proper-types-in-defines-in-multixact.c.patch
Description: Binary data
I've noticed this patch and had a quick look at it. As far as I
understand, this bug
does not lead to an incorrect matching, resulting only in degradation in
speed.
Anyway, consider this patch useful, hope it will be committed soon.
--
Best regards,
Maxim Orlov.
r MultiXactOffset or MultiXactId.
As always, reviews and opinions are very welcome!
--
Best regards,
Maxim Orlov.
v1-0001-Refactor-pg_resetwal-t-001_basic.pl.patch
Description: Binary data
resource usage, but I don't think this increase should be significant.
--
Best regards,
Maxim Orlov.
?
Sorry for a late reply. Thanks for an explanation. This is sounds
reasonable to me.
Svetlana had addressed this in the patch v2.
--
Best regards,
Maxim Orlov.
not. Maybe I
miss something?
--
Best regards,
Maxim Orlov.
v4-0001-Invalidate-Buffer-By-Bufnum.patch
Description: Binary data
-> Index Scan using btg_x_y_idx on btg
-(6 rows)
+ -> Seq Scan on btg
+(5 rows)
... and so on.
So, my proposal is simple. I think we need not just "ANALYZE btg", but
"VACUUM ANALYZE btg", to get rid of zeroed pages in this particular
case. PFA corresponding
be a "chicken and the egg" situation. It is very hard to
split overall 64xid patch into smaller pieces
with each part been a meaningful and beneficial for current 32xids Postgres.
Anyway, thanks for reply, appreciate it.
--
Best regards,
Maxim Orlov.
://www.postgresql.org/message-id/flat/CAOBaU_YrnH_Jqo46NhaJ7uRBiWWEcS40VNRQxgFbqYo9kApUsg%40mail.gmail.com
--
Best regards,
Maxim Orlov.
v1-0001-Add-Index-level-REINDEX-with-multiple-jobs.patch
Description: Binary data
age-id/CACG=ezZe1NQSCnfHOr78AtAZxJZeCvxrts0ygrxYwe=pyyj...@mail.gmail.com
[1]
https://www.postgresql.org/message-id/flat/CAJ7c6TPDOYBYrnCAeyndkBktO0WG2xSdYduTF0nxq%2BvfkmTF5Q%40mail.gmail.com
--
Best regards,
Maxim Orlov.
v1-0002-Switch-to-FullTransactionId-for-XLogRecord-xl_xid.patch
Description: Binar
bug fixed by commit 74cf7d46.
>
I'm pretty much sure it was, but, unfortunately, there are no way to 100%
confirm
this. All I know, they're using PG13 now.
--
Best regards,
Maxim Orlov.
---++-+-+--
bar | 8192 | 16384 | 0 | 8192 | 16384 |
| |0
(1 row)
... and so on
--
Best regards,
Maxim Orlov.
0001-Add-warning-if-datfrozenxid-or-datminmxid-is-not-set.patch
Description: Binary data
segno = (int) strtol(clde->d_name, NULL, 16);
+ segno = strtoi64(clde->d_name, NULL, 16);
--
Best regards,
Maxim Orlov.
v61-0003-Make-use-FullTransactionId-in-2PC-filenames.patch
Description: Binary data
v61-0004-Add-SLRU-tests-for-64-bit-page-case.patch
Description: Binary data
v
It would be nice to
correct this in the next release.
--
Best regards,
Maxim Orlov.
bled. It's
weird.
AFICS, the only reason for this behaviour is becouse of transaction
wraparound. It may occur while the feature is disabled end it is safe to
simply remove all the data from previous period. If we switch to
FullTransactionId in commit_ts we can overcome this limitation. Bu
nks a lot!
https://commitfest.postgresql.org/45/4609/
--
Best regards,
Maxim Orlov.
Back to the patch v11. I don’t understand a bit, what we should do next?
Make a separate thread or put this one on commitfest?
--
Best regards,
Maxim Orlov.
m for WAL sync methods instead of defines.
--
Best regards,
Maxim Orlov.
v11-0001-Fix-conflicting-types-for-sync_method.patch
Description: Binary data
27;int' vs 'DataDirSyncMethod' (aka 'enum
DataDirSyncMethod')
extern PGDLLIMPORT int sync_method;
...
As a solution, I suggest renaming sync_method in xlog module to
wal_sync_method. In fact,
appropriate GUC for this variable, called "wal_sync_method" and I s
section there to have
an explicit error instead of the compiler
warnings if somehow we decide to add another one worker type?
So, should we mark this thread as RfC?
--
Best regards,
Maxim Orlov.
So, in my view, consider split these commands.
Hope, that helps. Again, I'm +1 for this patch.
--
Best regards,
Maxim Orlov.
one thing still waiting to be improved if foreach loop. It is
not very handy to have a bunch of
similar calls foreach, forboth, forthree and etc. It will be ideal to have
single foreach interface, but I don't know how
to do it without overall interface of the loop.
Any opinions are very wel
On Mon, 20 Mar 2023 at 22:31, Gregory Stark (as CFM)
wrote:
>
> So is that other thread tracked in a different commitfest entry and
> this one completely redundant? I'll mark it Rejected then?
>
Yep, it appears so.
--
Best regards,
Maxim Orlov.
I did a look at the patch, and it seems to be in a good condition.
It implements declared functionality with no visible defects.
As for test, I think it is possible to implement "signals" case in tap
tests. But let the actual commiter decide does it worth it or not.
--
Best regards,
Maxim Orlov.
Id in
PGPROC and patch become too
big to handle here.
So I decided to keep it simple for now and use wrap logic trick and calc
FullTransactionId on current epoch,
since the span of active xids cannot exceed one epoch at any given time.
Patches 1 and 2 are the same as above.
--
Best regards,
Maxim
MultiXactOffset from 32 to 64 bits, and eliminate pg_multixact/members
> wraparound, while keeping multi-xids 32 bits wide.
>
> Yes, you're totally correct. If it will be committable that way, I'm all
for that.
--
Best regards,
Maxim Orlov.
ithout changing segments
naming.
2. Use the larger segment file names in async.c, to lift the current 8 GB
limit on the max number of pending notifications.
3. Extend pg_xact to 64-bits.
4. Extend pg_subtrans to 64-bits.
5. Extend pg_multixact so that pg_multixact/members is addressed by 64-bit
offsets.
6.
$ ls pg_commit_ts/
00C2
Either I do not understand something, or the files from pg_commit_ts
directory are not copied.
--
Best regards,
Maxim Orlov.
ot;, exit_code_buf);
+ SetVariable(pset.vars, "SHELL_ERROR", "true");
After this changes, I think, we make this patch RfC, shall we?
--
Best regards,
Maxim Orlov.
offsets
> * pg_multixact/members
> * pg_subtrans
> * pg_notify
> * pg_serial
Hi! We do ignore these values, since in order to pg_upgrade the server it
must be properly stopped and no transactions can outlast this moment.
--
Best regards,
Maxim Orlov.
Unfortunately, there is a fail in FreeBSD
https://cirrus-ci.com/task/6466749487382528
Maybe, this patch is need to be rebased?
--
Best regards,
Maxim Orlov.
sed in conjunction with the
threshold_xid. We must use spinlock to sync.
b). When the threshold_timestamp is used without conjunction with the
threshold_xid. In this case, we use atomic values.
--
Best regards,
Maxim Orlov.
On Wed, 25 Jan 2023 at 16:52, Robert Haas wrote:
> On Wed, Jan 25, 2023 at 3:52 AM Maxim Orlov wrote:
>
> Well, that's something we - and ideally you, as the patch author -
> need to analyze and figure out. We can't just take a shot and hope for
> the best.
>
I than
e variable is using
conjointly. So, I'm not
sure, is it completely safe to remove mutex. Actually, removing mutex and
switch to atomics
was my first choice. I've run all the tests and no problems were found.
But, at that time I choose
to be more conservative. Anyway, here is the new v
ld_xid, but is this really a problem?
Thoughts?
--
Best regards,
Maxim Orlov.
0001-PGPRO-7624-use-atomic-old_snapshot_threshold.patch
Description: Binary data
On Fri, 13 Jan 2023 at 07:50, Corey Huinker wrote:
>
> I named it wait_result_to_exit_code(), but I welcome suggestions of a
> better name.
>
Thanks! But CF bot still not happy. I think, we should address issues from
here https://cirrus-ci.com/task/5391002618298368
--
Best re
[23:19:52.197] meson-generated_.._psqlscanslash.c.obj : error LNK2019:
unresolved external symbol WSTOPSIG referenced in function evaluate_backtick
[23:19:52.197] src\bin\psql\psql.exe : fatal error LNK1120: 2 unresolved
externals
I belive, we need proper includes.
--
Best regards,
Maxim Orlov.
haps, I didn't make myself clear. Your solution is perfectly adapted to
our needs.
But all Windows since 2000 already have an environment variable
OS=Windows_NT. So, if env OS is defined and equal Windows_NT, this had to
be Windows.
May we use it in our case? I don't insist, just asking.
--
Best regards,
Maxim Orlov.
uot;OS"? I do not know much about Windows, but I think
this is kind of standard environment variable there.
--
Best regards,
Maxim Orlov.
Hi!
Here is a new patch set.
I've added comments and make use GetClogDirName call in copy_subdir_files.
--
Best regards,
Maxim Orlov.
v52-0003-Make-pg_upgrade-from-32-bit-to-64-bit-SLRU.patch
Description: Binary data
v52-0002-Use-64-bit-pages-representation-in-SLRU-callers.patch
Descri
On Fri, 6 Jan 2023 at 09:51, Japin Li wrote:
>
> For v51-0003. We can use GetClogDirName instead of GET_MAJOR_VERSION in
> copy_subdir_files().
>
Of course! Tanks! I'll address this in the next iteration, v52.
--
Best regards,
Maxim Orlov.
increase of an amount of SLRU pages per segment.
And as for other SLRUs, they cannot survive pg_upgrade mostly by the fact,
that cluster must be stopped upon upgrade.
Thus, no conversion needed.
--
Best regards,
Maxim Orlov.
egards,
Maxim Orlov.
1 - 100 of 158 matches
Mail list logo
