Re: Querying one partition in a function takes locks on all partitions

2025-03-28 Thread Evgeny Morozov
On 23/03/2025 2:35 pm, David Rowley wrote: >> alter table entity_2 add column new_column text; > Is this just an example command? You can't add a column to a > partition directly. Yes, it was just the simplest way I could think of to take an exclusive lock. But on this note: I guess it takes a lo

Querying one partition in a function takes locks on all partitions

2025-03-21 Thread Evgeny Morozov
I have a list-partitioned table. When I query the base table but filter by the partition column in a regular SQL query this takes a lock only on the one partition being queried, as I expect. However, when the exact same SQL query is run fom a DB function, with the partition ID passed in as argument

Re: "PANIC: could not open critical system index 2662" - twice

2023-06-19 Thread Evgeny Morozov
There haven't been any updates posted to https://www.postgresql.org/message-id/20230509040203.z6mvijumv7wxcuib%40awork3.anarazel.de so I just wanted to check if there is any update on the status of the patch? Can we expect it in PostgreSQL 15.4? Thanks.

Re: "PANIC: could not open critical system index 2662" - twice

2023-05-17 Thread Evgeny Morozov
On 17/05/2023 1:39 am, Andres Freund wrote: > Try to prevent the DROP DATABASE from getting cancelled :/. I still don't know why that's happening. I mean, I know why it gets cancelled (the client timeout we set in Npgsql), but I don't know why the drop does not succeed within 30 seconds. We could,

Re: "PANIC: could not open critical system index 2662" - twice

2023-05-16 Thread Evgeny Morozov
On 9/05/2023 3:32 am, Andres Freund wrote: > Attached is a rough prototype of that idea (only using datconnlimit == > -2 for now). > I guess we need to move this to -hackers. Perhaps I'll post subsequent > versions below > https://www.postgresql.org/message-id/20230314174521.74jl6ffqsee5mtug%40awor

Re: "PANIC: could not open critical system index 2662" - twice

2023-05-10 Thread Evgeny Morozov
On 10/05/2023 6:39 am, Kirk Wolak wrote: > It could be as simple as creating temp tables in the other database > (since I believe pg_class was hit). We do indeed create temp tables, both in other databases and in the ones being tested. (We also create non-temp tables there.) > > Also, not sure if t

Re: "PANIC: could not open critical system index 2662" - twice

2023-05-09 Thread Evgeny Morozov
On 8/05/2023 11:04 pm, Andres Freund wrote: > Are you using any extensions? Only plpgsql. > Do you have any chance to figure out what statements were running > concurrently with the DROP DATABASE? No. Is there some way to log that, other than just logging all statements (which seems impractical)?

Re: "PANIC: could not open critical system index 2662" - twice

2023-05-08 Thread Evgeny Morozov
On 8/05/2023 9:47 pm, Andres Freund wrote: > Did you have any occasions where CREATE or DROP DATABASE was interrupted? > Either due the connection being terminated or a crash? I've uploaded an edited version of the PG log for the time as https://objective.realityexists.net/temp/log-extract-2023-05

Re: "PANIC: could not open critical system index 2662" - twice

2023-05-08 Thread Evgeny Morozov
On 8/05/2023 4:24 am, Michael Paquier wrote: > here are the four things running in parallel so as I can get a failure > in loading a critical index when connecting Wow, that is some amazing detective work! We do indeed create tables during our tests, specifically partitions of tables copied from t

Re: "PANIC: could not open critical system index 2662" - twice

2023-05-07 Thread Evgeny Morozov
On 6/05/2023 11:13 pm, Thomas Munro wrote: > Did you previously run this same workload on versions < 15 and never > see any problem? Yes, kind of. We have a test suite that creates one test DB and runs a bunch of tests on it. Two of these tests, however, create another DB each (also by cloning the

Re: "PANIC: could not open critical system index 2662" - twice

2023-05-06 Thread Evgeny Morozov
On 6/05/2023 12:34 pm, Thomas Munro wrote: > So it does indeed look like something unknown has replaced 32KB of > data with 32KB of zeroes underneath us. Are there more non-empty > files that are all-zeroes? Something like this might find them: > > for F in base/1414389/* > do > if [ -s $F ] &&

Re: "PANIC: could not open critical system index 2662" - twice

2023-05-06 Thread Evgeny Morozov
On 6/05/2023 1:06 am, Thomas Munro wrote: > Next can you share the file base/1414389/2662? ("5" was from the wrong > database.) Right - I should have realised that! base/1414389/2662 is indeed all nulls, 32KB of them. I included the file anyway in https://objective.realityexists.net/temp/pgstuff2

Re: "PANIC: could not open critical system index 2662" - twice

2023-05-05 Thread Evgeny Morozov
On 5/05/2023 10:38 am, Andrew Gierth wrote: > sudo -u postgres psql -w -p 5434 -d "options='-P'" > (make that -d "dbname=whatever options='-P'" if you need to specify > some database name; or use PGOPTIONS="-P" in the environment.) Thanks, good to know! Unfortunately that also fails: # sudo -u p

Re: "PANIC: could not open critical system index 2662" - twice

2023-05-05 Thread Evgeny Morozov
On 5/05/2023 2:02 am, Thomas Munro wrote: > On Fri, May 5, 2023 at 11:15 AM Thomas Munro wrote: >> What does select >> pg_relation_filepath('pg_class_oid_index') show in the corrupted >> database, base/5/2662 or something else? > Oh, you can't get that far, but perhaps you could share the > pg_fil

Re: "PANIC: could not open critical system index 2662" - twice

2023-05-04 Thread Evgeny Morozov
On 4/05/2023 6:42 pm, Laurenz Albe wrote: > On Thu, 2023-05-04 at 15:49 +0000, Evgeny Morozov wrote: >> Well, the problem happened again! Kind of... This time PG has not >> crashed with the PANIC error in the subject, but pg_dumping certain DBs >> again fails with &g

Re: "PANIC: could not open critical system index 2662" - twice

2023-05-04 Thread Evgeny Morozov
On 14/04/2023 10:42 am, Alban Hertroys wrote: > Your problem coincides with a thread at freebsd-current with very > similar data corruption after a recent OpenZFS import: blocks of all > zeroes, but also missing files. So, perhaps these problems are related? > Apparently, there was a recent fix for

Re: "PANIC: could not open critical system index 2662" - twice

2023-04-14 Thread Evgeny Morozov
> Hmm, I am not certain. The block was filled with zeros from your error > message, and I think such blocks don't trigger a checksum warning. OK, so data_checksums=on might not have made any difference in this case? > So if your disk replaces a valid block with zeros (filesystem check > after cr

Re: "PANIC: could not open critical system index 2662" - twice

2023-04-13 Thread Evgeny Morozov
On 13/04/2023 5:02 pm, Laurenz Albe wrote: > It means that if the error is caused by a faulty disk changing your data, > you'll notice as soon as you touch the page. > > That would perhaps not have made a lot of difference in your case, > except that the error message would have been different and

Re: "PANIC: could not open critical system index 2662" - twice

2023-04-12 Thread Evgeny Morozov
On 12/04/2023 2:35 am, Michael Paquier wrote: > initdb does not enable checksums by default, requiring a > -k/--data-checksums, so likely this addition comes from from your > environment. Indeed, turns out we had it in init_db_options. > However, the docs say "Only >> data pages are protected by

Re: "PANIC: could not open critical system index 2662" - twice

2023-04-11 Thread Evgeny Morozov
> No idea about the former, but bad hardware is a good enough explanation. > As to keeping it from happening: use good hardware. Alright, thanks, I'll just keep my fingers crossed that it doesn't happen again then! > Also: Use checksums. PostgreSQL offers data checksums[1]. Some filesystems also

"PANIC: could not open critical system index 2662" - twice

2023-04-06 Thread Evgeny Morozov
Our PostgreSQL 15.2 instance running on Ubuntu 18.04 has crashed with this error: 2023-04-05 09:24:03.448 UTC [15227] ERROR:  index "pg_class_oid_index" contains unexpected zero page at block 0 2023-04-05 09:24:03.448 UTC [15227] HINT:  Please REINDEX it. ... 2023-04-05 13:05:25.018 UTC [15437] ro