Re: FileFallocate misbehaving on XFS

2025-03-10 Thread Michael Harris
On Tue, 4 Mar 2025 at 23:05, Jakub Wartak wrote: > out of curiosity, did Redhat provide any further (deeper) information > on why too big preallocation (or what heuristics) on XFS can cause > such issues? Hi Jakub So far I don't have any feedback from RH. Unfortunately there is a long corporate

Re: FileFallocate misbehaving on XFS

2025-01-30 Thread Michael Harris
Hello All An update on this. Earlier in this thread, Jakub had suggested remounting the XFS filesystems with the mount option allocsize=1m. I've done that now, on a few systems that have been experiencing this error multiple times a day, and it does seem to stop the errors from occurring. It has

Re: FileFallocate misbehaving on XFS

2025-01-03 Thread Michael Harris
Hi Andres On Wed, 1 Jan 2025 at 02:31, Andres Freund wrote: > Note that there's > a) a few hours between messages, whereas previous they were more frequent > b) f_bfree increased substantially. > > I assume that somewhere around 2AM some script prunes old partitions? Correct. Data is imported co

Re: FileFallocate misbehaving on XFS

2024-12-12 Thread Michael Harris
Hi Andres On Fri, 13 Dec 2024 at 08:38, Andres Freund wrote: > > Another interesting snippet: the application has a number of ETL > > workers going at once. The actual number varies depending on a number > > of factors but might be somewhere from 10 - 150. Each worker will have > > a single postg

Re: FileFallocate misbehaving on XFS

2024-12-11 Thread Michael Harris
Hi Andres On Thu, 12 Dec 2024 at 10:50, Andres Freund wrote: > Just to make sure - you're absolutely certain that you actually have space at > the time of the errors? As sure as I can be. The RHEL8 system that I took prints from yesterday has > 1.5TB free. I can't see it varying by that much. I

Re: FileFallocate misbehaving on XFS

2024-12-10 Thread Michael Harris
Hi again On Wed, 11 Dec 2024 at 12:09, Michael Harris wrote: > But another system I can access has multiple databases with ongoing > imports, yet all the errors bar one relate to one directory. > I will collect some data from that system and post it shortly. I've attached the sa

Re: FileFallocate misbehaving on XFS

2024-12-10 Thread Michael Harris
Hi Jakub On Tue, 10 Dec 2024 at 22:36, Jakub Wartak wrote: > Yay, reflink=0, that's pretty old fs ?! This particular filesystem was created on Centos 7, and retained when the system was upgraded to RL9. So yes probably pretty old! > Could you get us maybe those below commands too? (or from any

Re: FileFallocate misbehaving on XFS

2024-12-10 Thread Michael Harris
Hi Andres On Wed, 11 Dec 2024 at 03:09, Andres Freund wrote: > I think it's implied, but I just want to be sure: This was one of the affected > systems? Yes, correct. > Any chance to get df output? I'm mainly curious about the number of used > inodes. Sorry, I could swear I had included that a

Re: FileFallocate misbehaving on XFS

2024-12-09 Thread Michael Harris
Hi Andres Following up on the earlier question about OS upgrade paths - all the cases reported so far are either on RL8 (Kernel 4.18.0) or were upgraded to RL9 (kernel 5.14.0) and the affected filesystems were preserved. In fact the RL9 systems were initially built as Centos 7, and then when that

Re: FileFallocate misbehaving on XFS

2024-12-09 Thread Michael Harris
Tue, 10 Dec 2024 at 17:28, Michael Harris wrote: > > Hi Andres > > Following up on the earlier question about OS upgrade paths - all the > cases reported so far are either on RL8 (Kernel 4.18.0) or were > upgraded to RL9 (kernel 5.14.0) and the affected filesystems were > pre

Re: FileFallocate misbehaving on XFS

2024-12-09 Thread Michael Harris
Hi Tomas On Mon, 9 Dec 2024 at 21:06, Tomas Vondra wrote: > Sounds more like an XFS bug/behavior, so it's not clear to me what we > could do about it. I mean, if the filesystem reports bogus out-of-space, > is there even something we can do? I don't disagree that it's most likely an XFS issue. H

Re: FileFallocate misbehaving on XFS

2024-12-09 Thread Michael Harris
Hi Andres On Tue, 10 Dec 2024 at 03:31, Andres Freund wrote: > Were those pg_upgrades done with pg_upgrade --clone? Or have been, on the same > filesystem, in the past? No, our procedure is to use --link. > I found some references for bugs that were fixed in 5.13. But I think at least > some of

FileFallocate misbehaving on XFS

2024-12-09 Thread Michael Harris
Hello PG Hackers Our application has recently migrated to PG16, and we have experienced some failed upgrades. The upgrades are performed using pg_upgrade and have failed during the phase where the schema is restored into the new cluster, with the following error: pg_restore: error: could not exec

Re: ANALYZE ONLY

2024-09-22 Thread Michael Harris
On Sun, 22 Sept 2024 at 23:09, David Rowley wrote: > I'd like to push this soon, so if anyone has any last-minute feedback, > please let me know in the next few days. Many thanks for your updates and assistance. Looks good. Agreed, I was probably too conservative in some of the doc updates. Tha

Re: ANALYZE ONLY

2024-09-19 Thread Michael Harris
Thanks for the feedback, and sorry it has taken a few days to respond. On Mon, 16 Sept 2024 at 12:29, jian he wrote: > in https://www.postgresql.org/docs/current/ddl-inherit.html > <<< > Commands that do database maintenance and tuning (e.g., REINDEX, > VACUUM) typically only work on individual,

Re: ANALYZE ONLY

2024-09-11 Thread Michael Harris
Thanks for the feedback. On Tue, 10 Sept 2024 at 22:03, torikoshia wrote: > Regarding the addition of partition descendant tables, should we also > update the below comment on expand_vacuum_rel? Currently it refers only > partitions: > > | * Given a VacuumRelation, fill in the table OID if it wa

Re: ANALYZE ONLY

2024-09-09 Thread Michael Harris
Thanks for the feedback David. On Mon, 9 Sept 2024 at 11:27, David Rowley wrote: > You've written "was" (past tense), but then the existing text uses > "will" (future tense). I guess if the point in time is after parse and > before work has been done, then that's correct, but I think using "is" >

Re: ANALYZE ONLY

2024-08-31 Thread Michael Harris
fields - for 'reviewer' should I have put the people who have provided feedback on this thread? Is there anything else I need to do? Thanks again Cheers Mike On Fri, 30 Aug 2024 at 18:44, David Rowley wrote: > > Hi Michael, > > On Fri, 23 Aug 2024 at 22:01, Michael Harris

Re: ANALYZE ONLY

2024-08-31 Thread Michael Harris
Hello Atsushi & Melih Thank you both for your further feedback. On Thu, 29 Aug 2024 at 21:31, Melih Mutlu wrote: > I believe moving "[ ONLY ]" to the definitions of table_and_columns below, as > you did with "[ * ]", might be better to be consistent with other places (see > [1]) Agreed. I hav

Re: ANALYZE ONLY

2024-08-23 Thread Michael Harris
Thanks for the feedback Melih, On Thu, 22 Aug 2024 at 21:26, Melih Mutlu wrote: > It seems like extended_relation_expr allows "tablename *" syntax too. That > should be added in docs as well. (Same for VACUUM doc) I had included it in the parameter description but had missed it from the synopsi

Re: ANALYZE ONLY

2024-08-22 Thread Michael Harris
22 Aug 2024 at 13:38, Michael Harris wrote: > > Would we want to apply that change to VACUUM too? That might be a > > bit drastic, especially if you had a multi-level inheritance structure > > featuring > > large tables. > > I think they'd need to work the same

Re: ANALYZE ONLY

2024-08-21 Thread Michael Harris
Tom Lane wrote: > Yeah, my thought was to change the behavior so it's consistent in > that case too. This doesn't seem to me like a change that'd > really cause anybody serious problems: ANALYZE without ONLY would > do more than before, but it's work you probably want done anyway. Would we want

Re: ANALYZE ONLY

2024-08-21 Thread Michael Harris
Thank you all for the replies & discussion. It sounds like more are in favour of using the ONLY syntax attached to the tables is the best way to go, with the main advantages being: - consistency with other commands - flexibility in allowing to specify whether to include partitions for individual

ANALYZE ONLY

2024-08-19 Thread Michael Harris
Hello Postgres Hackers An application that I am developing uses Postgresql, and includes a fairly large number of partitioned tables which are used to store time series data. The tables are partitioned by time, and typically there is only one partition at a time - the current partition - that is