On Wed, Nov 22, 2023 at 2:28 PM Tom Lane <t...@sss.pgh.pa.us> wrote:
> "Efrain J. Berdecia" <ejberde...@yahoo.com> writes: > > Thanks, the issue we've run into, which I guess could be really a setup > issue, with running a COPY command while executing pg_restore, is that if > we are restoring a large table (bigger than 500GB) our WAL directory can > grow to be very large. > > I would think that if the pg_restore or COPY command was able to support > a batch-size option, this should allow postgres to either archive or remove > wal files and prevent having to re-size the WAL directory for a one time > refresh operation. > > I'm trying to gage how feasible would be to start looking at > contributing to add such a feature to either the COPY command or pg_restore. > > Given the shortage of other complaints, I tend to agree with Adrian > that there's not likely to be much interest in adding complexity > to pg_restore (or COPY) to address this. You should probably look > harder at the idea that you have some configuration problem that's > triggering your WAL bloat. If COPY can run you out of WAL space, > then so could any future bulk insert or update. > What OP needs, I think, since I'd use it, too, is "pg_bulkload without the intrusive hacks and restrictions".
