On Wed, Apr 8, 2015 at 11:53 AM, Robert Haas <robertmh...@gmail.com> wrote: > On Mon, Apr 6, 2015 at 1:51 PM, Jim Nasby <jim.na...@bluetreble.com> wrote: >> In any case, I don't think it would be terribly difficult to allow a bit >> more than 1GB in a StringInfo. Might need to tweak palloc too; ISTR there's >> some 1GB limits there too. > > The point is, those limits are there on purpose. Changing things > arbitrarily wouldn't be hard, but doing it in a principled way is > likely to require some thought. For example, in the COPY OUT case, > presumably what's happening is that we palloc a chunk for each > individual datum, and then palloc a buffer for the whole row. Now, we > could let the whole-row buffer be bigger, but maybe it would be better > not to copy all of the (possibly very large) values for the individual > columns over into a row buffer before sending it. Some refactoring > that avoids the need for a potentially massive (1.6TB?) whole-row > buffer would be better than just deciding to allow it.
I think that something to be aware of is that this is as well going to require some rethinking of the existing libpq functions that are here to fetch a row during COPY with PQgetCopyData, to make them able to fetch chunks of data from one row. -- Michael -- Sent via pgsql-hackers mailing list (pgsql-hackers@postgresql.org) To make changes to your subscription: http://www.postgresql.org/mailpref/pgsql-hackers
