Thank you. I meant to try PQExpBuffer — as you say, it fits much better with existing libpq style. But then I hit on the callback idea which was even more efficient, by a fair margin. It was also considerably simpler both inside libpq and in the client code, eliminating all sorts of awkward questions about who deallocates the buffer in which situations. So I ditched the "dynamic buffer" concept and went with the callback.
One other possible alternative suggests itself: instead of taking a callback and a context pointer, the function could probably just return a struct: status/size, and buffer. Then the caller would have to figure out whether there's a line in the buffer, and if so, process it. It seems like more work for the client code, but it may make the compiler's optimisation work easier. Jeroen On Fri, 3 Mar 2023 at 16:52, Jelte Fennema <postg...@jeltef.nl> wrote: > On Thu, 2 Mar 2023 at 20:45, Jeroen Vermeulen <jtv...@gmail.com> wrote: > > I'm attaching a diff now. This is not a patch, it's just a discussion > piece. > > Did you try with PQExpBuffer? I still think that probably fits better > in the API design of libpq. Obviously if it's significantly slower > than the callback approach in this patch then it's worth considering > using the callback approach. Overall it definitely seems reasonable to > me to have an API that doesn't do an allocation per row. >
