Re: Adding Support for Copy callback functionality on COPY TO api

2022-10-10 Thread Nathan Bossart
On Tue, Oct 11, 2022 at 11:52:03AM +0900, Michael Paquier wrote: > Okay. So, I have reviewed the whole thing, added a description of all > the fields of BeginCopyTo() in its top comment, tweaked a few things > and added in the module an extra NOTICE with the number of processed > rows. The result

Re: Adding Support for Copy callback functionality on COPY TO api

2022-10-10 Thread Michael Paquier
On Mon, Oct 10, 2022 at 05:06:39PM -0700, Nathan Bossart wrote: > Yeah, I named it that way because I figured we might want a test for the > COPY FROM callback someday. Okay. So, I have reviewed the whole thing, added a description of all the fields of BeginCopyTo() in its top comment, tweaked a

Re: Adding Support for Copy callback functionality on COPY TO api

2022-10-10 Thread Michael Paquier
On Wed, Sep 30, 2020 at 01:48:12PM +0500, Andrey V. Lepikhov wrote: > Your code almost exactly the same as proposed in [1] as part of 'Fast COPY > FROM' command. But it seems there are differences. > > [1] > https://www.postgresql.org/message-id/flat/3d0909dc-3691-a576-208a-90986e55489f%40postgre

Re: Adding Support for Copy callback functionality on COPY TO api

2022-10-10 Thread Nathan Bossart
On Tue, Oct 11, 2022 at 09:01:41AM +0900, Michael Paquier wrote: > I like these toy modules, they provide test coverage while acting as a > template for new developers. I am wondering whether it should have > something for the copy from callback, actually, as it is named > "test_copy_callbacks" bu

Re: Adding Support for Copy callback functionality on COPY TO api

2022-10-10 Thread Michael Paquier
On Mon, Oct 10, 2022 at 09:38:59AM -0700, Nathan Bossart wrote: > This new callback allows the use of COPY TO's machinery in extensions. A > couple of generic use-cases are listed upthread [0], and one concrete > use-case is the aws_s3 extension [1]. FWIW, I understand that the proposal is to hav

Re: Adding Support for Copy callback functionality on COPY TO api

2022-10-10 Thread Nathan Bossart
On Mon, Oct 10, 2022 at 12:41:40PM +0530, Bharath Rupireddy wrote: > IIUC, COPY TO callback helps move a table's data out of postgres > server. Just wondering, how is it different from existing solutions > like COPY TO ... PROGRAM/FILE, logical replication, pg_dump etc. that > can move a table's da

Re: Adding Support for Copy callback functionality on COPY TO api

2022-10-10 Thread Bharath Rupireddy
On Sun, Oct 9, 2022 at 2:44 AM Nathan Bossart wrote: > > Sorry for the noise. There was an extra #include in v4 that I've removed > in v5. IIUC, COPY TO callback helps move a table's data out of postgres server. Just wondering, how is it different from existing solutions like COPY TO ... PROGRAM

Re: Adding Support for Copy callback functionality on COPY TO api

2022-10-08 Thread Nathan Bossart
On Sat, Oct 08, 2022 at 10:37:41AM -0700, Nathan Bossart wrote: > Yeah, that makes more sense. It actually simplifies things a bit, too. Sorry for the noise. There was an extra #include in v4 that I've removed in v5. -- Nathan Bossart Amazon Web Services: https://aws.amazon.com >From 6e80d4113

Re: Adding Support for Copy callback functionality on COPY TO api

2022-10-08 Thread Nathan Bossart
On Sat, Oct 08, 2022 at 02:11:38PM +0900, Michael Paquier wrote: > Using an ereport(NOTICE) to show the data reported in the callback is > fine by me. How about making the module a bit more modular, by > passing as argument a regclass and building a list of arguments with > it? You may want to ho

Re: Adding Support for Copy callback functionality on COPY TO api

2022-10-07 Thread Michael Paquier
On Fri, Oct 07, 2022 at 02:48:24PM -0700, Nathan Bossart wrote: > Here is an attempt at adding such a test module. Using an ereport(NOTICE) to show the data reported in the callback is fine by me. How about making the module a bit more modular, by passing as argument a regclass and building a lis

Re: Adding Support for Copy callback functionality on COPY TO api

2022-10-07 Thread Nathan Bossart
On Fri, Oct 07, 2022 at 03:49:31PM +0900, Michael Paquier wrote: > Perhaps there should be a module in src/test/modules/ to provide a > short, still useful, example of what this could achieve? Here is an attempt at adding such a test module. -- Nathan Bossart Amazon Web Services: https://aws.ama

Re: Adding Support for Copy callback functionality on COPY TO api

2022-10-06 Thread Michael Paquier
On Tue, Aug 02, 2022 at 04:49:19PM -0700, Nathan Bossart wrote: > I've rebased this patch and will register it in the next commitfest > shortly. Perhaps there should be a module in src/test/modules/ to provide a short, still useful, example of what this could achieve? -- Michael signature.asc De

Re: Adding Support for Copy callback functionality on COPY TO api

2022-08-02 Thread Nathan Bossart
On Wed, Sep 30, 2020 at 04:41:51PM +0900, Michael Paquier wrote: > This feedback has not been answered after two weeks, so I have marked > the patch as returned with feedback. I've rebased this patch and will register it in the next commitfest shortly. -- Nathan Bossart Amazon Web Services: http

Re: Adding Support for Copy callback functionality on COPY TO api

2020-09-30 Thread Andrey V. Lepikhov
On 7/2/20 2:41 AM, Sanaba, Bilva wrote: Hi hackers, Currently, the COPY TO api does not support callback functions, while the COPY FROM api does. The COPY TO code does, however, include placeholders for supporting callbacks in the future. Rounding out the support of callback functions to bot

Re: Adding Support for Copy callback functionality on COPY TO api

2020-09-30 Thread Michael Paquier
On Mon, Sep 14, 2020 at 04:28:12PM -0700, Soumyadeep Chakraborty wrote: > I think a similar change should also be applied to BeginCopyFrom() and > CopyFrom(). Or even better, such code could be refactored to have a > separate destination type COPY_PIPE. This of course, will be a separate > patch. I

Re: Adding Support for Copy callback functionality on COPY TO api

2020-09-14 Thread Soumyadeep Chakraborty
Hi Bilva, Thank you for registering this patch! I had a few suggestions: 1. Please run pg_indent[1] on your code. Make sure you add copy_data_destination_cb to src/tools/pgindent/typedefs.list. Please run pg_indent on only the files you changed (it will take files as command line args) 2. For f