Hi, On Thu, Oct 3, 2024 at 2:23 AM Fujii Masao <masao.fu...@oss.nttdata.com> wrote: > > Hi, > > Currently, file_fdw updates several columns in the pg_stat_progress_copy view, > like relid and bytes_processed, but it doesn't track tuples_processed or > tuples_skipped. Monitoring these would be particularly useful when handling > large data sets via file_fdw, as it helps track the progress of scan. > > The attached patch updates file_fdw to add support for reporting > the number of tuples processed and skipped (due to on_error = 'ignore') > in the pg_stat_progress_copy view. What are your thoughts?
While the patch works fine and looks good to me, in the first place, it seems to me that the fact that file_fdw uses the COPY progress itself doesn't work properly. For example, unlike COPY command, queries could have multiple scans on one or more flie_fdw foreign tables when joining tables. I found the discussion for that[1]: there was a proposal of disabling COPY progress for file_fdw but the votes are split. I think it would be better to consider if we really want to support COPY progress for file_fdw before supporting more progress information. [1] https://www.postgresql.org/message-id/flat/20230119054703.GB13860%40telsasoft.com Regards, -- Masahiko Sawada Amazon Web Services: https://aws.amazon.com
