select limit error in file_fdw

[Erik Rijkers]

Hello,

I have noticed that since ffa4cbd623, a foreign table that pulls data
from a PROGRAM (in this case an unzip call) will fail if there is a 
LIMIT on the SELECT
(while succeeding without LIMIT). Below is an example.
(Table size matters, so larger machines than mine may need more than 
those 100000 rows.)

A pre-ffa4cbd623 instance did not have this problem (and neither does 
11.1)

This seems like a bug to me.

--------------- 8< ------------------------------
#!/bin/bash

# service=prod
  service=dev11  # this is latest dev11 (in 11.1 it still worked;
                 # commit ffa4cbd623 looks pertinent)

csv_file=/tmp/t.txt

echo "select n from generate_series(1, 100000) as f(n)" | psql -qtAX 
service=$service > $csv_file

zip ~/t.zip $csv_file

echo "
drop   server if     exists test_server cascade;
create server if not exists test_server foreign data wrapper file_fdw;
create schema if not exists tmp;
drop   foreign table if exists tmp.t cascade;
create foreign table           tmp.t (n int)
server test_server
options (
    program       'unzip -p \"/home/aardvark/t.zip\" \"tmp/t.txt\"'
  , format 'csv'
  , header 'TRUE'
  , delimiter E'\t'
);
" | psql -X service=$service

# this works OK:
echo "table tmp.t;" | psql -Xa  service=$service | head

# this fails in latest dev11 :
echo "table tmp.t limit 10;" | psql -Xa service=$service
--------------- 8< ------------------------------

Output:

updating: tmp/t.txt (deflated 63%)
DROP SERVER
CREATE SERVER
CREATE SCHEMA
DROP FOREIGN TABLE
CREATE FOREIGN TABLE
table tmp.t;
   n
--------
      2
      3
      4
      5
      6
      7
      8
table tmp.t limit 10;
ERROR:  program "unzip -p "/home/aardvark/t.zip" "tmp/t.txt"" failed
DETAIL:  child process exited with exit code 141

it would be nice to get this working again.


Thanks,

Erik Rijkers