a-agmon opened a new pull request, #16387: URL: https://github.com/apache/datafusion/pull/16387
Partly closes #16303 The purpose of this PR is to enable using CREATE command with glob pattern and a URL scheme - i.e., ``` CREATE EXTERNAL TABLE ee3 STORED AS CSV LOCATION 's3://tests/data/file-*-1.csv'; CREATE EXTERNAL TABLE pp STORED AS PARQUET LOCATION 's3://tests/data-p/te*'; ``` Its currently possible to create an external table using this syntax just for local files: ``` CREATE EXTERNAL TABLE aa STORED AS CSV LOCATION '/Users/aa/projects/tmdb/tmdb_*.csv'; ``` Therefore, the purpose here is to enable support for glob support also for remote url scheme. The implementation involves some sort of workaround - it intercepts `create_plan()`, and when the table involves a glob pattern and remote scheme then it creates it as a ListingTable. Part of the reason for this approach is the fact that DataFusion core modules use ListingTable::parse() method in its core modules, which only takes a glob pattern when it invovles local files. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: github-unsubscr...@datafusion.apache.org For additional commands, e-mail: github-h...@datafusion.apache.org
