Re: Increasing parallelism of queries while using file fdw and partitions

2018-12-18 Thread Patrick Mulrooney
Justin, Thanks for the idea. I pulled down the source for multicorn and added that to it. I do not see parallel queries in the analyze output (unless I force it and then it only gets one worker), but it does look like it is reading more than one file at once if I go with a non-partitioned table

Re: Increasing parallelism of queries while using file fdw and partitions

2018-12-18 Thread Justin Pryzby
On Tue, Dec 18, 2018 at 08:39:36PM -0800, Patrick Mulrooney wrote: > Wondering if anyone had any thoughts on how to tweak my setup to get it to > read many files at once instead of one at a time when using file fdw and > partitions. I found this: https://www.postgresql.org/docs/current/parallel-s

Increasing parallelism of queries while using file fdw and partitions

2018-12-18 Thread Patrick Mulrooney
Wondering if anyone had any thoughts on how to tweak my setup to get it to read many files at once instead of one at a time when using file fdw and partitions. We have a bunch of data tied up in files (each file > 4M rows, 5,000+ files per year) that I would like to be able to query directly us