On Thu, Feb 20, 2020 at 04:11:39PM +0530, Amit Kapila wrote:
On Thu, Feb 20, 2020 at 5:12 AM David Fetter <da...@fetter.org> wrote:
On Fri, Feb 14, 2020 at 01:41:54PM +0530, Amit Kapila wrote:
> This work is to parallelize the copy command and in particular "Copy
> <table_name> from 'filename' Where <condition>;" command.
Apropos of the initial parsing issue generally, there's an interesting
approach taken here: https://github.com/robertdavidgraham/wc2
Thanks for sharing. I might be missing something, but I can't figure
out how this can help here. Does this in some way help to allow
multiple workers to read and tokenize the chunks?
I think the wc2 is showing that maybe instead of parallelizing the
parsing, we might instead try using a different tokenizer/parser and
make the implementation more efficient instead of just throwing more
CPUs on it.
I don't know if our code is similar to what wc does, maytbe parsing
csv is more complicated than what wc does.
regards
--
Tomas Vondra http://www.2ndQuadrant.com
PostgreSQL Development, 24x7 Support, Remote DBA, Training & Services
