Hello Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/23627
to look at the new patch set (#9).
Change subject: IMPALA-14553: Run schema eval concurrently
......................................................................
IMPALA-14553: Run schema eval concurrently
The majority of time spent in generate-schema-statements.py is in
eval_section for schema operations that shell out, often uploading files
via the hadoop CLI or generating data files. These operations should be
independent.
Runs eval_section at the beginning so we don't repeat it for each row in
test_vectors, and executes them in parallel via a ThreadPool.
Also collects existing tables into a set to optimize lookup.
Speeds up functional-query devdata load by ~1 min - and slightly slows
down parallel TPC-H because functional-query starts sooner, resulting in
Loading TPC-H data OK (Took: 0 min 22 sec)
Loading functional-query data OK (Took: 1 min 22 sec)
Loading TPC-DS data OK (Took: 1 min 35 sec)
Change-Id: I2a78d05fd6a0005c83561978713237da2dde6af2
---
M testdata/bin/generate-schema-statements.py
1 file changed, 135 insertions(+), 74 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/27/23627/9
--
To view, visit http://gerrit.cloudera.org:8080/23627
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I2a78d05fd6a0005c83561978713237da2dde6af2
Gerrit-Change-Number: 23627
Gerrit-PatchSet: 9
Gerrit-Owner: Michael Smith <[email protected]>
Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
