prantogg opened a new pull request, #57:
URL: https://github.com/apache/sedona-spatialbench/pull/57
This PR introduces the following changes:
1. Running a command like this will automatically create 10 files for each
large table (in parallel)
```bash
spatialbench-cli -s 100 --parts=10
```
2. When `--parts` option is specified, each table is written out to
output_dir/table/table.part.extension
```
/sf1-parquet/trip/trip.1.parquet
/sf1-parquet/trip/trip.2.parquet
```
3. If the target output file already exists, then it will not be
regenerated. This allows you to kill an instance of tpchgen-cli and then run
the same command to pick back up where it left off
4. Atomic file generation - spatialbench-cli writes to a file named
".inprogress" and when complete renames the file
5. Refactors Zone table generation code for better readability
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
