Krisztian Szucs created ARROW-7965:
--------------------------------------
Summary: [Python] Hold a reference to the dataset factory for
later reuse
Key: ARROW-7965
URL: https://issues.apache.org/jira/browse/ARROW-7965
Project: Apache Arrow
Issue Type: Improvement
Components: Python
Reporter: Krisztian Szucs
Provide a more intuitive way to construct nested dataset:
```python
# instead of using confusing factory function
dataset([
factory("s3://old-taxi-data", format="parquet"),
factory("local/path/to/new/data", format="csv")
])
# let the user to construct a new dataset directly from dataset objects
dataset([
dataset("s3://old-taxi-data", format="parquet"),
dataset("local/path/to/new/data", format="csv")
])
```
In the future we might want to introduce a new Dataset class which wraps
functionality of both the dataset actory and the materialized dataset enabling
optimizations over rediscovery of already materialized datasets.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
