Below are the minutes of the call.
Best,
Rok
Present:
Dewey Dunnington, Raul Cumplido, Will Jones, Jonathan Keane, Matt Topol,
Rok Mihevc, Ian Joiner
Agenda:
1. Minimal C++/C interface
2. Naming the C++ Compute Engine
3. Change to bucket creation behaviour in S3FileSystem
Notes:
1. Dewey is asking about the use of the C Data interface. How do you
normally use it? What helpers are needed?
Discussion:
- Interface should be easily accessible to Rust and Julia
implementations.
- We should start a discussion on the mailing list about where to put
it and how to use it to make sure current users are sufficiently pinged.
See mailing list thread [1].
- DuckDB community might also be interested in and/or have input.
- Bitpacking seems to appear in most implementations.
- Perhaps stabilizing the streaming interface might be a good idea at
this point.
2. Following ML discussion the C++ Compute Engine is now named Acero. Here
is the announcement issue: [2].
3. Currently S3FileSystem automatically creates or deletes buckets which is
risky in industry settings. Will proposes a change for the current
behaviour to be opt-in and error by default if a destructive action would
happen. Please discuss on the ML [3] or PR [4].
[1] https://lists.apache.org/thread/6vt7btpbrkm554otxqz7oxzdlpfwkhwc
https://lists.apache.org/thread/02sdm4jmqg2z98kr1mg2yq13q858xbx6
[2] https://issues.apache.org/jira/browse/ARROW-16632
[3] https://lists.apache.org/thread/8p9cy5ny9rtx7lwkhzjpn2rw2c146pxd
[4] https://github.com/apache/arrow/pull/13206
