This is an automated email from the ASF dual-hosted git repository.
lidavidm pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/arrow-adbc.git
The following commit(s) were added to refs/heads/main by this push:
new 7130ab2 [Docs] Update CONTRIBUTING.md (#113)
7130ab2 is described below
commit 7130ab284b9c91e8024af21e5d2ac0fc6ec64434
Author: David Li <[email protected]>
AuthorDate: Thu Sep 8 11:18:06 2022 -0400
[Docs] Update CONTRIBUTING.md (#113)
---
CONTRIBUTING.md | 105 ++++++++++++++++++-----
c/{drivers/postgres => driver_manager}/README.md | 23 ++++-
c/drivers/flight_sql/README.md | 55 ++++++++++++
c/drivers/postgres/CMakeLists.txt | 2 +-
c/drivers/postgres/README.md | 38 +++++++-
c/drivers/{postgres => sqlite}/README.md | 17 +++-
ci/conda_env_cpp.txt | 2 +
python/adbc_driver_manager/README.md | 51 +++++++++++
python/adbc_driver_postgres/README.md | 61 +++++++++++++
python/adbc_driver_postgres/pyproject.toml | 2 +-
10 files changed, 319 insertions(+), 37 deletions(-)
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md
index 5a9daac..9c05ade 100644
--- a/CONTRIBUTING.md
+++ b/CONTRIBUTING.md
@@ -26,6 +26,9 @@ https://github.com/apache/arrow-adbc/issues
## Building
+In general: static checks, such as linting and formatting, are
+enforced via [pre-commit](https://pre-commit.com/).
+
### C/C++
The libraries here are all **individual** CMake projects.
@@ -44,15 +47,13 @@ $ cmake ../../c/driver_manager
$ make -j
```
-The SQLite3 and Apache Arrow Flight SQL drivers can be built
-similarly. Both drivers require an installation of the Arrow C++
-libraries (in the case of the Flight SQL driver, with Flight SQL
-enabled), and of course, the SQLite3 driver requires an installation
-of SQLite.
+All libraries here can be built similarly. For information on what
+they do and their dependencies, see their individual READMEs.
-To find dependencies, use CMake options such as `CMAKE_PREFIX_PATH`.
-A list of dependencies for Conda (conda-forge) is included, and can be
-used as follows:
+To specify where dependencies are to the build, use standard CMake
+options such as [`CMAKE_PREFIX_PATH`][cmake-prefix-path]. A list of
+dependencies for Conda (conda-forge) is included, and can be used as
+follows:
```shell
$ conda create -n adbc -c conda-forge --file ci/conda_env_cpp.txt
@@ -67,6 +68,7 @@ Some of Arrow's build options are supported (under a
different prefix):
Usually it is more convenient to explicitly set this to `OFF` for
development.
+All libraries use the same build options to enable tests.
For example, to build and run tests for the SQLite3 driver:
```shell
@@ -78,19 +80,80 @@ $ make -j
$ ctest
```
+Tests use [Googletest][gtest]. Some libraries may have additional
+test-time dependencies. For instance, the Postgres and Flight SQL
+drivers require servers to test against. See their individual READMEs
+for details.
+
+[cmake-prefix-path]:
https://cmake.org/cmake/help/latest/variable/CMAKE_PREFIX_PATH.html
+[gtest]: https://github.com/google/googletest/
+
+### GLib
+
+The GLib bindings use the [Meson][meson] build system.
+
+A build of the [driver manager](./c/driver_manager/README.md) is
+required. For example, if the libraries are installed to
+`$HOME/local`:
+
+```shell
+$ meson setup \
+ --pkg-config-path=$HOME/local/lib/pkgconfig \
+ --prefix=$HOME/local \
+ build/glib \
+ glib
+$ meson install -C build/glib
+```
+
+A list of dependencies for Conda (conda-forge) is included, and can be
+used as follows:
+
+```shell
+$ conda create -n adbc -c conda-forge --file ci/conda_env_glib.txt
+$ conda activate adbc
+```
+
+
+[meson]: https://mesonbuild.com/
+
+### Go
+
+Go libraries are a standard Go project.
+
+```shell
+$ cd go/adbc
+$ go build -v ./...
+$ go test -v ./...
+```
+
+### Java
+
+The Java components are a standard Maven project.
+
+```shell
+$ cd java/
+# Build and run tests
+$ mvn clean install
+```
+
### Python
-The Python driver manager is managed with [Poetry][poetry] and
-requires a C++ compiler. (The C++ sources for the driver manager are
-inlined into the Cython compilation process—so there is no need to
-separately build the driver manager.)
+Python libraries are managed with [Poetry][poetry]. See individual
+READMEs for additional dependencies. In general, that means all
+projects can be built as follows:
```shell
$ cd python
-$ pip install -r adbc_driver_manager/requirements-dev.txt
$ pip install -e adbc_driver_manager
```
+Or directly via poetry:
+
+```shell
+$ cd python/adbc_driver_manager
+$ poetry install
+```
+
When adding/updating dependencies, please regenerate the lockfile:
```shell
@@ -99,20 +162,16 @@ $ poetry update
$ poetry export --dev -o requirements-dev.txt
```
-To run tests, you will need a build of the SQLite3 driver above.
+Tests use [pytest][pytest]. Some libraries may have additional
+test-time dependencies. See their individual READMEs for details.
```shell
-$ export LD_LIBRARY_PATH=path/to/sqlite/driver/
-$ pytest -vvx adbc_driver_manager
+$ pytest -vvx
```
[poetry]: https://python-poetry.org
+[pytest]: https://docs.pytest.org/
-### Java
+### Ruby
-The Java components are a standard Maven project.
-
-```shell
-$ cd java/
-$ mvn clean install
-```
+The Ruby libraries are bindings around the GLib libraries.
diff --git a/c/drivers/postgres/README.md b/c/driver_manager/README.md
similarity index 52%
copy from c/drivers/postgres/README.md
copy to c/driver_manager/README.md
index 8138536..5b50557 100644
--- a/c/drivers/postgres/README.md
+++ b/c/driver_manager/README.md
@@ -17,9 +17,24 @@
under the License.
-->
-# libpq ADBC Driver
+# ADBC Driver Manager
-With credit to 0x0L's [pgeon](https://github.com/0x0L/pgeon) for the overall
approach.
+The driver manager provides a library that implements the ADBC API,
+but dynamically loads and manages the actual driver libraries
+underneath. Applications can use this to work with multiple drivers
+that would otherwise clash. Language-specific bindings can target
+this library to avoid having to manage individual libraries for every
+driver.
-This implements an ADBC driver that wraps
[libpq](https://www.postgresql.org/docs/14/libpq.html).
-This is still a work in progress.
+## Building
+
+Dependencies: none.
+
+See [CONTRIBUTING.md](../../CONTRIBUTING.md) for details.
+
+## Testing
+
+The test suite requires the SQLite driver to be available. On
+Linux/MacOS, you can build the driver, and then install it, or set
+`LD_LIBRARY_PATH`/`DYLD_LIBRARY_PATH` before running the test suite.
+Similarly, on Windows, you can set `PATH`.
diff --git a/c/drivers/flight_sql/README.md b/c/drivers/flight_sql/README.md
new file mode 100644
index 0000000..b1ac261
--- /dev/null
+++ b/c/drivers/flight_sql/README.md
@@ -0,0 +1,55 @@
+<!---
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+-->
+
+# ADBC Flight SQL Driver
+
+This implements an ADBC driver that wraps Arrow Flight SQL. This is
+still a work in progress.
+
+## Building
+
+Dependencies: Flight SQL itself. This can be installed with your
+favorite package manager, by installing the Arrow C++ libraries.
+
+See [CONTRIBUTING.md](../../CONTRIBUTING.md) for details.
+
+## Testing
+
+A running instance of the Flight SQL test server from the Arrow source
+tree is required. This means [building Arrow with
+tests](https://arrow.apache.org/docs/developers/cpp/building.html):
+
+```shell
+# Using a checkout of Arrow
+$ cd arrow/
+$ mkdir build && cd build
+$ cmake ../cpp -DARROW_FLIGHT=ON -DARROW_FLIGHT_SQL=ON -DARROW_BUILD_TESTS=ON
+$ cmake --build .
+$ ./debug/flight-sql-test-server
+Server listening on localhost:31337
+```
+
+Then, to run the tests, set the environment variable specifying the
+Flight location before running tests:
+
+```shell
+# From a build of the driver
+$ export ADBC_FLIGHT_SQL_LOCATION=grpc://localhost:31337
+$ ctest
+```
diff --git a/c/drivers/postgres/CMakeLists.txt
b/c/drivers/postgres/CMakeLists.txt
index 2d107bc..43bfa41 100644
--- a/c/drivers/postgres/CMakeLists.txt
+++ b/c/drivers/postgres/CMakeLists.txt
@@ -40,7 +40,7 @@ add_arrow_lib(adbc_driver_postgres
OUTPUTS
ADBC_LIBRARIES
SHARED_LINK_LIBS
- ${LIBPQ_LIBRARIES}
+ ${LIBPQ_LINK_LIBRARIES}
STATIC_LINK_LIBS
${LIBPQ_STATIC_LIBRARIES})
include_directories(SYSTEM ${REPOSITORY_ROOT})
diff --git a/c/drivers/postgres/README.md b/c/drivers/postgres/README.md
index 8138536..9e1c21e 100644
--- a/c/drivers/postgres/README.md
+++ b/c/drivers/postgres/README.md
@@ -17,9 +17,39 @@
under the License.
-->
-# libpq ADBC Driver
+# ADBC libpq Driver
-With credit to 0x0L's [pgeon](https://github.com/0x0L/pgeon) for the overall
approach.
+With credit to 0x0L's [pgeon](https://github.com/0x0L/pgeon) for the
+overall approach.
-This implements an ADBC driver that wraps
[libpq](https://www.postgresql.org/docs/14/libpq.html).
-This is still a work in progress.
+This implements an ADBC driver that wraps [libpq][libpq]. This is
+still a work in progress.
+
+[libpq]: https://www.postgresql.org/docs/14/libpq.html
+
+## Building
+
+Dependencies: libpq itself. This can be installed with your favorite
+package manager.
+
+See [CONTRIBUTING.md](../../CONTRIBUTING.md) for details.
+
+## Testing
+
+A running instance of Postgres is required. For example, using Docker:
+
+```shell
+$ docker run -it --rm \
+ -e POSTGRES_PASSWORD=password \
+ -e POSTGRES_DB=tempdb \
+ -p 5432:5432 \
+ postgres
+```
+
+Then, to run the tests, set the environment variable specifying the
+Postgres URI before running tests:
+
+```shell
+$ export
ADBC_POSTGRES_TEST_URI=postgres://localhost:5432/postgres?user=postgres&password=password
+$ ctest
+```
diff --git a/c/drivers/postgres/README.md b/c/drivers/sqlite/README.md
similarity index 71%
copy from c/drivers/postgres/README.md
copy to c/drivers/sqlite/README.md
index 8138536..f8aea1c 100644
--- a/c/drivers/postgres/README.md
+++ b/c/drivers/sqlite/README.md
@@ -17,9 +17,18 @@
under the License.
-->
-# libpq ADBC Driver
+# ADBC SQLite Driver
-With credit to 0x0L's [pgeon](https://github.com/0x0L/pgeon) for the overall
approach.
+This driver provides an interface to
+[SQLite](https://sqlite.org/index.html) using ADBC.
-This implements an ADBC driver that wraps
[libpq](https://www.postgresql.org/docs/14/libpq.html).
-This is still a work in progress.
+## Building
+
+Dependencies: SQLite itself. This can be installed with your favorite
+package manager.
+
+See [CONTRIBUTING.md](../../CONTRIBUTING.md) for details.
+
+## Testing
+
+No additional dependencies are required.
diff --git a/ci/conda_env_cpp.txt b/ci/conda_env_cpp.txt
index 1c5f906..cfdf3ec 100644
--- a/ci/conda_env_cpp.txt
+++ b/ci/conda_env_cpp.txt
@@ -20,4 +20,6 @@ cmake
compilers
gmock>=1.10.0
gtest>=1.10.0
+libpq
ninja
+libsqlite
diff --git a/python/adbc_driver_manager/README.md
b/python/adbc_driver_manager/README.md
new file mode 100644
index 0000000..10de050
--- /dev/null
+++ b/python/adbc_driver_manager/README.md
@@ -0,0 +1,51 @@
+<!---
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+-->
+
+# ADBC Driver Manager for Python
+
+This package contains bindings for the ADBC Driver Manager, as well as
+a [DBAPI 2.0/PEP 249-compatible][dbapi] interface on top. The DBAPI 2.0
+interface additionally requires PyArrow, and exposes a number of
+extensions mimicking those of [Turbodbc][turbodbc] or
+[DuckDB][duckdb]'s Python packages to allow you to retrieve Arrow
+Table objects instead of being limited to the row-oriented API of the
+base DBAPI interface.
+
+[dbapi]: https://peps.python.org/pep-0249/
+[duckdb]: https://duckdb.org/docs/api/python/overview
+[turbodbc]: https://turbodbc.readthedocs.io/en/latest/
+
+## Building
+
+Dependencies: a C++ compiler.
+
+See [CONTRIBUTING.md](../../CONTRIBUTING.md) for details.
+
+## Testing
+
+The [SQLite driver](../../c/drivers/sqlite/README.md) must be loadable
+at runtime (e.g. it must be on `LD_LIBRARY_PATH`, `DYLD_LIBRARY_PATH`,
+or `PATH`).
+
+See [CONTRIBUTING.md](../../CONTRIBUTING.md) for details.
+
+```shell
+$ export LD_LIBRARY_PATH=path/to/sqlite/driver/
+$ pytest -vvx
+```
diff --git a/python/adbc_driver_postgres/README.md
b/python/adbc_driver_postgres/README.md
new file mode 100644
index 0000000..17d6b4b
--- /dev/null
+++ b/python/adbc_driver_postgres/README.md
@@ -0,0 +1,61 @@
+<!---
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+-->
+
+# ADBC libpq Driver for Python
+
+This package contains bindings for the [libpq
+driver](../../c/drivers/postgres/README.md), using the [driver
+manager](../adbc_driver_manager/README.md) to provide a [DBAPI 2.0/PEP
+249-compatible][dbapi] interface on top.
+
+[dbapi]: https://peps.python.org/pep-0249/
+
+## Building
+
+Dependencies: a build of the libpq driver.
+
+Set the environment variable `ADBC_POSTGRES_LIBRARY` to the directory
+containing `libadbc_driver_postgres.so` (this library does not yet
+support Windows/MacOS) before running `poetry build`.
+
+See [CONTRIBUTING.md](../../CONTRIBUTING.md) for details on the
+general build process.
+
+## Testing
+
+A running instance of Postgres is required. For example, using Docker:
+
+```shell
+$ docker run -it --rm \
+ -e POSTGRES_PASSWORD=password \
+ -e POSTGRES_DB=tempdb \
+ -p 5432:5432 \
+ postgres
+```
+
+Then, to run the tests, set the environment variable specifying the
+Postgres URI before running tests:
+
+```shell
+$ export
ADBC_POSTGRES_TEST_URI=postgres://localhost:5432/postgres?user=postgres&password=password
+$ pytest -vvx
+```
+
+See [CONTRIBUTING.md](../../CONTRIBUTING.md) for details on the
+general test process.
diff --git a/python/adbc_driver_postgres/pyproject.toml
b/python/adbc_driver_postgres/pyproject.toml
index deb3184..4dd7a62 100644
--- a/python/adbc_driver_postgres/pyproject.toml
+++ b/python/adbc_driver_postgres/pyproject.toml
@@ -39,5 +39,5 @@ python = ">=3.8"
pytest = "^7.1.2"
[build-system]
-requires = ["poetry-core>=1.0.0"]
+requires = ["poetry-core>=1.0.0", "setuptools"]
build-backend = "poetry.core.masonry.api"
