This is an automated email from the ASF dual-hosted git repository. jiayu pushed a commit to branch branch-0.2.0 in repository https://gitbox.apache.org/repos/asf/sedona-db.git
commit 94b3ba53265e28118035c5f74bacb478a35e4671 Author: Dewey Dunnington <[email protected]> AuthorDate: Fri Dec 12 10:07:28 2025 -0600 chore(docs/reference/sql): Use Quarto to render SQL docs pages (#434) Co-authored-by: Copilot <[email protected]> --- .github/workflows/packaging.yml | 4 +- .gitignore | 2 + ci/scripts/build-docs.sh | 11 ++ docs/README.md | 17 ++ docs/contributors-guide.md | 70 +++++++ .../functions/.gitignore} | 15 +- .../_extensions/function-listing/_extension.yml} | 19 +- .../function-listing/function-listing.lua | 43 +++++ .../_extensions/render-meta-and-examples.lua | 137 +++++++++++++ .../functions/_matplotlib_defaults.py} | 17 +- docs/reference/functions/_quarto.yml | 46 +++++ docs/reference/functions/_render_examples.py | 120 ++++++++++++ docs/reference/functions/_render_listing.py | 83 ++++++++ docs/reference/functions/_render_meta.py | 213 +++++++++++++++++++++ .../functions/index.qmd} | 17 +- docs/reference/functions/st_analyze_agg.qmd | 49 +++++ docs/reference/functions/st_buffer.qmd | 81 ++++++++ docs/reference/functions/st_intersection.qmd | 54 ++++++ docs/requirements.txt | 1 + 19 files changed, 950 insertions(+), 49 deletions(-) diff --git a/.github/workflows/packaging.yml b/.github/workflows/packaging.yml index 978e2e0f..e9837fe8 100644 --- a/.github/workflows/packaging.yml +++ b/.github/workflows/packaging.yml @@ -98,6 +98,8 @@ jobs: with: python-version: "3.x" + - uses: quarto-dev/quarto-actions/setup@v2 + - name: Use stable Rust id: rust run: | @@ -119,7 +121,7 @@ jobs: - name: Install dev SedonaDB Python run: | - pip install python/sedonadb/ -vv + pip install "python/sedonadb/[geopandas]" -v - name: Build documentation run: | diff --git a/.gitignore b/.gitignore index 6a4f4a03..232ccf0f 100644 --- a/.gitignore +++ b/.gitignore @@ -49,3 +49,5 @@ __pycache__ # .env file for release management dev/release/.env + +/.luarc.json diff --git a/ci/scripts/build-docs.sh b/ci/scripts/build-docs.sh index cb127077..f6d333d2 100755 --- a/ci/scripts/build-docs.sh +++ b/ci/scripts/build-docs.sh @@ -33,6 +33,17 @@ for notebook in $(find "${SEDONADB_DIR}/docs" -name "*.ipynb"); do jupyter nbconvert --to markdown "${notebook}" done +# Clean + build SQL function documentation +pushd "${SEDONADB_DIR}/docs/reference/functions" + +# Remove built markdown files (they confuse quarto) +find . -name "*.md" -delete + +# Render the Quarto project +quarto render + +popd + pushd "${SEDONADB_DIR}" if mkdocs build --strict ; then echo "Success!" diff --git a/docs/README.md b/docs/README.md index 7a7cb81b..476af347 100644 --- a/docs/README.md +++ b/docs/README.md @@ -30,6 +30,23 @@ pip install -e "python/sedonadb/[test]" -vv pip install -r docs/requirements.txt ``` +The SQL function documentation is a [Quarto](https://quarto.org) project that must be rendered +at least once to generate the Markdown files required by mkdocs. This may be done with: + +```shell +cd docs/reference/functions +quarto render +``` + +When iterating on documentation, it is usually best to use the `mkdocs` commands directly: + * `mkdocs serve` - Start the live-reloading docs server. * `mkdocs build` - Build the documentation site. * `mkdocs -h` - Print help message and exit. + +The official documentation is built using a script which may be useful when building the documentation +locally for the first time: + +```shell +ci/scripts/build-docs.sh +``` diff --git a/docs/contributors-guide.md b/docs/contributors-guide.md index a3eb79a5..e81f4b95 100644 --- a/docs/contributors-guide.md +++ b/docs/contributors-guide.md @@ -336,3 +336,73 @@ To contribute to the SedonaDB documentation: * `mkdocs build` - Build the documentation site. * `mkdocs -h` - Print help message and exit. 1. Push your changes and open a pull request. + +SQL function reference is special: because we provide so many functions, we have +a specialized syntax for documenting them. The minimum required documentation for +a function is a file `docs/reference/functions/function_name.qmd`: + + --- + title: ST_FunctionName + description: A brief one sentence description of what the function does. + kernels: + - returns: geometry + args: [geometry] + --- + + ## Examples + + ```sql + SELECT ST_FunctionName(ST_Point(0, 1)) AS val; + ``` + +After writing this file, the `.md` file may be rendered using [Quarto](https://quarto.org): + +```shell +cd docs/reference/functions +quarto render +``` + +This command (1) expands `description` and `kernels` to a templated representation, +(2) checks and renders the result of the SQL examples, and (3) executes any +[Python code chunks](https://quarto.org/docs/computations/python.html). These may +be used to render figures that demonstrate visually what a function does or how its +parameters affect the result. + +The `kernels` section of the frontmatter allows multiple implementations of a function +to be documented. For example, many functions include implementations for geometry +*and* geography or allow extra arguments to be supplied to customize behaviour. As +an example, the frontmatter for `ST_Buffer()` is: + + --- + title: ST_Buffer + description: > + Computes a geometry that represents all points whose distance from the input + geometry is less than or equal to a specified distance. + kernels: + - returns: geometry + args: + - geometry + - name: distance + type: float64 + description: Radius of the buffer + - returns: geometry + args: + - geometry + - name: distance + type: float64 + - name: params + type: utf8 + description: Space-separated `key=value` parameters. + --- + +This illustrates a few ways in which arguments can be defined: + +- By the string `geometry`, `geography`, or `raster`. These are expanded to a full + definition by quarto but are so common that we allow abbreviating them to avoid + typing `description: Input geometry` for every single function. +- With a YAML object of `name` / `type` / `description`. The type names are lowercase + Arrow type names which should be identical to those printed when executing a query + in SedonaDB. + +The build system for function documentation is a work in progress, so be sure to ask +if you run into problems or have any questions about the syntax! diff --git a/docs/requirements.txt b/docs/reference/functions/.gitignore similarity index 82% copy from docs/requirements.txt copy to docs/reference/functions/.gitignore index 6bd80c39..d835f3f1 100644 --- a/docs/requirements.txt +++ b/docs/reference/functions/.gitignore @@ -15,15 +15,6 @@ # specific language governing permissions and limitations # under the License. -griffe -mike -mkdocs -mkdocs-git-revision-date-localized-plugin -mkdocs-glightbox -mkdocs-macros-plugin -mkdocs-material -mkdocstrings[python] -nbconvert -pyproj -ruff -lonboard +/.quarto/ +*.md +*_files/ diff --git a/docs/requirements.txt b/docs/reference/functions/_extensions/function-listing/_extension.yml similarity index 82% copy from docs/requirements.txt copy to docs/reference/functions/_extensions/function-listing/_extension.yml index 6bd80c39..a218f5f7 100644 --- a/docs/requirements.txt +++ b/docs/reference/functions/_extensions/function-listing/_extension.yml @@ -15,15 +15,10 @@ # specific language governing permissions and limitations # under the License. -griffe -mike -mkdocs -mkdocs-git-revision-date-localized-plugin -mkdocs-glightbox -mkdocs-macros-plugin -mkdocs-material -mkdocstrings[python] -nbconvert -pyproj -ruff -lonboard +title: Function Listing +author: SedonaDB +version: 1.0.0 +quarto-required: ">=1.4.0" +contributes: + shortcodes: + - function-listing.lua diff --git a/docs/reference/functions/_extensions/function-listing/function-listing.lua b/docs/reference/functions/_extensions/function-listing/function-listing.lua new file mode 100644 index 00000000..460d1570 --- /dev/null +++ b/docs/reference/functions/_extensions/function-listing/function-listing.lua @@ -0,0 +1,43 @@ +-- Licensed to the Apache Software Foundation (ASF) under one +-- or more contributor license agreements. See the NOTICE file +-- distributed with this work for additional information +-- regarding copyright ownership. The ASF licenses this file +-- to you under the Apache License, Version 2.0 (the +-- "License"); you may not use this file except in compliance +-- with the License. You may obtain a copy of the License at +-- +-- http://www.apache.org/licenses/LICENSE-2.0 +-- +-- Unless required by applicable law or agreed to in writing, +-- software distributed under the License is distributed on an +-- "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +-- KIND, either express or implied. See the License for the +-- specific language governing permissions and limitations +-- under the License. + +return { + ["function-listing"] = function(args, kwargs, meta) + -- Get the file pattern from arguments (default to "st_*.qmd") + local file_pattern = args[1] or "st_*.qmd" + + -- Execute _render_listing.py with the file pattern + local cmd = string.format("python3 _render_listing.py '%s'", file_pattern) + local handle = io.popen(cmd, "r") + + if not handle then + return pandoc.Para{pandoc.Str("Error: Could not execute _render_listing.py")} + end + + local result = handle:read("*all") + handle:close() + + -- Parse the markdown result and return as pandoc blocks + if result and result ~= "" then + local doc_result = pandoc.read(result, "markdown") + return doc_result.blocks + end + + -- Fallback if no result + return pandoc.Para{pandoc.Str("No functions found matching pattern: " .. file_pattern)} + end +} diff --git a/docs/reference/functions/_extensions/render-meta-and-examples.lua b/docs/reference/functions/_extensions/render-meta-and-examples.lua new file mode 100644 index 00000000..51e1781f --- /dev/null +++ b/docs/reference/functions/_extensions/render-meta-and-examples.lua @@ -0,0 +1,137 @@ +-- Licensed to the Apache Software Foundation (ASF) under one +-- or more contributor license agreements. See the NOTICE file +-- distributed with this work for additional information +-- regarding copyright ownership. The ASF licenses this file +-- to you under the Apache License, Version 2.0 (the +-- "License"); you may not use this file except in compliance +-- with the License. You may obtain a copy of the License at +-- +-- http://www.apache.org/licenses/LICENSE-2.0 +-- +-- Unless required by applicable law or agreed to in writing, +-- software distributed under the License is distributed on an +-- "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +-- KIND, either express or implied. See the License for the +-- specific language governing permissions and limitations +-- under the License. + +-- auto-description.lua +-- Automatically adds a Description section based on frontmatter using _render_meta.py +-- Also processes SQL code blocks using _render_examples.py + +local function render_sql_example(sql_code) + -- Create a temporary file for the SQL + local temp_file = os.tmpname() + + -- Write SQL content to temporary file + local temp_handle = io.open(temp_file, "w") + if not temp_handle then + return {pandoc.CodeBlock(sql_code, pandoc.Attr("", {"sql"}, {}))} + end + + temp_handle:write(sql_code) + temp_handle:close() + + -- Execute _render_examples.py using stdin (with - argument) + local cmd = string.format("python3 _render_examples.py - < '%s'", temp_file) + local handle = io.popen(cmd, "r") + if not handle then + os.remove(temp_file) + return {pandoc.CodeBlock(sql_code, pandoc.Attr("", {"sql"}, {}))} + end + + local result = handle:read("*all") + handle:close() + os.remove(temp_file) + + -- Parse the markdown result and return as pandoc blocks + if result and result ~= "" then + local doc_result = pandoc.read(result, "markdown") + return doc_result.blocks + end + + -- Fallback to original code block if rendering fails + return {pandoc.CodeBlock(sql_code, pandoc.Attr("", {"sql"}, {}))} +end + +local function render_meta_content(doc) + -- Use Pandoc's JSON encoding (may not work perfectly) + local json_content = pandoc.json.encode(doc.meta) + + -- Create a temporary file for the JSON + local temp_file = os.tmpname() + + -- Write JSON content to temporary file + local temp_handle = io.open(temp_file, "w") + if not temp_handle then + return pandoc.Para{pandoc.Str("Error: Could not create temporary file")} + end + + temp_handle:write(json_content) + temp_handle:close() + + -- Pass JSON directly to _render_meta.py (JSON is valid YAML) + local cmd = string.format("python3 _render_meta.py - < '%s'", temp_file) + + local handle = io.popen(cmd, "r") + if not handle then + os.remove(temp_file) + return pandoc.Para{pandoc.Str("Error: Could not execute _render_meta.py")} + end + + local result = handle:read("*all") + if not result then + result = "Error: Could not read output from _render_meta.py" + end + + handle:close() + os.remove(temp_file) -- Parse the markdown result and return as pandoc blocks + if result and result ~= "" then + local doc_result = pandoc.read(result, "markdown") + return doc_result.blocks + end + + return {} +end + +function Pandoc(doc) + local description = doc.meta.description + + if description and description ~= "" then + -- First pass: Process all existing SQL code blocks before generating new content + local new_blocks = {} + for i, block in ipairs(doc.blocks) do + if block.t == "CodeBlock" and block.classes and #block.classes > 0 and block.classes[1] == "sql" then + local rendered_blocks = render_sql_example(block.text) + -- Add all rendered blocks to new_blocks + for _, rendered_block in ipairs(rendered_blocks) do + table.insert(new_blocks, rendered_block) + end + else + table.insert(new_blocks, block) + end + end + doc.blocks = new_blocks + + -- Second pass: Generate content using _render_meta.py + local meta_blocks = render_meta_content(doc) + + if #meta_blocks > 0 then + -- Insert after title (first header) if it exists, otherwise at the beginning + local insert_pos = 1 + for i, block in ipairs(doc.blocks) do + if block.t == "Header" and block.level == 1 then + insert_pos = i + 1 + break + end + end + + -- Insert the generated blocks + for j, block in ipairs(meta_blocks) do + table.insert(doc.blocks, insert_pos + j - 1, block) + end + end + end + + return doc +end diff --git a/docs/requirements.txt b/docs/reference/functions/_matplotlib_defaults.py similarity index 82% copy from docs/requirements.txt copy to docs/reference/functions/_matplotlib_defaults.py index 6bd80c39..4baaa176 100644 --- a/docs/requirements.txt +++ b/docs/reference/functions/_matplotlib_defaults.py @@ -15,15 +15,8 @@ # specific language governing permissions and limitations # under the License. -griffe -mike -mkdocs -mkdocs-git-revision-date-localized-plugin -mkdocs-glightbox -mkdocs-macros-plugin -mkdocs-material -mkdocstrings[python] -nbconvert -pyproj -ruff -lonboard +import matplotlib.pyplot as plt + +plt.rcParams["axes.labelsize"] = 8 +plt.rcParams["xtick.labelsize"] = 8 +plt.rcParams["ytick.labelsize"] = 8 diff --git a/docs/reference/functions/_quarto.yml b/docs/reference/functions/_quarto.yml new file mode 100644 index 00000000..331ca253 --- /dev/null +++ b/docs/reference/functions/_quarto.yml @@ -0,0 +1,46 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +# This quarto project allows `quarto render` from this directory to be +# used such that it takes care of executing code blocks to create figures +# and render standardized frontmatter defining function arguments into +# usage/argument documentation. +project: + title: "Function Reference" +format: + gfm: + # Increase the quality of the figures (the default of 72 is not very + # appealing) + fig-dpi: 300 + +# Use jupyter to render python code blocks. We could also use knitr, which +# is more flexible but requires an R install available. +engine: jupyter + +# By default, don't echo the Python code for Python code blocks. We use these +# blocks primarily to create figures, not to demo Python code and its output. +# individual code blocks can add `#| echo: true` to override this default. +execute: + echo: false + +# These filters take care of rendering ```sql code blocks, rendering the +# standardized description/usage/arguments based on document frontmatter, +# and providing the function-listing shortcode used to list all the functions +# in index.qmd. +filters: + - _extensions/render-meta-and-examples.lua + - _extensions/function-listing diff --git a/docs/reference/functions/_render_examples.py b/docs/reference/functions/_render_examples.py new file mode 100644 index 00000000..1bf1f963 --- /dev/null +++ b/docs/reference/functions/_render_examples.py @@ -0,0 +1,120 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +import sedonadb + +sd = sedonadb.connect() + + +def render_examples(examples, width=80, ascii=False): + """Renders examples to stdout using SedonaDB for Python + + This takes an iterable of example SQL strings and renders the SQL + and the result to stdout. All examples are run in the same + context. + + Output: + + ```` + + ```sql + --- examples that don't return a result are accumulated + SET some.config = true; + --- when a query does return a result, we end the sql code block + --- and print the result + SELECT one as 1; + ``` + + ``` + <output> + ``` + + ```` + """ + try: + examples_iter = iter(examples) + while True: + render_examples_iter_until_result(examples_iter, width=width, ascii=ascii) + except StopIteration: + pass + + +def render_examples_iter_until_result(examples_iter, width=80, ascii=False): + example = next(examples_iter) + + # Open the block where the SQL is printed + print("\n```sql") + while example is not None: + # Execute the example to get a row count. If this is a resultless + # statement (no rows, no cols), don't execute it again to print. This + # allows SET and CREATE TABLE statements to "set up" examples. We + # could also look for a trailing semicolon (e.g., only print results + # of statements without a trailing semicolon) if this approach is + # problematic. + + # Echo the example + print(example.strip()) + + # Parse it + df = sd.sql(example) + + # Execute and check emptiness + if df.execute() == 0 and not df.schema.names: + example = next(examples_iter, None) + continue + + # Close the ```sql block + print("```\n") + + # Print the result block (executes the query again) + print("```") + df.show(limit=None, width=width, ascii=ascii) + print("```") + return + + # If we're here, none of the statements had any output, so we need to close + # the sql block + print("```\n") + + +if __name__ == "__main__": + import argparse + import sys + + parser = argparse.ArgumentParser(description="Render SedonaDB SQL examples") + parser.add_argument( + "examples", + nargs="+", + help=( + "SQL strings to be rendered or `-` to read from stdin. " + "When reading from stdin, multiple examples may be separated by " + "with `----` on its own line." + ), + ) + parser.add_argument("--width", type=int, default=80) + parser.add_argument("--ascii", default=False, action="store_true") + + args = parser.parse_args(sys.argv[1:]) + if args.examples == ["-"]: + args.examples = sys.stdin.read().split("\n----\n") + + try: + render_examples(args.examples, width=args.width, ascii=args.ascii) + except Exception as e: + raise ValueError( + f"Failed to render examples:\n{'\n----\n'.join(args.examples)}" + ) from e diff --git a/docs/reference/functions/_render_listing.py b/docs/reference/functions/_render_listing.py new file mode 100644 index 00000000..464e0d01 --- /dev/null +++ b/docs/reference/functions/_render_listing.py @@ -0,0 +1,83 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +import io +from pathlib import Path +import yaml + +from _render_meta import render_meta + + +def render_listing(paths): + """Lists meta information for zero or more filenames to function .qmd files + + Output: + + ``` + ## [ST_Name](st_name.md) + + <description> + + ## Usage + + return_type ST_Name(arg_name: arg_type) + ``` + """ + for file in paths: + raw_meta = read_frontmatter(file) + link_href = file.name.replace(".qmd", ".md") + print(f"\n## [{raw_meta['title']}]({link_href})\n\n") + + render_meta(raw_meta, level=2, arguments=False) + + +def read_frontmatter(path): + frontmatter = io.StringIO() + with open(path) as f: + for line in f: + if line.strip() == "---": + break + for line in f: + if line.strip() == "---": + break + frontmatter.write(line) + frontmatter.seek(0) + return yaml.safe_load(frontmatter) + + +def collect_files(file_glob): + return list(sorted(Path(__file__).parent.rglob(file_glob))) + + +if __name__ == "__main__": + import argparse + import sys + import yaml + + parser = argparse.ArgumentParser(description="Render SedonaDB SQL function listing") + parser.add_argument( + "files", + nargs="+", + help="Files or globs of files whose frontmatter should be included in the listing", + ) + args = parser.parse_args(sys.argv[1:]) + + in_files: list[Path] = [] + for file_or_glob in args.files: + in_files.extend(collect_files(file_or_glob)) + + render_listing(in_files) diff --git a/docs/reference/functions/_render_meta.py b/docs/reference/functions/_render_meta.py new file mode 100644 index 00000000..e3465a70 --- /dev/null +++ b/docs/reference/functions/_render_meta.py @@ -0,0 +1,213 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +import io + + +def render_meta(raw_meta, level=1, usage=True, arguments=True): + """Render parsed YAML frontmatter into a standardized template + + Output: + + ``` + <description> + + ## Usage + + return_type ST_Name(arg_name: arg_type) + + ## Arguments + + - **arg_name** (arg_type): Arg description + ``` + + """ + if "description" in raw_meta: + render_description(raw_meta["description"]) + + if "kernels" in raw_meta: + for kernel in raw_meta["kernels"]: + kernel["args"] = expand_args(kernel["args"]) + + if usage: + render_usage(raw_meta["title"], raw_meta["kernels"], level) + + if arguments: + render_args(raw_meta["kernels"], level=level) + + +def render_description(description): + print(to_str(description).strip()) + + +def render_usage(name, kernels, level): + print(f"\n{heading(level + 1)} Usage\n") + print("\n```sql") + for kernel in kernels: + args = ", ".join(render_usage_arg(arg) for arg in kernel["args"]) + print(f"{to_str(kernel['returns'])} {to_str(name)}({args})") + print("```") + + +def render_usage_arg(arg): + if arg["default"] is not None: + return f"{arg['name']}: {arg['type']} = {arg['default']}" + else: + return f"{arg['name']}: {arg['type']}" + + +def render_args(kernels, level): + try: + expanded_args = {} + for kernel in reversed(kernels): + args_dict = {arg["name"]: arg for arg in kernel["args"]} + expanded_args.update({k: v for k, v in args_dict.items() if v is not None}) + except Exception as e: + raise ValueError( + f"Failed to consolidate argument documentation from kernels:\n{kernels}" + ) from e + + print(f"\n{heading(level + 1)} Arguments\n") + for arg in expanded_args.values(): + print( + f"- **{to_str(arg['name'])}** ({to_str(arg['type'])}): {to_str(arg['description'])}" + ) + + +def expand_args(args): + """Normalizes frontmatter-specified arguments + + This enables shorthand argument definitions like "geometry" to coexist + with more verbosely defined argument definitions. + """ + args = [expand_arg(arg) for arg in args] + return deduplicate_common_arg_combinations(args) + + +def deduplicate_common_arg_combinations(expanded_args): + """Transform argument names for binary functions with multiple geometry inputs + + Functions like ST_Intersects() that accept two geometries or two geographies + will both have auto-generated function names [geom, geom]. This function + renames them to [geomA, geomB]. + """ + all_names = [arg["name"] for arg in expanded_args] + if all_names[:2] == ["geom", "geom"]: + all_names[:2] = ["geomA", "geomB"] + elif all_names[:2] == ["geog", "geog"]: + all_names[:2] = ["geogA", "geogB"] + + return [ + { + "name": new_name, + "type": arg["type"], + "description": arg["description"], + "default": arg["default"], + } + for arg, new_name in zip(expanded_args, all_names) + ] + + +def expand_arg(arg): + """Ensure each arg definition is a dict with keys type, name, description, + and default + """ + if isinstance(arg, dict): + arg = {k: to_str(v) for k, v in arg.items()} + else: + arg = to_str(arg) + arg = { + "type": arg, + "name": DEFAULT_ARG_NAMES[arg], + "description": DEFAULT_ARG_DESCRIPTIONS.get(arg, None), + } + + if "default" not in arg: + arg["default"] = None + if "description" not in arg: + arg["description"] = None + + return arg + + +# Define some default argument names/descriptions so that we can abbreviate +# the ubiquitous "geometry" input type argument without repeating the string +# "geom: Input geometry" for every single function. +DEFAULT_ARG_NAMES = { + "geometry": "geom", + "geography": "geog", + "raster": "rast", +} + +DEFAULT_ARG_DESCRIPTIONS = { + "geometry": "Input geometry", + "geography": "Input geography", + "raster": "Input raster", +} + + +def heading(level): + return "#" * level + " " + + +def to_str(v): + """Convert a value to a string + + Because we call this from within a Quarto filter and use pandoc's built-in + JSON output, we might get a pandoc JSONified AST here. Parsing the AST here + is slightly easier than getting the Quarto filter to recreate the initial + YAML frontmatter. + """ + if isinstance(v, str): + return v + + if isinstance(v, dict): + if v["t"] == "Str": + return v["c"] + if v["t"] == "Code": + return f"`{v['c'][1]}`" + elif v["t"] == "Space": + return " " + elif v["t"] == "Para": + return "".join(to_str(item) for item in v["c"]) + else: + raise ValueError(f"Unhandled type in Pandoc ast convert: {v}") + else: + return "".join(to_str(item) for item in v) + + +if __name__ == "__main__": + import argparse + import sys + import yaml + + parser = argparse.ArgumentParser(description="Render SedonaDB SQL function header") + parser.add_argument( + "meta", + help=( + "Function yaml metadata (e.g., frontmatter for a function doc page). " + "Use `-` to read stdin." + ), + ) + + args = parser.parse_args(sys.argv[1:]) + if args.meta == "-": + args.meta = sys.stdin.read() + + with io.StringIO(args.meta) as f: + raw_meta = yaml.safe_load(f) + render_meta(raw_meta) diff --git a/docs/requirements.txt b/docs/reference/functions/index.qmd similarity index 82% copy from docs/requirements.txt copy to docs/reference/functions/index.qmd index 6bd80c39..7c2ea8c6 100644 --- a/docs/requirements.txt +++ b/docs/reference/functions/index.qmd @@ -1,3 +1,4 @@ +--- # Licensed to the Apache Software Foundation (ASF) under one # or more contributor license agreements. See the NOTICE file # distributed with this work for additional information @@ -15,15 +16,7 @@ # specific language governing permissions and limitations # under the License. -griffe -mike -mkdocs -mkdocs-git-revision-date-localized-plugin -mkdocs-glightbox -mkdocs-macros-plugin -mkdocs-material -mkdocstrings[python] -nbconvert -pyproj -ruff -lonboard +title: "SQL Function Reference" +--- + +{{< function-listing st_*.qmd >}} diff --git a/docs/reference/functions/st_analyze_agg.qmd b/docs/reference/functions/st_analyze_agg.qmd new file mode 100644 index 00000000..e9b1de6a --- /dev/null +++ b/docs/reference/functions/st_analyze_agg.qmd @@ -0,0 +1,49 @@ +--- +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +title: ST_Analyze_Agg +description: Compute the statistics of geometries for the input geometry. +kernels: + - returns: struct + args: [geometry] +--- + +## Description + +`ST_Analyze_Agg()` provides a high-level summary of its input geometries. The fields of +its struct return type are: + +- `count`: Number of input geometries +- `minx`, `miny`, `maxx`, `maxy`: Minimum bounding rectangle (envelope) +- `mean_size_in_bytes` +- `mean_points_per_geometry` +- `puntal_count`: Number of point or multipoint geometries +- `lineal_count`: Number of line or multilinestring geometries +- `polygonal_count`: Number of polygon or multipolygon geometries +- `geometrycollection_count`: Number of geometrycollection geometries +- `mean_envelope_width` +- `mean_envelope_height` +- `mean_envelope_area` + +## Examples + +```sql +SELECT ST_Analyze_Agg( + ST_GeomFromText('MULTIPOINT(1.1 101.1,2.1 102.1,3.1 103.1,4.1 104.1,5.1 105.1,6.1 106.1,7.1 107.1,8.1 108.1,9.1 109.1,10.1 110.1)') + ) AS analyze +``` diff --git a/docs/reference/functions/st_buffer.qmd b/docs/reference/functions/st_buffer.qmd new file mode 100644 index 00000000..64f807c8 --- /dev/null +++ b/docs/reference/functions/st_buffer.qmd @@ -0,0 +1,81 @@ +--- +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +title: ST_Buffer +description: > + Computes a geometry that represents all points whose distance from the input + geometry is less than or equal to a specified distance. +kernels: + - returns: geometry + args: + - geometry + - name: distance + type: float64 + description: Radius of the buffer + - returns: geometry + args: + - geometry + - name: distance + type: float64 + - name: params + type: utf8 + description: > + Space-separated `key=value` parameters. Supported parameters include `quad_segs`, + `endcap`, `join`, `mitre_limit`, and `side`. These parameters are identical to + the PostGIS buffer parameter strings. +--- + +## Examples + +```sql +SELECT ST_Buffer( + ST_GeomFromText('POLYGON ((10 10, 11 10, 10 11, 10 10))'), + 1.0 +) AS geom; +``` + +```{python} +import _matplotlib_defaults +import sedonadb + +sd = sedonadb.connect() + +sd.sql(""" +SELECT ST_GeomFromText('POLYGON ((10 10, 11 10, 10 11, 10 10))') as geom +""").to_view("input") + +df = sd.sql("SELECT *, ST_Buffer(geom, 1.0) AS result FROM input").to_pandas() + +ax = df.geom.plot(alpha=0.5) +df.result.plot(ax=ax, facecolor='none', edgecolor='red') +``` + +```sql +SELECT ST_Buffer( + ST_GeomFromText('POLYGON ((10 10, 11 10, 10 11, 10 10))'), + 1.0, + 'quad_segs=2' +) AS geom; +``` + +```{python} +df = sd.sql("SELECT *, ST_Buffer(geom, 1.0, 'quad_segs=2') AS result FROM input").to_pandas() + +ax = df.geom.plot(alpha=0.5) +df.result.plot(ax=ax, facecolor='none', edgecolor='red') +``` diff --git a/docs/reference/functions/st_intersection.qmd b/docs/reference/functions/st_intersection.qmd new file mode 100644 index 00000000..2bdc47be --- /dev/null +++ b/docs/reference/functions/st_intersection.qmd @@ -0,0 +1,54 @@ +--- +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +title: ST_Intersection +description: Compute the intersection of two geometries or geographies. +kernels: + - returns: geometry + args: [geometry, geometry] + - returns: geography + args: [geography, geography] +--- + +## Examples + +```sql +SELECT ST_Intersection( + ST_GeomFromText('POLYGON ((1 1, 11 1, 11 11, 1 11, 1 1))'), + ST_GeomFromText('POLYGON ((0 0, 10 0, 10 10, 0 10, 0 0))') +) AS val; +``` + +```{python} +import _matplotlib_defaults +import sedonadb + +sd = sedonadb.connect() + +sd.sql(""" +SELECT + ST_GeomFromText('POLYGON ((1 1, 11 1, 11 11, 1 11, 1 1))') as geom_a, + ST_GeomFromText('POLYGON ((0 0, 10 0, 10 10, 0 10, 0 0))') as geom_b +""").to_view("input") + +df = sd.sql("SELECT *, ST_Intersection(geom_a, geom_b) as result FROM input").to_pandas() + +ax = df.geom_a.plot(alpha=0.5) +df.geom_b.plot(ax=ax, alpha=0.5) +df.result.plot(ax=ax, facecolor='none', edgecolor='red') +``` diff --git a/docs/requirements.txt b/docs/requirements.txt index 6bd80c39..53f6716c 100644 --- a/docs/requirements.txt +++ b/docs/requirements.txt @@ -16,6 +16,7 @@ # under the License. griffe +ipykernel mike mkdocs mkdocs-git-revision-date-localized-plugin
