This is an automated email from the ASF dual-hosted git repository.
jiayu pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/sedona.git
The following commit(s) were added to refs/heads/master by this push:
new c9efdf8f1e [DOC] Clean up the Sphinx documentation build warnings and
errors (#2191)
c9efdf8f1e is described below
commit c9efdf8f1e80bba5db67ed528c2e756e365f3939
Author: Feng Zhang <[email protected]>
AuthorDate: Tue Jul 29 20:05:11 2025 -0700
[DOC] Clean up the Sphinx documentation build warnings and errors (#2191)
* [DOC] Clean up the Sphinx documentation build warnings and errors
1. Fixed duplicate toctree references - Removed duplicate entries from
sedona.rst that were already in index.rst
2. Created missing _static directory - The build was expecting this
directory but it didn't exist
3. Fixed NumPy 2.0 compatibility issue - Added mock imports for
pyspark.pandas modules that were failing due to deprecated
np.NaN usage
4. Fixed docstring formatting errors in:
- ST_Angle function - Fixed indentation and list formatting
- ST_WeightedDistanceBandColumn - Converted @param to :param format
- SedonaPyDeck.create_choropleth_map - Fixed parameter order and
indentation
- STAC client methods - Converted to proper Sphinx :param format
- CollectionClient methods - Fixed parameter documentation format
5. Added modules.rst to toctree - Fixed warning about document not being
included
6. Mocked missing raster modules - Added raster_serde and sedona_raster
to mock imports
The remaining 7 warnings are:
- 2 mocked object detections (expected for the raster modules)
- 5 cross-reference ambiguity warnings (due to classes existing in both
shapely1 and shapely2 modules, which appears to be
intentional for compatibility)
* clean up the rest of the warnings
* adjust color and theme
* fix pre-commit lint
* address copilot comments
* Update python/sedona/spark/stac/client.py
Co-authored-by: Copilot <[email protected]>
---------
Co-authored-by: Copilot <[email protected]>
---
python/sedona/doc/_static/color-scheme-info.md | 65 +++++
python/sedona/doc/_static/custom.css | 321 +++++++++++++++++++++++++
python/sedona/doc/_templates/layout.html | 17 ++
python/sedona/doc/conf.py | 42 +++-
python/sedona/doc/index.rst | 26 +-
python/sedona/doc/sedona.rst | 13 -
python/sedona/spark/maps/SedonaPyDeck.py | 14 +-
python/sedona/spark/sql/st_functions.py | 32 +--
python/sedona/spark/stac/client.py | 77 +++---
python/sedona/spark/stac/collection_client.py | 142 ++++++-----
10 files changed, 588 insertions(+), 161 deletions(-)
diff --git a/python/sedona/doc/_static/color-scheme-info.md
b/python/sedona/doc/_static/color-scheme-info.md
new file mode 100644
index 0000000000..dddee5805e
--- /dev/null
+++ b/python/sedona/doc/_static/color-scheme-info.md
@@ -0,0 +1,65 @@
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one
+ or more contributor license agreements. See the NOTICE file
+ distributed with this work for additional information
+ regarding copyright ownership. The ASF licenses this file
+ to you under the Apache License, Version 2.0 (the
+ "License"); you may not use this file except in compliance
+ with the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing,
+ software distributed under the License is distributed on an
+ "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ KIND, either express or implied. See the License for the
+ specific language governing permissions and limitations
+ under the License.
+ -->
+
+# Apache Sedona Python Documentation Color Scheme
+
+## Updated Color Palette (Material Design Deep Orange + Black)
+
+### Primary Colors
+
+- **Deep Orange Primary**: `#ff5722` (used for headers, navigation, primary
buttons)
+- **Deep Orange Dark**: `#d84315` (used for hover states, headings)
+
+### Accent Colors
+
+- **Black Accent**: `#000000` (used for links, current navigation, highlights)
+- **Black Accent Dark**: `#333333` (used for hover states on links)
+
+### Supporting Colors
+
+- **Dark Gray**: `#263238` (sidebar background)
+- **Light Gray**: `#f5f5f5` (code backgrounds)
+- **Secondary Gray**: `#e0e0e0` (borders, dividers)
+- **Text**: `#333333` (main content text)
+- **White**: `#ffffff` (content backgrounds)
+
+## Color Usage
+
+### Navigation
+
+- Header: Deep Orange (`#ff5722`)
+- Sidebar: Dark Gray (`#263238`)
+- Current page: Deep Orange with Black accent border
+- Links: Black accent (`#000000`)
+
+### Content
+
+- Headings: Deep Orange Dark (`#d84315`)
+- Links: Black accent (`#000000`)
+- Code blocks: Light Gray backgrounds
+- Admonitions: Deep Orange borders with light orange backgrounds
+- API documentation: Black accent borders
+
+### Interactive Elements
+
+- Buttons: Black accent backgrounds
+- Hover states: Darker variants of respective colors
+- Search focus: Black accent border
+
+This color scheme uses Material Design deep orange primary with black accents
for a professional, high-contrast appearance.
diff --git a/python/sedona/doc/_static/custom.css
b/python/sedona/doc/_static/custom.css
new file mode 100644
index 0000000000..e278f3dc98
--- /dev/null
+++ b/python/sedona/doc/_static/custom.css
@@ -0,0 +1,321 @@
+/* Apache Sedona Custom Theme Styling */
+
+/* Color Variables */
+:root {
+ --sedona-primary: #ff5722;
+ --sedona-primary-dark: #d84315;
+ --sedona-accent: #000000;
+ --sedona-accent-dark: #333333;
+ --sedona-secondary: #e0e0e0;
+ --sedona-text: #333333;
+ --sedona-light-gray: #f5f5f5;
+ --sedona-white: #ffffff;
+ --sedona-dark: #263238;
+}
+
+/* Global Styling */
+body {
+ font-family: 'Roboto', 'Segoe UI', 'Helvetica Neue', Arial, sans-serif
!important;
+ color: var(--sedona-text) !important;
+}
+
+/* Header Styling */
+.wy-nav-top {
+ background: var(--sedona-primary) !important;
+ color: var(--sedona-white) !important;
+}
+
+.wy-nav-top a {
+ color: var(--sedona-white) !important;
+ font-weight: 500;
+}
+
+/* Sidebar Styling */
+.wy-nav-side {
+ background: var(--sedona-dark) !important;
+ width: 280px !important;
+}
+
+.wy-menu-vertical {
+ background: var(--sedona-dark) !important;
+}
+
+.wy-menu-vertical a {
+ background: transparent !important;
+ color: #ffffff !important;
+ border-bottom: none !important;
+ padding: 8px 24px !important;
+ transition: all 0.2s ease !important;
+}
+
+.wy-menu-vertical a:hover {
+ background: rgba(255, 255, 255, 0.08) !important;
+ color: var(--sedona-white) !important;
+}
+
+.wy-menu-vertical .current > a {
+ background: var(--sedona-primary) !important;
+ color: var(--sedona-white) !important;
+ border-right: 3px solid var(--sedona-accent) !important;
+}
+
+.wy-menu-vertical .toctree-l1 > a {
+ font-weight: 500 !important;
+ color: #ffffff !important;
+}
+
+.wy-menu-vertical .toctree-l2 > a {
+ padding-left: 40px !important;
+}
+
+.wy-menu-vertical .toctree-l3 > a {
+ padding-left: 56px !important;
+}
+
+/* Hide Contents: caption in left navigation bar */
+.wy-menu-vertical .caption,
+.wy-menu-vertical .toctree-wrapper .caption {
+ display: none !important;
+}
+
+/* Main Content Area */
+.wy-nav-content-wrap {
+ margin-left: 280px !important;
+ background: var(--sedona-white) !important;
+}
+
+.wy-nav-content {
+ background: var(--sedona-white) !important;
+ padding: 20px !important;
+}
+
+.document {
+ max-width: 1200px !important;
+ margin: 0 auto !important;
+}
+
+/* Typography */
+h1, h2, h3, h4, h5, h6 {
+ color: var(--sedona-primary-dark) !important;
+ font-weight: 500 !important;
+ margin-top: 2rem !important;
+ margin-bottom: 1rem !important;
+}
+
+h1 {
+ font-size: 2.5rem !important;
+ border-bottom: 2px solid var(--sedona-secondary) !important;
+ padding-bottom: 0.5rem !important;
+}
+
+h2 {
+ font-size: 2rem !important;
+ border-bottom: 1px solid var(--sedona-secondary) !important;
+ padding-bottom: 0.3rem !important;
+}
+
+h3 {
+ font-size: 1.5rem !important;
+}
+
+/* Code Blocks */
+.highlight {
+ background: var(--sedona-light-gray) !important;
+ border: 1px solid var(--sedona-secondary) !important;
+ border-radius: 4px !important;
+ margin: 1rem 0 !important;
+}
+
+.highlight pre {
+ background: transparent !important;
+ border: none !important;
+ padding: 16px !important;
+}
+
+code {
+ background: var(--sedona-light-gray) !important;
+ color: var(--sedona-primary-dark) !important;
+ padding: 2px 4px !important;
+ border-radius: 3px !important;
+ font-family: 'Roboto Mono', 'Consolas', 'Monaco', monospace !important;
+}
+
+/* Tables */
+.wy-table-responsive table td,
+.wy-table-responsive table th {
+ white-space: normal !important;
+ background: var(--sedona-white) !important;
+ border: 1px solid var(--sedona-secondary) !important;
+}
+
+.wy-table-responsive table th {
+ background: var(--sedona-light-gray) !important;
+ font-weight: 500 !important;
+}
+
+/* Admonitions */
+.admonition {
+ border-left: 4px solid var(--sedona-primary) !important;
+ background: rgba(255, 87, 34, 0.05) !important;
+ border-radius: 0 4px 4px 0 !important;
+ margin: 1rem 0 !important;
+}
+
+.admonition-title {
+ background: rgba(255, 87, 34, 0.1) !important;
+ color: var(--sedona-primary-dark) !important;
+ font-weight: 500 !important;
+}
+
+/* Links */
+a {
+ color: var(--sedona-accent) !important;
+ text-decoration: none !important;
+}
+
+a:hover {
+ color: var(--sedona-accent-dark) !important;
+ text-decoration: underline !important;
+}
+
+/* Navigation Breadcrumbs */
+.wy-breadcrumbs {
+ background: transparent !important;
+ padding: 0 !important;
+ margin-bottom: 24px !important;
+}
+
+.wy-breadcrumbs li {
+ color: #666 !important;
+}
+
+.wy-breadcrumbs li a {
+ color: var(--sedona-accent) !important;
+}
+
+/* Footer */
+.rst-footer-buttons {
+ margin-top: 2rem !important;
+ padding-top: 2rem !important;
+ border-top: 1px solid var(--sedona-secondary) !important;
+}
+
+/* Mobile Responsiveness */
+@media screen and (max-width: 768px) {
+ .wy-nav-side {
+ width: 280px !important;
+ }
+
+ .wy-nav-content-wrap {
+ margin-left: 0 !important;
+ }
+}
+
+/* Hero Section Styling */
+.sedona-hero {
+ background: linear-gradient(135deg, var(--sedona-primary) 0%,
var(--sedona-primary-dark) 100%) !important;
+ color: var(--sedona-white) !important;
+ padding: 3rem 2rem !important;
+ margin: -20px -20px 2rem -20px !important;
+ border-radius: 0 0 8px 8px !important;
+ text-align: center !important;
+}
+
+.sedona-hero h1 {
+ color: var(--sedona-white) !important;
+ font-size: 3rem !important;
+ font-weight: 300 !important;
+ margin: 0 0 1rem 0 !important;
+ border: none !important;
+ padding: 0 !important;
+}
+
+.hero-subtitle {
+ font-size: 1.25rem !important;
+ opacity: 0.9 !important;
+ margin: 0 !important;
+ font-weight: 300 !important;
+}
+
+/* Custom Apache Sedona Branding */
+.wy-side-nav-search {
+ background: var(--sedona-primary) !important;
+}
+
+.wy-side-nav-search input[type=text] {
+ border: 1px solid rgba(255, 255, 255, 0.2) !important;
+ background: rgba(255, 255, 255, 0.1) !important;
+ color: var(--sedona-white) !important;
+}
+
+.wy-side-nav-search input[type=text]:focus {
+ border-color: var(--sedona-accent) !important;
+ background: var(--sedona-white) !important;
+ color: var(--sedona-text) !important;
+}
+
+/* Add Sedona Logo Styling */
+.wy-side-nav-search .wy-dropdown > a:before {
+ content: "🌍 ";
+ margin-right: 8px;
+}
+
+/* Improve content spacing */
+.document .section {
+ margin-bottom: 2rem;
+}
+
+/* Style for Python API documentation */
+.py.class, .py.method, .py.function {
+ border-left: 3px solid var(--sedona-accent);
+ padding-left: 12px;
+ margin: 1rem 0;
+}
+
+.py.class > dt, .py.method > dt, .py.function > dt {
+ background: rgba(0, 0, 0, 0.05) !important;
+ border-radius: 4px;
+ padding: 8px 12px;
+}
+
+/* Version notice styling */
+.version {
+ background: var(--sedona-light-gray);
+ border: 1px solid var(--sedona-secondary);
+ border-radius: 4px;
+ padding: 12px;
+ margin: 1rem 0;
+}
+
+/* Toctree caption styling - make Contents: white for better contrast (main
content only) */
+.document .toctree-wrapper .caption,
+.rst-content .document .toctree-wrapper .caption {
+ color: var(--sedona-white) !important;
+ font-weight: 500 !important;
+ background: var(--sedona-primary) !important;
+ padding: 8px 16px !important;
+ margin: 0 -16px 16px -16px !important;
+ border-radius: 4px !important;
+}
+
+/* Enhanced button styling */
+.btn, .rst-content .btn {
+ background: var(--sedona-accent) !important;
+ border-color: var(--sedona-accent) !important;
+ color: var(--sedona-white) !important;
+}
+
+.btn:hover, .rst-content .btn:hover {
+ background: var(--sedona-accent-dark) !important;
+ border-color: var(--sedona-accent-dark) !important;
+}
+
+/* Navigation prev/next buttons */
+.rst-footer-buttons .btn-neutral {
+ background: var(--sedona-primary) !important;
+ color: var(--sedona-white) !important;
+}
+
+.rst-footer-buttons .btn-neutral:hover {
+ background: var(--sedona-primary-dark) !important;
+}
diff --git a/python/sedona/doc/_templates/layout.html
b/python/sedona/doc/_templates/layout.html
new file mode 100644
index 0000000000..480cf48205
--- /dev/null
+++ b/python/sedona/doc/_templates/layout.html
@@ -0,0 +1,17 @@
+{% extends "!layout.html" %}
+
+{% block extrahead %}
+ {{ super() }}
+ <!-- Google Fonts for Apache Sedona styling -->
+ <link
href="https://fonts.googleapis.com/css2?family=Roboto:wght@300;400;500;700&family=Roboto+Mono:wght@400;500&display=swap";
rel="stylesheet">
+
+ <!-- Additional meta tags for better rendering -->
+ <meta name="viewport" content="width=device-width, initial-scale=1.0">
+ <meta name="theme-color" content="#1565c0">
+{% endblock %}
+
+{% block document %}
+ <div class="sedona-content-wrapper">
+ {{ super() }}
+ </div>
+{% endblock %}
diff --git a/python/sedona/doc/conf.py b/python/sedona/doc/conf.py
index 2918684fee..60467b59ab 100644
--- a/python/sedona/doc/conf.py
+++ b/python/sedona/doc/conf.py
@@ -39,9 +39,30 @@ extensions = [
"sphinx_rtd_theme", # Read the Docs theme
]
+# Mock imports to handle NumPy 2.0 compatibility issues with PySpark and
missing dependencies
+autodoc_mock_imports = [
+ "pyspark.pandas",
+ "pyspark.pandas.indexes",
+ "pyspark.pandas.indexes.base",
+ "pyspark.pandas.series",
+ "pyspark.pandas.strings",
+ "pystac",
+ "sedona.spark.raster.raster_serde",
+ "sedona.spark.raster.sedona_raster",
+]
+
templates_path = ["_templates"]
exclude_patterns = ["_build", "Thumbs.db", ".DS_Store"]
+# Suppress specific warnings
+suppress_warnings = [
+ # Suppress warnings about mocked objects (these are intentionally mocked)
+ "autodoc.mocked_object",
+ # Suppress warnings about multiple targets for cross-references
+ # This is expected due to shapely1/shapely2 compatibility layer
+ "ref.python",
+]
+
autodoc_default_options = {
"members": True,
"undoc-members": True,
@@ -55,9 +76,28 @@ html_theme = "sphinx_rtd_theme"
html_static_path = ["_static"]
html_title = f"{project} Documentation"
+# Custom CSS to match Apache Sedona branding
+html_css_files = [
+ "custom.css",
+]
+
html_theme_options = {
"collapse_navigation": False,
"navigation_depth": 4,
- "style_external_links": True,
+ "style_external_links": False,
"titles_only": False,
+ "logo_only": False,
+ "prev_next_buttons_location": "bottom",
+ "vcs_pageview_mode": "",
+ # Custom colors to match Sedona theme
+ "style_nav_header_background": "#ff5722",
+}
+
+# Apache Sedona Branding
+html_logo = None # We'll use CSS for logo styling
+html_favicon = None
+
+# Additional context for templates
+html_context = {
+ "display_github": False,
}
diff --git a/python/sedona/doc/index.rst b/python/sedona/doc/index.rst
index 4b7c3f7b71..be9bba3be7 100644
--- a/python/sedona/doc/index.rst
+++ b/python/sedona/doc/index.rst
@@ -3,20 +3,34 @@
You can adapt this file completely to your liking, but it should at least
contain the root `toctree` directive.
-sedona-python documentation
-===========================
+Apache Sedona™ Python API Documentation
+========================================
-Introduction to Sedona Python Packages
-======================================
+.. raw:: html
-Apache Sedona provides a suite of Python packages for spatial data processing
and analysis. These packages include modules for working with geospatial data,
integrating with big data frameworks, and performing advanced spatial
computations.
+ <div class="sedona-hero">
+ <h1>🌍 Apache Sedona™ Python API</h1>
+ <p class="hero-subtitle">A cluster computing system for processing
large-scale spatial data</p>
+ </div>
-The following documentation covers the available Python packages in Sedona,
including their submodules and functionalities.
+Welcome to Apache Sedona™
+==========================
+
+Apache Sedona™ is a cluster computing system for processing large-scale
spatial data. Sedona extends existing cluster computing systems, such as Apache
Spark, Apache Flink, and Snowflake, with a set of out-of-the-box distributed
Spatial Datasets and Spatial SQL that efficiently load, process, and analyze
large-scale spatial data across machines.
+
+This documentation covers the Python API for Apache Sedona, providing
comprehensive guides and references for:
+
+- **Spatial RDDs**: Distributed spatial data structures
+- **Spatial DataFrames**: Spark DataFrame integration with spatial operations
+- **Spatial Functions**: Built-in spatial analysis and processing functions
+- **Visualization**: Map creation and spatial data visualization tools
+- **GeoPandas Integration**: Seamless integration with the GeoPandas ecosystem
.. toctree::
:maxdepth: 2
:caption: Contents:
+ modules
sedona.flink
sedona.geopandas
sedona.spark
diff --git a/python/sedona/doc/sedona.rst b/python/sedona/doc/sedona.rst
index a8c4594349..d1508904d6 100644
--- a/python/sedona/doc/sedona.rst
+++ b/python/sedona/doc/sedona.rst
@@ -1,19 +1,6 @@
sedona package
==============
-Subpackages
------------
-
-.. toctree::
- :maxdepth: 4
-
- sedona.flink
- sedona.geopandas
- sedona.spark
- sedona.stac
- sedona.stats
- sedona.utils
-
Submodules
----------
diff --git a/python/sedona/spark/maps/SedonaPyDeck.py
b/python/sedona/spark/maps/SedonaPyDeck.py
index 2fdc3af302..1aea3df622 100644
--- a/python/sedona/spark/maps/SedonaPyDeck.py
+++ b/python/sedona/spark/maps/SedonaPyDeck.py
@@ -48,16 +48,18 @@ class SedonaPyDeck:
):
"""
Create a pydeck map with a choropleth layer added
- :param elevation_col: Optional elevation for the polygons
+
:param df: SedonaDataFrame to plot on the choropleth map.
:param fill_color: color scheme to fill the map with.
- If no color scheme is given, a default color scheme is created
using the 'plot_col' column as the quantizing column
+ If no color scheme is given, a default color scheme is created
using the 'plot_col' column as the quantizing column
:param plot_col: Column to be used to create a default color scheme.
If fill_color is provided, this parameter is ignored.
- :param initial_view_state:
- :param map_style:
- :param map_provider:
+ :param initial_view_state: Initial view state for the map
+ :param map_style: Map style to use
+ :param map_provider: Map provider to use
+ :param elevation_col: Optional elevation for the polygons
:param api_keys: Optional dictionary of API keys for Map providers
- :return: A pydeck Map object with choropleth layer added:
+ :param stroked: Whether to stroke the polygons
+ :return: A pydeck Map object with choropleth layer added
"""
for field in df.schema.fields:
diff --git a/python/sedona/spark/sql/st_functions.py
b/python/sedona/spark/sql/st_functions.py
index 43153377b7..08fc20c350 100644
--- a/python/sedona/spark/sql/st_functions.py
+++ b/python/sedona/spark/sql/st_functions.py
@@ -2339,13 +2339,16 @@ def ST_Angle(
) -> Column:
"""
Returns the computed angle between vectors formed by given geometries in
radian. Range of result is between 0 and 2 * pi.
+
3 Variants:
- Angle(Point1, Point2, Point3, Point4)
- Computes angle formed by vectors formed by Point1-Point2 and
Point3-Point4
- Angle(Point1, Point2, Point3)
- Computes angle formed by angle Point1-Point2-Point3
- Angle(Line1, Line2)
- Computes angle between vectors formed by S1-E1 and S2-E2, where S
and E are start and endpoints.
+
+ - Angle(Point1, Point2, Point3, Point4)
+ Computes angle formed by vectors formed by Point1-Point2 and
Point3-Point4
+ - Angle(Point1, Point2, Point3)
+ Computes angle formed by angle Point1-Point2-Point3
+ - Angle(Line1, Line2)
+ Computes angle between vectors formed by S1-E1 and S2-E2, where S and
E are start and endpoints.
+
:param g1: Point or Line
:param g2: Point or Line
:param g3: Point or None
@@ -2692,16 +2695,15 @@ def ST_WeightedDistanceBandColumn(
Weights will be distance^alpha.
-
- @param geometry: name of the geometry column
- @param threshold: Distance threshold for considering neighbors
- @param alpha: alpha to use for inverse distance weights. Computation is
dist^alpha. Default is -1.0
- @param include_zero_distance_neighbors: whether to include neighbors that
are 0 distance. If 0 distance neighbors are
+ :param geometry: name of the geometry column
+ :param threshold: Distance threshold for considering neighbors
+ :param alpha: alpha to use for inverse distance weights. Computation is
dist^alpha. Default is -1.0
+ :param include_zero_distance_neighbors: whether to include neighbors that
are 0 distance. If 0 distance neighbors are
included, values are infinity as per the floating point spec (divide
by 0)
- @param include_self: whether to include self in the list of neighbors
- @param self_weight: the value to use for the self weight. Default is 1.0
- @param use_spheroid: whether to use a cartesian or spheroidal distance
calculation. Default is false
- @param attributes: the attributes to save in the neighbor column.
+ :param include_self: whether to include self in the list of neighbors
+ :param self_weight: the value to use for the self weight. Default is 1.0
+ :param use_spheroid: whether to use a cartesian or spheroidal distance
calculation. Default is false
+ :param attributes: the attributes to save in the neighbor column.
"""
if isinstance(alpha, float):
diff --git a/python/sedona/spark/stac/client.py
b/python/sedona/spark/stac/client.py
index ac954ea17f..103dc65396 100644
--- a/python/sedona/spark/stac/client.py
+++ b/python/sedona/spark/stac/client.py
@@ -36,12 +36,8 @@ class Client:
This class method creates an instance of the Client class with the
given URL.
- Parameters:
- - url (str): The URL of the STAC API to connect to.
- Example: "https://planetarycomputer.microsoft.com/api/stac/v1";
-
- Returns:
- - Client: An instance of the Client class connected to the specified
URL.
+ :param url: The URL of the STAC API to connect to. Example:
"https://planetarycomputer.microsoft.com/api/stac/v1";
+ :return: An instance of the Client class connected to the specified
URL.
"""
return cls(url)
@@ -52,12 +48,8 @@ class Client:
This method creates an instance of the CollectionClient class for the
given collection ID,
allowing interaction with the specified collection in the STAC API.
- Parameters:
- - collection_id (str): The ID of the collection to retrieve.
- Example: "aster-l1t"
-
- Returns:
- - CollectionClient: An instance of the CollectionClient class for the
specified collection.
+ :param collection_id: The ID of the collection to retrieve. Example:
"aster-l1t"
+ :return: An instance of the CollectionClient class for the specified
collection.
"""
return CollectionClient(self.url, collection_id)
@@ -68,7 +60,7 @@ class Client:
This method fetches the root catalog from the STAC API, providing
access to all collections and items.
Returns:
- - dict: The root catalog of the STAC API.
+ dict: The root catalog of the STAC API.
"""
# Implement logic to fetch and return the root catalog
return CollectionClient(self.url, None)
@@ -88,39 +80,32 @@ class Client:
"""
Searches for items in the specified collection with optional filters.
- Parameters:
- - ids (Union[str, list]): A variable number of item IDs to filter the
items.
- Example: "item_id1" or ["item_id1", "item_id2"]
-
- - collection_id (Optional[str]): The ID of the collection to search in.
- Example: "aster-l1t"
-
- - bbox (Optional[list]): A list of bounding boxes for filtering the
items.
- Each bounding box is represented as a list of four float values:
[min_lon, min_lat, max_lon, max_lat].
- Example: [[-180.0, -90.0, 180.0, 90.0]] # This bounding box covers
the entire world.
-
- - geometry (Optional[Union[str, BaseGeometry, List[Union[str,
BaseGeometry]]]]): Shapely geometry object(s) or WKT string(s) for spatial
filtering.
- Can be a single geometry, WKT string, or a list of geometries/WKT
strings.
- If both bbox and geometry are provided, geometry takes precedence.
- Example: Polygon(...) or "POLYGON((0 0, 1 0, 1 1, 0 1, 0 0))" or
[Polygon(...), Polygon(...)]
-
- - datetime (Optional[Union[str, python_datetime.datetime, list]]): A
single datetime, RFC 3339-compliant timestamp,
- or a list of date-time ranges for filtering the items. The datetime
can be specified in various formats:
- - "YYYY" expands to ["YYYY-01-01T00:00:00Z", "YYYY-12-31T23:59:59Z"]
- - "YYYY-mm" expands to ["YYYY-mm-01T00:00:00Z",
"YYYY-mm-<last_day>T23:59:59Z"]
- - "YYYY-mm-dd" expands to ["YYYY-mm-ddT00:00:00Z",
"YYYY-mm-ddT23:59:59Z"]
- - "YYYY-mm-ddTHH:MM:SSZ" remains as ["YYYY-mm-ddTHH:MM:SSZ",
"YYYY-mm-ddTHH:MM:SSZ"]
- - A list of date-time ranges can be provided for multiple intervals.
- Example: "2020-01-01T00:00:00Z" or python_datetime.datetime(2020, 1,
1) or [["2020-01-01T00:00:00Z", "2021-01-01T00:00:00Z"]]
-
- - max_items (Optional[int]): The maximum number of items to return
from the search, even if there are more matching results.
- Example: 100
-
- - return_dataframe (bool): If True, return the result as a Spark
DataFrame instead of an iterator of PyStacItem objects.
- Example: True
-
- Returns:
- - Union[Iterator[PyStacItem], DataFrame]: An iterator of PyStacItem
objects or a Spark DataFrame that match the specified filters.
+ :param ids: A variable number of item IDs to filter the items.
+ Example: "item_id1" or ["item_id1", "item_id2"]
+ :param collection_id: The ID of the collection to search in.
+ Example: "aster-l1t"
+ :param bbox: A list of bounding boxes for filtering the items.
+ Each bounding box is represented as a list of four float values:
[min_lon, min_lat, max_lon, max_lat].
+ Example: [[-180.0, -90.0, 180.0, 90.0]] # This bounding box
covers the entire world.
+ :param geometry: Shapely geometry object(s) or WKT string(s) for
spatial filtering.
+ Can be a single geometry, WKT string, or a list of geometries/WKT
strings.
+ If both bbox and geometry are provided, geometry takes precedence.
+ Example: Polygon(...) or "POLYGON((0 0, 1 0, 1 1, 0 1, 0 0))" or
[Polygon(...), Polygon(...)]
+ :param datetime: A single datetime, RFC 3339-compliant timestamp,
+ or a list of date-time ranges for filtering the items. The
datetime can be specified in various formats:
+
+ - "YYYY" expands to ["YYYY-01-01T00:00:00Z",
"YYYY-12-31T23:59:59Z"]
+ - "YYYY-mm" expands to ["YYYY-mm-01T00:00:00Z",
"YYYY-mm-<last_day>T23:59:59Z"]
+ - "YYYY-mm-dd" expands to ["YYYY-mm-ddT00:00:00Z",
"YYYY-mm-ddT23:59:59Z"]
+ - "YYYY-mm-ddTHH:MM:SSZ" remains as ["YYYY-mm-ddTHH:MM:SSZ",
"YYYY-mm-ddTHH:MM:SSZ"]
+ - A list of date-time ranges can be provided for multiple intervals.
+
+ Example: "2020-01-01T00:00:00Z" or python_datetime.datetime(2020,
1, 1) or [["2020-01-01T00:00:00Z", "2021-01-01T00:00:00Z"]]
+ :param max_items: The maximum number of items to return from the
search, even if there are more matching results.
+ Example: 100
+ :param return_dataframe: If True, return the result as a Spark
DataFrame instead of an iterator of PyStacItem objects.
+ Example: True
+ :return: An iterator of PyStacItem objects or a Spark DataFrame that
match the specified filters.
"""
if collection_id:
client = self.get_collection(collection_id)
diff --git a/python/sedona/spark/stac/collection_client.py
b/python/sedona/spark/stac/collection_client.py
index 43e10eeba0..5efd6f93de 100644
--- a/python/sedona/spark/stac/collection_client.py
+++ b/python/sedona/spark/stac/collection_client.py
@@ -33,15 +33,15 @@ def get_collection_url(url: str, collection_id:
Optional[str] = None) -> str:
If the collection ID is provided and the URL starts with 'http' or
'https', the collection ID
is appended to the URL. Otherwise, an exception is raised.
- Parameters:
- - url (str): The base URL of the STAC collection.
- - collection_id (Optional[str]): The optional collection ID to append to
the URL.
+ Args:
+ url (str): The base URL of the STAC collection.
+ collection_id (Optional[str]): The optional collection ID to append to
the URL.
Returns:
- - str: The constructed collection URL.
+ str: The constructed collection URL.
Raises:
- - ValueError: If the URL does not start with 'http' or 'https' and a
collection ID is provided.
+ ValueError: If the URL does not start with 'http' or 'https' and a
collection ID is provided.
"""
if not collection_id:
return url
@@ -68,11 +68,11 @@ class CollectionClient:
This method ensures that certain attributes are nested under the
'properties' key
in the item dictionary. If the 'properties' key does not exist, it is
initialized.
- Parameters:
- - item_dict (dict): The dictionary representation of a STAC item.
+ Args:
+ item_dict (dict): The dictionary representation of a STAC item.
Returns:
- - dict: The updated item dictionary with specified attributes moved to
'properties'.
+ dict: The updated item dictionary with specified attributes moved
to 'properties'.
"""
# List of attributes to move to 'properties'
attributes_to_move = [
@@ -107,19 +107,19 @@ class CollectionClient:
"""
This function applies spatial and temporal filters to a Spark
DataFrame using safe parameterized operations.
- Parameters:
- - df (DataFrame): The input Spark DataFrame to be filtered.
- - bbox (Optional[list]): A list of bounding boxes for filtering the
items.
- Each bounding box is represented as a list of four float values:
[min_lon, min_lat, max_lon, max_lat].
- Example: [[-180.0, -90.0, 180.0, 90.0]] # This bounding box covers
the entire world.
- - geometry (Optional[list]): A list of geometry objects (Shapely or
WKT) for spatial filtering.
- If both bbox and geometry are provided, geometry takes precedence.
- - datetime (Optional[list]): A list of date-time ranges for filtering
the items.
- Each date-time range is represented as a list of two strings in ISO
8601 format: [start_datetime, end_datetime].
- Example: [["2020-01-01T00:00:00Z", "2021-01-01T00:00:00Z"]] # This
interval covers the entire year of 2020.
+ Args:
+ df (DataFrame): The input Spark DataFrame to be filtered.
+ bbox (Optional[list]): A list of bounding boxes for filtering the
items.
+ Each bounding box is represented as a list of four float
values: [min_lon, min_lat, max_lon, max_lat].
+ Example: [[-180.0, -90.0, 180.0, 90.0]] # This bounding box
covers the entire world.
+ geometry (Optional[list]): A list of geometry objects (Shapely or
WKT) for spatial filtering.
+ If both bbox and geometry are provided, geometry takes
precedence.
+ datetime (Optional[list]): A list of date-time ranges for
filtering the items.
+ Each date-time range is represented as a list of two strings
in ISO 8601 format: [start_datetime, end_datetime].
+ Example: [["2020-01-01T00:00:00Z", "2021-01-01T00:00:00Z"]] #
This interval covers the entire year of 2020.
Returns:
- - DataFrame: The filtered Spark DataFrame.
+ DataFrame: The filtered Spark DataFrame.
The function uses Spark SQL column operations and functions instead of
string concatenation
to prevent SQL injection vulnerabilities. Spatial and temporal
conditions are combined using OR logic.
@@ -281,14 +281,14 @@ class CollectionClient:
It then expands the date string to cover the entire time period for
that date.
- Parameters:
- - date_str (str): The date string to expand.
+ Args:
+ date_str (str): The date string to expand.
Returns:
- - list: A list containing the start and end datetime strings in ISO
8601 format.
+ list: A list containing the start and end datetime strings in ISO
8601 format.
Raises:
- - ValueError: If the date string format is invalid.
+ ValueError: If the date string format is invalid.
Examples:
- "2017" expands to ["2017-01-01T00:00:00Z", "2017-12-31T23:59:59Z"]
@@ -325,23 +325,20 @@ class CollectionClient:
Returns an iterator of items. Each item has the supplied item ID
and/or optional spatial and temporal extents.
This method loads the collection data from the specified collection
URL and applies
- optional filters to the data. The filters include:
- - IDs: A list of item IDs to filter the items. If not provided, no ID
filtering is applied.
- - bbox (Optional[list]): A list of bounding boxes for filtering the
items.
- - geometry (Optional[Union[str, BaseGeometry, List[Union[str,
BaseGeometry]]]]): Shapely geometry object(s) or WKT string(s) for spatial
filtering.
- Can be a single geometry, WKT string, or a list of geometries/WKT
strings.
- If both bbox and geometry are provided, geometry takes precedence.
- - datetime (Optional[Union[str, python_datetime.datetime, list]]): A
single datetime, RFC 3339-compliant timestamp,
- or a list of date-time ranges for filtering the items.
- - max_items (Optional[int]): The maximum number of items to return
from the search, even if there are more matching results.
-
- Returns:
- - Iterator[PyStacItem]: An iterator of PyStacItem objects that match
the specified filters.
- If no filters are provided, the iterator contains all items in the
collection.
-
- Raises:
- - RuntimeError: If there is an error loading the data or applying the
filters, a RuntimeError
- is raised with a message indicating the failure.
+ optional filters to the data.
+
+ :param ids: A list of item IDs to filter the items. If not provided,
no ID filtering is applied.
+ :param bbox: A list of bounding boxes for filtering the items.
+ :param geometry: Shapely geometry object(s) or WKT string(s) for
spatial filtering.
+ Can be a single geometry, WKT string, or a list of geometries/WKT
strings.
+ If both bbox and geometry are provided, geometry takes precedence.
+ :param datetime: A single datetime, RFC 3339-compliant timestamp,
+ or a list of date-time ranges for filtering the items.
+ :param max_items: The maximum number of items to return from the
search, even if there are more matching results.
+ :return: An iterator of PyStacItem objects that match the specified
filters.
+ If no filters are provided, the iterator contains all items in the
collection.
+ :raises RuntimeError: If there is an error loading the data or
applying the filters, a RuntimeError
+ is raised with a message indicating the failure.
"""
try:
df = self.load_items_df(bbox, geometry, datetime, ids, max_items)
@@ -377,25 +374,23 @@ class CollectionClient:
optional spatial and temporal filters to the data. The spatial filter
is applied using
a bounding box, and the temporal filter is applied using a date-time
range.
- Parameters:
- - bbox (Optional[list]): A list of bounding boxes for filtering the
items.
- Each bounding box is represented as a list of four float values:
[min_lon, min_lat, max_lon, max_lat].
- Example: [[-180.0, -90.0, 180.0, 90.0]] # This bounding box covers
the entire world.
- - geometry (Optional[Union[str, BaseGeometry, List[Union[str,
BaseGeometry]]]]): Shapely geometry object(s) or WKT string(s) for spatial
filtering.
- Can be a single geometry, WKT string, or a list of geometries/WKT
strings.
- If both bbox and geometry are provided, geometry takes precedence.
- Example: Polygon(...) or "POLYGON((0 0, 1 0, 1 1, 0 1, 0 0))" or
[Polygon(...), Polygon(...)]
- - datetime (Optional[Union[str, python_datetime.datetime, list]]): A
single datetime, RFC 3339-compliant timestamp,
- or a list of date-time ranges for filtering the items.
- Example: "2020-01-01T00:00:00Z" or python_datetime.datetime(2020, 1,
1) or [["2020-01-01T00:00:00Z", "2021-01-01T00:00:00Z"]]
-
- Returns:
- - DataFrame: A Spark DataFrame containing the filtered items. If no
filters are provided,
- the DataFrame contains all items in the collection.
-
- Raises:
- - RuntimeError: If there is an error loading the data or applying the
filters, a RuntimeError
- is raised with a message indicating the failure.
+ :param ids: A variable number of item IDs to filter the items.
+ Example: "item_id1" or ["item_id1", "item_id2"]
+ :param bbox: A list of bounding boxes for filtering the items.
+ Each bounding box is represented as a list of four float values:
[min_lon, min_lat, max_lon, max_lat].
+ Example: [[-180.0, -90.0, 180.0, 90.0]] # This bounding box
covers the entire world.
+ :param geometry: Shapely geometry object(s) or WKT string(s) for
spatial filtering.
+ Can be a single geometry, WKT string, or a list of geometries/WKT
strings.
+ If both bbox and geometry are provided, geometry takes precedence.
+ Example: Polygon(...) or "POLYGON((0 0, 1 0, 1 1, 0 1, 0 0))" or
[Polygon(...), Polygon(...)]
+ :param datetime: A single datetime, RFC 3339-compliant timestamp,
+ or a list of date-time ranges for filtering the items.
+ Example: "2020-01-01T00:00:00Z" or python_datetime.datetime(2020,
1, 1) or [["2020-01-01T00:00:00Z", "2021-01-01T00:00:00Z"]]
+ :param max_items: The maximum number of items to return from the
search.
+ :return: A Spark DataFrame containing the filtered items. If no
filters are provided,
+ the DataFrame contains all items in the collection.
+ :raises RuntimeError: If there is an error loading the data or
applying the filters, a RuntimeError
+ is raised with a message indicating the failure.
"""
try:
df = self.load_items_df(bbox, geometry, datetime, ids, max_items)
@@ -422,18 +417,17 @@ class CollectionClient:
optional spatial and temporal filters to the data. The filtered data
is then saved
to the specified output path in Parquet format.
- Parameters:
- - output_path (str): The path where the Parquet file will be saved.
- - spatial_extent (Optional[SpatialExtent]): A spatial extent object
that defines the
- bounding box for filtering the items. If not provided, no spatial
filtering is applied.
- - temporal_extent (Optional[TemporalExtent]): A temporal extent object
that defines the
- date-time range for filtering the items. If not provided, no
temporal filtering is applied.
- To match a single datetime, you can set the start and end datetime
to the same value in the datetime.
- Here is an example: [["2020-01-01T00:00:00Z",
"2020-01-01T00:00:00Z"]]
-
- Raises:
- - RuntimeError: If there is an error loading the data, applying the
filters, or saving the
- DataFrame to Parquet format, a RuntimeError is raised with a message
indicating the failure.
+ :param ids: A list of item IDs to filter the items. If not provided,
no ID filtering is applied.
+ :param output_path: The path where the Parquet file will be saved.
+ :param bbox: A bounding box for filtering the items. If not provided,
no spatial filtering is applied.
+ :param geometry: Shapely geometry object(s) or WKT string(s) for
spatial filtering.
+ If both bbox and geometry are provided, geometry takes precedence.
+ :param datetime: A temporal extent that defines the
+ date-time range for filtering the items. If not provided, no
temporal filtering is applied.
+ To match a single datetime, you can set the start and end datetime
to the same value in the datetime.
+ Example: [["2020-01-01T00:00:00Z", "2020-01-01T00:00:00Z"]]
+ :raises RuntimeError: If there is an error loading the data, applying
the filters, or saving the
+ DataFrame to Parquet format, a RuntimeError is raised with a
message indicating the failure.
"""
try:
df = self.get_dataframe(
@@ -457,11 +451,11 @@ class CollectionClient:
The expected input schema of the loaded dataframe (df) can be found
here:
https://sedona.apache.org/latest-snapshot/api/sql/Stac/#usage
- Parameters:
- - df (DataFrame): The input DataFrame with an assets column.
+ Args:
+ df (DataFrame): The input DataFrame with an assets column.
Returns:
- - DataFrame: The DataFrame with a consistent schema for the assets
column.
+ DataFrame: The DataFrame with a consistent schema for the assets
column.
"""
from pyspark.sql.functions import col, explode, struct
