aminghadersohi opened a new pull request, #35018: URL: https://github.com/apache/superset/pull/35018
<!--- Please write the PR title following the conventions at https://www.conventionalcommits.org/en/v1.0.0/ Example: fix(dashboard): load charts correctly --> ### SUMMARY <!--- Describe the change below, including rationale and design decisions --> This PR enhances the BaseDAO layer to support structured programmatic filtering, enabling external API integrations (like MCP services) to query Superset data without Flask-AppBuilder dependencies. The changes are fully backward compatible with zero breaking changes. #### Key Enhancements **1. Structured Filtering System** - Added `ColumnOperator` model with 14 filter operators (eq, neq, gt, gte, lt, lte, like, ilike, in, nin, null, nnull, sw, nsw) - Enables type-safe, programmatic filtering without FAB web dependencies - Supports complex queries with multiple filters and operators **2. Enhanced Query Methods** - New `list()` method with comprehensive filtering, sorting, pagination, and search - New `count()` method for filtered record counting - Schema discovery via `get_filterable_columns_and_operators()` for API documentation **3. UUID Support & Flexible ID Handling** - Automatic UUID type conversion for gradual migration to UUID primary keys - Support for non-standard ID field names across models - Enhanced `find_by_id()`/`find_by_ids()` with string/UUID flexibility **4. Model-Specific Extensions** - ChartDAO: Added filterable fields for `tags`, `owner`, `viz_type`, `datasource_name` - DashboardDAO: Added filterable fields for `tags`, `owner`, `published`, `favorite` - DatasetDAO: Added filterable fields for `tags`, `owner` #### Design Rationale The existing DAO layer lacked programmatic filtering capabilities needed by external services. This enhancement: - Maintains complete backward compatibility (all existing code continues working) - Follows Superset's patterns and conventions - Provides type safety with MyPy compliance - Enables self-documenting APIs through schema discovery - Supports the ongoing UUID migration initiative ### BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF <!--- Skip this if not applicable --> N/A - Backend infrastructure change with no UI impact ### TESTING INSTRUCTIONS <!--- Required! What steps can be taken to manually verify the changes? --> 1. **Run the comprehensive test suite:** ```bash # Run the new DAO tests (2400+ lines of test coverage) pytest tests/unit_tests/dao/ ``` 2. **Verify backward compatibility:** ```bash # Existing DAO tests should all pass unchanged pytest tests/unit_tests/daos/ pytest tests/integration_tests/charts/ pytest tests/integration_tests/dashboards/ pytest tests/integration_tests/datasets/ ``` 3. **Test programmatic filtering (Python shell example):** ```python from superset.daos.chart import ChartDAO from superset.daos.base import ColumnOperator, ColumnOperatorEnum # Test filtering charts, total = ChartDAO.list( column_operators=[ ColumnOperator(col="viz_type", opr=ColumnOperatorEnum.eq, value="line") ], page_size=10 ) # Test schema discovery schema = ChartDAO.get_filterable_columns_and_operators() assert "viz_type" in schema ``` 4. **Verify type checking:** ```bash pre-commit run mypy ``` ### ADDITIONAL INFORMATION <!--- Check any relevant boxes with "x" --> <!--- HINT: Include "Fixes #nnn" if you are fixing an existing issue --> - [ ] Has associated issue: - [ ] Required feature flags: - [ ] Changes UI - [ ] Includes DB Migration (follow approval process in [SIP-59](https://github.com/apache/superset/issues/13351)) - [ ] Migration is atomic, supports rollback & is backwards-compatible - [ ] Confirm DB migration upgrade and downgrade tested - [ ] Runtime estimates and downtime expectations provided - [x] Introduces new feature or API - New DAO methods: `list()`, `count()`, `get_filterable_columns_and_operators()` - New filtering system via `ColumnOperator` and `ColumnOperatorEnum` - Fully backward compatible - no breaking changes - [ ] Removes existing feature or API **Additional Notes:** - This PR is the foundation for the MCP (Model Context Protocol) service integration - Comprehensive test coverage: 2400+ lines of tests covering all operators and edge cases - Follows Superset's ongoing initiatives: UUID migration, type safety (MyPy), and API standardization - Zero performance impact on existing code paths - new features are opt-in via optional parameters -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
