rusackas opened a new issue, #32625: URL: https://github.com/apache/superset/issues/32625
*Please make sure you are familiar with the SIP process documented* [here](https://github.com/apache/superset/issues/5602). The SIP will be numbered by a committer upon acceptance. ## [DRAFT - SIP] Proposal for a new Apache Superset Developer Portal<title> ### Motivation Apache Superset is evolving to support a plugin-based architecture as part of [SIP-151](https://github.com/apache/superset/issues/31932). With this change, there is a need for comprehensive and well-structured documentation tailored to developers building extensions, integrations, and customizations. The current documentation is fragmented across multiple sources, making it difficult for contributors and developers to find relevant information efficiently. It gives no distinction between, or single entry to, documentation created for users, developers, and administrators. ### Proposed Change This SIP proposes the creation of a **Superset Developer Portal**, which will serve as a single, authoritative source of documentation for the Superset SDK and plugin ecosystem. Additionally, it will lay the foundation for an eventual overhaul of Superset’s broader documentation, restructuring content into dedicated sections for Developers, Users, Contributors, and Installation/Deployment. A key goal of this initiative is to **grow the Superset community** by making it easier for both Superset and Superset plugin developers to contribute, and by empowering users and administrators to easily find all the documentation they need to successfully run and use Superset. This SIP aims to align on the requirements and select the best tooling for implementing a modern, centralized Developer Portal for Apache Superset. After doing some tool comparison (see grid below), the tool we’re proposing is Docusaurus. While by stroke of luck that happens to be the tool we’re using on our current docs site, it also seems to be the only one to meet the broad requirements we have. Key requirements of the Superset Developer Portal include: - Providing well-organized, **text and code-snippet-based documentation**, inspired by the [VS Code extension SDK doc](https://code.visualstudio.com/api)s. - Serving as a **single repository** for documentation, consolidating: - GitHub Wiki - Current documentation site - Various README files - _…and more!_ - Supporting **MDX** (Markdown with JSX) to embed interactive components in documentation, reducing reliance on screenshots. - Allowing embedding or replacement of **React Storybook**, enabling developers to interact with UI components directly. - Integration with **automated testing (E2E tests)** to capture and embed static screenshots, ensuring documentation stays up to date. - Implement **multi-instance versioning** to allow users to reference the correct documentation relative to: - Superset core versions - Monorepo package versions (e.g., components, themes, SDK, etc.) - Site search, or preferably an **AI-powered search** tool (if ASF guidelines permit) for efficient navigation. [side note, Kapa.ai now has a DPA and features in place to accomplish this] - An improved contributor experience by providing: - A way to **contribute via PRs** to markdown/MDX files. - A mechanism for users to highlight and suggest changes interactively. - **Automated documentation generation** using JSDoc/docstrings, and sidecar scripts, and integrate an **API playground** similar to Swagger UI. - Support for **required tooling** (GHA deployment, Matomo analytics, etc) ### New or Changed Public Interfaces - Introduction of a new **Developer Portal** website. - Potential **deprecation of Superset’s GitHub Wiki** in favor of structured documentation. - Inclusion of **interactive documentation components** using MDX and potentially replacing Storybook. - **Automated API documentation** generation from JSDoc/docstrings. ### New dependencies - **Docusaurus 3.0** (based on tool evaluation below) - **MDX support** for embedding interactive components. - **Search indexing tool**, potentially AI-powered (Kapa.ai based on recent ASF approvals). - **Automated screenshot and API doc generation tools** (TBD). ### Migration Plan and Compatibility - The Developer Portal will initially launch alongside existing documentation. Depending on the tool selected, this new portal may be included with the current documentation deployment, or require its own publication lifecycle (e.g. GitHub actions) and may be hosted on a new subdomain (subject to ASF Infra guidance and approvals). It may be plausible to host the new site at https://superset.apache.org/docs/, which is not used by the current documentation site (https://superset.apache.org/docs/intro/ is the current entry point). - The Developer Portal will initially be populated with documentation around the new Extensions Architecture, related SDKs, and contribution guidelines for the Developer Portal itself. - Documentation from the GitHub Wiki, Readme files, and the legacy Docs site will later be migrated to the new platform, and thus deprecating/removing those other resources, adding redirects where warranted. - Versioning strategies will be determined to ensure backward-compatibility and that users are consuming docs relevant to the version they have installed. We will focus early efforts on versioning component parts of the docs relative to their NPM/Python package counterparts (e.g. plugins, packages, SDK versions). - Contributions will be continue to allow markdown/MDX-based updates via GitHub PRs, but we will assess any/all alternative means of gathering documentation suggestions/feedback/contribution ### Rejected Alternatives - **Keeping the current documentation** structure: This was deemed insufficient due to fragmentation and difficulty in navigation. - **Using only React Storybook** for interactive documentation: Storybook alone does not address broader documentation needs. - **Maintaining multiple sources of truth**: A single Developer Portal will be more maintainable and user-friendly. - **All other tools** seen in this table besides Docusaurus. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
