i-love-thinking opened a new issue, #32741:
URL: https://github.com/apache/superset/issues/32741
## [SIP] Proposal for Migrating from npm + Lerna to pnpm
### Motivation
Apache Superset’s frontend currently relies on **npm** for dependency
management and **Lerna** for monorepo publishing and workspace organization.
While Lerna has served us well, it has experienced periods of limited
maintenance and slower updates. Additionally, we still face challenges with
large `node_modules` directories, long install times, and complex multi-package
workflows.
**pnpm** offers a compelling alternative that consolidates both package
management and monorepo workspace features in a single tool. Some key benefits
include:
- **Smaller `node_modules` footprint**: pnpm’s content-addressable store
greatly reduces disk usage.
- **Faster installations**: pnpm’s symlink-based architecture speeds up the
dependency installation process.
- **Actively maintained**: pnpm is supported by the npm team, ensuring
ongoing development and timely fixes.
- **Built-in workspace management**: pnpm can replace Lerna’s monorepo
features without additional overhead.
### Description of the problem to be solved
1. **Large `node_modules` directories**: Both local development environments
and CI pipelines suffer from bloated install footprints.
2. **Long install times**: As dependencies grow, `npm install` becomes
slower and more resource-intensive.
3. **Maintenance overhead**: Using two tools—npm and Lerna—for monorepo
management can lead to redundant configuration and potential version
mismatches.
4. **Future-proofing**: Lerna’s maintenance status has fluctuated, creating
uncertainty for the project’s long-term needs.
### Proposed Change
1. **Adopt pnpm Workspaces**
- Migrate from Lerna’s monorepo setup (`lerna.json` and associated
scripts) to a **`pnpm-workspace.yaml`** configuration.
- Remove or deprecate Lerna-specific commands in favor of pnpm’s built-in
workspace features.
2. **Replace npm with pnpm**
- Update all `npm install` and `npm run` scripts to use **pnpm**.
- Validate and adjust any scripts or hooks to ensure they function under
pnpm.
3. **Integrate Changesets for Versioning & Publishing** (optional but
recommended)
- If we still want automated changelog generation and version bumping,
incorporate [Changesets](https://github.com/changesets/changesets).
- Configure pnpm to run Changesets during CI to publish packages.
4. **Update CI/CD**
- Switch CI steps from `npm install` to `pnpm install`.
- Evaluate caching strategies (e.g., caching the pnpm store) to maximize
build performance.
5. **Documentation**
- Provide clear migration steps for developers (e.g., uninstall Lerna
globally if used, install pnpm, and switch to `pnpm` commands).
- Update any references to Lerna or npm in project documentation,
READMEs, and onboarding guides.
### New or Changed Public Interfaces
- **No direct impact on Superset’s REST endpoints, dashboards, or CLI** is
anticipated.
- Developer-facing scripts (e.g., `lerna publish` or `npm run build`) will
be replaced by `pnpm run publish` or `pnpm run build`, necessitating
documentation updates.
### New Dependencies
- **pnpm**
- License: MIT
- Actively maintained by the npm team and open-source community.
- **(Optional) Changesets**
- License: MIT
- Actively maintained and widely adopted for multi-package versioning and
changelog generation.
### Migration Plan and Compatibility
1. **Local Environment**
- Developers will install pnpm globally (`corepack enable pnpm` or
others).
- Replace Lerna commands (e.g., `lerna bootstrap`, `lerna publish`) with
pnpm equivalents (`pnpm install`, `pnpm publish`).
2. **CI Environments**
- Update pipelines to install pnpm and run `pnpm install` instead of `npm
install` or `lerna bootstrap`.
- Validate build scripts, tests, and publishing flows under pnpm.
3. **Backward Compatibility**
- Removing Lerna does not affect final build artifacts or runtime usage.
- The main changes are internal to developer workflows and CI processes.
### Rejected Alternatives
1. **Continue using npm + Lerna**
- Maintains the status quo but does not address large `node_modules`,
slower install times, or the uncertain maintenance status of Lerna.
2. **Nx or Turborepo**
- Both offer monorepo solutions, but either introduce additional layers
of complexity or lack out-of-the-box publishing features (like Nx or
Turborepo).
---
By migrating from **npm + Lerna** to **pnpm**, Apache Superset can simplify
its monorepo workflows, reduce disk usage, speed up installation times, and
rely on a single, well-maintained solution for both dependency management and
workspace organization. This proposal aims to streamline developer workflows
and position Superset for future growth and maintenance.
#32692
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
