morningman opened a new pull request, #4413:
URL: https://github.com/apache/doris-website/pull/4413
## Why
`japanese-translation.yml` and `korean-translation.yml` are about 210
identical lines with five logical differences, and three of those (the label
check before model calls, the exact `paths` input, the fixed locale) had been
added to the Korean side only. Comparing them also turned up two behaviours
worth fixing:
- **Master movement discarded whole batches after the model calls.**
Generation checked master only after the last page, and any move discarded
everything. Two of the last 12 Japanese runs failed this way: on 10-18 after 50
minutes of generation; on 10-19 after 25 minutes, although master had already
moved before generation started (the generate job waited 40 minutes for a
runner after discovery). Most of those moves were unrelated merges.
- **The Japanese workflow's `locale` input was a second way to run Korean
generation**, with the Japanese defaults: manual `generate`, no label check,
`japanese-translation-*` artifacts.
## What changed
### 1. One reusable workflow
`.github/workflows/translation-run.yml` (`workflow_call`) holds the
discovery and generation jobs. The Japanese and Korean workflows keep only
their triggers, manual defaults, the lock and one call with their locale and
artifact prefix. Secrets are passed explicitly, not inherited. Mode and filters
are set once as workflow-level env instead of two to four times per file.
Artifact names are unchanged (`japanese-translation-plan-<run>`,
`korean-translation-generation-<run>`, …).
### 2. Same inputs and checks for both languages
- Japanese gains the exact `paths` input and the label check before auth
loading, which is now `translate-$I18N_LOCALE`.
- The Japanese `locale` input is removed. Each workflow runs only its own
language.
### 3. Unrelated master merges no longer discard a batch
New `scripts/i18n/master-drift.cjs` decides what a master move means for a
batch:
- **Unrelated merge:** the batch stays valid. The branch stays on the source
commit, the PR body records the newer master, and the PR checks run against
current master.
- **Change to a batch page** (English source, target or state shard): that
page is dropped and stays pending for the next run. A Dev/4.x copy is dropped
with its primary. The publisher stops instead.
- **Change to translation configuration, routing identity or the pipeline:**
the batch is discarded before any further model call. This covers the locale's
scope, glossary, protection exceptions, inventory and top-level state files,
`translations/config.json`, `config/i18n.js`, `docusaurus.config.js`,
`versions.json` and `scripts/i18n/`. A master that no longer contains the
source commit is discarded too.
When the checks run:
- Generation checks master before the first model call, before each page and
after the last one. The publisher checks again before pushing.
- The generate job checks out current master instead of the discovery commit
and replans, so a long runner queue no longer dooms the batch before it starts.
### 4. The replan reuses an unchanged discovery plan
`workflow-scan.cjs` records what a plan depends on: the checkout, the
open-PR reservations and the filters. The generate job reuses the downloaded
discovery plan when all three are unchanged, which skips the 2.5–4.7 minute
recount of every native document. The planner is deterministic.
### 5. Lock name
The shared concurrency group is renamed from `japanese-translation-ja` to
`docs-translation`.
### Considered and not adopted
A `blob:none` checkout. Reconciliation reads the managed files of every open
translation PR with `git show`. In a partial clone each of those blobs is a
separate network fetch, which costs more than the ~30 s the checkout would save
once several translation PRs are open.
## Verification
- All 21 translation test suites listed in Build Check pass on Node 20.20.2
(219 tests).
- New tests cover:
- drift detection against a real local bare remote: unrelated merge, page
change, moved source, configuration change, rewritten master;
- generation stopping before any model call on a batch-wide change, both
before the batch and before a page;
- a page changed before its turn being skipped, and a page changed after
its model call being dropped with its Dev/4.x copy;
- the manifest being kept when a batch is discarded after the last page;
- the plan fingerprint, and the replan reusing a discovery plan without
recounting;
- the label check failing for a missing label;
- the caller/shared workflow contract.
- Mutation checks: each of these breaks a test — removing the per-page
check, moving it inside the per-page `catch`, keeping copies of dropped pages,
turning rename detection back on, removing the ancestor check.
- `scripts/i18n/simulate-workflow.cjs` (local bare remote, GitHub CLI and
Codex mocks) passes for the default, incremental and ten-page fallback
scenarios, and for the new `I18N_SIM_MASTER_DRIFT=unrelated|page|config`.
- `unrelated` publishes and notes the newer master in the PR body.
- `page` skips the page with no model request.
- `config` stops before any model request.
- The default scenario failed on master before this PR (three requests
instead of two). Its page, `engines.md`, declares `"language": "en"`, and the
Codex mock copied it. The mock now localizes a declared front matter language,
as a real model does.
- actionlint 1.7.12 reports nothing new. The two remaining findings also
exist on master: the `queue` concurrency key (newer than actionlint) and the
age of `actions/setup-python@v4`.
## Merge note
Merge while no Japanese or Korean translation run is queued or running.
Because the lock is renamed, a run started under the old name and a run started
under the new one could otherwise overlap. Today's scheduled runs start at
16:23 (Japanese) and 16:43 UTC (Korean).
🤖 Generated with [Claude Code](https://claude.com/claude-code)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
