sunchao commented on code in PR #5924: URL: https://github.com/apache/datafusion-comet/pull/5924#discussion_r4011591942
########## docs/source/contributor-guide/bug_triage.md: ########## @@ -76,6 +76,65 @@ A bug should be escalated to a higher priority if: - A `priority:medium` bug is reported by multiple users or affects a common workload → consider escalating to `priority:high` - A `priority:low` CI flake is blocking PR merges consistently → escalate to `priority:medium` +- A bug turns out to be a `regression` from the most recent release → consider escalating one + level, because users who upgrade are exposed to it without changing anything on their side + +## Regression Label + +| Label | Description | +| ------------ | ------------------------------------------------------- | +| `regression` | A bug that did not affect the most recent Comet release | + +Apply `regression` to a bug when a workload that behaved correctly on the most recent release +behaves incorrectly on `main`. That covers wrong results, a new failure, a new crash, and the case +where an expression used to fall back to Spark (and was therefore correct) and now runs natively +with a wrong answer. It also covers a loss of safety: a query that failed with a clear error on the +last release and now returns silently wrong data is a regression, even though it never produced the +right answer on either version. + +A defect that already shipped in the most recent release is **not** a regression, no matter how +recently it was reported. Neither is a defect in a feature added after that release: a workload +running on the release cannot reach code that did not exist yet. + +`regression` is orthogonal to priority. A regression still gets the priority label its symptoms +earn, and it is an escalation trigger rather than a priority of its own. It applies only to bugs. + +### Determining the Comparison Point + +Always compare against the most recent release **tag**, resolved at triage time rather than +hard-coded, so the comparison point moves forward as Comet ships: + +```bash +LATEST_RELEASE=$(gh release view --repo apache/datafusion-comet --json tagName --jq .tagName) +git fetch --tags +git log -1 --format=%cI "$LATEST_RELEASE" +``` + +Compare against the **tag's commit date**, not the release's publication date — commits that land +between the two are not in the release. + +### Establishing Regression Status + +Work through these in order and stop at the first definite answer: + +1. **Issue creation date.** An issue opened before the tag was cut describes behavior that shipped + in that release. Not a regression. +2. **Is the defective code present at the tag?** `git show "$LATEST_RELEASE:<path>"`, + `git grep <pattern> "$LATEST_RELEASE"`, or `git diff "$LATEST_RELEASE"..HEAD -- <path>`. If the + defective logic is there verbatim, not a regression. Review Comment: ### Correctness [P2] Could we check released reachability before treating identical defective code as conclusive? An existing Rust kernel can be unreachable on the release because the expression falls back to Spark, then become reachable after a serde or support-level change. The kernel is still verbatim at the tag, so this rule stops at `not a regression` and never reaches the fallback-to-native case that the definition explicitly counts as a regression. The same shortcut can bypass the dependency comparison for unchanged callers. Please require evidence that the affected path actually behaved the same on the release, or continue as `unclear`, in both this guide and Step 4b of the skill. ########## docs/source/contributor-guide/bug_triage.md: ########## @@ -76,6 +76,65 @@ A bug should be escalated to a higher priority if: - A `priority:medium` bug is reported by multiple users or affects a common workload → consider escalating to `priority:high` - A `priority:low` CI flake is blocking PR merges consistently → escalate to `priority:medium` +- A bug turns out to be a `regression` from the most recent release → consider escalating one + level, because users who upgrade are exposed to it without changing anything on their side + +## Regression Label + +| Label | Description | +| ------------ | ------------------------------------------------------- | +| `regression` | A bug that did not affect the most recent Comet release | + +Apply `regression` to a bug when a workload that behaved correctly on the most recent release +behaves incorrectly on `main`. That covers wrong results, a new failure, a new crash, and the case +where an expression used to fall back to Spark (and was therefore correct) and now runs natively +with a wrong answer. It also covers a loss of safety: a query that failed with a clear error on the +last release and now returns silently wrong data is a regression, even though it never produced the +right answer on either version. + +A defect that already shipped in the most recent release is **not** a regression, no matter how +recently it was reported. Neither is a defect in a feature added after that release: a workload +running on the release cannot reach code that did not exist yet. + +`regression` is orthogonal to priority. A regression still gets the priority label its symptoms +earn, and it is an escalation trigger rather than a priority of its own. It applies only to bugs. + +### Determining the Comparison Point + +Always compare against the most recent release **tag**, resolved at triage time rather than +hard-coded, so the comparison point moves forward as Comet ships: + +```bash +LATEST_RELEASE=$(gh release view --repo apache/datafusion-comet --json tagName --jq .tagName) +git fetch --tags +git log -1 --format=%cI "$LATEST_RELEASE" +``` + +Compare against the **tag's commit date**, not the release's publication date — commits that land +between the two are not in the release. + +### Establishing Regression Status + +Work through these in order and stop at the first definite answer: + +1. **Issue creation date.** An issue opened before the tag was cut describes behavior that shipped + in that release. Not a regression. Review Comment: ### Correctness [P2] Could we use the issue date only as a hint and verify the affected behavior at the tag before concluding `not a regression`? A bug can be reported against `main` before the release commit is cut on a separate branch, without that change being included in the release. The current `1.0.0` tag and `main` do in fact have [diverged histories](https://github.com/apache/datafusion-comet/compare/3a7a2c437cc771621b6040a308657573dbc1b9c2...812b4c318137ebf36a764b29d6ab3edc05cf1201). An older issue can also have been fixed in the release and then recur. This unconditional first exit skips every source/reproducer check and can miss the regression label and escalation for either case. Please make the same correction in Step 4b of the skill. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
