Hi all,
A short follow-up to the earlier announcement of Ignite PR Checker [1]. A
few
people asked how it compares to the teamcity-bot day to day, so here are the
differences that actually change the workflow.
1) A stricter, less noisy definition of "blocker".
A test is reported as broken-by-your-PR only if it fails in ALL of the
last N
finished runs of its suite on the branch (default 3), is clean across
the last
~100 master runs, AND still fails in the most recent finished run — a
passing
re-run clears it automatically. That removes almost all of the single-run
flapping. Concrete example from this week: PR #13296 had 19 failing
suites and
~35 failing tests; the checker reported 0 blockers, because every one of
them
either fails on master too or passes on re-run. The signal is a handful
of
tests, not the raw red count.
2) You work entirely inside the PR — under your own accounts, no bot
identity.
Comment `/run-all` (or `/run-all top`, or `/top`) on a pull request and
the whole
RunAll chain is queued under YOUR TeamCity token; the verdict comes back
as a PR
comment and/or a JIRA visa from YOUR account. There is no shared
technical
account to provision and nothing is attributed to a bot — every action
is the
person who asked, authenticated with their own ci2/JIRA/GitHub tokens.
3) It re-runs the noise for you.
Opt-in: when a run you triggered finishes with blockers or broken suites
(timeout/OOM/crash), those suites are re-run automatically (up to 2
attempts),
and the PR comment narrates the waves with a live ETA. A green re-run
clears the
blocker before the verdict is finalised.
4) It catches what a green build can hide.
- broken suites (compilation/timeout/crash) are surfaced, not silently
dropped;
- a suite that ran far fewer tests than the same suite on master is
flagged —
tests that never ran can't fail — using master as the baseline rather
than
TeamCity's pinned-reference metric, which false-alarms on PR branches
long
after a legitimate test-count change;
- a run that was cancelled after getting through most of its suites is
still
analysed (as partial), instead of showing "no run";
- if commits were pushed after the analysed run, the verdict is marked
stale
with a one-click re-run.
5) No infrastructure to babysit.
No database, a single small self-updating process, Apache-2.0. Tokens
are stored
encrypted and only while a standing option is switched on.
Live for contributors: https://ignite-pr-checker.is-a.dev — log in with a
ci2
access token. The flaky-tests board and the status page are public (no
login).
Source and issues: https://github.com/anton-vinogradov/ignite-pr-checker.
It is meant to complement the existing tooling, not replace anything
overnight —
feedback and PRs are very welcome.
Thanks,
Anton
[1] <link to your first announcement thread>
On Sat, Jul 11, 2026 at 12:09 AM Anton Vinogradov <[email protected]> wrote:
> Hi Igniters,
>
> I've built a small tool that answers one question about a pull request:
> which test failures are actually caused by the change, and which were
> already broken or flaky before it.
>
> https://ignite-pr-checker.is-a.dev
>
> What it does:
>
> * Analyses the PR's latest RunAll on TeamCity (ci2) and splits the failed
> tests into:
> - blockers: fail consistently in the branch's recent runs and never fail
> in the last ~100 master runs — attributed to the PR;
> - recently started failing: a fresh break to watch, not yet steady;
> - filtered noise: pre-existing on master or flapping on the same code.
> * Shows suites that didn't produce a reliable run (compilation, execution
> timeout, OOM, interrupted chain) separately instead of mining them for
> bogus failures.
> * Verdicts update live while a RunAll is still running — failures of
> already-finished suites are folded in, no need to wait ~4h for the chain.
> * Groups blockers by root cause (one broken thing often shows up as dozens
> of failing tests), with pass/fail history strips per test.
> * Can trigger RunAll / re-run individual suites or whole sections right
> from the page, and post the verdict ("visa") as a comment to the
> IGNITE ticket — manually, once on run completion, or automatically for
> every run you trigger (opt-in).
> * A "flaky tests" page keeps a fix-master queue: tests currently failing
> on master, ranked by fail rate, linked to the failing master runs.
>
> How it relates to the TC Bot: same core idea, and on identical builds the
> verdicts match the bot's. The differences are that it watches all open PRs
> continuously (no per-request visa needed), reads TeamCity on the fly with
> no database, and anchors every verdict to the newest finished run.
>
> Login is your own ci2 access token; it stays encrypted in your session
> cookie and is never stored server-side. Everything runs under your account.
>
> Source (Apache-2.0): https://github.com/anton-vinogradov/ignite-pr-checker
>
> The tool is young and I'd really appreciate feedback — issues, discussions
> in the repo, or just a reply here. If you find a case where the verdict is
> wrong or confusing, that's exactly what I want to hear about.
>
> Thanks,
> Anton
>
