Community · E2 · artifact verified
Classify CI failures as flaky or real
A GitHub Action that classifies test failures as regression, flaky, environment, or unknown: deterministic signals first, one structured Jev choice second, and a local policy that never auto-reruns tests or masks failures.
01 · Role in the system
What Jev does here
A red build is not always a product regression. The Action collects current results and run history, computes deterministic signals itself, and asks Jev for a structured failure_type choice - with calibrated probability - through a provider factory that supports the Vercel AI Gateway, the native TypeSafe endpoint, or any compatible endpoint. A local policy then applies the rules; the classification is guidance for humans and downstream steps, not a silent greenwash: the Action never auto-reruns tests and never masks failures. Ships as a released GitHub Action with CI, unit tests over the decision mode, and configurable provider and signals.
02 · Control boundary
Where Jev sits
Deterministic signals plus one structured failure-type choice; a local policy owns the verdict, and the tool refuses to rerun or mask anything.
Code owns the loop, permissions, thresholds, validation, and side effects. Jev owns only the bounded judgments described above.
03 · Known limits
What this evidence does not prove
- Classification quality depends on the history the collector can see; a repository with sparse runs gives the model less to work with.
- Judgment calls leave the repository only through the configured provider; custom-compatible endpoints must be HTTPS.
- No accuracy benchmark is published in the README; the unit tests cover the decision plumbing, not classification precision.
04 · Attribution
Public sources
This is a Community record: the project was published by a third-party community author.
- JevForge ↗Community · github · public · checked 2026-09-25