Community · E2 · artifact verified
Let DSPy compose rule graphs with typed judge fallbacks
rulesmith lets DSPy build decision graphs automatically: given a task and labeled examples, it searches for a small rules program - cheap field tests settle what they can, a judge answers typed questions on the rest, GEPA keeps what scores better.
01 · Role in the system
What Jev does here
The compiled rules file reads like honest triage: cheap tests on input fields first ('has_company_logo == 0 and length of company_profile == 0: fraudulent'), and an always-fallback asking the judge 'Is this job posting a scam?' with typed fraudulent/genuine criteria. On job-fraud detection the same 4B judge under rulesmith beat DSPy's tuned classifier by 6.7 points using 29% of the model calls, and tied it on clothing reviews with 6%. The benchmarks doc publishes every dataset - including where rulesmith does not help: one-sentence inputs tie DSPy, and a TF-IDF classifier still beats it on job postings. An independent research prototype, explicitly not affiliated with TypeSafe; judges can be local or hosted.
02 · Control boundary
Where Jev sits
Jev as the residual judge behind compiled rules: DSPy+GEPA search composes cheap field tests with typed fallback questions, so most inputs never reach a model call and the judge handles only the genuinely ambiguous residue.
Code owns the loop, permissions, thresholds, validation, and side effects. Jev owns only the bounded judgments described above.
03 · Known limits
What this evidence does not prove
- Independent research prototype, not affiliated with TypeSafe; 6 stars.
- Published datasets show cases where rulesmith does not help (one-sentence inputs, TF-IDF wins).
- Judge can be local or hosted; hosted-Jev numbers are author-run.
04 · Attribution
Public sources
This is a Community record: the project was published by a third-party community author.
- bradAGI ↗Community · github · public · checked 2026-10-13