Terminal
Terminal/Skill
Audit existing AI evaluation evidence, graders, datasets, and run reliability before trusting a score or optimizing an application. Not a generic code or security review.
By Confident AI (DeepEval)
Works with
npx skills add confident-ai/eval-skills --skill eval-auditRead the operating agreement. Inspect artifacts; do not produce findings from a checklist alone.
For bundled local tools and exact commands, read toolkit setup. Run the workspace installer for this skill; it preserves existing customized files.
Find the runner command, actual app entrypoint, cases, reference origins, full traces, human labels, grader definition/version, and raw results. Reuse connected observability tools or local exports. For the managed track read hosted workflow guidance.
Classify findings as blocking (invalid claim), important (weak evidence), or improvement. Each needs an artifact reference, observed consequence, and concrete fix. Mark unavailable evidence as unknown, not passed.
Correcting a grader requires retaining old grades and regrading all compared stored outputs. Correcting app execution may require rerunning the baseline. Version the change, show whether rankings move, and return to the appropriate stage. Do not tune the app to work around a mislabeled reference.
Deliver the recomputed result, confidence limits, failure/error breakdown, prioritized findings, and the next executable step. If trustworthy, reuse the pipeline.