evaluation AI Agent Skills
Browse 21 skills related to evaluation
evaluating-llms-harness
davila7Evaluates LLMs across 60+ academic benchmarks (MMLU, HumanEval, GSM8K, TruthfulQA, HellaSwag). Use when benchmarking model quality, comparing models, reporting academic results, or tracking training progress. Industry standard used by EleutherAI, HuggingFace, and major labs. Supports HuggingFace, vLLM, APIs.
phoenix-observability
davila7Open-source AI observability platform for LLM tracing, evaluation, and monitoring. Use when debugging LLM applications with detailed traces, running evaluations on datasets, or monitoring production AI systems with real-time insights.
langsmith-observability
davila7LLM observability platform for tracing, evaluation, and monitoring. Use when debugging LLM applications, evaluating model outputs against datasets, monitoring production systems, or building systematic testing pipelines for AI applications.
evaluating-code-models
davila7Evaluates code generation models across HumanEval, MBPP, MultiPL-E, and 15+ benchmarks with pass@k metrics. Use when benchmarking code models, comparing coding abilities, testing multi-language support, or measuring code generation quality. Industry standard from BigCode Project used by HuggingFace leaderboards.
nemo-evaluator-sdk
davila7Evaluates LLMs across 100+ benchmarks from 18+ harnesses (MMLU, HumanEval, GSM8K, safety, VLM) with multi-backend execution. Use when needing scalable evaluation on local Docker, Slurm HPC, or cloud platforms. NVIDIA's enterprise-grade platform with container-first architecture for reproducible benchmarking.
trulens-evaluation-setup
trueraConfigure feedback functions and selectors for TruLens evaluations
trulens-running-evaluations
trueraExecute TruLens evaluations and view results
trulens-dataset-curation
trueraCreate and curate evaluation datasets with ground truth for TruLens
trulens-evaluation-workflow
trueraSystematically evaluate your LLM application with TruLens
evaluating-code-models
sangrokjungEvaluates code generation models across HumanEval, MBPP, MultiPL-E, and 15+ benchmarks with pass@k metrics. Use when benchmarking code models, comparing coding abilities, testing multi-language support, or measuring code generation quality. Industry standard from BigCode Project used by HuggingFace leaderboards.
evaluating-llms-harness
sangrokjungEvaluates LLMs across 60+ academic benchmarks (MMLU, HumanEval, GSM8K, TruthfulQA, HellaSwag). Use when benchmarking model quality, comparing models, reporting academic results, or tracking training progress. Industry standard used by EleutherAI, HuggingFace, and major labs. Supports HuggingFace, vLLM, APIs.
skills-eval
atholaEvaluate and improve Claude skill quality through auditing. Use when reviewing skill quality, preparing skills for production, or auditing existing skills. Do not use when creating new skills (use modular-skills) or writing prose (use writing-clearly-and-concisely). Use this skill before shipping any skill to production.
methodology-curator
atholaSurfaces expert frameworks and proven methodologies before creating OR evaluating skills, hooks, agents, or commands. Helps select approaches from domain masters. Use when starting creation workflows, evaluating methodology gaps, or seeking domain expertise. Do not use if you already have a specific methodology or are fixing syntax/structural issues.
golden-dataset
yonatangrossGolden dataset lifecycle patterns for curation, versioning, quality validation, and CI integration. Use when building evaluation datasets, managing dataset versions, validating quality scores, or integrating golden tests into pipelines.
assess
yonatangrossAssesses and rates quality 0-10 with pros/cons analysis. Use when evaluating code, designs, or approaches.
scoring
SimHackerMotto: Style counts. Context matters. Story trumps numbers.
evaluator
SimHackerIndependent assessment without debate context — adversarial loop prevents gaming
rubric
SimHackerMeasurable criteria translating qualitative debate to quantitative scores
researcher-role-skill
enunoProfessional technical solutions research skill for investigating technologies, analyzing best practices, evaluating feasibility, and producing comprehensive reports with actionable recommendations for development initiatives.
gpd-training-evaluation
chuongdlbStable-Baselines3 PPO training pipeline — environment setup, callbacks, reward thresholds, model saving, evaluation, inference loop.
eval-harness
gabrielgadeaEvaluation-driven development framework. Define measurable criteria, create a test harness, measure before/after. Use when quality must be quantified, not just asserted.