llm-evaluation
wshobsonImplement comprehensive evaluation strategies for LLM applications using automated metrics, human feedback, and benchmarking. Use when testing LLM performance, measuring AI application quality, or establishing evaluation frameworks.
agent-evaluation
davila7Testing and benchmarking LLM agents including behavioral testing, capability assessment, reliability metrics, and production monitoring—where even top agents achieve less than 50% on real-world benchmarks Use when: agent testing, agent evaluation, benchmark agents, agent reliability, test agent.
evaluating-llms-harness
davila7Evaluates LLMs across 60+ academic benchmarks (MMLU, HumanEval, GSM8K, TruthfulQA, HellaSwag). Use when benchmarking model quality, comparing models, reporting academic results, or tracking training progress. Industry standard used by EleutherAI, HuggingFace, and major labs. Supports HuggingFace, vLLM, APIs.
evaluating-code-models
davila7Evaluates code generation models across HumanEval, MBPP, MultiPL-E, and 15+ benchmarks with pass@k metrics. Use when benchmarking code models, comparing coding abilities, testing multi-language support, or measuring code generation quality. Industry standard from BigCode Project used by HuggingFace leaderboards.
nemo-evaluator-sdk
davila7Evaluates LLMs across 100+ benchmarks from 18+ harnesses (MMLU, HumanEval, GSM8K, safety, VLM) with multi-backend execution. Use when needing scalable evaluation on local Docker, Slurm HPC, or cloud platforms. NVIDIA's enterprise-grade platform with container-first architecture for reproducible benchmarking.
cirq
davila7Quantum computing framework for building, simulating, optimizing, and executing quantum circuits. Use this skill when working with quantum algorithms, quantum circuit design, quantum simulation (noiseless or noisy), running on quantum hardware (Google, IonQ, AQT, Pasqal), circuit optimization and compilation, noise modeling and characterization, or quantum experiments and benchmarking (VQE, QAOA, QPE, randomized benchmarking).
llm-evaluation
sickn33Implement comprehensive evaluation strategies for LLM applications using automated metrics, human feedback, and benchmarking. Use when testing LLM performance, measuring AI application quality, or establishing evaluation frameworks.
V3 Performance Optimization
ruvnetAchieve aggressive v3 performance targets: 2.49x-7.47x Flash Attention speedup, 150x-12,500x search improvements, 50-75% memory reduction. Comprehensive benchmarking and optimization suite.
benchmark-kernel
flashinfer-aiGuide for benchmarking FlashInfer kernels with CUPTI timing
social-media-analyzer
alirezarezvaniSocial media campaign analysis and performance tracking. Calculates engagement rates, ROI, and benchmarks across platforms. Use for analyzing social media performance, calculating engagement rate, measuring campaign ROI, comparing platform metrics, or benchmarking against industry standards.
running-performance-tests
Execute load testing, stress testing, and performance benchmarking. Use when performing specialized testing. Trigger with phrases like "run load tests", "test performance", or "benchmark the system".
tbench
coderTerminal-Bench integration for Mux agent benchmarking and failure analysis
criterium
Use this skill when users ask about benchmarking Clojure code, measuring performance, profiling execution time, or using the criterium library. Covers the 0.5.x API including bench macro, bench plans, viewers, domain analysis, and argument generation.
knowledge-worker-salaries
danielmiesslerComprehensive global knowledge worker salary data with total market value calculations, sector breakdowns, geographic comparisons, and authoritative sources. USE WHEN discussing knowledge worker compensation, salary benchmarking, economic analysis of professional labor markets, or AI impact on wages.
defeatbeta-analyst
defeat-betaComprehensive financial analysis using 60+ data endpoints. Analyze company fundamentals, financial statements, valuation metrics, profitability ratios, growth trends, and industry comparisons. Use for: (1) fundamental analysis and DCF modeling, (2) financial statement analysis, (3) valuation and ratio analysis, (4) growth and profitability assessment, (5) industry benchmarking, or any deep financial research tasks.
benchmarking-analyst
a5c-aiBenchmarking analysis skill for performance comparison and best practice identification.
reputation-intelligence
a5c-aiReputation measurement and benchmarking platform integration
transportation-spend-analyzer
a5c-aiFreight spend analysis and benchmarking skill for cost optimization and carrier negotiation support
performance-benchmark-suite
a5c-aiSDK performance benchmarking and regression detection
gpu-benchmarking
a5c-aiExpert skill for automated GPU performance benchmarking and regression detection. Design micro-benchmarks, measure kernel execution time with CUDA events, calculate achieved vs theoretical performance, generate comparison reports, detect regressions in CI/CD, and profile power/thermal characteristics.
transfer-pricing-analyzer
a5c-aiIntercompany transfer pricing analysis skill with benchmarking and documentation generation
energy-auditor
a5c-aiProcess energy audit skill for consumption analysis, benchmarking, and efficiency improvement identification
comp-benchmarking
a5c-aiAnalyze market compensation data and establish competitive pay structures
network-testing
a5c-aiComprehensive network testing, benchmarking, and performance validation skill
logistics-kpi-tracker
a5c-aiComprehensive logistics performance measurement skill with KPI tracking, benchmarking, and improvement recommendations
rb-benchmarker
a5c-aiRandomized benchmarking skill for gate fidelity characterization
evaluating-code-models
sangrokjungEvaluates code generation models across HumanEval, MBPP, MultiPL-E, and 15+ benchmarks with pass@k metrics. Use when benchmarking code models, comparing coding abilities, testing multi-language support, or measuring code generation quality. Industry standard from BigCode Project used by HuggingFace leaderboards.
evaluating-llms-harness
sangrokjungEvaluates LLMs across 60+ academic benchmarks (MMLU, HumanEval, GSM8K, TruthfulQA, HellaSwag). Use when benchmarking model quality, comparing models, reporting academic results, or tracking training progress. Industry standard used by EleutherAI, HuggingFace, and major labs. Supports HuggingFace, vLLM, APIs.
go-test-expert
GoogleCloudPlatformExpert in Go testing patterns, table-driven tests, httptest, benchmarking, and fuzzing. Activates for "test", "fail", "benchmark", "debug", "fuzz".
benchmarking
Prorise-coolBenchmarking and competitive analysis techniques. Compares performance, processes, and practices against industry standards, competitors, and best-in-class organizations.
V3 Performance Optimization
proffesor-for-testingAchieve aggressive v3 performance targets: 2.49x-7.47x Flash Attention speedup, 150x-12,500x search improvements, 50-75% memory reduction. Comprehensive benchmarking and optimization suite.
performance-optimization
ynulihaoApply systematic performance optimization techniques when writing or reviewing code. Use when optimizing hot paths, reducing latency, improving throughput, fixing performance regressions, or when the user mentions performance, optimization, speed, latency, throughput, profiling, or benchmarking.
llm-evaluation
MicrockImplement comprehensive evaluation strategies for LLM applications using automated metrics, human feedback, and benchmarking. Use when testing LLM performance, measuring AI application quality, or establishing evaluation frameworks.
financial-analysis
LerianStudioComprehensive financial analysis workflow covering ratio analysis, trend analysis, benchmarking, and variance analysis. Delivers documented, audit-ready insights.
profiling-optimization
aj-geddesProfile application performance, identify bottlenecks, and optimize hot paths using CPU profiling, flame graphs, and benchmarking. Use when investigating performance issues or optimizing critical code paths.
WebAssembly Testing
PramodDuttaTesting WebAssembly modules including compilation verification, memory management, interop testing, and performance benchmarking of WASM components.
optimization-phase
marcusgollValidates production readiness through performance benchmarking, accessibility audits, security reviews, and code quality checks. Use after implementation phase completes, before deployment, or when conducting quality gates for features. (project)
Historical Cost Analyzer
datadrivenconstructionAnalyze historical construction costs for benchmarking, trend analysis, and estimating calibration. Compare projects, track escalation, identify patterns.
performance-benchmark-specialist
manutejPerformance benchmarking expertise for shell tools, covering benchmark design, statistical analysis (min/max/mean/median/stddev), performance targets (<100ms, >90% hit rate), workspace generation, and comprehensive reporting
llm_evaluation
vuralserhat86Implement comprehensive evaluation strategies for LLM applications using automated metrics, human feedback, and benchmarking. Use when testing LLM performance, measuring AI application quality, or establishing evaluation frameworks.
performance-monitor
404kidwizExpert in observing, benchmarking, and optimizing AI agents. Specializes in token usage tracking, latency analysis, and quality evaluation metrics. Use when optimizing agent costs, measuring performance, or implementing evals. Triggers include "agent performance", "token usage", "latency optimization", "eval", "agent metrics", "cost optimization", "agent benchmarking".
competitive-analyst
zenobi-usExpert competitive analyst specializing in competitor intelligence, strategic analysis, and market positioning. Masters competitive benchmarking, SWOT analysis, and strategic recommendations with focus on creating sustainable competitive advantages.
optimizing-r
jeremy-allenR performance profiling, benchmarking, and optimization strategies. Use this skill when code is running slowly, comparing alternative implementations, deciding between dplyr/data.table/base R, or implementing parallel processing. Covers profvis and bench usage, performance workflow, parallel processing with in_parallel(), data backend selection, modern purrr patterns (list_rbind, walk), and common performance anti-patterns to avoid.
Performance Analysis and Optimization
ShunsukeHayashiCPU profiling, benchmarking, and memory analysis for Rust applications. Use when code is slow, memory usage is high, or optimization is needed.
Performance Thinker
omer-metinPerformance optimization mindset - knowing when to optimize, how to measure, where bottlenecks hide, and when "fast enough" is the right answer. Use when "slow, performance, optimize, profiling, benchmark, latency, throughput, cache, n+1, bottleneck, memory leak, too slow, speed up, response time, performance, optimization, profiling, caching, latency, throughput, big-o, benchmarking" are mentioned.
agent-evaluation
omer-metinTesting and benchmarking LLM agents including behavioral testing, capability assessment, reliability metrics, and production monitoring—where even top agents achieve less than 50% on real-world benchmarks. Use when "agent testing, agent evaluation, benchmark agents, agent reliability, test agent, testing, evaluation, benchmark, agents, reliability, quality" are mentioned.
eval-recipes-runner
rysweetRun Microsoft's eval-recipes benchmarks to validate amplihack improvements against baseline agents. Auto-activates when testing improvements, running evals, or benchmarking changes.
model-evaluation-benchmark
rysweetAutomated reproduction of comprehensive model evaluation benchmarks following the Benchmark Suite V3. Auto-activates for model benchmarking, comparison evaluation, or performance testing between AI models.