Create standardized test questions that comply with the CAC evaluation system specifications. Use this Skill when the user needs to add a new question, create a test question, or uses triggers such as "create question", "new question", or "add question". Supports four question banks: coding, mathematics, logic, and comprehensive.
创建符合 CAC 评测系统规范的标准化测试题目。
创建题目前,请提供:
NNN-problem-name 格式python scripts/validate_questions.py{题库名}/{difficulty}-test/NNN-problem-name/
├── README.md # 人类阅读的完整文档
├── meta.yaml # 机器读取的元数据
├── prompt.md # 发给被测模型的 prompt
├── reference.md # 标准答案/评判依据
└── test-results/ # 测试结果目录
详细格式规范见 question_format_guide.md。 编号规则见 numbering_rules.md。 文件模板见 templates/ 目录。
id: {category}-{difficulty}-{number}
brief: 题目简短描述
category: math | code | logic | comprehensive
difficulty: base | advanced | final | final+
scoring_std:
max_score: 10
indicators:
- accuracy # 准确性
- completeness # 完整性
| 类型 | 可用指标 |
|------|----------|
| 代码题 | ans_correct, code_quality, efficiency, robustness |
| 理论题 | completeness, accuracy, clarity, depth |
| 设计题 | ans_correct, example_quality, completeness, practicality |
纯净的题目文本,直接发给被测模型,不含元数据。
标准答案和评判依据,供评判模型参考。
| 类别 | 路径 | category |
|------|------|----------|
| 代码 | 代码能力基准测试题库/ | code |
| 数理 | 数理能力基准测试题库/ | math |
| 逻辑 | 自然语言与逻辑能力基准测试题库/ | logic |
| 综合 | 综合能力测评/ | comprehensive |
参考现有题目:
数理能力基准测试题库/base-test/001-chicken-rabbit-cage/代码能力基准测试题库/base-test/001-simple-calculator/自然语言与逻辑能力基准测试题库/base-test/001-age-multiple-reasoning/Category:science-education