批量数据采集技能,负责分批并发调度 researcher agent 抓取所有数据源。
此技能用于指导 /weekly 命令如何高效、分批、并发地从多个数据源采集信息。
作为批量调度器,负责:
researcher agent.opencode/REFERENCE.md 获取完整数据源列表generateHNUrls(start_date, end_date)import { generateHNUrls } from '.opencode/utils.mjs'
// Hacker News - 每天一个 URL
const hnUrls = generateHNUrls(start_date, end_date)
// 返回: [
// "https://news.ycombinator.com/front?day=2026-03-22",
// "https://news.ycombinator.com/front?day=2026-03-23",
// ...
// ]
将所有 URL 按优先级分为 3 批,每批 10-12 个 URL:
| 批次 | 数据源类型 | 来源 | | ----------- | ------------------- | ------------------------------------------ | | Batch 1 | Important Resources | REFERENCE.md 中 "Important Resources" 部分 | | Batch 2 | Blogs & Websites | REFERENCE.md 中 "Blogs & Websites" 部分 | | Batch 3 | KOL & Influencers | REFERENCE.md 中 "KOL & Influencers" 部分 |
| 参数 | 值 | 说明 |
| ------------------- | ------------ | -------------- |
| max_parallel | 5 | 每轮最大并发数 |
| batch_interval | 3s | 批次间等待时间 |
| domain_rate_limit | 2 req/domain | 同域名限流 |
对每个批次:
max_parallel 分轮(如 12 个 URL 分 3 轮:5 + 5 + 2)researcher agentbatch_interval,再启动下一轮调用 researcher 的参数格式:
url: https://news.ycombinator.com/front?day=2026-03-22
source_name: Hacker News
week_id: Y26W12
start_date: 2026-03-22
end_date: 2026-03-28
current_date: 2026-03-25
timezone: UTC+0
并发调用示例(伪代码):
# Batch 1: Important Resources
并行调用:
- researcher(url: "https://news.ycombinator.com/front?day=2026-03-22", source_name: "Hacker News")
- researcher(url: "https://news.ycombinator.com/front?day=2026-03-23", source_name: "Hacker News")
- researcher(url: "https://drafts.miantiao.me/", source_name: "Miantiao Drafts")
- researcher(url: "https://www.solidot.org/search?tid=151", source_name: "Solidot")
- ...
等待 Batch 1 全部完成
# Batch 2: Blogs & Websites
并行调用:
- researcher(url: "https://www.anthropic.com/engineering", source_name: "Anthropic Engineering")
- researcher(url: "https://claude.com/blog", source_name: "Claude Blog")
- ...
等待 Batch 2 全部完成
# Batch 3: KOL & Influencers
并行调用:
- researcher(url: "https://baoyu.io/", source_name: "Baoyu")
- ...
等待 Batch 3 全部完成
所有日志统一写入 logs/weekly-{week_id}.log,仅用于人类审计,不作为恢复依据。
日志格式:
[2026-03-25T12:34:56Z] [PHASE1] [INFO] 开始抓取 Hacker News
[2026-03-25T12:35:10Z] [PHASE1] [OK] Hacker News - 5 篇文章
[2026-03-25T12:35:15Z] [PHASE1] [FAIL] daily.dev - 429 Too Many Requests (retried 2)
日志级别:
| 级别 | 用途 |
| ------ | -------------------------------- |
| INFO | 阶段/任务开始 |
| OK | 任务成功(含文章数) |
| FAIL | 任务失败(含错误原因和重试次数) |
所有批次完成后,在日志末尾生成汇总:
[2026-03-25T12:45:00Z] [PHASE1] [SUMMARY] 总数据源: 30 | 成功: 27 (X篇) | 失败: 3
Search for places (restaurants, cafes, etc.) via Google Places API proxy on localhost.
Interact with GitHub using the `gh` CLI. Use `gh issue`, `gh pr`, `gh run`, and `gh api` for issues, PRs, CI runs, and advanced queries.
Create or update AgentSkills. Use when designing, structuring, or packaging skills with scripts, references, and assets.
Start voice calls via the OpenClaw voice-call plugin.
Notion API for creating and managing pages, databases, and blocks.
Gemini CLI for one-shot Q&A, summaries, and generation.
Category:developer