groq-prod-checklist

Overview

Complete pre-launch checklist for deploying Groq-powered applications to production. Covers API key security, model selection, rate limit planning, fallback strategies, and monitoring setup. Work top-to-bottom: each section is a gate that must be green before the go-live verification runs.

Deep code (fallback function, health-check endpoint, go-live script) lives in references/ so this file stays scannable — drill in when you reach that step.

Prerequisites

• Staging environment tested with Groq API
• Groq Developer or Enterprise plan (free tier is not suitable for production)
• Production API key created in console.groq.com
• Monitoring and alerting infrastructure ready

Instructions

Read the target app's Groq integration and config, then walk each gate below. Tick every box; an unchecked item is a launch blocker.

1. API Key & Auth

• [ ] Production API key stored in secret manager (not .env files)
• [ ] Key is NOT shared with development or staging environments
• [ ] Key rotation procedure documented and tested
• [ ] Pre-commit hook blocks gsk_ pattern in code

2. Model Selection

• [ ] Production model chosen and tested (recommend llama-3.3-70b-versatile)
• [ ] Fallback model configured (llama-3.1-8b-instant)
• [ ] Deprecated model IDs removed (check deprecations)
• [ ] max_tokens set to actual expected output size (not context max)

3. Rate Limit Planning

• [ ] Production rate limits known (check console.groq.com/settings/limits)
• [ ] Estimated peak RPM < 80% of limit
• [ ] Estimated peak TPM < 80% of limit
• [ ] Exponential backoff with retry-after header implemented
• [ ] Request queue for burst protection (p-queue or similar)

4. Error Handling & Fallback

• [ ] All Groq error types caught (Groq.APIError, Groq.APIConnectionError)
• [ ] 429 errors retried with backoff
• [ ] 5xx errors retried with backoff
• [ ] 401 errors trigger alert (key may be revoked)
• [ ] Network timeouts configured (default 60s may be too long)
• [ ] Circuit breaker pattern for sustained failures
• [ ] Fallback-to-degradation wrapper in place — see the completionWithFallback pattern in references/implementation.md

5. Health Check

• [ ] /api/health (or /healthz) probes Groq with a 1-token request and returns 503 when degraded — full route in references/implementation.md

6. Monitoring Setup

• [ ] Latency histogram (p50, p95, p99)
• [ ] Token throughput counter (tokens/sec by model)
• [ ] Error rate by status code (429, 5xx)
• [ ] Rate limit remaining gauge (from response headers)
• [ ] Cost tracking (tokens * price per million)
• [ ] Alert: latency p95 > 1s (Groq normally < 200ms)
• [ ] Alert: error rate > 5%
• [ ] Alert: rate limit remaining < 10%

7. Spending Controls

• [ ] Monthly spending cap set in Groq Console
• [ ] Budget alerts at 50%, 80%, 95%
• [ ] Auto-pause enabled when cap is reached

8. Documentation

• [ ] Incident runbook created (see groq-incident-runbook)
• [ ] Key rotation SOP documented
• [ ] On-call knows how to check status.groq.com
• [ ] Rollback procedure tested

9. Go-Live Verification

Run the pre-flight curl script against production — status, key, health endpoint, and rate-limit headroom must all pass. Full script and pass/fail table in references/go-live.md.

Output

Working through this skill produces a go / no-go launch decision:

• A completed checklist where every box is ticked or explicitly waived with a reason.
• A green go-live verification run (all four pre-flight checks passing).
• The alert matrix (below) wired into your monitoring stack.

Any unchecked security or auth item (Sections 1, 2) is a hard blocker; unchecked monitoring or spending items (Sections 6, 7) are P3 blockers that may launch with a tracked follow-up.

Error Handling

Wire these alerts before go-live so production failures page the right severity:

| Alert | Condition | Severity | |-------|-----------|----------| | API errors spike | 5xx rate > 5/min | P1 | | Latency degraded | p95 > 1000ms | P2 | | Rate limited | 429 count > 5/min | P2 | | Auth failure | Any 401 error | P1 | | Spending near cap | >90% of monthly budget | P3 |

Examples

Minimal fallback skeleton — try the primary model, fall back to the fast model on 429/5xx:

try {
  return await groq.chat.completions.create({ model: "llama-3.3-70b-versatile", messages, timeout: 15_000 });
} catch (err: any) {
  if (err.status === 429 || err.status >= 500) {
    return await groq.chat.completions.create({ model: "llama-3.1-8b-instant", messages, timeout: 10_000 });
  }
  throw err;
}

• Full fallback-with-degradation function and health-check endpoint: references/implementation.md
• Complete go-live verification script and result interpretation: references/go-live.md

Resources

• Groq Status Page
• Groq Rate Limits
• Groq Spend Limits
• Groq Models (check deprecations)

Next Steps

Once launched, keep the integration current: schedule model-deprecation reviews against the Groq deprecations page, and for version upgrades follow the groq-upgrade-migration skill. If an incident fires an alert above, escalate through the groq-incident-runbook.