async-jobs

Background task processing with Celery, ARQ, Redis and Temporal. This skill is a wrapper, not a manual: Celery and ARQ document their own product well, so what lives here is our delta, the thresholds, working config, ordering constraints and tool-choice rules we picked. Product mechanics are linked, not restated.

Start with Read("references/ork-delta.md").

Quick Reference

| Topic | Where our part lives | |-------|----------------------| | Configuration | references/celery-config.md, rules/jobs-task-queue.md | | Task Routing | references/ork-delta.md (queue taxonomy, prefetch tiers, Redis priority) | | Canvas Workflows | rules/celery-canvas.md | | Retry Strategies | references/ork-delta.md (backoff cap, idempotency layers, lock TTLs) | | Scheduling | rules/jobs-scheduling.md, references/ork-delta.md (beat process model) | | Monitoring | references/ork-delta.md (alert thresholds, histogram buckets) | | Result Backends | rules/jobs-monitoring.md, references/ork-delta.md (return contract) | | ARQ Patterns | rules/jobs-task-queue.md, references/ork-delta.md (budgets, pool ownership) | | Temporal Workflows | rules/temporal-workflows.md | | Temporal Activities | rules/temporal-activities.md |

10 topic areas, 6 rule files in rules/, house delta in references/ork-delta.md.

Quick Start

@app.task(bind=True, max_retries=3, default_retry_delay=60)
def process_payment(self, order_id: str):
    try:
        return gateway.charge(order_id)
    except TransientError as exc:
        raise self.retry(exc=exc, countdown=2 ** self.request.retries * 60)

Load more examples: Read("references/quick-start-examples.md") for Celery retry task and ARQ/FastAPI integration patterns.

Upstream coverage (do not restate)

Fetch these when you need product mechanics. The right-hand column is the part we keep, because it is a house threshold, a working config or an ordering constraint that upstream cannot know.

| Topic | First-party source | House subset stays in | |-------|--------------------|-----------------------| | Celery settings, serializers, time limits, worker flags | https://docs.celeryq.dev/en/stable/userguide/configuration.html and .../optimizing.html | references/celery-config.md, rules/jobs-task-queue.md | | Queue declarations, router classes, Redis priority mechanics | https://docs.celeryq.dev/en/stable/userguide/routing.html | references/ork-delta.md | | chain / group / chord / signature semantics | https://docs.celeryq.dev/en/stable/userguide/canvas.html | rules/celery-canvas.md keeps the house canvas subset. Its si()-in-chords guidance is UNVERIFIED and contested: confirm the argument-passing behaviour against the upstream canvas page before relying on it | | autoretry_for, retry_backoff, Reject, task base classes | https://docs.celeryq.dev/en/stable/userguide/tasks.html | references/ork-delta.md, rules/jobs-task-queue.md | | Beat schedules, crontab syntax, DatabaseScheduler | https://docs.celeryq.dev/en/stable/userguide/periodic-tasks.html and https://django-celery-beat.readthedocs.io/en/latest/ | rules/jobs-scheduling.md keeps our beat_schedule shapes; references/ork-delta.md keeps the process model | | Flower flags, inspect, signal names | https://docs.celeryq.dev/en/stable/userguide/monitoring.html, https://docs.celeryq.dev/en/stable/userguide/signals.html, https://flower.readthedocs.io/en/latest/config.html | references/ork-delta.md | | Result backend, AsyncResult, custom states | https://docs.celeryq.dev/en/stable/userguide/configuration.html | rules/jobs-monitoring.md keeps our status endpoints and update_state() usage | | Per-task rate_limit, control.rate_limit, Redis Lua | https://docs.celeryq.dev/en/stable/userguide/workers.html, https://redis.io/docs/latest/develop/programmability/eval-intro/ | references/ork-delta.md | | ARQ WorkerSettings, enqueue_job, _defer_by / _defer_until, Job status | https://arq-docs.helpmanual.io/ | rules/jobs-task-queue.md keeps the worker skeleton; references/ork-delta.md keeps the budgets | | FastAPI lifespan and dependency wiring | https://fastapi.tiangolo.com/advanced/events/ | references/ork-delta.md | | Distributed locks with SET NX EX | https://redis.io/docs/latest/commands/set/ | references/ork-delta.md |

Configuration

Load: Read("references/celery-config.md").

| Decision | Recommendation | |----------|----------------| | Serializer | JSON (never pickle) | | Ack mode | Late ack (task_acks_late=True) | | Prefetch | 1 for fair, 4-8 for throughput | | Time limit | soft < hard (540 / 600) | | Timezone | UTC always |

Task Routing

| Decision | Recommendation | |----------|----------------| | Queue count | 5: critical / high / default / low / bulk | | Priority levels | 0-9, with all four Redis priority switches set together | | Worker assignment | Dedicated worker per queue | | Prefetch | 1 critical, 2 high, 4 default, 8 low/bulk | | Routing | Router class once past 5 routing rules |

Canvas Workflows

Load: Read("rules/celery-canvas.md").

| Decision | Recommendation | |----------|----------------| | Sequential | Chain with s() | | Parallel | Group for independent tasks | | Fan-in | Chord (all header tasks must succeed for the body to run) | | Ignore input | Use si() immutable signature | | Error in chain | Reject stops the chain, retry continues it | | Partial failures | Return an error dict from chord header tasks |

Retry Strategies

| Decision | Recommendation | |----------|----------------| | Retry delay | Exponential backoff, jitter on, capped at 600s | | Max retries | 3-5 for transient, 0 for permanent | | Idempotency | Redis marker (86400s TTL) plus the vendor idempotency key | | Failed tasks | DLQ for manual review | | Singleton | Redis lock with a TTL longer than the hard time limit |

Scheduling

Load: Read("rules/jobs-scheduling.md").

| Decision | Recommendation | |----------|----------------| | Schedule type | Crontab for time-based, float interval for frequency | | Dynamic | DatabaseScheduler (django-celery-beat) | | Overlap | Redis lock, 3600s default and 7200s for long jobs | | Beat process | Separate process; embedded --beat is development only | | Timezone | UTC always |

Monitoring

| Decision | Recommendation | |----------|----------------| | Dashboard | Flower with persistent storage | | Metrics | Prometheus wired to task_prerun / task_postrun / task_failure | | Health | Broker reachable, at least one worker, queue depths | | Alerting | critical > 100, default > 5000, workers < 1 | | Autoscale | Queue depth > 500 |

Result Backends

Load: Read("rules/jobs-monitoring.md").

| Decision | Recommendation | |----------|----------------| | Status storage | Redis result backend, status and small JSON only | | Large results | S3 or database, task returns a reference dict | | Progress | Custom states with update_state() | | Result query | AsyncResult with state checks |

ARQ Patterns

Load: Read("rules/jobs-task-queue.md").

| Decision | Recommendation | |----------|----------------| | Simple async | ARQ (native async), max_jobs=10, job_timeout=300 | | Pool ownership | FastAPI lifespan, never a per-request create_pool | | Complex workflows | Celery (chains, chords, DLQ, per-task rate limits) | | In-process quick | FastAPI BackgroundTasks, under 30s, non-critical only | | LLM workflows | LangGraph, not Celery |

Tool Selection

Load: Read("references/quick-start-examples.md") for the full tool comparison table (ARQ, Celery, RQ, Dramatiq, FastAPI BackgroundTasks).

Anti-Patterns (FORBIDDEN)

Load details: Read("references/anti-patterns.md") for the full list.

Key rules: never run long tasks in request handlers, never block on results inside tasks, never store large results in Redis, always use idempotency for retried tasks.

Temporal Workflows

Load: Read("rules/temporal-workflows.md").

| Decision | Recommendation | |----------|----------------| | Workflow ID | Business-meaningful, idempotent | | Determinism | Use workflow.random(), workflow.now() | | I/O | Always via activities, never directly |

Temporal Activities

Load: Read("rules/temporal-activities.md").

| Decision | Recommendation | |----------|----------------| | Activity timeout | start_to_close for most cases | | Error handling | Non-retryable for business errors | | Testing | WorkflowEnvironment.start_local() for integration tests |

Related Skills

• ork:python-backend - FastAPI, asyncio, SQLAlchemy patterns
• ork:langgraph - LangGraph workflow patterns (use for LLM workflows, not Celery)
• ork:distributed-systems - Resilience patterns, circuit breakers
• ork:monitoring-observability - Metrics and alerting

Capability Details

Load details: Read("references/capability-details.md") for the keyword index and problem-to-capability mapping.