senior-data-engineer

21.8k

davila7

World-class data engineering skill for building scalable data pipelines, ETL/ELT systems, and data infrastructure. Expertise in Python, SQL, Spark, Airflow, dbt, Kafka, and modern data stack. Includes data modeling, pipeline orchestration, data quality, and DataOps. Use when designing data architectures, building data pipelines, optimizing data workflows, or implementing data governance.

pipeline-assistant

8.6k

redpanda-data

This skill should be used when users need to create or fix Redpanda Connect pipeline configurations. Trigger when users mention "config", "pipeline", "YAML", "create a config", "fix my config", "validate my pipeline", or describe a streaming pipeline need like "read from Kafka and write to S3".

component-search

8.6k

redpanda-data

This skill should be used when users need to discover Redpanda Connect components for their streaming pipelines. Trigger when users ask about finding inputs, outputs, processors, or other components, or when they mention specific technologies like "kafka consumer", "postgres output", "http server", or ask "which component should I use for X".

senior-data-engineer

2.2k

alirezarezvani

Data engineering skill for building scalable data pipelines, ETL/ELT systems, and data infrastructure. Expertise in Python, SQL, Spark, Airflow, dbt, Kafka, and modern data stack. Includes data modeling, pipeline orchestration, data quality, and DataOps. Use when designing data architectures, building data pipelines, optimizing data workflows, implementing data governance, or troubleshooting data issues.

kafka-stream-processor

1.5k

Jeremylongshore Claude Code Plugins Plus Skills Kafka Stream Processor

Process Kafka stream processor operations. Auto-activating skill for Data Pipelines. Triggers on: Kafka stream processor, Kafka stream processor Part of the Data Pipelines skill category. Use when working with Kafka stream processor functionality. Trigger with phrases like "Kafka stream processor", "Kafka processor", "Kafka".

kafka-producer-consumer

1.5k

Jeremylongshore Claude Code Plugins Plus Skills Kafka Producer Consumer

Kafka Producer Consumer - Auto-activating skill for Backend Development. Triggers on: kafka producer consumer, kafka producer consumer Part of the Backend Development skill category.

asyncapi-docs

376

a5c-ai

AsyncAPI specification handling for event-driven API documentation. Parse, validate, and generate documentation for message-based APIs including Kafka, MQTT, WebSocket, and AMQP systems.

Kafka Topic Designer

376

a5c-ai

Designs and optimizes Apache Kafka topics and configurations

streaming-data

296

ancoleman

Build event streaming and real-time data pipelines with Kafka, Pulsar, Redpanda, Flink, and Spark. Covers producer/consumer patterns, stream processing, event sourcing, and CDC across TypeScript, Python, Go, and Java. When building real-time systems, microservices communication, or data integration pipelines.

using-message-queues

296

ancoleman

Async communication patterns using message brokers and task queues. Use when building event-driven systems, background job processing, or service decoupling. Covers Kafka (event streaming), RabbitMQ (complex routing), NATS (cloud-native), Redis Streams, Celery (Python), BullMQ (TypeScript), Temporal (workflows), and event sourcing patterns.

building-with-kafka-strimzi

150

panaversity

Use when building event-driven systems with Apache Kafka on Kubernetes. Triggers include EDA patterns, Kafka producers/consumers, Strimzi operator deployment, Schema Registry, transactions, exactly-once semantics. NOT for general messaging (use Dapr pub/sub for abstraction).

building-with-dapr

150

panaversity

Use when building distributed microservices with Dapr sidecar architecture. Triggers include Dapr components, service invocation, state management, pub/sub, secrets, bindings, configuration, actors, and workflows. NOT for direct infrastructure clients (use building-with-kafka-strimzi instead).

spring-boot-event-driven-patterns

128

giuseppe-trisciuoglio

Provides Event-Driven Architecture (EDA) patterns in Spring Boot using ApplicationEvent, @EventListener, and Kafka. Use when building loosely-coupled microservices with domain events, transactional event listeners, and distributed messaging patterns.

spring-boot-saga-pattern

128

giuseppe-trisciuoglio

Provides distributed transaction patterns using the Saga Pattern in Spring Boot microservices. Use when building microservices requiring transaction management across multiple services, handling compensating transactions, ensuring eventual consistency, or implementing choreography or orchestration-based sagas with Spring Boot, Kafka, or Axon Framework.

Message Queue Testing

71

PramodDutta

Testing message queue implementations including RabbitMQ, SQS, and Kafka with delivery guarantees, ordering, dead letter queues, and consumer testing.

data-pipeline-engineer

43

curiositech

Expert data engineer for ETL/ELT pipelines, streaming, data warehousing. Activate on: data pipeline, ETL, ELT, data warehouse, Spark, Kafka, Airflow, dbt, data modeling, star schema, streaming data, batch processing, data quality. NOT for: API design (use api-architect), ML training (use ML skills), dashboards (use design skills).

kafka-stream-processing

40

manutej

Complete guide for Apache Kafka stream processing including producers, consumers, Kafka Streams, connectors, schema registry, and production deployment

Kafka Engineer

39

404kidwiz

Expert in Apache Kafka, event streaming, and real-time data pipelines. Specializes in Kafka Connect, KSQL, and Schema Registry.

message_queues

39

vuralserhat86

Async communication patterns using message brokers and task queues. Use when building event-driven systems, background job processing, or service decoupling. Covers Kafka (event streaming), RabbitMQ (complex routing), NATS (cloud-native), Redis Streams, Celery (Python), BullMQ (TypeScript), Temporal (workflows), and event sourcing patterns.

ops-devops-platform

34

vasilyu1983

Production-grade DevOps and platform engineering patterns: Kubernetes, Terraform, containers, GitOps, CI/CD, observability, incident response, security hardening, and cloud-native operations (AWS, GCP, Azure, Kafka).

stream-processing

31

melodic-software

Use when designing real-time data processing systems, choosing stream processing frameworks, or implementing event-driven architectures. Covers Kafka, Flink, and streaming patterns.

event-architect

29

omer-metin

Event sourcing and CQRS expert for AI memory systems. Use when "event sourcing, event store, cqrs, nats jetstream, kafka events, event projection, replay events, event schema, event-sourcing, cqrs, nats, kafka, projections, event-driven, memory-architecture, ml-memory" mentioned.

senior-data-engineer

28

hainamchung

World-class data engineering skill for building scalable data pipelines, ETL/ELT systems, and data infrastructure. Expertise in Python, SQL, Spark, Airflow, dbt, Kafka, and modern data stack. Includes data modeling, pipeline orchestration, data quality, and DataOps. Use when designing data architectures, building data pipelines, optimizing data workflows, or implementing data governance.

event-driven

27

cosmix

Event-driven architecture patterns including message queues, pub/sub, event sourcing, CQRS, and sagas. Use when implementing async messaging, distributed transactions, event stores, command query separation, domain events, integration events, data streaming, choreography, orchestration, or integrating with RabbitMQ, Kafka, Apache Pulsar, AWS SQS, AWS SNS, NATS, event buses, or message brokers.

kafka-development

23

Mindrally

Best practices and guidelines for Apache Kafka event streaming and distributed messaging

kafka

22

mjunaidca

Apache Kafka on Kubernetes with Strimzi (KRaft mode, no ZooKeeper). This skill should be used when users ask to deploy Kafka clusters, build producers/consumers, implement event-driven patterns, or debug Kafka issues. Includes tested manifests and Makefile for one-command deployment.

event-driven-architecture

16

travisjneuman

Kafka, RabbitMQ, SQS/SNS, event sourcing, CQRS, saga patterns, dead letter queues, and idempotency. Use when designing asynchronous systems, implementing message-driven workflows, or building event streaming pipelines.

kafka-development-practices

15

oimiragieo

Applies general coding standards and best practices for Kafka development with Scala.

DigitalOcean Managed Databases

15

bobmatnyc

DigitalOcean Managed Databases for PostgreSQL, MySQL, Redis, MongoDB, Kafka, OpenSearch, and Valkey. Use when provisioning, scaling, or operating managed database clusters on DigitalOcean.

testcontainers-go

10

testcontainers

A comprehensive guide for using Testcontainers for Go to write reliable integration tests with Docker containers in Go projects. Supports 62+ pre-configured modules for databases, message queues, cloud services, and more. Use this skill when writing Go integration tests, setting up test databases (PostgreSQL, MySQL, Redis, MongoDB), testing with message queues (Kafka, RabbitMQ), or creating container-based test infrastructure. Covers modules, generic containers, networking, cleanup, wait strategies, CI/CD integration, and common anti-patterns.

nats-messaging

9

TerminalSkills

Build distributed messaging systems with NATS — pub/sub, request/reply, JetStream persistent messaging, and key-value store. Use when someone asks to "set up message queue", "pub/sub system", "event-driven architecture", "NATS messaging", "distributed messaging", "microservice communication", "message broker", or "replace Kafka/RabbitMQ with something simpler". Covers core NATS, JetStream, KV store, and object store.

kafka

9

TerminalSkills

Build event-driven systems with Apache Kafka. Use when a user asks to set up message streaming, implement event sourcing, build pub/sub systems, process real-time data streams, or connect microservices with async messaging.

Stream

8

simota

ETL/ELTパイプライン設計、データフロー可視化、バッチ/ストリーミング選定、Kafka/Airflow/dbt設計。データパイプライン構築、データ品質管理が必要な時に使用。

complexity-review

8

Eferro Augmented Lean Delivery Complexity Review

Reviews technical proposals and architectural decisions against 30 complexity dimensions. Use when user proposes technical solutions, designs systems, or evaluates architectural choices. Questions necessity of scale, consistency, resilience, and other complexity drivers. Pushes for simplest viable approach. Use when: - User proposes specific technologies (Kafka, microservices, event sourcing, etc.) - Designing new systems or features - Evaluating architectural trade-offs - Someone says "we need" followed by complex infrastructure Do NOT use when: - Simple feature implementation (use hamburger-method or story-splitting instead) - Work is already planned and simple - User asks "how to" implement something specific (use micro-steps-coach instead)

Spring Boot Modulith

6

joaquimscosta

Spring Modulith 2.0 implementation for bounded contexts in Spring Boot 4. Use it when structuring application modules, implementing @ApplicationModuleListener for event-driven communication, testing with the Scenario API, enforcing module boundaries, or externalizing events to Kafka/AMQP. For decisions about modular monolith architecture, see the domain-driven-design skill.

event-driven-architect

6

Dexploarer

Design event-driven architectures using Kafka, RabbitMQ, event sourcing, CQRS, and saga patterns. Activates when users need help with event-driven design, message queues, event sourcing, or asynchronous communication patterns.

surreal-sync

5

24601

Data migration and synchronization to SurrealDB from MongoDB, PostgreSQL, MySQL, Neo4j, Kafka, and JSONL. Full and incremental CDC sync. Part of the surreal-skills collection.

data-pipelines

5

ZLStas

Apply Data Pipelines Pocket Reference practices (James Densmore). Covers Infrastructure (Ch 1-2: warehouses, lakes, cloud), Patterns (Ch 3: ETL, ELT, CDC), DB Ingestion (Ch 4: MySQL, PostgreSQL, MongoDB, full/incremental), File Ingestion (Ch 5: CSV, JSON, cloud storage), API Ingestion (Ch 6: REST, pagination, rate limiting), Streaming (Ch 7: Kafka, Kinesis, event-driven), Storage (Ch 8: Redshift, BigQuery, Snowflake), Transforms (Ch 9: SQL, Python, dbt), Validation (Ch 10: Great Expectations, schema checks), Orchestration (Ch 11: Airflow, DAGs, scheduling), Monitoring (Ch 12: SLAs, alerting), Best Practices (Ch 13: idempotency, backfilling, error handling). Trigger on "data pipeline", "ETL", "ELT", "data ingestion", "Airflow", "dbt", "data warehouse", "Kafka streaming", "CDC", "data orchestration".

data-pipelines

5

booklib-ai

Apply Data Pipelines Pocket Reference practices (James Densmore). Covers Infrastructure (Ch 1-2: warehouses, lakes, cloud), Patterns (Ch 3: ETL, ELT, CDC), DB Ingestion (Ch 4: MySQL, PostgreSQL, MongoDB, full/incremental), File Ingestion (Ch 5: CSV, JSON, cloud storage), API Ingestion (Ch 6: REST, pagination, rate limiting), Streaming (Ch 7: Kafka, Kinesis, event-driven), Storage (Ch 8: Redshift, BigQuery, Snowflake), Transforms (Ch 9: SQL, Python, dbt), Validation (Ch 10: Great Expectations, schema checks), Orchestration (Ch 11: Airflow, DAGs, scheduling), Monitoring (Ch 12: SLAs, alerting), Best Practices (Ch 13: idempotency, backfilling, error handling). Trigger on "data pipeline", "ETL", "ELT", "data ingestion", "Airflow", "dbt", "data warehouse", "Kafka streaming", "CDC", "data orchestration".

spring-maven-modular

5

mduongvandinh

Maven Modular Architecture with profiles for optional components. Enable/disable modules like Redis, Kafka, RabbitMQ dynamically.

spring-boot-full-stack

5

mduongvandinh

Complete Java Spring Boot skill set for building enterprise applications. Includes modular architecture with optional components: - PostgreSQL database with JPA/Hibernate + Flyway migration - Redis caching (optional) - Kafka/RabbitMQ messaging (optional, choose one) - JWT + OAuth2 authentication (optional OAuth2) - RBAC authorization (optional) - TDD with Mockito - Spec-First Development with OpenSpec

Testing Spring Apps

4

gitwalter

JUnit 5 patterns, Spring Boot Test slices (@WebMvcTest, @DataJpaTest), MockMvc for API testing, Testcontainers for integration tests (PostgreSQL, Redis, Kafka), Mockito patterns, Test fixtures with @TestConfiguration

spring-kafka-integration

4

olehsvyrydov

[Extends backend-developer] Kafka specialist for Spring/Reactor. Use for Kafka producers/consumers, DLT, retry mechanisms, transactional outbox, event sourcing. Covers Spring Kafka 4.x and Reactor Kafka 1.3.x. Invoke alongside backend-developer.

senior-data-engineer

4

ngxtm

World-class data engineering skill for building scalable data pipelines, ETL/ELT systems, and data infrastructure. Expertise in Python, SQL, Spark, Airflow, dbt, Kafka, and modern data stack. Includes data modeling, pipeline orchestration, data quality, and DataOps. Use when designing data architectures, building data pipelines, optimizing data workflows, or implementing data governance.

devcontainers

4

digitalocean-labs

Set up local development environments with production parity for DigitalOcean App Platform. Use when setting up local dev, adding devcontainer to a project, running App Platform apps locally, or configuring backing services (Postgres, Redis, Kafka, S3).

managed-db-services

4

digitalocean-labs

Configure DigitalOcean Managed MySQL, MongoDB, Valkey, Kafka, and OpenSearch for App Platform. Use when setting up non-PostgreSQL databases, configuring trusted sources, or troubleshooting database connectivity.

spring-testing

4

gitwalter

JUnit 5 patterns, Spring Boot Test slices (@WebMvcTest, @DataJpaTest), MockMvc for API testing, Testcontainers for integration tests (PostgreSQL, Redis, Kafka), Mockito patterns, Test fixtures with @TestConfiguration

clickhouse-streaming

4

Dandedotdev Dotfiles Clickhouse Streaming

Use when ingesting continuous data streams from Kafka, RabbitMQ, or Kinesis into ClickHouse. Covers backpressure handling, exactly-once semantics, stream processing patterns, and performance optimization. NOT for database replication (see clickhouse-cdc) or batch ETL (see clickhouse-patterns).