tensorrt-llm

21.8k

davila7

Optimizes LLM inference with NVIDIA TensorRT for maximum throughput and lowest latency. Use for production deployment on NVIDIA GPUs (A100/H100), when you need 10-100x faster inference than PyTorch, or for serving models with quantization (FP8/INT4), in-flight batching, and multi-GPU scaling.

serving-llms-vllm

21.8k

davila7

Serves LLMs with high throughput using vLLM's PagedAttention and continuous batching. Use when deploying production LLM APIs, optimizing inference latency/throughput, or serving models with limited GPU memory. Supports OpenAI-compatible endpoints, quantization (GPTQ/AWQ/FP8), and tensor parallelism.

sglang

21.8k

davila7

Fast structured generation and serving for LLMs with RadixAttention prefix caching. Use for JSON/regex outputs, constrained decoding, agentic workflows with tool calls, or when you need 5× faster inference than vLLM with prefix sharing. Powers 300,000+ GPUs at xAI, AMD, NVIDIA, and LinkedIn.

llama-cpp

21.8k

davila7

Runs LLM inference on CPU, Apple Silicon, and consumer GPUs without NVIDIA hardware. Use for edge deployment, M1/M2/M3 Macs, AMD/Intel GPUs, or when CUDA is unavailable. Supports GGUF quantization (1.5-8 bit) for reduced memory and 4-10× speedup vs PyTorch on CPU.

runtime-skills

819

llama-farm

Universal Runtime best practices for PyTorch inference, Transformers models, and FastAPI serving. Covers device management, model loading, memory optimization, and performance tuning.

model-serving

296

ancoleman

LLM and ML model deployment for inference. Use when serving models in production, building AI APIs, or optimizing inference. Covers vLLM (LLM serving), TensorRT-LLM (GPU optimization), Ollama (local), BentoML (ML deployment), Triton (multi-model), LangChain (orchestration), LlamaIndex (RAG), and streaming patterns.

machine-learning-engineer

39

404kidwiz

Use when user needs ML model deployment, production serving infrastructure, optimization strategies, and real-time inference systems. Designs and implements scalable ML systems with focus on reliability and performance.

machine-learning-engineer

36

zenobi-us

Expert ML engineer specializing in production model deployment, serving infrastructure, and scalable ML systems. Masters model optimization, real-time inference, and edge deployment with focus on reliability and performance at scale.

ai-llm

34

vasilyu1983

Production LLM engineering skill. Covers strategy selection (prompting vs RAG vs fine-tuning), dataset design, PEFT/LoRA, evaluation workflows, deployment handoff to inference serving, and lifecycle operations with cost/safety controls.

ai-llm-inference

34

vasilyu1983

Operational patterns for LLM inference: latency budgeting, tail-latency control, caching, batching/scheduling, quantization/compression, parallelism, and reliable serving at scale. Emphasizes production-grade performance, cost control, and observability.

model-serving-kubernetes

10

BagelHole

Deploy ML models on Kubernetes with KServe (formerly KFServing) and NVIDIA Triton Inference Server. Includes canary deployments, autoscaling, model versioning, A/B testing, and GPU resource management for production model serving.

vllm-server

10

BagelHole

Deploy and manage vLLM for high-throughput LLM inference. Configure continuous batching, tensor parallelism, quantization, and OpenAI-compatible API endpoints for production LLM serving.

vllm

9

TerminalSkills

High-throughput LLM serving engine with PagedAttention for efficient memory management. Serves open-source models with OpenAI-compatible API, continuous batching, tensor parallelism, and quantization support. Optimized for production inference workloads.

ml-engineer

5

agent-skills-hub

Build production ML systems with PyTorch 2.x, TensorFlow, and modern ML frameworks. Implements model serving, feature engineering, A/B testing, and monitoring. Use PROACTIVELY for ML model deployment, inference optimization, or production ML infrastructure.

deploy-ml-model-serving

2

pjt222

Deploy machine learning models to production serving infrastructure using MLflow, BentoML, or Seldon Core with REST/gRPC endpoints, implement autoscaling, monitoring, and A/B testing capabilities for high-performance model inference at scale. Use when deploying trained models for real-time inference, setting up REST or gRPC prediction APIs, implementing autoscaling for variable load, running A/B tests between model versions, or migrating from batch to real-time inference.

deploy-ml-model-serving

2

pjt222

Deploy machine learning models to production serving infrastructure using MLflow, BentoML, or Seldon Core with REST/gRPC endpoints, implement autoscaling, monitoring, and A/B testing capabilities for high-performance model inference at scale. Use when deploying trained models for real-time inference, setting up REST or gRPC prediction APIs, implementing autoscaling for variable load, running A/B tests between model versions, or migrating from batch to real-time inference.

build-feature-store

2

pjt222

Build a feature store using Feast for centralized feature management, configure offline and online stores for batch and real-time serving, define feature views with transformations, and implement point-in-time correct joins for ML pipelines. Use when managing features for multiple ML models, ensuring training-serving consistency, serving low-latency features for real-time inference, reusing feature definitions across projects, or building a feature catalog for discovery and governance.

build-feature-store

2

pjt222

Build a feature store using Feast for centralized feature management, configure offline and online stores for batch and real-time serving, define feature views with transformations, and implement point-in-time correct joins for ML pipelines. Use when managing features for multiple ML models, ensuring training-serving consistency, serving low-latency features for real-time inference, reusing feature definitions across projects, or building a feature catalog for discovery and governance.

ML Engineer

oki3505F

Build production ML systems with PyTorch 2.x, TensorFlow, and modern ML frameworks. Implements model serving, feature engineering, A/B testing, and monitoring. Use PROACTIVELY for ML model deployment, inference optimization, or production ML infrastructure.

ML Engineer

haniakrim21

Build production ML systems with PyTorch 2.x, TensorFlow, and modern ML frameworks. Implements model serving, feature engineering, A/B testing, and monitoring. Use proactively for ML model deployment, inference optimization, or production ML infrastructure.

together-dedicated-containers

zainhas

Deploy custom Dockerized inference workloads on Together AI's managed GPU infrastructure using Dedicated Container Inference (DCI). Tools include Jig CLI for building/deploying, Sprocket SDK for request handling, and a private container registry. Use when users need custom model serving, containerized inference, Docker-based GPU workloads, or workloads beyond standard model endpoints.

serving-llms-vllm

AXGZ21

Serves LLMs with high throughput using vLLM's PagedAttention and continuous batching. Use when deploying production LLM APIs, optimizing inference latency/throughput, or serving models with limited GPU memory. Supports OpenAI-compatible endpoints, quantization (GPTQ/AWQ/FP8), and tensor parallelism.