gguf-quantization

21.8k

davila7

GGUF format and llama.cpp quantization for efficient CPU/GPU inference. Use when deploying models on consumer hardware, Apple Silicon, or when needing flexible quantization from 2-8 bit without GPU requirements.

llama-cpp

21.8k

davila7

Runs LLM inference on CPU, Apple Silicon, and consumer GPUs without NVIDIA hardware. Use for edge deployment, M1/M2/M3 Macs, AMD/Intel GPUs, or when CUDA is unavailable. Supports GGUF quantization (1.5-8 bit) for reduced memory and 4-10× speedup vs PyTorch on CPU.

pytorch-model-recovery

62

letta-ai

This skill should be used when reconstructing PyTorch models from weight files (state dictionaries), checkpoint files, or partial model artifacts. It applies when the agent needs to infer model architecture from saved weights, rebuild models without original source code, or recover models from corrupted/incomplete saves. Use this skill for tasks involving torch.load, state_dict reconstruction, architecture inference, or model recovery in CPU-constrained environments.

onnx

9

TerminalSkills

Open Neural Network Exchange format for model interoperability across frameworks. Export models from PyTorch, TensorFlow, and other frameworks to ONNX, optimize with ONNX Runtime, and deploy for cross-platform inference on CPU, GPU, and edge devices.

triton

9

TerminalSkills

NVIDIA Triton Inference Server for deploying AI models at scale. Supports multiple frameworks (ONNX, TensorRT, PyTorch, TensorFlow), model ensembles, dynamic batching, model versioning, and GPU/CPU inference with high throughput and low latency.

llama-cpp

AXGZ21

Runs LLM inference on CPU, Apple Silicon, and consumer GPUs without NVIDIA hardware. Use for edge deployment, M1/M2/M3 Macs, AMD/Intel GPUs, or when CUDA is unavailable. Supports GGUF quantization (1.5-8 bit) for reduced memory and 4-10× speedup vs PyTorch on CPU.