eda-checklist

Systematic checklist for exploratory data analysis. Work through each section in order. Check off items as you complete them.

Role in ds plugin: This skill provides the systematic methodology for tabular EDA -- what to check and in what order. It is invoked by /ds:eda at step 6 (Data Quality Checks). The exploratory-data-analysis skill handles file type detection and format-specific analysis for non-tabular scientific formats. These skills are complementary: eda-checklist answers "what to check in tabular data" while exploratory-data-analysis answers "how to analyze any file format."

Structure

• [ ] Shape and types -- Row count, column count, dtypes, memory usage
• [ ] Index structure -- Is the index meaningful (datetime, ID) or just a range?
• [ ] Column naming -- Consistent naming convention? Any ambiguous names?

Missing Data

• [ ] Missing rates per column -- Compute df.isnull().mean() and sort descending
• [ ] Missing patterns -- Use a missing data heatmap to check if missingness is random (MCAR), conditional (MAR), or structural (MNAR)
• [ ] Columns >50% missing -- Flag for potential removal or imputation investigation

Duplicates

• [ ] Exact duplicate rows -- df.duplicated().sum()
• [ ] Near-duplicate detection -- Check key columns for records that differ only in minor fields

Target Analysis

• [ ] Target distribution -- Class balance (classification) or distribution shape (regression)
• [ ] Target-feature correlations -- Rank features by correlation with target
• [ ] Point-biserial correlation -- For categorical features vs. continuous target

Numeric Distributions

• [ ] Summary statistics -- Mean, median, std, min, max, percentiles (p5, p25, p75, p95)
• [ ] Skewness -- Flag features with |skewness| > 2 for potential log transform
• [ ] Outlier detection -- IQR method (below Q1-1.5IQR or above Q3+1.5IQR) and z-score method

Categorical Distributions

• [ ] Value counts -- Top values and their frequencies
• [ ] Rare categories -- Categories with <1% frequency
• [ ] High cardinality -- Columns with >50 unique values (consider encoding strategy)

Temporal Patterns

• [ ] Time range -- Earliest and latest dates
• [ ] Gaps -- Missing time periods or irregular frequency
• [ ] Seasonality -- Weekly, monthly, or yearly patterns
• [ ] Trend -- Increasing or decreasing over time

Correlations

• [ ] Numeric correlation matrix -- Pearson correlation heatmap
• [ ] Top correlated pairs -- Pairs with |r| > 0.8 (potential multicollinearity)
• [ ] Multicollinearity check -- VIF (variance inflation factor) for features >10
• [ ] Cross-feature patterns -- Scatter matrix for top features

Red Flags

• [ ] Constant/near-constant -- Columns with single value or >99% same value
• [ ] ID-like columns -- Columns with all unique values (potential identifiers, not features)
• [ ] Leakage suspects -- Features with suspiciously high target correlation (>0.95)
• [ ] Suspicious distributions -- Features that look derived from the target