fix-integration-bug

This skill is for fixing bugs in existing integrations. Follow this workflow to reproduce, test, and fix integration issues.

1. Reproduce the Bug

First, understand and reproduce the issue:

# Run with console logging to see actual trace output
BRAINTRUST_ENABLE_TRACE_CONSOLE_LOG=true bundle exec appraisal provider ruby examples/provider.rb

# Or create a minimal reproduction script
BRAINTRUST_ENABLE_TRACE_CONSOLE_LOG=true bundle exec appraisal provider ruby -e '
  require "braintrust"
  # minimal reproduction code
'

2. Appraisal Commands

Test against specific gem versions:

# Install dependencies for all appraisals
bundle exec appraisal install

# Run tests for specific appraisal
bundle exec appraisal provider rake test

# Run single test file
bundle exec appraisal provider rake test TEST=test/braintrust/trace/provider_test.rb

# Run with specific seed (useful for reproducing flaky test failures from CI)
bundle exec appraisal provider rake test[12345]

# Run all appraisals
bundle exec appraisal rake test

# Re-record VCR cassettes
VCR_MODE=all bundle exec appraisal provider rake test

3. Add Failing Test Case

Write a test that reproduces the bug:

def test_bug_description
  # Arrange: Set up the scenario that triggers the bug

  # Act: Call the method

  # Assert: Verify expected behavior (this should FAIL initially)
end

4. Add Example Case (if applicable)

Add a case to the internal example that exercises the buggy code path:

• Location: examples/internal/provider.rb
• Purpose: Demonstrates the fix works end-to-end
• Follow existing example patterns (nest under root span, print output)

5. TDD Fix Cycle

1. Run failing test: bundle exec appraisal provider rake test
2. Implement minimal fix in lib/braintrust/trace/contrib/provider.rb
3. Run tests again (should pass)
4. Lint: bundle exec rake lint:fix
5. Run all appraisals: bundle exec appraisal rake test

6. Verify with MCP

Query traces to confirm the fix:

mcp__braintrust__btql_query(query: "SELECT input, output, metrics FROM project_logs LIMIT 5")

Reference Files

• Integrations: lib/braintrust/trace/contrib/{openai,anthropic,ruby_llm}.rb
• Tests: test/braintrust/trace/{openai,anthropic,ruby_llm}_test.rb
• Examples: examples/internal/{openai,anthropic,ruby_llm}.rb
• VCR cassettes: test/fixtures/vcr_cassettes/provider/