boot-analysis

Analyze and optimize the kernel boot process from bootloader to kernel ready.

Purpose

Understanding the boot sequence is critical for debugging initialization issues, optimizing boot time, and ensuring proper subsystem ordering. This skill provides tools for analyzing boot logs, verifying checkpoint progression, and identifying boot failures.

When to Use

• Boot failures: Kernel hangs or crashes during initialization
• Initialization order issues: Subsystems initialized in wrong order
• Boot time optimization: Reducing time from bootloader to ready
• Checkpoint verification: Confirming all subsystems initialize correctly
• Boot regression analysis: New code breaks boot sequence
• Understanding boot flow: Learning how kernel initialization works

Breenix Boot Sequence

Phase 1: Bootloader Handoff

What happens:

• Bootloader (bootloader crate) loads kernel
• Sets up initial page tables
• Provides memory map
• Transfers control to kernel entry point

Entry point: kernel/src/main.rs kernel_main()

Initial state:

// CPU in Long Mode (64-bit)
// Interrupts disabled
// Paging enabled (bootloader setup)
// Stack ready
// Physical memory mapped at offset

Typical log output:

[Bootloader messages]
Loading kernel...
Jumping to kernel entry point...

Phase 2: Early Initialization

Subsystems initialized (in order):

1. Logger

[ INFO] Breenix OS starting...

• Serial output configured
• Framebuffer initialized
• Log level set

2. GDT (Global Descriptor Table)

[ INFO] GDT initialized

• Kernel/user code segments
• Kernel/user data segments
• TSS (Task State Segment)

3. IDT (Interrupt Descriptor Table)

[ INFO] IDT initialized

• Exception handlers (divide by zero, page fault, etc.)
• Interrupt handlers (timer, keyboard)
• Double fault handler with IST stack

4. PIC (Programmable Interrupt Controller)

[ INFO] PIC initialized

• Remapped to avoid conflicts
• All interrupts masked initially

Phase 3: Memory Subsystem

5. Frame Allocator

[ INFO] Physical memory: 94 MiB usable
[DEBUG] Frame allocator initialized

• Reads bootloader memory map
• Identifies usable regions
• Initializes frame tracking

6. Heap Allocator

[ INFO] Heap: 1024 KiB

• Sets up kernel heap
• Enables dynamic allocation
• #[global_allocator] now functional

7. Virtual Memory

[DEBUG] Page table initialized

• Kernel page table setup
• Higher-half kernel mapping
• Recursive mapping if used

8. Kernel Stacks

[DEBUG] Kernel stack allocator initialized

• Stack bitmap allocator
• Guard pages configured
• IST stacks for exceptions

Phase 4: Device Drivers

9. Timer (PIT)

[ INFO] Timer initialized at 100 Hz

• Configures Programmable Interval Timer
• Sets interrupt frequency
• Starts tick counting

10. RTC (Real-Time Clock)

[ INFO] RTC initialized: 2025-10-23 12:34:56 UTC

• Reads hardware clock
• Caches boot time
• Enables wall-clock time APIs

11. Serial Input

[ INFO] Serial input interrupts enabled

• UART receive interrupts
• Input buffer ready
• Command processing available

12. Keyboard

[ INFO] Keyboard initialized

• PS/2 keyboard driver
• Scancode processing
• Key event generation

Phase 5: System Infrastructure

13. Interrupts Enabled

[ INFO] Enabling interrupts...

• Unmasks timer interrupt
• Unmasks keyboard interrupt
• System becomes responsive

14. System Calls

[ INFO] System call infrastructure initialized

• INT 0x80 handler registered
• Syscall dispatcher ready
• SWAPGS configured

15. Threading

[ INFO] Threading subsystem initialized

• Scheduler initialized
• Idle thread created
• Context switch infrastructure ready

16. Process Management

[ INFO] Process management initialized

• Process manager ready
• PID allocation working
• Fork/exec infrastructure initialized

Phase 6: Testing (if enabled)

17. POST (Power-On Self Test)

[ INFO] Running POST tests...
=== Memory Test ===
✅ MEMORY TEST COMPLETE
...
🎯 KERNEL_POST_TESTS_COMPLETE 🎯

• Validates subsystems
• Runs self-checks
• Confirms kernel health

18. Userspace Tests (if configured)

RING3_SMOKE: creating hello_time userspace process
[ INFO] Process created: PID 1
USERSPACE OUTPUT: Hello from userspace!

• Creates test processes
• Verifies userspace execution
• Tests system calls

Phase 7: Kernel Ready

Final state:

[ INFO] Kernel initialization complete
[ INFO] System ready

• All subsystems operational
• Ready for interactive use or more tests
• Idle loop or wait for input

Boot Analysis Techniques

Technique 1: Extract Boot Timeline

Using log-analysis skill:

# Get all initialization messages in order
grep "initialized\|INITIALIZED\|Initializing" logs/breenix_20251023_*.log

# Or more comprehensive
grep -E "INFO|WARN|ERROR" logs/latest.log | less

Expected sequence:

1. GDT initialized
2. IDT initialized
3. PIC initialized
4. Physical memory info
5. Timer initialized
6. RTC initialized
7. Interrupts enabled
8. Threading initialized
9. Process management initialized

Technique 2: Find Boot Checkpoint Failures

Identify last successful checkpoint:

# Find last "initialized" message
grep "initialized" logs/breenix_*.log | tail -10

# Or find last successful operation
grep "SUCCESS\|✅\|complete" logs/breenix_*.log | tail -10

If boot hangs:

• Last checkpoint shows how far boot progressed
• Next subsystem is where hang occurs
• Focus debugging on that subsystem

Technique 3: Compare Boot Sequences

Working vs broken boot:

# Extract initialization sequence
grep "initialized\|Initializing" working.log > working_boot.txt
grep "initialized\|Initializing" broken.log > broken_boot.txt

# Compare
diff -u working_boot.txt broken_boot.txt

Look for:

• Missing initialization steps
• Different initialization order
• New error messages
• Stops at different point

Technique 4: Time Boot Stages

Add timing checkpoints:

let start = kernel::time::get_monotonic_ms();

// Initialize subsystem
gdt::init();

let elapsed = kernel::time::get_monotonic_ms() - start;
log::info!("GDT initialization took {}ms", elapsed);

Analyze timing:

• Which stages are slow?
• Where can we optimize?
• Any unexpected delays?

Technique 5: Verify Subsystem Dependencies

Check initialization order:

// Memory must be initialized before heap
assert!(frame_allocator.is_initialized());
heap::init(); // Safe now

// GDT must be before IDT
gdt::init();
idt::init(); // Can reference GDT segments

// Interrupts must be off during sensitive operations
assert!(!are_enabled());

Common Boot Issues

Issue 1: Boot Hang

Symptoms:

• Kernel boots partway then stops
• No error message, just hangs
• Some subsystems initialized, others not

Diagnosis:

# Find last successful operation
grep "initialized\|complete" logs/latest.log | tail -1

# Check if interrupts were enabled prematurely
grep "Enabling interrupts" logs/latest.log

# Look for infinite loops
grep "WARN\|ERROR" logs/latest.log

Common causes:

1. Interrupts enabled too early

   • Timer interrupt fires before handler ready
   • Solution: Ensure all handlers registered before enabling

2. Deadlock during initialization

   • Lock acquired, never released
   • Solution: Check lock usage during boot

3. Infinite loop in subsystem init

   • Waiting for condition that never happens
   • Solution: Add timeouts or debug why condition fails

Fix patterns:

// Add checkpoint logging
log::info!("About to initialize subsystem X");
subsystem_x::init();
log::info!("Subsystem X initialized successfully");

// If hangs between checkpoints, focus on subsystem_x::init()

Issue 2: Boot Panic

Symptoms:

PANIC: [message]
Stack trace: ...

Diagnosis:

# Get panic message
grep "PANIC" logs/latest.log

# Get context
grep -B20 "PANIC" logs/latest.log

Common causes:

1. Assertion failure

   assert!(condition); // Failed during boot
   

   Check if assertion is correct or if precondition not met

2. Unwrap on None/Err

   let value = option.unwrap(); // Panic if None
   

   Use proper error handling during boot

3. Out of memory

   allocation error: Layout { ... }
   

   Increase heap size or defer allocation

Issue 3: Wrong Initialization Order

Symptoms:

• Later subsystem fails
• "Not initialized" error
• Double fault or page fault

Example:

// BAD - heap used before initialization
use alloc::vec::Vec;
let v = Vec::new(); // Panic! Heap not initialized yet
heap::init();

// GOOD
heap::init();
use alloc::vec::Vec;
let v = Vec::new(); // OK

Diagnosis:

• Check dependency chain
• Verify order matches requirements
• Look for use-before-init patterns

Issue 4: Boot Regression

Symptoms:

• Kernel booted before, now doesn't
• Recent change broke boot
• Used to reach checkpoint X, now stops at Y

Diagnosis:

# Find which commit broke it
git bisect start
git bisect bad HEAD
git bisect good last_working_commit

# Test each commit
kernel-debug-loop/scripts/quick_debug.py \
  --signal "KERNEL READY" \
  --timeout 15

Fix:

• Identify the breaking commit
• Understand what it changed
• Fix or revert

Boot Optimization

Current Boot Time

Measure with kernel-debug-loop:

# Time to specific checkpoint
kernel-debug-loop/scripts/quick_debug.py \
  --signal "Kernel initialization complete" \
  --timeout 30

Typical times (approximate):

• Early init (GDT, IDT, PIC): ~10ms
• Memory subsystem: ~50ms
• Drivers (timer, keyboard): ~20ms
• Threading/processes: ~10ms
• POST tests: ~100ms (if enabled)
• Total boot: ~200-500ms to ready state

Optimization Strategies

1. Defer non-critical initialization

// Don't initialize during boot if not needed
// keyboard::init(); // Defer until first use

// Or lazy initialization
pub fn get_keyboard() -> &'static Keyboard {
    static INIT: Once = Once::new();
    INIT.call_once(|| {
        keyboard::init();
    });
    &KEYBOARD
}

2. Parallelize independent operations

// Currently serial:
timer::init();
rtc::init();
keyboard::init();

// Could be parallel (if truly independent):
// Note: Difficult in kernel without threading during boot

3. Reduce logging verbosity

// Debug builds: verbose
#[cfg(debug_assertions)]
log::debug!("Detailed info");

// Release builds: minimal
#[cfg(not(debug_assertions))]
log::info!("Essential info only");

4. Optimize expensive operations

// Identify slow operations with timing
let start = time::get_monotonic_ms();
expensive_operation();
let elapsed = time::get_monotonic_ms() - start;
if elapsed > 10 {
    log::warn!("Slow operation: {}ms", elapsed);
}

Integration with Other Skills

With kernel-debug-loop

# Fast iteration on boot fixes
kernel-debug-loop/scripts/quick_debug.py \
  --signal "BOOT_CHECKPOINT" \
  --timeout 10

With log-analysis

# Extract boot sequence
echo '"initialized"' > /tmp/log-query.txt
./scripts/find-in-logs

# Find boot failures
echo '"PANIC|FAULT|ERROR"' > /tmp/log-query.txt
./scripts/find-in-logs

With systematic-debugging

Document boot issues:

# Problem
Kernel hangs after "Enabling interrupts"

# Root Cause
Timer interrupt handler called before scheduler ready

# Solution
Initialize scheduler before enabling interrupts

# Evidence
Before: Hang
After: Boot completes successfully

Boot Checkpoints Reference

Essential checkpoints every boot should reach:

[✓] GDT initialized
[✓] IDT initialized
[✓] PIC initialized
[✓] Physical memory detected
[✓] Heap initialized
[✓] Timer initialized
[✓] Interrupts enabled
[✓] Threading initialized
[✓] Kernel ready

If boot stops before reaching a checkpoint, debug that subsystem.

Best Practices

1. Log initialization: Every subsystem should log successful init
2. Check prerequisites: Verify dependencies before initializing
3. Fail fast: Panic early if critical init fails
4. Add checkpoints: Mark progress through boot sequence
5. Time operations: Identify bottlenecks
6. Test changes: Verify boot still works after changes
7. Document order: Comment why init order matters

Quick Reference

Key Boot Files

kernel/src/main.rs                  - Entry point, boot orchestration
kernel/src/gdt.rs                   - GDT initialization
kernel/src/interrupts/mod.rs        - IDT initialization
kernel/src/memory/frame_allocator.rs - Physical memory
kernel/src/time/timer.rs            - Timer initialization
kernel/src/time/rtc.rs              - RTC initialization

Boot Signals to Watch

"GDT initialized"
"IDT initialized"
"Physical memory:"
"Heap:"
"Timer initialized"
"Enabling interrupts"
"Threading subsystem initialized"
"Kernel initialization complete"

Summary

Boot analysis requires:

• Understanding the complete boot sequence
• Identifying checkpoints and dependencies
• Using logs to diagnose failures
• Comparing working vs broken boots
• Optimizing slow operations
• Ensuring proper initialization order

A well-understood boot sequence makes debugging initialization issues straightforward.