basic1/.claude/skills/sd_card_best_practices.md

# SD Card Best Practices for RP2350 + FatFS

This document captures critical best practices for working with SD card operations in this project, based on lessons learned during development.

## 1. SPI Bus Contention (CRITICAL)

### ⚠️ CRITICAL: Display and SD Card Share the Same SPI Bus

**The display and SD card use the same SPI bus and CANNOT be accessed simultaneously.** Attempting to do so will cause the Pico to crash or behave unpredictably.

### Real-World Example: Game Launch Crash

The serial uploader originally crashed when launching games because:

1. `SerialUploader::launch_game()` writes file to SD (SPI)
2. Immediately calls `select_game_by_name()`
3. `LuaGame::load_script()` reads from SD (SPI)
4. **Meanwhile**, Core 1 is refreshing the display (also SPI)
5. **CRASH** due to simultaneous SPI access

### Solution: Wait for Display to Be Idle

Before any SD card operation that isn't already protected, ensure no display refresh is in progress:

```cpp
// In main loop:
if (serial_uploader.wants_to_launch_game() && !is_refresh_in_progress()) {
    // Safe to launch now - no SPI conflict with display
    bool game_launched = serial_uploader.complete_launch();
    // ...
}
```

### Key Patterns for Avoiding Contention

**Pattern 1: Check `is_refresh_in_progress()` before SD operations**
```cpp
if (!is_refresh_in_progress()) {
    uint prev_speed = sd_card_set_spi_speed();
    // SD card operations
    sd_card_restore_spi_speed(prev_speed);
}
```

**Pattern 2: Split operations into "prepare" and "execute" phases**
```cpp
// Phase 1: Prepare (safe, no SD access)
void prepare_operation() {
    state = READY_TO_EXECUTE;
}

// Phase 2: Execute (only when !is_refresh_in_progress())
bool execute_operation() {
    uint prev_speed = sd_card_set_spi_speed();
    // SD card operations here
    sd_card_restore_spi_speed(prev_speed);
}
```

**Pattern 3: Keep main loop responsive**
```cpp
while (1) {
    // Check if we need to do SD operation
    if (needs_sd_operation && !is_refresh_in_progress()) {
        perform_sd_operation();
    }

    // Don't sleep if waiting for SD operation window
    bool stay_awake = pending_refresh || needs_sd_operation;
    if (!stay_awake) {
        __wfi();  // Sleep until interrupt
    }
}
```

### When This Matters Most

- **Game loading**: Reading Lua scripts from SD during game launch
- **Serial uploads**: Writing files and immediately loading them
- **Save/load operations**: Writing game state to SD
- **Directory scanning**: Re-scanning games while display is active

### The Core Architecture

This project uses **dual-core display refresh**:
- **Core 0**: Main logic, input processing, game updates, SD card operations
- **Core 1**: Display refresh (writes framebuffer to display via SPI)

Core 1 runs asynchronously, so you must explicitly check `is_refresh_in_progress()` before SD operations.

## 2. SPI Speed Management

### Critical Rule: Always Set SD Card Speed Before Operations

The display and SD card share the same SPI bus but operate at different speeds:
- **Display**: 32 MHz (fast)
- **SD Card**: 12.5 MHz (slower, more reliable)

**ALWAYS wrap SD card operations with speed switching:**

```cpp
// Save current speed and switch to SD card speed
uint prev_speed = sd_card_set_spi_speed();

// ... SD card operations here ...
// f_open(), f_write(), f_read(), f_readdir(), etc.

// Restore previous speed for display
sd_card_restore_spi_speed(prev_speed);
```

### Why This Matters

Running SD card operations at the wrong SPI speed causes:
- Unreliable reads/writes
- Corrupted data
- `FR_DISK_ERR` errors from FatFS
- Hardware-level protocol failures (0xFF data responses)

### Where Speed Switching is Already Handled

These functions handle SPI speed internally (you don't need to wrap them):
- `LuaGameLoader::register_all_games()`
- All functions in `sd_card.c` (low-level operations)

### Where You MUST Handle Speed Switching

Any code that calls FatFS functions directly:
- `f_open()`, `f_close()`
- `f_read()`, `f_write()`
- `f_opendir()`, `f_readdir()`, `f_closedir()`
- `f_stat()`, `f_mkdir()`, `f_unlink()`
- `f_getfree()`, `f_sync()`

## 2. SD Card Write Protocol

### The Data Response Polling Issue

When writing to SD card with `CMD24` (write single block), the data response token may not arrive immediately. You must poll for it:

```cpp
// After sending data block and CRC:
uint8_t response = 0xFF;
for (int i = 0; i < 10; i++) {
    response = sd_card_transfer(0xFF);
    if (response != 0xFF) {
        break;  // Got the response
    }
}

// Check if data was accepted
if ((response & 0x1F) != SD_DATA_ACCEPTED) {
    // Write failed
}
```

**Why:** The SD card may need a few clock cycles before sending the data response token. Reading only once may return 0xFF (no response yet).

### Wait for Card Ready After CMD24

After sending the write command and before sending data:

```cpp
// After CMD24 command:
uint8_t ready_byte;
do {
    ready_byte = sd_card_transfer(0xFF);
    timeout_count++;
    if (timeout_count > 1000) {
        return false;  // Timeout
    }
} while (ready_byte != 0xFF);
```

This ensures the card is ready to receive the data block.

## 3. FatFS Best Practices

### Always Check Return Codes

```cpp
FRESULT fr = f_open(&fil, path, FA_CREATE_ALWAYS | FA_WRITE);
if (fr != FR_OK) {
    printf("ERROR: f_open failed: %d\n", fr);
    // Clean up and return
    return false;
}
```

Common FatFS error codes:
- `FR_OK (0)`: Success
- `FR_DISK_ERR (1)`: Low-level disk error (often SPI speed issue)
- `FR_NOT_READY (3)`: Card not initialized
- `FR_NO_FILE (4)`: File not found
- `FR_NO_PATH (5)`: Path not found
- `FR_EXIST (8)`: File/directory already exists

### Use FA_CREATE_ALWAYS to Overwrite

For rapid iteration (like our serial uploader):

```cpp
f_open(&fil, path, FA_CREATE_ALWAYS | FA_WRITE);
```

This overwrites existing files, perfect for development.

### Write in Chunks for Large Files

For files larger than 512 bytes, write in chunks:

```cpp
const uint32_t CHUNK_SIZE = 512;
uint32_t total_written = 0;

while (total_written < total_size) {
    uint32_t chunk_size = min(CHUNK_SIZE, total_size - total_written);
    UINT bytes_written;

    fr = f_write(&fil, buffer + total_written, chunk_size, &bytes_written);
    if (fr != FR_OK || bytes_written != chunk_size) {
        // Handle error
        break;
    }

    total_written += bytes_written;
}
```

### Always Sync and Close

```cpp
f_sync(&fil);  // Ensure data is written to card
f_close(&fil); // Close file and update directory
```

Skipping `f_sync()` can lead to data loss if power is lost.

## 4. Memory Management

### Clean Up After Re-scanning

When re-scanning games (like after upload), clean up old data:

```cpp
// Clear game launcher entries
game_launcher->clear_games();

// Clear Lua game factory data
LuaGameLoader::clear_factory_data();

// Re-scan
LuaGameLoader::register_all_games(game_launcher);
```

**Why:** Without cleanup, you get duplicate registrations and memory leaks.

### Delete Old Game Instances Before Creating New Ones

```cpp
if (selected_game) {
    delete selected_game;
    selected_game = nullptr;
}

// Now create new game
selected_game = factory(width, height, renderer, gui, input_manager);
```

**Critical for Lua games:** Each LuaGame has a Lua state. Not cleaning up the old one before creating a new one causes conflicts and freezes.

## 5. Debugging Tips

### Add Targeted Debug Output

When debugging SD operations, add prints at key points:

```cpp
printf("✓ Wrote %u bytes to %s\n", bytes_written, filepath);
```

But avoid spamming the console - it slows down operations significantly.

### Check Hardware Layer First

If FatFS returns `FR_DISK_ERR`, the issue is usually at the hardware level:
1. Check SPI speed (most common issue)
2. Check SD card write protection
3. Check physical SD card connection
4. Verify SD card is properly initialized

### Use Root Directory for Testing

When debugging writes, test with root directory first:

```cpp
FIL test_file;
if (f_open(&test_file, "/test.txt", FA_CREATE_ALWAYS | FA_WRITE) == FR_OK) {
    printf("Root write OK\n");
    f_close(&test_file);
    f_unlink("/test.txt");
}
```

This isolates directory-related issues.

## 6. Common Pitfalls

### ❌ DON'T: Access SD Card During Display Refresh (MOST CRITICAL)

```cpp
// BAD - Will crash the Pico!
void launch_game() {
    scan_games();  // Reads SD card
    selected_game = create_game();  // Reads Lua script from SD
}
// Called directly without checking if display is refreshing
```

### ✅ DO: Wait for Display to Be Idle

```cpp
// GOOD - Wait for safe window
if (!is_refresh_in_progress()) {
    uint prev_speed = sd_card_set_spi_speed();
    scan_games();
    selected_game = create_game();
    sd_card_restore_spi_speed(prev_speed);
}
```

### ❌ DON'T: Forget SPI Speed Management

```cpp
// BAD - Will fail or be unreliable
f_open(&fil, "/games/test.lua", FA_WRITE);
```

### ✅ DO: Always Switch Speeds

```cpp
// GOOD
uint prev_speed = sd_card_set_spi_speed();
f_open(&fil, "/games/test.lua", FA_WRITE);
// ... operations ...
sd_card_restore_spi_speed(prev_speed);
```

### ❌ DON'T: Assume Immediate Data Response

```cpp
// BAD - May get 0xFF (no response yet)
uint8_t response = sd_card_transfer(0xFF);
if (response != 0x05) {
    // Might incorrectly fail
}
```

### ✅ DO: Poll for Data Response

```cpp
// GOOD - Poll until response arrives
uint8_t response = 0xFF;
for (int i = 0; i < 10; i++) {
    response = sd_card_transfer(0xFF);
    if (response != 0xFF) break;
}
```

### ❌ DON'T: Skip Error Checking

```cpp
// BAD
f_write(&fil, buffer, size, &bytes_written);
f_close(&fil);
```

### ✅ DO: Check Every Return Value

```cpp
// GOOD
if (f_write(&fil, buffer, size, &bytes_written) != FR_OK) {
    printf("Write failed\n");
    f_close(&fil);
    return false;
}
```

### ❌ DON'T: Create New Games Without Cleanup

```cpp
// BAD - Memory leak and Lua state conflicts
selected_game = new LuaGame(...);
```

### ✅ DO: Clean Up First

```cpp
// GOOD
if (selected_game) {
    delete selected_game;
    selected_game = nullptr;
}
selected_game = new LuaGame(...);
```

## 7. Serial Upload Pattern

The serial uploader demonstrates the complete pattern:

```cpp
bool SerialUploader::write_file_to_sd() {
    // 1. Validate input
    if (!file_buffer || bytes_received == 0) return false;

    // 2. Set SD card SPI speed
    uint prev_speed = sd_card_set_spi_speed();

    // 3. Ensure directory exists
    f_mkdir("/games");

    // 4. Open file (overwrite mode for iteration)
    FIL fil;
    FRESULT fr = f_open(&fil, filepath, FA_CREATE_ALWAYS | FA_WRITE);
    if (fr != FR_OK) {
        sd_card_restore_spi_speed(prev_speed);
        return false;
    }

    // 5. Write in chunks
    const uint32_t CHUNK_SIZE = 512;
    uint32_t total_written = 0;
    while (total_written < bytes_received) {
        uint32_t chunk = min(CHUNK_SIZE, bytes_received - total_written);
        UINT written;

        fr = f_write(&fil, file_buffer + total_written, chunk, &written);
        if (fr != FR_OK || written != chunk) {
            f_close(&fil);
            sd_card_restore_spi_speed(prev_speed);
            return false;
        }

        total_written += written;
    }

    // 6. Sync and close
    f_sync(&fil);
    f_close(&fil);

    // 7. Restore display SPI speed
    sd_card_restore_spi_speed(prev_speed);

    return true;
}
```

## 9. Testing Checklist

When implementing new SD card functionality:

- [ ] **SPI bus contention checked** - verify `!is_refresh_in_progress()` before SD operations
- [ ] SPI speed switching is in place
- [ ] All FatFS return codes are checked
- [ ] Files are properly closed after operations
- [ ] Memory is cleaned up (no leaks)
- [ ] Error messages are informative
- [ ] Tested with both small and large files
- [ ] Tested overwriting existing files
- [ ] Tested with non-existent directories
- [ ] Verified data integrity (read back after write)

## Summary

The most important rules:
1. **⚠️ CRITICAL: Avoid SPI bus contention** - Check `!is_refresh_in_progress()` before SD operations (display and SD share SPI bus)
2. **Always manage SPI speed** around FatFS operations
3. **Poll for SD card responses** - don't assume immediate response
4. **Check error codes** on every operation
5. **Clean up memory** before creating new game instances
6. **Write in chunks** for large files
7. **Sync before closing** to ensure data is written

Following these practices will save hours of debugging SD card issues!