|
|
|
@@ -0,0 +1,189 @@
|
|
|
|
|
---
|
|
|
|
|
name: tactility-bluetooth
|
|
|
|
|
description: Use when debugging or extending Bluetooth/BLE in Tactility OS — BLE HID Host (central) for keyboards/mice/combo touchpads, Report ID stripping, mouse report parsing (Fosmon-style standard vs Logitech 12-bit), NimBLE scan cache including nameless/RPA devices, PUBLIC vs RANDOM addr_type fallback, and auto-connect/reconnect retry loops.
|
|
|
|
|
version: 1.0.0
|
|
|
|
|
author: Hermes Agent
|
|
|
|
|
license: MIT
|
|
|
|
|
metadata:
|
|
|
|
|
hermes:
|
|
|
|
|
tags: [tactility, bluetooth, ble, hid-host, nimble, esp32, esp32s3, combo-keyboard, auto-connect]
|
|
|
|
|
related_skills: [tactility-firmware, systematic-debugging]
|
|
|
|
|
---
|
|
|
|
|
|
|
|
|
|
# Tactility Bluetooth / BLE HID Host
|
|
|
|
|
|
|
|
|
|
## Overview
|
|
|
|
|
Tactility's BLE stack uses NimBLE host (ESP-IDF) on ESP32/S3. Central role (HID Host) connects to external keyboards, mice, and combo keyboard+touchpad devices. Relevant for:
|
|
|
|
|
|
|
|
|
|
- Fosmon Mini BT Keyboard w/ Touchpad (Amazon B00BX0YKX4 / https://a.co/d/0fg4vprm) — keyboard Report ID 1 + mouse/touchpad Report ID 2, 4-5 byte reports `[buttons, x, y, wheel, pan]`
|
|
|
|
|
- Generic combo devices that sleep, advertise without name, use RPA (Resolvable Private Address)
|
|
|
|
|
|
|
|
|
|
Builds require `device.py <id>` to generate sdkconfig; simulator POSIX path has no BT.
|
|
|
|
|
|
|
|
|
|
## When to Use
|
|
|
|
|
- Mouse doesn't move / jumps erratically on combo KB+touchpad
|
|
|
|
|
- Bluetooth won't reconnect to previously paired KB/mouse after sleep
|
|
|
|
|
- Adding new HID Host behavior, touchpad gestures, consumer reports
|
|
|
|
|
- Debugging `BLE_HS_EALREADY`, `addr_type`, bond loss, `hid_host_active` flag
|
|
|
|
|
- Changing scan filter, RSSI, adv parsing
|
|
|
|
|
|
|
|
|
|
Don't use for USB HID (`usb_host_hid`) or classic BT (Tactility uses BLE only for HID host).
|
|
|
|
|
|
|
|
|
|
## Architecture
|
|
|
|
|
|
|
|
|
|
### Key Files
|
|
|
|
|
- `Platforms/platform-esp32/source/drivers/bluetooth/esp32_ble_scan.cpp` — GAP DISC event handler, `scan_results[64]` + `scan_addrs[64]`, `ble_gap_disc_event_handler`, `ble_resolve_next_unnamed_peer`
|
|
|
|
|
- `Platforms/platform-esp32/source/drivers/bluetooth/esp32_ble.cpp` — advertising helpers, dispatch_enable/disable, `hid_host_active` atomic guard against simultaneous central connect
|
|
|
|
|
- `Platforms/platform-esp32/private/bluetooth/esp32_ble_internal.h` — `BleCtx`, `scan_mutex`, `hid_host_active`
|
|
|
|
|
- `Tactility/Source/bluetooth/BluetoothHidHost.cpp` — central connection state machine: service/char/disc, CCCD subscribe chain, Report Map read, type resolution, `NOTIFY_RX` dispatch, LVGL indev registration
|
|
|
|
|
- `Tactility/Source/bluetooth/Bluetooth.cpp` — `bt_event_bridge`, `SCAN_FINISHED -> autoConnectHidHost()`, `PROFILE_STATE_CHANGED IDLE -> rescan`
|
|
|
|
|
- `Tactility/Source/app/btmanage/*` + `btpeersettings/*` — UI, manual connect calls `hidHostConnect(addr)` directly
|
|
|
|
|
|
|
|
|
|
### Flow
|
|
|
|
|
```
|
|
|
|
|
DISC callback -> cacheScanAddr + scan_results -> BT_EVENT_PEER_FOUND -> getScanResults()
|
|
|
|
|
SCAN_FINISHED -> autoConnectHidHost() iterates scanResults, checks PairedDevice autoConnect + profile BT_PROFILE_HID_HOST -> hidHostConnect()
|
|
|
|
|
hidHostConnect: getCachedScanAddrType() for addr_type -> ble_gap_connect() -> GAP CONNECT -> security_initiate -> disc_all_svcs -> disc_all_chrs -> disc_all_dscs -> read Report Ref + Report Map -> type resolution -> CCCD subscribe -> ready
|
|
|
|
|
NOTIFY_RX: find rpt by valHandle -> switch type -> handleKeyboard/Mouse with reportId
|
|
|
|
|
```
|
|
|
|
|
|
|
|
|
|
## Combo Device Quirks — Fosmon Example
|
|
|
|
|
|
|
|
|
|
From Amazon B00BX0YKX4 (Fosmon Mini Bluetooth Keyboard QWERTY + Touchpad):
|
|
|
|
|
- Dual HID: keyboard (boot-like 8-byte mod+reserved+6keys) + mouse (3-5 byte relative)
|
|
|
|
|
- Uses Report IDs: ID 1 keyboard, ID 2 mouse; Report Map has two top-level collections `0x05 0x01 0x09 0x06 A1 01 85 01 ...` and `0x09 0x02 A1 01 85 02 ...`
|
|
|
|
|
- Some NimBLE stacks notify raw char value INCLUDING ID byte: `[0x02, buttons, x, y]` vs expected `[buttons, x, y]`
|
|
|
|
|
- Sleeps quickly, wakes on keypress with directed advertising (RPA, no name, ~30s window)
|
|
|
|
|
|
|
|
|
|
## Mouse Report Parsing — Pitfall & Fix (2026-07-13)
|
|
|
|
|
|
|
|
|
|
### Old Bug
|
|
|
|
|
```cpp
|
|
|
|
|
if (len >= 5) { // assumed Logitech 12-bit packed
|
|
|
|
|
raw_x = data[2] | ((data[3]&0x0F)<<8); sign-extend 12-bit
|
|
|
|
|
raw_y = (data[3]>>4) | (data[4]<<4);
|
|
|
|
|
}
|
|
|
|
|
```
|
|
|
|
|
Cheap touchpads send standard 4/5-byte `[btn, x8, y8, wheel, pan]`. Above decodes wheel byte as nibble → huge jumps.
|
|
|
|
|
|
|
|
|
|
### Fix Pattern (dual-heuristic)
|
|
|
|
|
1. Implement `*Internal(data,len,expected_report_id)` variants that strip leading ID if `expected_report_id !=0 && data[0]==id`.
|
|
|
|
|
2. For `plen>=5`, try both:
|
|
|
|
|
- std: `dx=(int8)p[1], dy=(int8)p[2]`
|
|
|
|
|
- 12-bit: `raw_x12 = p[2] | ((p[3]&0x0F)<<8)`, `raw_y12 = (p[3]>>4)|(p[4]<<4)` sign-extend 12-bit
|
|
|
|
|
3. Heuristic:
|
|
|
|
|
- `std_small = abs(dx)<=48 && abs(dy)<=48`, `s12_large = abs(raw_x12)>128 || abs(raw_y12)>128`
|
|
|
|
|
- If std_small && s12_large → prefer std (touchpad with wheel byte)
|
|
|
|
|
- If s12_large and raw_12 plausible `<=300` and std huge `>64` or `p[1]==0` → prefer 12-bit (Logitech)
|
|
|
|
|
- Else std
|
|
|
|
|
4. Always dispatch via Internal with `rpt.reportId` so ID stripped.
|
|
|
|
|
|
|
|
|
|
See `references/mouse-report-parsing.md`.
|
|
|
|
|
|
|
|
|
|
## BLE Scan Inclusive Caching — Pitfall & Fix
|
|
|
|
|
|
|
|
|
|
### Old Bug
|
|
|
|
|
`esp32_ble_scan.cpp:90` filtered nameless:
|
|
|
|
|
```cpp
|
|
|
|
|
if (!found && record.name[0] != '\0' && scan_count<64) // store only named
|
|
|
|
|
```
|
|
|
|
|
Bonded KB after sleep advertises only appearance+flags or directed (no name), uses RPA that rotates. Result:
|
|
|
|
|
- `scan_results` empty for that MAC
|
|
|
|
|
- `scan_addr_cache` misses → `getCachedScanAddrType()` fails → default PUBLIC, but device may be RANDOM → `ble_gap_connect rc!=0`
|
|
|
|
|
- `autoConnectHidHost()` never finds peer
|
|
|
|
|
|
|
|
|
|
### Fix
|
|
|
|
|
- Store ALL unique advertisers regardless of name. Still set `should_publish = name[0]!= '\0'` for UI cadence if needed, but cache must include nameless.
|
|
|
|
|
- On duplicate, always update `scan_addrs[i] = disc.addr` to track RPA rotation.
|
|
|
|
|
- `getScanResults()` now returns nameless too — UI shows "Unknown (aabbcc...)" which is useful for debugging; optionally filter weak RSSI in View layer.
|
|
|
|
|
|
|
|
|
|
See `references/scan-and-reconnect.md`.
|
|
|
|
|
|
|
|
|
|
## Reconnect — Addr Type & Retry
|
|
|
|
|
|
|
|
|
|
### Persistent auto-connect on boot (fix 2026-07-12)
|
|
|
|
|
Previous `autoConnectHidHost()` only connected if peer was seen in last scan. On reboot Fosmon is asleep → scan empty → no connect, and retry timer was only created inside `hidHostConnect()` so boot chain died after 1st scan.
|
|
|
|
|
|
|
|
|
|
Fix implemented in `BluetoothHidHost.cpp`:
|
|
|
|
|
- `ensureAutoConnTimer()` lazy creates `hid_autoconn_retry_timer`, called from `autoConnectHidHost()` itself (exists at boot)
|
|
|
|
|
- `hid_autoconn_empty_scan_count` tracks consecutive empty scans
|
|
|
|
|
- Phase 1: if peer in scanResults → connect with fresh addr_type + RSSI log
|
|
|
|
|
- Phase 2: not in scan but saved autoConnect peer exists:
|
|
|
|
|
- After `HID_AUTOCONN_DIRECT_AFTER=1` empty scan → direct `hidHostConnect(saved_addr)` even without advert — NimBLE does internal 8s scan/connect trying both PUBLIC/RANDOM
|
|
|
|
|
- On fail → reschedule timer 3s (fast), 10s, 30s (backoff) forever → catches wake on keypress (~30s directed adv window)
|
|
|
|
|
- Counter resets on successful connect, stops timer on connect
|
|
|
|
|
|
|
|
|
|
User confirmed: "keyboard mouse combo works great now" after mouse parse fix, but "If the board restart, it won't autoconnect" — this persistent direct-connect fixed it.
|
|
|
|
|
|
|
|
|
|
### Addr Type Fallback
|
|
|
|
|
NimBLE store can hold RANDOM, but cache lookup may miss. Pattern:
|
|
|
|
|
```cpp
|
|
|
|
|
ble_addr.type = PUBLIC; memcpy val
|
|
|
|
|
if (getCachedScanAddrType(...,&type)) type= cached;
|
|
|
|
|
rc = ble_gap_connect(..., PUBLIC, ...)
|
|
|
|
|
if (rc!=0 && type==PUBLIC) retry RANDOM;
|
|
|
|
|
if (rc!=0 && cached==RANDOM) retry PUBLIC;
|
|
|
|
|
```
|
|
|
|
|
Log which type used. Timeout 5s → 8s for slow wake.
|
|
|
|
|
|
|
|
|
|
### Periodic Retry Timer
|
|
|
|
|
Keyboard wakes on keypress for ~30s directed adv. If scan ended just before wake, single rescan misses it. Add `hid_autoconn_retry_timer` esp_timer (ESP_TIMER_TASK):
|
|
|
|
|
- Created lazily in `hidHostConnect`
|
|
|
|
|
- Stopped on successful connect
|
|
|
|
|
- In `autoConnectHidHost()` when device not found but autoConnect peers exist: `esp_timer_start_once(retry_timer, 3*1e6)`
|
|
|
|
|
- Callback `hidAutoConnRetryCb` starts scan if not scanning; `SCAN_FINISHED` -> `autoConnectHidHost()` again.
|
|
|
|
|
|
|
|
|
|
Prevent tight loop after manual disconnect? TODO in code: if user manually disconnects and autoConnect still true, it will keep retrying. Separate toggle or respect manual intent.
|
|
|
|
|
|
|
|
|
|
### `hid_host_active` Guard
|
|
|
|
|
`BleCtx.hid_host_active` prevents simultaneous central connections during `ble_resolve_next_unnamed_peer` name resolution (would fail `BLE_HS_EALREADY`). Set true in `hidHostConnect`, false on disconnect/connect fail.
|
|
|
|
|
|
|
|
|
|
## Build Env Pitfall (Hermes venv pollution)
|
|
|
|
|
|
|
|
|
|
From 2026-07-11/13 sessions:
|
|
|
|
|
- Hermes desktop venv Python 3.11 `pydantic` (3.11 `pydantic_core`) leaks via `PYTHONPATH` into ESP-IDF 3.9 env → `ModuleNotFoundError: pydantic_core._pydantic_core` or `TypeError: unsupported operand type(s) for |`
|
|
|
|
|
- Toolchain xtensa `xtensa-esp32s3-elf-gcc` not in PATH when PATH stripped to only idf env
|
|
|
|
|
- Also, if `ESP_IDF_VERSION` env unset, CMakeLists chooses simulator path → `Unknown CMake command idf_build_get_property` + SDL detection `SDL Threads needed` even after `device.py`
|
|
|
|
|
|
|
|
|
|
Fix wrapper (board-direct per user 2026-07-12 — **never simulator** for BT/driver changes, user said "Don't compile for simulator is a waste of time, build for the board directly"):
|
|
|
|
|
```bash
|
|
|
|
|
cd firmware
|
|
|
|
|
rm -rf build sdkconfig # required when lvgl.fontSize changed or switching python envs
|
|
|
|
|
env -u PYTHONPATH -u PYTHONHOME -u ESP_IDF_VERSION -u IDF_PATH \
|
|
|
|
|
PATH=$HOME/.espressif/python_env/idf5.3_py3.9_env/bin:$HOME/.espressif/tools/xtensa-esp-elf/esp-13.2.0_20240530/xtensa-esp-elf/bin:/Users/adolforeyna/esp/esp-idf/tools:/opt/homebrew/bin:/usr/bin:/bin \
|
|
|
|
|
python device.py waveshare-esp32-s3-rlcd
|
|
|
|
|
grep FONT_SIZE sdkconfig # verify 14/18/24 after font bump
|
|
|
|
|
# then
|
|
|
|
|
env -u PYTHONPATH -u PYTHONHOME \
|
|
|
|
|
PATH=...same... IDF_PATH=/Users/adolforeyna/esp/esp-idf ESP_IDF_VERSION=5.3 \
|
|
|
|
|
idf.py -p /dev/cu.usbmodem1101 flash
|
|
|
|
|
```
|
|
|
|
|
User preference: "Don't compile for simulator is a waste of time, build for the board directly." Enforced 2026-07-12.
|
|
|
|
|
For font bumps: always `rm -rf build` after editing `device.properties` lvgl.fontSize, else old `TT_LVGL_TEXT_FONT_DEFAULT_SIZE=14` stays cached.
|
|
|
|
|
|
|
|
|
|
Quick sanity without full build: `grep -rn "hidHostHandleMouseReport\|hid_host_ctx\|scan_results" firmware/ --include="*.cpp" --include="*.h"`
|
|
|
|
|
env -u PYTHONPATH -u PYTHONHOME -u ESP_IDF_VERSION -u IDF_PATH \
|
|
|
|
|
PATH=$HOME/.espressif/python_env/idf5.3_py3.9_env/bin:$HOME/.espressif/tools/xtensa-esp-elf/esp-13.2.0_20240530/xtensa-esp-elf/bin:/Users/adolforeyna/esp/esp-idf/tools:/opt/homebrew/bin:/usr/bin:/bin \
|
|
|
|
|
python device.py waveshare-esp32-s3-rlcd
|
|
|
|
|
# then
|
|
|
|
|
env -u PYTHONPATH ...same PATH... idf.py build
|
|
|
|
|
# flash
|
|
|
|
|
env -u PYTHONPATH ... idf.py -p /dev/cu.usbmodem1101 flash
|
|
|
|
|
```
|
|
|
|
|
User preference: "Don't compile for simulator is a waste of time, build for the board directly." Enforced 2026-07-12.
|
|
|
|
|
|
|
|
|
|
Quick sanity without full build: `grep -rn "hidHostHandleMouseReport\|hid_host_ctx\|scan_results" firmware/ --include="*.cpp" --include="*.h"`
|
|
|
|
|
|
|
|
|
|
## Common Pitfalls
|
|
|
|
|
1. Assuming 5-byte mouse = 12-bit. Always implement dual heuristic.
|
|
|
|
|
2. Forgetting Report ID strip — combo devices include ID. Pass `rpt.reportId` to handler.
|
|
|
|
|
3. Filtering scan to named only — breaks bonded auto-connect. Cache all, filter UI only.
|
|
|
|
|
4. Not updating `scan_addrs` on duplicate — RPA rotation missed.
|
|
|
|
|
5. Only trying PUBLIC — many keyboards use RANDOM RPA. Always fallback.
|
|
|
|
|
6. Single scan after disconnect — misses wake window. Use 3s retry timer.
|
|
|
|
|
7. `hid_host_active` not cleared on connect fail — blocks future name resolution.
|
|
|
|
|
|
|
|
|
|
## References
|
|
|
|
|
- `references/combo-keyboard-mouse.md` — Fosmon report map structure, ID table, sleep/wake behavior
|
|
|
|
|
- `references/mouse-report-parsing.md` — dual-decode code snippet + test vectors
|
|
|
|
|
- `references/scan-and-reconnect.md` — inclusive cache patch, addr_type matrix, retry timer pattern
|
Adolfo Reyna