|
|
|
@@ -12,6 +12,43 @@ Develop an app/workflow that lets Hermes communicate with Adolfo through the reM
|
|
|
|
|
## Current Status
|
|
|
|
|
Initial SSH-based proof of concept works.
|
|
|
|
|
|
|
|
|
|
2026-07-16 iMac dev spike:
|
|
|
|
|
- iMac SSH alias `imac` works as `adolforeyna@192.168.68.124`.
|
|
|
|
|
- Installed official reMarkable Paper Pro SDK `4.0.813` into `~/remarkable-dev/toolchains/rmpp-4.0.813` on the iMac.
|
|
|
|
|
- Verified SDK environment: `environment-setup-cortexa53-crypto-remarkable-linux`, compiler `aarch64-remarkable-linux-gcc`, Qt `6.5.2`.
|
|
|
|
|
- Built a CLI Hello World binary at `~/remarkable-dev/hello-rmpp/hello_rmpp` on the iMac.
|
|
|
|
|
- Built a visual Qt Quick Hello World binary at `~/remarkable-dev/hello-rmpp-qt/hello_rmpp_qt` on the iMac.
|
|
|
|
|
- Copied tarballs locally for archive/handoff:
|
|
|
|
|
- `/home/adolforeyna/remarkable-dev/hello-rmpp.tar.gz`
|
|
|
|
|
- `/home/adolforeyna/remarkable-dev/hello-rmpp-qt.tar.gz`
|
|
|
|
|
- Tablet SSH alias `remarkable` was initially stale (`.107`), but the Paper Pro was found at `192.168.68.128`.
|
|
|
|
|
- Deployed and ran `/home/root/hello_rmpp` on the Paper Pro successfully; output: `Hello from a custom reMarkable Paper Pro app!`
|
|
|
|
|
- Deployed `/home/root/hello_rmpp_qt` to the Paper Pro.
|
|
|
|
|
- Ran the visual Qt app with `QT_QPA_PLATFORM=epaper` after briefly stopping xochitl, then killed the app and restarted xochitl.
|
|
|
|
|
- User visually confirmed the official example worked on the Paper Pro screen.
|
|
|
|
|
- Visual app runtime logs confirmed the Paper Pro e-paper Qt platform initialized, including screen geometry `1620x2160` and touch range `2064x2832`.
|
|
|
|
|
- First Hermes Ink Chat prototype based on Qt Quick worked visually but was too slow for handwriting and only received finger touch through QML `MouseArea`.
|
|
|
|
|
- Paper Pro exposes separate input devices:
|
|
|
|
|
- `/dev/input/event2` = `Elan marker input`
|
|
|
|
|
- `/dev/input/event3` = `Elan touch input`
|
|
|
|
|
- Marker probe built/deployed at `/home/root/marker_probe`; marker axis ranges:
|
|
|
|
|
- `ABS_X`: `0..11180`, resolution `2832`
|
|
|
|
|
- `ABS_Y`: `0..15340`, resolution `2064`
|
|
|
|
|
- `ABS_PRESSURE`: `0..4096`
|
|
|
|
|
- `ABS_DISTANCE`: `0..65535`
|
|
|
|
|
- `ABS_TILT_X/Y`: `-9000..9000`
|
|
|
|
|
- Native C++ pen app now reads `/dev/input/event2`, maps marker coordinates through full-screen-to-local drawing-area offset, gates drawing on `BTN_TOUCH` or `ABS_PRESSURE >= 80` so hover/proximity does not draw, and throttles repaint to reduce e-paper churn.
|
|
|
|
|
- `Send to Hermes` saves SVG ink to `/home/root/hermes-chat/outbox/ink-native-*.svg` and updates `/home/root/hermes-chat/outbox/latest.txt`.
|
|
|
|
|
- Pulled latest SVG from tablet, converted to PNG, and Hermes vision read the handwriting as: `Hey there Pal!`
|
|
|
|
|
- App build `bd5b992d...` adds inbox polling: it checks `/home/root/hermes-chat/inbox/response.txt` every ~2s and updates the console when Hermes writes a response.
|
|
|
|
|
- Remaining work: create the automatic bridge loop that watches/pulls `latest.txt`, sends the image to Hermes/OCR, and writes `/home/root/hermes-chat/inbox/response.txt`; current test response was written manually.
|
|
|
|
|
- Pi-side sidecar logger installed on `aeropi5` under `~/remarkable-logger/` with user systemd timer `remarkable-logger.timer` every 30s. It SSH-polls Paper Pro `192.168.68.128`, logs JSONL to `~/remarkable-logger/events.jsonl`, writes `latest.json`/`state.json`, tracks online/offline, `xochitl` active state, most-recent modified document as current-document guess, and watched terms `diary,dairy,journal` for quasi-live document updates. First run found 364 docs and watched docs including `Journal2026`, `Journal`, `Daily Journaling`, `Daily Journal`, and `DairyGpt`.
|
|
|
|
|
- xochitl event-source probe: `journalctl -u xochitl` did not emit document-open/page-open entries at normal verbosity during a live monitor; only battery wake lines appeared. DBus introspection on xochitl unique names was access denied. `busctl --system monitor` works for system signals but no xochitl document signal was confirmed. xochitl keeps `.eventcache.db`/WAL with telemetry JSON fragments (`domain: document`, tool/template/actions), but committed DB had only generic telemetry and no document UUID/name/open event from the test. `.config/remarkable/xochitl.conf` had `LastOpen=` blank. Conclusion: rely on xochitl file mtimes/content/page `.rm` changes for current-document/quasi-live tracking rather than a documented/open xochitl event feed.
|
|
|
|
|
- Device-side sidecar installed on Paper Pro: `/etc/systemd/system/remarkable-sidecar.service` enabled/running; script `/home/root/remarkable-sidecar/remarkable_sidecar.sh`; daily text logs `/home/root/remarkable-sidecar/logs/YYYY-MM-DD.txt`; state snapshots `/home/root/remarkable-sidecar/state/`. It logs `online`, 60s `heartbeat`, best-effort `offline`, and `document_change` by comparing xochitl file mtimes every 20s. BusyBox `nc` lacks UDP, so ARM64 helper `/home/root/remarkable-sidecar/bin/udp_broadcast` was built with the reMarkable SDK on iMac and broadcasts to `192.168.68.255:49321`.
|
|
|
|
|
- Pi UDP listener installed on `aeropi5`: `~/remarkable-logger/remarkable_udp_listener.py` as user service `remarkable-udp-listener.service`, logging UDP JSONL to `~/remarkable-logger/udp/YYYY-MM-DD.jsonl` and latest IP to `~/remarkable-logger/latest_device.json`. Pi poller `remarkable_logger.py` was patched to use `latest_device.json` when `REMARKABLE_HOST` is not set, so DHCP/IP changes are caught by tablet broadcast. Duplicate UDP packets from Pi eth0+wlan0 are deduped within 2s.
|
|
|
|
|
- Sidecar verification after user highlighted a book and edited `Today's Goals`: Pi poller detected `ai-2027.pdf` at `2026-07-16T23:40:38Z` and `Today's Goals` at `2026-07-16T23:41:11Z`. Device sidecar initially only logged heartbeats because Paper Pro lacks `comm`; script was patched to use BusyBox `awk` set-difference. After restart/recovery it emitted `document_change` UDP/text lines for `ai-2027.pdf`, `Today's Goals`, and a later `Quick sheets` change.
|
|
|
|
|
- v6 `.rm` rendering spike: `rmrl` renders older/PDF-backed docs but fails on Paper Pro v6 pages with `UnsupportedVersion`. PyPI `rmscene` parses v6 `Line` items. Shader/highlighter colors are in `Line.color_rgba=(r,g,b,a)` while normal ink usually has `color_rgba=None` plus pen/color enums. Installed lightweight renderer `/home/adolforeyna/remarkable-logger/tools/render_rm_v6_color.py` with venv `/home/adolforeyna/remarkable-logger/.venv-render`; verified against Quick sheets page 66 and Journal2026 changed page.
|
|
|
|
|
|
|
|
|
|
Hermes can:
|
|
|
|
|
- Connect to the reMarkable via SSH alias `remarkable`.
|
|
|
|
|
- Locate xochitl documents under `/home/root/.local/share/remarkable/xochitl/`.
|
|
|
|
|
aeroreyna