---
Date: 2026-06-17
Author: Hermes
Tags: [project, remarkable, hermes, ssh, xochitl, app-development]
---

# reMarkable Hermes Communication App

## Purpose
Develop an app/workflow that lets Hermes communicate with Adolfo through the reMarkable Paper Pro: Adolfo writes on the tablet, Hermes reads/transcribes the response, then Hermes adds typed/color-coded responses back into the document or a dedicated app surface.

## Current Status
Initial SSH-based proof of concept works.

2026-07-16 iMac dev spike:
- iMac SSH alias `imac` works as `adolforeyna@192.168.68.124`.
- Installed official reMarkable Paper Pro SDK `4.0.813` into `~/remarkable-dev/toolchains/rmpp-4.0.813` on the iMac.
- Verified SDK environment: `environment-setup-cortexa53-crypto-remarkable-linux`, compiler `aarch64-remarkable-linux-gcc`, Qt `6.5.2`.
- Built a CLI Hello World binary at `~/remarkable-dev/hello-rmpp/hello_rmpp` on the iMac.
- Built a visual Qt Quick Hello World binary at `~/remarkable-dev/hello-rmpp-qt/hello_rmpp_qt` on the iMac.
- Copied tarballs locally for archive/handoff:
  - `/home/adolforeyna/remarkable-dev/hello-rmpp.tar.gz`
  - `/home/adolforeyna/remarkable-dev/hello-rmpp-qt.tar.gz`
- Tablet SSH alias `remarkable` was initially stale (`.107`), but the Paper Pro was found at `192.168.68.128`.
- Deployed and ran `/home/root/hello_rmpp` on the Paper Pro successfully; output: `Hello from a custom reMarkable Paper Pro app!`
- Deployed `/home/root/hello_rmpp_qt` to the Paper Pro.
- Ran the visual Qt app with `QT_QPA_PLATFORM=epaper` after briefly stopping xochitl, then killed the app and restarted xochitl.
- User visually confirmed the official example worked on the Paper Pro screen.
- Visual app runtime logs confirmed the Paper Pro e-paper Qt platform initialized, including screen geometry `1620x2160` and touch range `2064x2832`.
- First Hermes Ink Chat prototype based on Qt Quick worked visually but was too slow for handwriting and only received finger touch through QML `MouseArea`.
- Paper Pro exposes separate input devices:
  - `/dev/input/event2` = `Elan marker input`
  - `/dev/input/event3` = `Elan touch input`
- Marker probe built/deployed at `/home/root/marker_probe`; marker axis ranges:
  - `ABS_X`: `0..11180`, resolution `2832`
  - `ABS_Y`: `0..15340`, resolution `2064`
  - `ABS_PRESSURE`: `0..4096`
  - `ABS_DISTANCE`: `0..65535`
  - `ABS_TILT_X/Y`: `-9000..9000`
- Native C++ pen app now reads `/dev/input/event2`, maps marker coordinates through full-screen-to-local drawing-area offset, gates drawing on `BTN_TOUCH` or `ABS_PRESSURE >= 80` so hover/proximity does not draw, and throttles repaint to reduce e-paper churn.
- `Send to Hermes` saves SVG ink to `/home/root/hermes-chat/outbox/ink-native-*.svg` and updates `/home/root/hermes-chat/outbox/latest.txt`.
- Pulled latest SVG from tablet, converted to PNG, and Hermes vision read the handwriting as: `Hey there Pal!`
- App build `bd5b992d...` adds inbox polling: it checks `/home/root/hermes-chat/inbox/response.txt` every ~2s and updates the console when Hermes writes a response.
- Remaining work: create the automatic bridge loop that watches/pulls `latest.txt`, sends the image to Hermes/OCR, and writes `/home/root/hermes-chat/inbox/response.txt`; current test response was written manually.
- Pi-side sidecar logger installed on `aeropi5` under `~/remarkable-logger/` with user systemd timer `remarkable-logger.timer` every 30s. It SSH-polls Paper Pro `192.168.68.128`, logs JSONL to `~/remarkable-logger/events.jsonl`, writes `latest.json`/`state.json`, tracks online/offline, `xochitl` active state, most-recent modified document as current-document guess, and watched terms `diary,dairy,journal` for quasi-live document updates. First run found 364 docs and watched docs including `Journal2026`, `Journal`, `Daily Journaling`, `Daily Journal`, and `DairyGpt`.
- xochitl event-source probe: `journalctl -u xochitl` did not emit document-open/page-open entries at normal verbosity during a live monitor; only battery wake lines appeared. DBus introspection on xochitl unique names was access denied. `busctl --system monitor` works for system signals but no xochitl document signal was confirmed. xochitl keeps `.eventcache.db`/WAL with telemetry JSON fragments (`domain: document`, tool/template/actions), but committed DB had only generic telemetry and no document UUID/name/open event from the test. `.config/remarkable/xochitl.conf` had `LastOpen=` blank. Conclusion: rely on xochitl file mtimes/content/page `.rm` changes for current-document/quasi-live tracking rather than a documented/open xochitl event feed.
- Device-side sidecar installed on Paper Pro: `/etc/systemd/system/remarkable-sidecar.service` enabled/running; script `/home/root/remarkable-sidecar/remarkable_sidecar.sh`; daily text logs `/home/root/remarkable-sidecar/logs/YYYY-MM-DD.txt`; state snapshots `/home/root/remarkable-sidecar/state/`. It logs `online`, 60s `heartbeat`, best-effort `offline`, and `document_change` by comparing xochitl file mtimes every 20s. BusyBox `nc` lacks UDP, so ARM64 helper `/home/root/remarkable-sidecar/bin/udp_broadcast` was built with the reMarkable SDK on iMac and broadcasts to `192.168.68.255:49321`.
- Pi UDP listener installed on `aeropi5`: `~/remarkable-logger/remarkable_udp_listener.py` as user service `remarkable-udp-listener.service`, logging UDP JSONL to `~/remarkable-logger/udp/YYYY-MM-DD.jsonl` and latest IP to `~/remarkable-logger/latest_device.json`. Pi poller `remarkable_logger.py` was patched to use `latest_device.json` when `REMARKABLE_HOST` is not set, so DHCP/IP changes are caught by tablet broadcast. Duplicate UDP packets from Pi eth0+wlan0 are deduped within 2s.
- Sidecar verification after user highlighted a book and edited `Today's Goals`: Pi poller detected `ai-2027.pdf` at `2026-07-16T23:40:38Z` and `Today's Goals` at `2026-07-16T23:41:11Z`. Device sidecar initially only logged heartbeats because Paper Pro lacks `comm`; script was patched to use BusyBox `awk` set-difference. After restart/recovery it emitted `document_change` UDP/text lines for `ai-2027.pdf`, `Today's Goals`, and a later `Quick sheets` change.
- v6 `.rm` rendering spike: `rmrl` renders older/PDF-backed docs but fails on Paper Pro v6 pages with `UnsupportedVersion`. PyPI `rmscene` parses v6 `Line` items. Shader/highlighter colors are in `Line.color_rgba=(r,g,b,a)` while normal ink usually has `color_rgba=None` plus pen/color enums. Installed lightweight renderer `/home/adolforeyna/remarkable-logger/tools/render_rm_v6_color.py` with venv `/home/adolforeyna/remarkable-logger/.venv-render`; verified against Quick sheets page 66 and Journal2026 changed page.

Hermes can:
- Connect to the reMarkable via SSH alias `remarkable`.
- Locate xochitl documents under `/home/root/.local/share/remarkable/xochitl/`.
- Find the document named `Hermes` via `.metadata` `visibleName`.
- Download the document files locally.
- Read handwritten replies from the generated page thumbnail/image.
- Generate a PDF-backed update containing:
  - Adolfo's handwritten response transcribed into typed text.
  - Hermes' response in a different font/color.
- Upload the updated document back to the tablet.

## Important Findings
- reMarkable documents are stored as UUID-based companion files in xochitl:
  - `{UUID}.metadata`
  - `{UUID}.content`
  - `{UUID}.pagedata`
  - optional `{UUID}.pdf` / `{UUID}.epub`
  - `{UUID}/` directory containing page `.rm` stroke/scene files
  - optional `{UUID}.thumbnails/`
- The official xochitl docs say xochitl should not be running when accessing or changing stored documents.
- Replacing a PDF while xochitl is running can update files on disk but not reliably refresh the tablet UI.
- Restarting only `rm-sync` can detect changed files and rebuild `.tree`, but did not force the visible document to refresh.
- Closing/reopening the document can help, but cached annotation/stroke layers may still appear.
- Handwriting remains visible if the page `.rm` overlay files are left in the document UUID directory.
- A reliable PDF-replacement flow was:
  1. Stop xochitl briefly.
  2. Stop rm-sync if needed.
  3. Back up the `{UUID}*` document set.
  4. Remove stale `{UUID}/` page `.rm` overlay files and old thumbnails.
  5. Upload updated `.pdf`, `.content`, `.metadata`, `.pagedata`, and thumbnail.
  6. Start xochitl again.
  7. Verify xochitl and rm-sync are active.
- This works, but stopping xochitl is disruptive. It should not be the final UX.

## Tested Document
- Visible name: `Hermes`
- UUID: `b109ab4c-f21d-4c0e-9df7-248b20dfb9b0`
- Backups made on tablet during testing:
  - `/home/root/hermes-backups/20260617-122145`
  - `/home/root/hermes-backups/20260617-122818`
  - `/home/root/hermes-backups/20260617-123411`

## References
- xochitl documentation: https://developer.remarkable.com/documentation/xochitl
- Qt/e-paper application documentation: https://developer.remarkable.com/documentation/qt_epaper
- Official Qt Quick examples: https://github.com/reMarkable/remarkable-developer-examples
- Awesome reMarkable project index: https://github.com/reHackable/awesome-reMarkable
- Toltec community package repository: https://github.com/toltec-dev/toltec
- rmkit application framework: https://github.com/rmkit-dev/rmkit
- Oxide launcher/desktop environment: https://github.com/Eeems-Org/oxide
- libremarkable Rust low-level device framework: https://github.com/canselcik/libremarkable

## Design Direction
The better long-term approach is likely a dedicated reMarkable app or Qt Quick/e-paper application rather than repeatedly replacing xochitl PDF files.

Possible directions:
- Build a Qt/e-paper app that shows a Hermes conversation surface directly on the tablet.
- Use SSH or a local bridge process to exchange messages between the tablet and Hermes.
- Keep xochitl as the handwritten input surface only if we can safely read `.rm`/scene data and write generated `.rm`/scene content without restarting xochitl.
- Prefer a non-disruptive workflow where xochitl stays running and the tablet UI refreshes naturally.

## Open Questions
- Can we generate valid `.rm`/scene stroke data for typed or rendered Hermes replies so xochitl treats it like normal ink?
- Can a custom Qt/e-paper app coexist cleanly with xochitl on Paper Pro?
- What is the best transport: SSH polling, local service, webhook, file drop, or a small tablet-side daemon?
- Can handwriting recognition be done directly from `.rm` files instead of relying on thumbnails/screenshots?
- How should message history be persisted: in xochitl, a tablet app store, or the family brain?

## Next Steps
- Study the Qt/e-paper docs and sample app path.
- Determine how to build/deploy a minimal Paper Pro Qt app.
- Prototype a simple tablet-side Hermes chat screen.
- Investigate `.rm`/scene format generation only if xochitl integration remains preferable.
