Compare commits
12 Commits
main
..
6a1fe5dda6
| Author | SHA1 | Date | |
|---|---|---|---|
| 6a1fe5dda6 | |||
| d06f8ddbb5 | |||
| a2bb8e5132 | |||
| 9ee0599bec | |||
| c3689e82cc | |||
| 4c71c77bf6 | |||
| 0595e149d4 | |||
| 995297314d | |||
| 349034b8ba | |||
| 3085bac5f1 | |||
| b03cb2b244 | |||
| 3a2e6f3f11 |
@@ -39,12 +39,6 @@ print(f"undef {len(undef)} missing {missing}")
|
||||
|
||||
### New missing from v2/v3/v4/v5 UI passes
|
||||
|
||||
- `device_get_by_name`, `device_get_first_by_type` (out-param API) → **Not exported** on flashed 0.8.0-dev (verified via serial on .129: `E ELF: Can't find symbol device_get_by_name`). The firmware exports the **legacy** `device_find_by_name` / `device_find_first_by_type` instead. Gotcha: newer SDK/CDN headers declare only `device_get_*` (so `device_find_*` fails to compile), but the flashed firmware only exports `device_find_*` (so `device_get_*` fails at load). Fix: call the exported legacy `device_find_*` and declare them locally in the app source since the SDK header dropped them:
|
||||
```c
|
||||
extern struct Device* device_find_by_name(const char* name);
|
||||
extern struct Device* device_find_first_by_type(const struct DeviceType* type);
|
||||
```
|
||||
Verify the ACTUAL load on-device via serial (`/dev/cu.usbmodem*`, 115200) — `verify_symbols.py` compares against local firmware source and can report 0 missing even when the flashed firmware's export table differs.
|
||||
- `lv_obj_set_style_bg_grad_color/dir/stop` → **Not exported** 0.8.0-dev. Gradient fails `missing symbol`. Use solid `0x0E0E14`.
|
||||
- `LV_OPA_95` → **Not defined** (only 0/10/20/30/40/50/60/70/80/90/100/COVER/TRANSP). Use `COVER` or `90`.
|
||||
- `lv_arc_create`, `lv_arc_set_range/value/bg_angles`, `lv_arc_get_value` → **Not exported on 0.7.0-dev**, added `dff93cb6 Add lv_arc.h symbols (#496)` after 0.7 release. Symptom blue modal `Error / Application failed to start: missing symbol / OK`, heap healthy ~31KB not OOM. Fix horizontal slide using `lv_slider_create` + `lv_indev_get_point/active`. See `book-browser.md`.
|
||||
|
||||
@@ -9,42 +9,23 @@ Tactility external ELF apps — loaded at runtime via firmware symbol table. Eac
|
||||
Upstream: `https://github.com/TactilityProject/TactilityApps`
|
||||
Personal Gitea mirror: `https://git.reynafamily.com/adolforeyna/tactility_apps` (`personal` remote)
|
||||
|
||||
Dev board IP: `192.168.68.112` (S3, OS 0.8.0-dev, SDK 0.8.0-dev) — main deploy target. `.111` and `.114` also online. OS image is 0.8.0-dev so manifest should use `sdk=0.8.0-dev` or loader warns.
|
||||
Dev board IP: `192.168.68.112` (S3, OS 0.8.0-dev, SDK 0.8.0-dev) — main deploy target. `.111` also online. OS image is 0.8.0-dev so manifest should use `sdk=0.8.0-dev` or loader warns.
|
||||
|
||||
## Local Workstation
|
||||
|
||||
- Host: macOS 26.5.2
|
||||
- Project: `/Users/adolforeyna/Projects/electronics/tactility/tactility_apps`
|
||||
- ESP-IDF: `/Users/adolforeyna/esp/esp-idf` (v5.5.2), Python env `idf5.5_py3.9_env` at `~/.espressif/python_env/idf5.5_py3.9_env` (NOT `idf5.3_py3.9_env` — that env does not exist on this machine)
|
||||
|
||||
### Tactility SDK location (IMPORTANT)
|
||||
|
||||
- When you run `tactility.py ... build` **without** `--local-sdk`, it auto-downloads the SDK to the per-app cache:
|
||||
`Apps/<Name>/.tactility/<version>-<platform>/TactilitySDK` (e.g. `Apps/BookPlayer/.tactility/0.8.0-dev-esp32s3/TactilitySDK`).
|
||||
- **The freshly CDN-downloaded 0.8.0-dev SDK for esp32s3 is broken for this setup** — it fails at CMake with
|
||||
`Failed to resolve component 'app-module' required by component 'TactilitySDK': unknown name` and also renames the
|
||||
device API (`device_find_*` → `device_get_by_name/device_get_first_by_type` out-param style). It is a *different,
|
||||
newer* SDK than the working one.
|
||||
- **The known-good SDK is the locally-cached copy** already present in the other apps'
|
||||
`.tactility/0.8.0-dev-esp32s3/TactilitySDK` (the one that carries a `Drivers/` folder). It builds clean.
|
||||
- Workaround when a fresh download fails: copy the working cached SDK over the broken one:
|
||||
```bash
|
||||
SRC=Apps/PipecatVoice/.tactility/0.8.0-dev-esp32s3 # known-good copy
|
||||
DST=Apps/MyApp/.tactility/0.8.0-dev-esp32s3
|
||||
rm -rf "$DST/TactilitySDK"; cp -R "$SRC/TactilitySDK" "$DST/"
|
||||
```
|
||||
Then rebuild. (Resolving the root cause on the newer SDK is still TODO — consider pinning/downloading the cached zip
|
||||
so this isn't needed per-app.)
|
||||
- Host: M2 Air (14,2) macOS 26.5.2
|
||||
- Project: `/Users/adolforeyna/Projects/Tactility/apps`
|
||||
- ESP-IDF: `/Users/adolforeyna/esp/esp-idf`, Python env `idf5.3_py3.9_env` at `~/.espressif/python_env/idf5.3_py3.9_env`
|
||||
- Tactility SDK: `/Users/adolforeyna/Projects/Tactility/firmware/release/TactilitySDK` (built locally, `--local-sdk`)
|
||||
|
||||
### Hardware
|
||||
|
||||
| Board | IP | Port | Notes |
|
||||
|-------|-----|------|-------|
|
||||
| ES3C28P 2.8" color 240x320 | `192.168.68.112` | `/dev/cu.usbmodem101` | Primary deploy target, 16MB flash, OCT PSRAM, SD for ELF assets |
|
||||
| ES3C28P 2.8" color 240x320 | `192.168.68.114` | `/dev/cu.usbmodem101` | Secondary S3 (used for recent BookPlayer deploys) |
|
||||
| Waveshare RLCD 4.2" mono 400x300 ST7305 | `192.168.68.111` | `/dev/cu.usbmodem1101` | Secondary, compact density, SD threshold 60, font 18 |
|
||||
| Waveshare RLCD 4.2" mono 400x300 ST7305 | `192.168.68.111`? .1101 USB | `/dev/cu.usbmodem1101` | Secondary, compact density, mono threshold 60, font 18 |
|
||||
|
||||
User shorthand: "ending in 112" → `192.168.68.112`. Install is dashboard-port API (80), not dev port 6666.
|
||||
User shorthand: "ending in 112" → `192.168.68.112`.
|
||||
|
||||
## Build, Deploy, Run — Correct Env Wrapper
|
||||
|
||||
@@ -53,10 +34,11 @@ Hermes desktop Python 3.11 venv pollutes `PYTHONPATH` → `tactility.py` crashes
|
||||
Always:
|
||||
|
||||
```bash
|
||||
cd /Users/adolforeyna/Projects/electronics/tactility/tactility_apps
|
||||
cd /Users/adolforeyna/Projects/Tactility/apps
|
||||
unset PYTHONPATH; unset PYTHONHOME
|
||||
export IDF_PYTHON_ENV_PATH=/Users/adolforeyna/.espressif/python_env/idf5.5_py3.9_env
|
||||
export IDF_PYTHON_ENV_PATH=/Users/adolforeyna/.espressif/python_env/idf5.3_py3.9_env
|
||||
source /Users/adolforeyna/esp/esp-idf/export.sh
|
||||
export TACTILITY_SDK_PATH=/Users/adolforeyna/Projects/Tactility/firmware/release/TactilitySDK
|
||||
|
||||
# Build
|
||||
$IDF_PYTHON_ENV_PATH/bin/python tactility.py Apps/MyApp clean
|
||||
@@ -69,7 +51,6 @@ xtensa-esp32s3-elf-nm -D Apps/MyApp/build/cmake-build-esp32s3/MyApp.app.elf | gr
|
||||
python .claude/skills/tactility-app-development/scripts/verify_symbols.py Apps/MyApp
|
||||
|
||||
# Install to device (dashboard API port 80, not dev port 6666)
|
||||
# Preferred: use reyna-cli (see "Device discovery & management via reyna-cli" below)
|
||||
$IDF_PYTHON_ENV_PATH/bin/python tactility.py Apps/MyApp install 192.168.68.112 esp32s3
|
||||
# or dashboard directly:
|
||||
curl -X PUT http://192.168.68.112/api/apps/install -F "file=@build/MyApp.app"
|
||||
@@ -84,41 +65,6 @@ curl http://192.168.68.112/api/apps # list installed
|
||||
# If 6666 refused → use dashboard PUT above
|
||||
```
|
||||
|
||||
### Device discovery & management via reyna-cli
|
||||
|
||||
`reyna-cli` (at `/Users/adolforeyna/.local/bin/reyna-cli`) wraps the Tactility LAN web API with a device registry
|
||||
(`~/.hermes/local_devices.yaml`). Prefer it over raw IPs/curl for fleet discovery and management. Boards are
|
||||
addressed by their stable registry **ID** (not IP — DHCP IPs change); `devices list` maps IDs→IPs.
|
||||
|
||||
Registered Tactility boards (from `reyna-cli devices list`):
|
||||
|
||||
| ID | IP | Board |
|
||||
|----|----|-------|
|
||||
| `kidosos_a790` | `.129` | Personal 3.5-inch Tactility Board (Dev) — USB `/dev/cu.usbmodem*` for serial |
|
||||
| `kidosos_5c5c` | `.114` | Grace's 2.8-inch board |
|
||||
| `kidosos_5690` | `.107` | Elias's 2.8-inch board |
|
||||
| `kidosos_591c` | `.128` | Kitchen 2.8-inch board |
|
||||
| `esp32_screen` | `.123` | ESP32 Screen |
|
||||
| `little32` | `.122` | Little32 Kitchen ESP32 Screen |
|
||||
|
||||
Commands:
|
||||
|
||||
```bash
|
||||
reyna-cli tactility discover # probe all boards; online/offline + sysinfo
|
||||
reyna-cli devices list # registry (id, ip, display name, status)
|
||||
reyna-cli tactility sysinfo kidsos_a790 # firmware/version/heap/psram/SD
|
||||
reyna-cli tactility apps kidsos_a790 # installed apps
|
||||
reyna-cli tactility install kidsos_5c5c build/MyApp.app # PUT install by device id
|
||||
reyna-cli tactility run kidsos_5c5c one.tactility.myapp # launch app
|
||||
reyna-cli tactility report kidsos_a790 # apps + SD + bible/podcast usage
|
||||
# most commands accept --json for machine-readable output
|
||||
```
|
||||
|
||||
Use a serial monitor on the USB board for loader/ELF diagnostics: `/dev/cu.usbmodem*` @ 115200
|
||||
(e.g. `idf5.5_py3.9_env/bin/python -m serial.tools.miniterm /dev/cu.usbmodem31201 115200`). This is how you
|
||||
capture `E ELF: Can't find symbol X` on `.129` — `verify_symbols.py` only compares against local firmware
|
||||
SOURCE and can pass even when the flashed firmware's export table differs.
|
||||
|
||||
### manifest.properties
|
||||
|
||||
```properties
|
||||
@@ -157,7 +103,6 @@ Keep app source as pure C (no `auto`, no lambdas in `.c`) or rename to `.cpp`.
|
||||
Tactility firmware exports symbols via `ESP_ELFSYM_EXPORT` / `DEFINE_MODULE_SYMBOL` in `firmware/Modules/lvgl-module/source/symbols.c`. If ELF references an unexported symbol → device modal `Error / Application failed to start: missing symbol`, heap still healthy (~31KB free) → NOT OOM.
|
||||
|
||||
Known missing on 0.8.0-dev:
|
||||
- `device_get_by_name` / `device_get_first_by_type` (out-param API) → **Not exported** by flashed firmware; use legacy exported `device_find_by_name` / `device_find_first_by_type` (declare them locally — newer SDK headers dropped them). Gotcha: `verify_symbols.py` compares vs local firmware SOURCE and can pass even when the flashed firmware's export table differs — confirm on-device via serial (`/dev/cu.usbmodem*`, 115200): `E ELF: Can't find symbol X`.
|
||||
- `lv_font_montserrat_14/18` → use `lvgl_get_text_font(SMALL/DEFAULT/LARGE)` via `tactility/lvgl_fonts.h`
|
||||
- `lv_obj_set_style_bg_grad_color/dir/stop`, `LV_OPA_95/5`, `LV_SYMBOL_STAR` → use solid colors, COVER/90
|
||||
- `lv_obj_set_style_max_width` → use `LV_PCT(92)` fixed percents for gutters
|
||||
|
||||
@@ -32,10 +32,6 @@
|
||||
#define MAX_TITLE 128
|
||||
#define MAX_AUTHOR 128
|
||||
|
||||
// Fade-in/out applied at the start/end of each page's narration to avoid an
|
||||
// amplitude step (audible "pop") at page boundaries. In frames per channel.
|
||||
#define FADE_FRAMES 4096
|
||||
|
||||
typedef enum {
|
||||
STATE_IDLE,
|
||||
STATE_PLAYING,
|
||||
@@ -87,12 +83,6 @@ typedef struct {
|
||||
uint8_t* audio_buf; // Shared MP3 input and WAV buffer
|
||||
mp3d_sample_t* pcm_buf; // MP3 decoded pcm buffer
|
||||
|
||||
// Currently-open output stream format (so we can reuse it across page
|
||||
// changes instead of tearing the codec down/recreating it every page).
|
||||
uint32_t stream_rate;
|
||||
uint8_t stream_channels;
|
||||
uint8_t stream_bits;
|
||||
|
||||
TaskHandle_t playback_task_handle;
|
||||
} AppCtx;
|
||||
|
||||
@@ -125,15 +115,8 @@ static void play_wav(AppCtx* ctx);
|
||||
static void scan_books(AppCtx* ctx);
|
||||
|
||||
/* ─── Audio-stream helpers ─── */
|
||||
// The flashed firmware exports the legacy device_find_* lookup API; the newer
|
||||
// device_get_* (out-param) API is not yet exported on 0.8.0-dev, so calling it
|
||||
// fails at load with "missing symbol". The newer SDK headers dropped the legacy
|
||||
// declarations, so declare them here (symbols are resolved at runtime by the ELF
|
||||
// loader against the flashed firmware).
|
||||
extern struct Device* device_find_by_name(const char* name);
|
||||
extern struct Device* device_find_first_by_type(const struct DeviceType* type);
|
||||
|
||||
static bool find_audio_stream_device(AppCtx* ctx) {
|
||||
// Prefer device_find_first_by_type but keep compatibility with name lookup
|
||||
struct Device* dev = device_find_by_name("audio-stream");
|
||||
if (dev) {
|
||||
ctx->stream_dev = dev;
|
||||
@@ -151,20 +134,10 @@ static void close_stream_if_open(AppCtx* ctx) {
|
||||
if (ctx->stream_handle) {
|
||||
audio_stream_close(ctx->stream_handle);
|
||||
ctx->stream_handle = NULL;
|
||||
ctx->stream_rate = 0;
|
||||
ctx->stream_channels = 0;
|
||||
ctx->stream_bits = 0;
|
||||
}
|
||||
}
|
||||
|
||||
static bool open_output_stream(AppCtx* ctx, uint32_t sample_rate, uint8_t channels, uint8_t bits) {
|
||||
// Reuse the already-open stream when the format hasn't changed. This keeps the
|
||||
// codec alive across page changes, avoiding the audible pop caused by closing and
|
||||
// re-initialising the audio hardware on every page turn.
|
||||
if (ctx->stream_handle && ctx->stream_rate == sample_rate
|
||||
&& ctx->stream_channels == channels && ctx->stream_bits == bits) {
|
||||
return true;
|
||||
}
|
||||
close_stream_if_open(ctx);
|
||||
if (!ctx->stream_dev) return false;
|
||||
struct AudioStreamConfig cfg = {
|
||||
@@ -178,9 +151,6 @@ static bool open_output_stream(AppCtx* ctx, uint32_t sample_rate, uint8_t channe
|
||||
ctx->stream_handle = NULL;
|
||||
return false;
|
||||
}
|
||||
ctx->stream_rate = sample_rate;
|
||||
ctx->stream_channels = channels;
|
||||
ctx->stream_bits = bits;
|
||||
ESP_LOGI(TAG, "audio_stream output opened: %u Hz %u ch %u-bit", (unsigned)sample_rate, channels, bits);
|
||||
return true;
|
||||
}
|
||||
@@ -352,7 +322,6 @@ static void load_page(AppCtx* ctx, int page_index, bool start_audio) {
|
||||
/* ─── Return to Book Picker Screen ─── */
|
||||
static void return_to_picker(AppCtx* ctx) {
|
||||
wait_for_playback_task_to_exit(ctx);
|
||||
close_stream_if_open(ctx);
|
||||
|
||||
if (ctx->manifest_root) {
|
||||
cJSON_Delete(ctx->manifest_root);
|
||||
@@ -419,7 +388,6 @@ static void play_mp3(AppCtx* ctx) {
|
||||
bool eof = false;
|
||||
int sample_rate = 0;
|
||||
int channels = 0;
|
||||
size_t page_frames_written = 0;
|
||||
|
||||
ESP_LOGI(TAG, "Starting MP3 playback via audio-stream: %s (%d bytes)", ctx->current_audio_path, file_size);
|
||||
|
||||
@@ -474,30 +442,14 @@ static void play_mp3(AppCtx* ctx) {
|
||||
channels = info.channels;
|
||||
}
|
||||
|
||||
// Adjust Volume and apply a fade-in/out at the start/end of the page
|
||||
// so the transition to the next page doesn't click.
|
||||
// Adjust Volume
|
||||
int vol = ctx->volume;
|
||||
int16_t* samples_ptr = (int16_t*)ctx->pcm_buf;
|
||||
size_t sample_count = (size_t)samples * info.channels;
|
||||
uint8_t ch = (uint8_t)info.channels;
|
||||
size_t bytes_per_frame = (size_t)ch * sizeof(int16_t);
|
||||
size_t frames_in_chunk = (size_t)samples;
|
||||
float remaining_frames = (file_size > bytes_read_total)
|
||||
? (float)(file_size - bytes_read_total) / (float)bytes_per_frame : 0.0f;
|
||||
for (size_t i = 0; i < sample_count; ++i) {
|
||||
size_t frame = page_frames_written + i / ch;
|
||||
float fade_in = (frame < FADE_FRAMES) ? (float)frame / (float)FADE_FRAMES : 1.0f;
|
||||
float fade_out = 1.0f;
|
||||
float remaining = remaining_frames - (float)(i / ch);
|
||||
if (remaining < (float)FADE_FRAMES) {
|
||||
fade_out = (remaining > 0.0f) ? remaining / (float)FADE_FRAMES : 0.0f;
|
||||
}
|
||||
float gain = fade_in * fade_out;
|
||||
int32_t scaled = (int32_t)samples_ptr[i] * vol / 100;
|
||||
scaled = (int32_t)((float)scaled * gain);
|
||||
samples_ptr[i] = (int16_t)scaled;
|
||||
}
|
||||
page_frames_written += frames_in_chunk;
|
||||
|
||||
// Write via audio_stream (resampled to native 44100 internally)
|
||||
size_t offset = 0;
|
||||
@@ -533,16 +485,10 @@ static void play_mp3(AppCtx* ctx) {
|
||||
}
|
||||
|
||||
fclose(file);
|
||||
close_stream_if_open(ctx);
|
||||
|
||||
bool stopped_externally = (ctx->state == STATE_IDLE);
|
||||
|
||||
if (stopped_externally) {
|
||||
// User-initiated stop (prev/next, back, app close): tear the stream down.
|
||||
close_stream_if_open(ctx);
|
||||
}
|
||||
// On a natural page finish we deliberately LEAVE the stream open so the next
|
||||
// page reuses it, instead of closing+recreating the codec (which pops).
|
||||
|
||||
if (!stopped_externally) {
|
||||
tt_lvgl_lock(portMAX_DELAY);
|
||||
handle_audio_finished(ctx);
|
||||
@@ -621,7 +567,6 @@ static void play_wav(AppCtx* ctx) {
|
||||
ESP_LOGI(TAG, "Starting WAV via audio-stream: %s (%u Hz, %u ch)", ctx->current_audio_path, (unsigned int)header.sample_rate, (unsigned int)header.channels);
|
||||
|
||||
size_t total_played = 0;
|
||||
size_t page_frames_written = 0;
|
||||
bool stream_open = true;
|
||||
|
||||
while (total_played < data_size && ctx->state != STATE_IDLE) {
|
||||
@@ -643,29 +588,14 @@ static void play_wav(AppCtx* ctx) {
|
||||
size_t read_bytes = fread(ctx->audio_buf, 1, to_read, file);
|
||||
if (read_bytes == 0) break;
|
||||
|
||||
// Scaling Volume + fade-in/out at page boundaries to avoid a click.
|
||||
// Scaling Volume
|
||||
int vol = ctx->volume;
|
||||
int16_t* samples_ptr = (int16_t*)ctx->audio_buf;
|
||||
size_t sample_count = read_bytes / sizeof(int16_t);
|
||||
uint8_t ch = (header.channels > 0) ? header.channels : 1;
|
||||
size_t bytes_per_frame = (size_t)ch * sizeof(int16_t);
|
||||
size_t frames_in_chunk = read_bytes / bytes_per_frame;
|
||||
float remaining_frames = (data_size > total_played)
|
||||
? (float)(data_size - total_played) / (float)bytes_per_frame : 0.0f;
|
||||
for (size_t i = 0; i < sample_count; ++i) {
|
||||
size_t frame = page_frames_written + i / ch;
|
||||
float fade_in = (frame < FADE_FRAMES) ? (float)frame / (float)FADE_FRAMES : 1.0f;
|
||||
float fade_out = 1.0f;
|
||||
float remaining = remaining_frames - (float)(i / ch);
|
||||
if (remaining < (float)FADE_FRAMES) {
|
||||
fade_out = (remaining > 0.0f) ? remaining / (float)FADE_FRAMES : 0.0f;
|
||||
}
|
||||
float gain = fade_in * fade_out;
|
||||
int32_t scaled = (int32_t)samples_ptr[i] * vol / 100;
|
||||
scaled = (int32_t)((float)scaled * gain);
|
||||
samples_ptr[i] = (int16_t)scaled;
|
||||
}
|
||||
page_frames_written += frames_in_chunk;
|
||||
|
||||
// Write via audio_stream
|
||||
size_t offset = 0;
|
||||
@@ -699,16 +629,10 @@ static void play_wav(AppCtx* ctx) {
|
||||
}
|
||||
|
||||
fclose(file);
|
||||
close_stream_if_open(ctx);
|
||||
|
||||
bool stopped_externally = (ctx->state == STATE_IDLE);
|
||||
|
||||
if (stopped_externally) {
|
||||
// User-initiated stop (prev/next, back, app close): tear the stream down.
|
||||
close_stream_if_open(ctx);
|
||||
}
|
||||
// On a natural page finish we deliberately LEAVE the stream open so the next
|
||||
// page reuses it, instead of closing+recreating the codec (which pops).
|
||||
|
||||
if (!stopped_externally) {
|
||||
tt_lvgl_lock(portMAX_DELAY);
|
||||
handle_audio_finished(ctx);
|
||||
|
||||
@@ -0,0 +1,16 @@
|
||||
cmake_minimum_required(VERSION 3.20)
|
||||
|
||||
include($ENV{IDF_PATH}/tools/cmake/project.cmake)
|
||||
|
||||
if (DEFINED ENV{TACTILITY_SDK_PATH})
|
||||
set(TACTILITY_SDK_PATH $ENV{TACTILITY_SDK_PATH})
|
||||
else()
|
||||
set(TACTILITY_SDK_PATH "../../release/TactilitySDK")
|
||||
message(WARNING "⚠️ TACTILITY_SDK_PATH environment variable is not set, defaulting to ${TACTILITY_SDK_PATH}")
|
||||
endif()
|
||||
|
||||
include("${TACTILITY_SDK_PATH}/TactilitySDK.cmake")
|
||||
set(EXTRA_COMPONENT_DIRS ${TACTILITY_SDK_PATH})
|
||||
|
||||
project(ImmichElias)
|
||||
tactility_project(ImmichElias)
|
||||
@@ -4,5 +4,6 @@ set(CJSON_SOURCE "$ENV{IDF_PATH}/components/json/cJSON/cJSON.c")
|
||||
idf_component_register(
|
||||
SRCS ${SOURCE_FILES} ${CJSON_SOURCE}
|
||||
INCLUDE_DIRS Source "$ENV{IDF_PATH}/components/json/cJSON"
|
||||
REQUIRES TactilitySDK lwip
|
||||
REQUIRES TactilitySDK esp_http_client
|
||||
)
|
||||
target_compile_options(${COMPONENT_LIB} PRIVATE -Wno-format-truncation -Wno-unused-but-set-variable -Wno-unused-function -Wno-unused-variable)
|
||||
@@ -0,0 +1,988 @@
|
||||
#include <tt_app.h>
|
||||
#include <tt_lvgl.h>
|
||||
#include <tt_lvgl_toolbar.h>
|
||||
#include <tactility/lvgl_fonts.h>
|
||||
|
||||
#include <esp_log.h>
|
||||
#include <esp_http_client.h>
|
||||
#include <esp_heap_caps.h>
|
||||
#include <esp_random.h>
|
||||
#include <cJSON.h>
|
||||
|
||||
#include <string.h>
|
||||
#include <stdlib.h>
|
||||
#include <stdio.h>
|
||||
#include <sys/stat.h>
|
||||
#include <sys/types.h>
|
||||
#include <unistd.h>
|
||||
#include <dirent.h>
|
||||
#include <time.h>
|
||||
|
||||
#include <freertos/FreeRTOS.h>
|
||||
#include <freertos/task.h>
|
||||
|
||||
static const char* TAG = "ImmichElias";
|
||||
|
||||
#define IMMICH_BASE_URL "http://192.168.68.110:2283"
|
||||
#define IMMICH_API_KEY "tactility-elias-2af3d9c44b72f987f41afc3438c3dd740862417797a3e5a6"
|
||||
#define IMMICH_PERSON_ID "cbece6a9-720f-482a-a945-c9ba5ca41fd4"
|
||||
#define PERSON_NAME "Elias"
|
||||
#define PERSON_COUNT_STR "10739"
|
||||
|
||||
#define RESIZE_PROXY_URL "http://192.168.68.110:8106"
|
||||
|
||||
#define MAX_PHOTOS 100
|
||||
#define JSON_BUF_SIZE (128 * 1024)
|
||||
#define THUMB_BUF_SIZE (512 * 1024)
|
||||
|
||||
#define FALLBACK_W 320
|
||||
#define FALLBACK_H 240
|
||||
|
||||
typedef struct {
|
||||
char id[64];
|
||||
char name[32];
|
||||
char count_str[16];
|
||||
} PersonInfo;
|
||||
|
||||
static const PersonInfo PERSONS[] = {
|
||||
{"cbece6a9-720f-482a-a945-c9ba5ca41fd4", "Elias", "10739"},
|
||||
{"ad40f024-c34b-4ec6-a19e-2fa157b259e5", "Grace", ""},
|
||||
{"6d9e4ae6-143d-43c9-bd42-437e3edfe2df", "Alicia", ""},
|
||||
{"3ddbc731-e857-473c-8226-8af97e5cb86e", "Adolfo", ""},
|
||||
};
|
||||
#define PERSONS_COUNT (sizeof(PERSONS)/sizeof(PERSONS[0]))
|
||||
|
||||
typedef struct {
|
||||
char id[64];
|
||||
char thumbnail_url[256];
|
||||
char fileCreatedAt[64];
|
||||
char originalFileName[128];
|
||||
char type[16];
|
||||
int width;
|
||||
int height;
|
||||
} Photo;
|
||||
|
||||
static Photo photos[MAX_PHOTOS];
|
||||
static int photo_count = 0;
|
||||
|
||||
typedef struct {
|
||||
uint8_t* buf;
|
||||
size_t cap;
|
||||
size_t len;
|
||||
} HttpBuf;
|
||||
|
||||
static esp_err_t http_evt_handler(esp_http_client_event_t* evt) {
|
||||
HttpBuf* hb = (HttpBuf*)evt->user_data;
|
||||
if (hb == NULL) return ESP_OK;
|
||||
if (evt->event_id == HTTP_EVENT_ON_DATA && evt->data && evt->data_len > 0) {
|
||||
size_t copy_len = (size_t)evt->data_len;
|
||||
if (hb->len + copy_len > hb->cap) {
|
||||
size_t remaining = (hb->cap > hb->len) ? hb->cap - hb->len : 0;
|
||||
if (remaining < copy_len) copy_len = remaining;
|
||||
}
|
||||
if (copy_len > 0) {
|
||||
memcpy(hb->buf + hb->len, evt->data, copy_len);
|
||||
hb->len += copy_len;
|
||||
}
|
||||
}
|
||||
return ESP_OK;
|
||||
}
|
||||
|
||||
static int http_get_with_auth(const char* url, HttpBuf* out) {
|
||||
if (out) out->len = 0;
|
||||
esp_http_client_config_t cfg = {
|
||||
.url = url,
|
||||
.event_handler = http_evt_handler,
|
||||
.user_data = out,
|
||||
.timeout_ms = 8000,
|
||||
.buffer_size = 4096,
|
||||
.buffer_size_tx = 1024,
|
||||
};
|
||||
esp_http_client_handle_t client = esp_http_client_init(&cfg);
|
||||
if (!client) return -1;
|
||||
esp_http_client_set_method(client, HTTP_METHOD_GET);
|
||||
esp_http_client_set_header(client, "x-api-key", IMMICH_API_KEY);
|
||||
esp_err_t err = esp_http_client_perform(client);
|
||||
int status = esp_http_client_get_status_code(client);
|
||||
esp_http_client_cleanup(client);
|
||||
return (err == ESP_OK) ? status : -1;
|
||||
}
|
||||
|
||||
static int http_get_no_auth(const char* url, HttpBuf* out) {
|
||||
if (out) out->len = 0;
|
||||
esp_http_client_config_t cfg = {
|
||||
.url = url,
|
||||
.event_handler = http_evt_handler,
|
||||
.user_data = out,
|
||||
.timeout_ms = 8000,
|
||||
.buffer_size = 4096,
|
||||
.buffer_size_tx = 1024,
|
||||
};
|
||||
esp_http_client_handle_t client = esp_http_client_init(&cfg);
|
||||
if (!client) return -1;
|
||||
esp_http_client_set_method(client, HTTP_METHOD_GET);
|
||||
esp_err_t err = esp_http_client_perform(client);
|
||||
int status = esp_http_client_get_status_code(client);
|
||||
ESP_LOGI(TAG, "GET noauth %s status %d len %d", url, status, out ? (int)out->len : -1);
|
||||
esp_http_client_cleanup(client);
|
||||
return (err == ESP_OK) ? status : -1;
|
||||
}
|
||||
|
||||
static int http_post_json(const char* url, const char* json_body, HttpBuf* out) {
|
||||
if (out) out->len = 0;
|
||||
esp_http_client_config_t cfg = {
|
||||
.url = url,
|
||||
.event_handler = http_evt_handler,
|
||||
.user_data = out,
|
||||
.timeout_ms = 8000,
|
||||
.buffer_size = 4096,
|
||||
.buffer_size_tx = 2048,
|
||||
};
|
||||
esp_http_client_handle_t client = esp_http_client_init(&cfg);
|
||||
if (!client) return -1;
|
||||
esp_http_client_set_method(client, HTTP_METHOD_POST);
|
||||
esp_http_client_set_header(client, "x-api-key", IMMICH_API_KEY);
|
||||
esp_http_client_set_header(client, "Content-Type", "application/json");
|
||||
esp_http_client_set_post_field(client, json_body, strlen(json_body));
|
||||
esp_err_t err = esp_http_client_perform(client);
|
||||
int status = esp_http_client_get_status_code(client);
|
||||
esp_http_client_cleanup(client);
|
||||
return (err == ESP_OK) ? status : -1;
|
||||
}
|
||||
|
||||
static void shuffle_photos(void) {
|
||||
if (photo_count <= 1) return;
|
||||
for (int i = photo_count - 1; i > 0; i--) {
|
||||
uint32_t rnd = esp_random();
|
||||
int j = rnd % (i + 1);
|
||||
if (i != j) {
|
||||
Photo tmp = photos[i];
|
||||
photos[i] = photos[j];
|
||||
photos[j] = tmp;
|
||||
}
|
||||
}
|
||||
ESP_LOGI(TAG, "Photos shuffled random, count %d", photo_count);
|
||||
}
|
||||
|
||||
static int fetch_person_assets(const char* person_id) {
|
||||
if (!person_id || person_id[0]=='\0') person_id = PERSONS[0].id;
|
||||
char* json_buf = heap_caps_malloc(JSON_BUF_SIZE, MALLOC_CAP_SPIRAM | MALLOC_CAP_8BIT);
|
||||
if (!json_buf) json_buf = heap_caps_malloc(JSON_BUF_SIZE, MALLOC_CAP_DEFAULT | MALLOC_CAP_8BIT);
|
||||
if (!json_buf) return -1;
|
||||
HttpBuf hb = { .buf = (uint8_t*)json_buf, .cap = JSON_BUF_SIZE - 1, .len = 0 };
|
||||
char url[256];
|
||||
snprintf(url, sizeof(url), "%s/api/search/metadata", IMMICH_BASE_URL);
|
||||
// Pick random page to get variety from 10k+ photos
|
||||
int total_estimate = 10739;
|
||||
// Try to use count from PERSONS if available
|
||||
for (int pi=0; pi<PERSONS_COUNT; pi++) {
|
||||
if (strcmp(PERSONS[pi].id, person_id)==0 && PERSONS[pi].count_str[0]!='\0') {
|
||||
total_estimate = atoi(PERSONS[pi].count_str);
|
||||
if (total_estimate<=0) total_estimate=10739;
|
||||
break;
|
||||
}
|
||||
}
|
||||
int max_page = total_estimate / MAX_PHOTOS;
|
||||
if (max_page < 1) max_page = 1;
|
||||
int random_page = (esp_random() % max_page) + 1;
|
||||
char body[320];
|
||||
snprintf(body, sizeof(body), "{\"personIds\":[\"%s\"],\"size\":%d,\"page\":%d}", person_id, MAX_PHOTOS, random_page);
|
||||
ESP_LOGI(TAG, "Fetching person %s assets page %d/%d size %d", person_id, random_page, max_page, MAX_PHOTOS);
|
||||
int status = http_post_json(url, body, &hb);
|
||||
if (status != 200) { heap_caps_free(json_buf); return -1; }
|
||||
if (hb.len >= JSON_BUF_SIZE) hb.len = JSON_BUF_SIZE - 1;
|
||||
json_buf[hb.len] = '\0';
|
||||
cJSON* root = cJSON_Parse(json_buf);
|
||||
heap_caps_free(json_buf);
|
||||
if (!root) return -1;
|
||||
cJSON* assets = cJSON_GetObjectItem(root, "assets");
|
||||
cJSON* items = NULL;
|
||||
if (assets) items = cJSON_GetObjectItem(assets, "items");
|
||||
else { items = cJSON_GetObjectItem(root, "items"); if (!items && cJSON_IsArray(root)) items = root; }
|
||||
if (!items || !cJSON_IsArray(items)) { cJSON_Delete(root); return -1; }
|
||||
int n = cJSON_GetArraySize(items);
|
||||
photo_count = 0;
|
||||
for (int i = 0; i < n && photo_count < MAX_PHOTOS; i++) {
|
||||
cJSON* it = cJSON_GetArrayItem(items, i);
|
||||
if (!it) continue;
|
||||
cJSON* jid = cJSON_GetObjectItem(it, "id");
|
||||
if (!jid || !cJSON_IsString(jid)) continue;
|
||||
// Filter only IMAGE type to avoid video that won't display
|
||||
cJSON* jt_tmp = cJSON_GetObjectItem(it, "type");
|
||||
if (jt_tmp && cJSON_IsString(jt_tmp)) {
|
||||
if (strcmp(jt_tmp->valuestring, "IMAGE") != 0) {
|
||||
ESP_LOGI(TAG, "Skipping non-IMAGE asset %s type %s", jid->valuestring, jt_tmp->valuestring);
|
||||
continue;
|
||||
}
|
||||
}
|
||||
Photo* p = &photos[photo_count];
|
||||
memset(p, 0, sizeof(Photo));
|
||||
strncpy(p->id, jid->valuestring, sizeof(p->id)-1);
|
||||
cJSON* jw = cJSON_GetObjectItem(it, "width"); if (jw && cJSON_IsNumber(jw)) p->width = jw->valueint;
|
||||
cJSON* jh = cJSON_GetObjectItem(it, "height"); if (jh && cJSON_IsNumber(jh)) p->height = jh->valueint;
|
||||
cJSON* jfc = cJSON_GetObjectItem(it, "fileCreatedAt"); if (jfc && cJSON_IsString(jfc)) strncpy(p->fileCreatedAt, jfc->valuestring, sizeof(p->fileCreatedAt)-1);
|
||||
cJSON* jfn = cJSON_GetObjectItem(it, "originalFileName"); if (jfn && cJSON_IsString(jfn)) strncpy(p->originalFileName, jfn->valuestring, sizeof(p->originalFileName)-1);
|
||||
cJSON* jt = cJSON_GetObjectItem(it, "type"); if (jt && cJSON_IsString(jt)) strncpy(p->type, jt->valuestring, sizeof(p->type)-1);
|
||||
photo_count++;
|
||||
}
|
||||
cJSON_Delete(root);
|
||||
shuffle_photos();
|
||||
return photo_count;
|
||||
}
|
||||
|
||||
static bool ensure_dir_exists(const char* file_path) {
|
||||
char dir[320];
|
||||
strncpy(dir, file_path, sizeof(dir)-1);
|
||||
dir[sizeof(dir)-1] = '\0';
|
||||
char* last_slash = strrchr(dir, '/');
|
||||
if (!last_slash) return true;
|
||||
*last_slash = '\0';
|
||||
if (strlen(dir) == 0) return true;
|
||||
struct stat st;
|
||||
if (stat(dir, &st) == 0 && S_ISDIR(st.st_mode)) return true;
|
||||
char tmp[320]; tmp[0]='\0';
|
||||
if (dir[0]=='/') strcpy(tmp, "/");
|
||||
char work[320];
|
||||
strncpy(work, dir, sizeof(work)-1);
|
||||
work[sizeof(work)-1]='\0';
|
||||
char* p = work;
|
||||
if (p[0]=='/') p++;
|
||||
char* start = p;
|
||||
while (1) {
|
||||
char* slash = strchr(start, '/');
|
||||
bool last = false;
|
||||
if (slash) *slash='\0'; else last=true;
|
||||
if (strlen(start)>0) {
|
||||
if (strlen(tmp)>0 && tmp[strlen(tmp)-1]!='/') strncat(tmp, "/", sizeof(tmp)-strlen(tmp)-1);
|
||||
strncat(tmp, start, sizeof(tmp)-strlen(tmp)-1);
|
||||
mkdir(tmp, 0777);
|
||||
}
|
||||
if (last) break;
|
||||
*slash='/';
|
||||
start = slash+1;
|
||||
if (*start=='\0') break;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
typedef struct {
|
||||
AppHandle app;
|
||||
volatile bool visible;
|
||||
volatile bool stop_requested;
|
||||
// Picker (persons) like Bible book selector
|
||||
lv_obj_t* picker_wrapper;
|
||||
lv_obj_t* lst_persons;
|
||||
// Player (photos)
|
||||
lv_obj_t* player_wrapper;
|
||||
lv_obj_t* header_bar;
|
||||
lv_obj_t* ctrl_bar;
|
||||
lv_obj_t* lbl_book_title;
|
||||
lv_obj_t* lbl_indicator;
|
||||
lv_obj_t* img_page;
|
||||
lv_obj_t* lbl_info;
|
||||
lv_obj_t* lbl_status;
|
||||
lv_obj_t* btn_prev;
|
||||
lv_obj_t* btn_next;
|
||||
lv_obj_t* btn_reload;
|
||||
lv_obj_t* btn_back;
|
||||
lv_timer_t* slideshow_timer;
|
||||
lv_timer_t* autohide_timer;
|
||||
int current_idx;
|
||||
char status_text[256];
|
||||
volatile bool fetch_in_progress;
|
||||
volatile bool thumb_in_progress;
|
||||
uint8_t* thumb_buf;
|
||||
size_t thumb_len;
|
||||
TaskHandle_t fetch_task;
|
||||
TaskHandle_t thumb_task;
|
||||
char saved_img_path[320];
|
||||
volatile bool has_image;
|
||||
int disp_w;
|
||||
int disp_h;
|
||||
int img_w;
|
||||
int img_h;
|
||||
// Current person
|
||||
char current_person_id[64];
|
||||
char current_person_name[32];
|
||||
} AppCtx;
|
||||
|
||||
static bool get_cache_dir_path(AppCtx* ctx, char* out_dir, size_t out_size) {
|
||||
if (ctx && ctx->app) {
|
||||
size_t sz = out_size;
|
||||
tt_app_get_user_data_child_path(ctx->app, "cache", out_dir, &sz);
|
||||
if (out_dir[0]!='\0') { mkdir(out_dir, 0777); ensure_dir_exists(out_dir); return true; }
|
||||
}
|
||||
snprintf(out_dir, out_size, "/sdcard/tactility/immich_cache");
|
||||
mkdir(out_dir, 0777);
|
||||
return true;
|
||||
}
|
||||
|
||||
static bool get_cache_path_for_asset(AppCtx* ctx, const char* assetId, char* out_path, size_t out_size) {
|
||||
if (!assetId || !out_path || out_size==0) return false;
|
||||
int w = ctx ? ctx->img_w : FALLBACK_W;
|
||||
int h = ctx ? ctx->img_h : FALLBACK_H;
|
||||
if (w<=0) w=FALLBACK_W;
|
||||
if (h<=0) h=FALLBACK_H;
|
||||
if (w>800) w=800;
|
||||
if (h>600) h=600;
|
||||
if (ctx && ctx->app) {
|
||||
char child[160];
|
||||
snprintf(child, sizeof(child), "cache/%s_%dx%d.png", assetId, w, h);
|
||||
size_t sz = out_size;
|
||||
tt_app_get_user_data_child_path(ctx->app, child, out_path, &sz);
|
||||
if (out_path[0]!='\0') { ensure_dir_exists(out_path); return true; }
|
||||
}
|
||||
snprintf(out_path, out_size, "/sdcard/tactility/immich_cache/%s_%dx%d.png", assetId, w, h);
|
||||
ensure_dir_exists(out_path);
|
||||
return true;
|
||||
}
|
||||
|
||||
static bool get_cache_path_legacy(AppCtx* ctx, const char* assetId, char* out_path, size_t out_size) {
|
||||
if (!assetId || !out_path) return false;
|
||||
if (ctx && ctx->app) {
|
||||
char child[128];
|
||||
snprintf(child, sizeof(child), "cache/%s.png", assetId);
|
||||
size_t sz = out_size;
|
||||
tt_app_get_user_data_child_path(ctx->app, child, out_path, &sz);
|
||||
if (out_path[0]!='\0') { ensure_dir_exists(out_path); return true; }
|
||||
}
|
||||
snprintf(out_path, out_size, "/sdcard/tactility/immich_cache/%s.png", assetId);
|
||||
ensure_dir_exists(out_path);
|
||||
return true;
|
||||
}
|
||||
|
||||
static bool get_image_save_path(AppCtx* ctx, char* out_path, size_t out_size) {
|
||||
if (ctx && ctx->app) {
|
||||
size_t sz = out_size;
|
||||
tt_app_get_user_data_child_path(ctx->app, "immich_cur.png", out_path, &sz);
|
||||
if (out_path[0]!='\0') { ensure_dir_exists(out_path); return true; }
|
||||
}
|
||||
snprintf(out_path, out_size, "/sdcard/immich_cur.png");
|
||||
return true;
|
||||
}
|
||||
|
||||
static const char* save_buffer_to_cache(AppCtx* ctx, const char* assetId, const uint8_t* buf, size_t len, char* out_saved_path, size_t out_saved_size) {
|
||||
char path[320];
|
||||
if (!get_cache_path_for_asset(ctx, assetId, path, sizeof(path))) return NULL;
|
||||
ensure_dir_exists(path);
|
||||
FILE* f = fopen(path, "wb");
|
||||
if (!f) {
|
||||
ESP_LOGW(TAG, "fopen fail cache %s", path);
|
||||
return NULL;
|
||||
}
|
||||
size_t written = fwrite(buf, 1, len, f);
|
||||
fclose(f);
|
||||
ESP_LOGI(TAG, "Cached %d bytes to %s written %d (req %dx%d)", (int)len, path, (int)written, ctx ? ctx->img_w : 0, ctx ? ctx->img_h : 0);
|
||||
if (written != len) return NULL;
|
||||
if (out_saved_path && out_saved_size>0) { strncpy(out_saved_path, path, out_saved_size-1); out_saved_path[out_saved_size-1]='\0'; }
|
||||
char legacy[320];
|
||||
if (get_image_save_path(ctx, legacy, sizeof(legacy))) {
|
||||
FILE* lf = fopen(legacy, "wb");
|
||||
if (lf) { fwrite(buf,1,len,lf); fclose(lf); }
|
||||
}
|
||||
return out_saved_path;
|
||||
}
|
||||
|
||||
static bool load_from_cache(AppCtx* ctx, const char* assetId, char* out_path, size_t out_size, size_t* out_file_size) {
|
||||
char path[320];
|
||||
if (!get_cache_path_for_asset(ctx, assetId, path, sizeof(path))) return false;
|
||||
struct stat st;
|
||||
if (stat(path, &st)==0 && st.st_size>0 && st.st_size <= THUMB_BUF_SIZE) {
|
||||
FILE* f = fopen(path, "rb");
|
||||
if (f) { fclose(f); strncpy(out_path, path, out_size-1); out_path[out_size-1]='\0'; if(out_file_size) *out_file_size=(size_t)st.st_size; ESP_LOGI(TAG,"Cache HIT %s size %d (sized)", path, (int)st.st_size); return true; }
|
||||
}
|
||||
// Try legacy without size
|
||||
if (!get_cache_path_legacy(ctx, assetId, path, sizeof(path))) return false;
|
||||
if (stat(path, &st)==0 && st.st_size>0 && st.st_size <= THUMB_BUF_SIZE) {
|
||||
FILE* f = fopen(path, "rb");
|
||||
if (f) { fclose(f); strncpy(out_path, path, out_size-1); out_path[out_size-1]='\0'; if(out_file_size) *out_file_size=(size_t)st.st_size; ESP_LOGI(TAG,"Cache HIT %s size %d (legacy)", path, (int)st.st_size); return true; }
|
||||
}
|
||||
return false;
|
||||
}
|
||||
|
||||
static int scan_cache_dir_for_photos(AppCtx* ctx) {
|
||||
char dir_path[320];
|
||||
if (!get_cache_dir_path(ctx, dir_path, sizeof(dir_path))) return 0;
|
||||
DIR* d = opendir(dir_path);
|
||||
if (!d) return 0;
|
||||
photo_count = 0;
|
||||
struct dirent* ent;
|
||||
while ((ent = readdir(d))!=NULL && photo_count < MAX_PHOTOS) {
|
||||
if (ent->d_name[0]=='.') continue;
|
||||
size_t len = strlen(ent->d_name);
|
||||
if (len<5) continue;
|
||||
const char* ext = ent->d_name + len -4;
|
||||
if (!(ext[0]=='.' && (ext[1]=='p'||ext[1]=='P') && (ext[2]=='n'||ext[2]=='N') && (ext[3]=='g'||ext[3]=='G'))) continue;
|
||||
char id_buf[64];
|
||||
strncpy(id_buf, ent->d_name, sizeof(id_buf)-1);
|
||||
id_buf[sizeof(id_buf)-1]='\0';
|
||||
// Strip size suffix _WxH if present
|
||||
char* us = strrchr(id_buf, '_');
|
||||
if (us) {
|
||||
// Check if after _ is like 320x240.png
|
||||
char* dot = strrchr(id_buf, '.');
|
||||
if (dot) *dot='\0';
|
||||
// If us contains x, truncate at _
|
||||
// Actually id may contain _, but we want id part before _
|
||||
// For cache files like <id>_<w>x<h>.png, we want id before _
|
||||
// The id itself is uuid without _, so first _ is separator
|
||||
// So keep part before _
|
||||
if (us != id_buf) {
|
||||
// Ensure remaining part after _ contains x
|
||||
if (strchr(us, 'x') || strchr(us, 'X')) {
|
||||
*us='\0';
|
||||
} else {
|
||||
// No x, it was legacy without size, keep as is but already stripped extension
|
||||
}
|
||||
}
|
||||
} else {
|
||||
char* dot2 = strrchr(id_buf, '.');
|
||||
if (dot2) *dot2='\0';
|
||||
}
|
||||
if (strlen(id_buf)==0) continue;
|
||||
// Deduplicate: check if id already exists
|
||||
bool dup=false;
|
||||
for (int i=0;i<photo_count;i++) if (strcmp(photos[i].id, id_buf)==0) {dup=true; break;}
|
||||
if (dup) continue;
|
||||
Photo* p=&photos[photo_count];
|
||||
memset(p,0,sizeof(Photo));
|
||||
strncpy(p->id, id_buf, sizeof(p->id)-1);
|
||||
strncpy(p->originalFileName, ent->d_name, sizeof(p->originalFileName)-1);
|
||||
strncpy(p->type, "IMAGE", sizeof(p->type)-1);
|
||||
photo_count++;
|
||||
}
|
||||
closedir(d);
|
||||
if (photo_count>0) shuffle_photos();
|
||||
return photo_count;
|
||||
}
|
||||
|
||||
static int fetch_png_from_proxy(const char* assetId, uint8_t* buf, size_t cap, size_t* out_len, int req_w, int req_h) {
|
||||
if (req_w<=0) req_w=FALLBACK_W;
|
||||
if (req_h<=0) req_h=FALLBACK_H;
|
||||
if (req_w>800) req_w=800;
|
||||
if (req_h>600) req_h=600;
|
||||
char url[384];
|
||||
snprintf(url, sizeof(url), "%s/resize?assetId=%s&w=%d&h=%d&format=png", RESIZE_PROXY_URL, assetId, req_w, req_h);
|
||||
ESP_LOGI(TAG, "Fetching proxy PNG %s (req %dx%d)", url, req_w, req_h);
|
||||
HttpBuf hb = { .buf = buf, .cap = cap, .len = 0 };
|
||||
int status = http_get_no_auth(url, &hb);
|
||||
if (status==200 && hb.len>0) {
|
||||
if (out_len) *out_len=hb.len;
|
||||
ESP_LOGI(TAG, "Proxy PNG OK %d bytes for %dx%d", (int)hb.len, req_w, req_h);
|
||||
return (int)hb.len;
|
||||
}
|
||||
ESP_LOGE(TAG, "Proxy PNG fail id %s %dx%d status %d", assetId, req_w, req_h, status);
|
||||
return -1;
|
||||
}
|
||||
|
||||
static void start_fetch_task(AppCtx* ctx);
|
||||
static void start_thumb_task(AppCtx* ctx);
|
||||
static void ui_update_all(AppCtx* ctx);
|
||||
static void thumb_task_fn(void* arg);
|
||||
static void fetch_task_fn(void* arg);
|
||||
static void hide_chrome(AppCtx* ctx);
|
||||
static void show_chrome(AppCtx* ctx);
|
||||
|
||||
static void detect_display_size(AppCtx* ctx, lv_obj_t* parent) {
|
||||
if (!ctx) return;
|
||||
int w=FALLBACK_W, h=FALLBACK_H;
|
||||
lv_display_t* disp=NULL;
|
||||
if (parent) disp = lv_obj_get_display(parent);
|
||||
if (!disp) disp = lv_display_get_default();
|
||||
if (disp) {
|
||||
int dw = lv_display_get_horizontal_resolution(disp);
|
||||
int dh = lv_display_get_vertical_resolution(disp);
|
||||
if (dw>0 && dh>0) { w=dw; h=dh; }
|
||||
}
|
||||
if (w==FALLBACK_W && h==FALLBACK_H) {
|
||||
int dw2 = lv_display_get_horizontal_resolution(NULL);
|
||||
int dh2 = lv_display_get_vertical_resolution(NULL);
|
||||
if (dw2>0 && dh2>0) { w=dw2; h=dh2; }
|
||||
}
|
||||
ctx->disp_w=w; ctx->disp_h=h;
|
||||
// Black screen fix: 480x320 PNG decodes ~600KB, causes internal low 8723 bytes -> decoder fail -> black screen
|
||||
// Use 320x240 (decoded ~300KB) for stability, widget still fullscreen 480x320 background
|
||||
// 2x would be 480x320 but OOM, so keep 320x240 as best stable, or 240x180 for extra safety
|
||||
int target_w = 320;
|
||||
int target_h = 240;
|
||||
ctx->img_w=target_w; ctx->img_h=target_h;
|
||||
ESP_LOGI(TAG, "Display detected %dx%d -> img request %dx%d FULLSCREEN bg (320x240 stable, 480x320=2x OOM)", w,h,target_w,target_h);
|
||||
}
|
||||
|
||||
static void hide_chrome(AppCtx* ctx) {
|
||||
if (!ctx) return;
|
||||
if (!tt_lvgl_lock(pdMS_TO_TICKS(500))) return;
|
||||
// Bible-style: only bottom controls, no top header
|
||||
if (ctx->ctrl_bar) lv_obj_add_flag(ctx->ctrl_bar, LV_OBJ_FLAG_HIDDEN);
|
||||
tt_lvgl_unlock();
|
||||
}
|
||||
static void show_chrome(AppCtx* ctx) {
|
||||
if (!ctx) return;
|
||||
if (!tt_lvgl_lock(pdMS_TO_TICKS(500))) return;
|
||||
if (ctx->ctrl_bar) lv_obj_remove_flag(ctx->ctrl_bar, LV_OBJ_FLAG_HIDDEN);
|
||||
tt_lvgl_unlock();
|
||||
if (tt_lvgl_lock(pdMS_TO_TICKS(300))) {
|
||||
if (ctx->autohide_timer) { lv_timer_reset(ctx->autohide_timer); lv_timer_resume(ctx->autohide_timer); }
|
||||
tt_lvgl_unlock();
|
||||
}
|
||||
}
|
||||
static void autohide_timer_cb(lv_timer_t* timer) {
|
||||
AppCtx* ctx = (AppCtx*)lv_timer_get_user_data(timer);
|
||||
if (!ctx || !ctx->visible || ctx->stop_requested) return;
|
||||
hide_chrome(ctx);
|
||||
if (tt_lvgl_lock(pdMS_TO_TICKS(200))) { lv_timer_pause(timer); tt_lvgl_unlock(); }
|
||||
}
|
||||
static void slideshow_timer_cb(lv_timer_t* timer) {
|
||||
AppCtx* ctx = (AppCtx*)lv_timer_get_user_data(timer);
|
||||
if (!ctx || !ctx->visible || ctx->stop_requested) return;
|
||||
if (ctx->fetch_in_progress || ctx->thumb_in_progress) return;
|
||||
if (photo_count==0) return;
|
||||
ESP_LOGI(TAG, "Slideshow auto-advance 60s %dx%d", ctx->img_w, ctx->img_h);
|
||||
ctx->current_idx++;
|
||||
if (ctx->current_idx >= photo_count) ctx->current_idx=0;
|
||||
ctx->thumb_len=0; ctx->has_image=false;
|
||||
snprintf(ctx->status_text, sizeof(ctx->status_text), "Slideshow %d/%d %dx%d", ctx->current_idx+1, photo_count, ctx->img_w, ctx->img_h);
|
||||
ui_update_all(ctx);
|
||||
show_chrome(ctx);
|
||||
start_thumb_task(ctx);
|
||||
}
|
||||
|
||||
static void ui_update_all(AppCtx* ctx) {
|
||||
if (!ctx || !ctx->visible || ctx->stop_requested) return;
|
||||
if (!tt_lvgl_lock(pdMS_TO_TICKS(1000))) return;
|
||||
if (!ctx->visible || ctx->stop_requested) { tt_lvgl_unlock(); return; }
|
||||
if (ctx->lbl_book_title) lv_label_set_text_fmt(ctx->lbl_book_title, "%s - %s photos", PERSON_NAME, PERSON_COUNT_STR);
|
||||
if (ctx->lbl_indicator) {
|
||||
if (photo_count>0) lv_label_set_text_fmt(ctx->lbl_indicator, "%d/%d", ctx->current_idx+1, photo_count);
|
||||
else lv_label_set_text(ctx->lbl_indicator, "0/0");
|
||||
}
|
||||
if (ctx->lbl_info) {
|
||||
if (photo_count>0 && ctx->current_idx < photo_count) {
|
||||
Photo* p=&photos[ctx->current_idx];
|
||||
lv_label_set_text_fmt(ctx->lbl_info, "%s\n%.24s\n%dx%d disp %dx%d img %dx%d", p->originalFileName, p->fileCreatedAt, p->width, p->height, ctx->disp_w, ctx->disp_h, ctx->img_w, ctx->img_h);
|
||||
} else lv_label_set_text(ctx->lbl_info, ctx->status_text);
|
||||
}
|
||||
if (ctx->lbl_status) {
|
||||
if (ctx->thumb_len>0 && ctx->has_image) lv_label_set_text_fmt(ctx->lbl_status, "PNG %d bytes %dx%d", (int)ctx->thumb_len, ctx->img_w, ctx->img_h);
|
||||
else lv_label_set_text(ctx->lbl_status, ctx->status_text);
|
||||
}
|
||||
if (ctx->img_page) {
|
||||
if (ctx->has_image && ctx->saved_img_path[0]!='\0') {
|
||||
FILE* test = fopen(ctx->saved_img_path, "rb");
|
||||
if (test) {
|
||||
fclose(test);
|
||||
char lvgl_path[384];
|
||||
snprintf(lvgl_path, sizeof(lvgl_path), "A:%s", ctx->saved_img_path);
|
||||
lv_image_set_src(ctx->img_page, lvgl_path);
|
||||
lv_obj_clear_flag(ctx->img_page, LV_OBJ_FLAG_HIDDEN);
|
||||
// Fullscreen background: widget covers entire display
|
||||
lv_obj_set_size(ctx->img_page, ctx->disp_w > 0 ? ctx->disp_w : ctx->img_w, ctx->disp_h > 0 ? ctx->disp_h : ctx->img_h);
|
||||
lv_obj_center(ctx->img_page);
|
||||
ESP_LOGI(TAG, "lv_img_set_src %s %dx%d", lvgl_path, ctx->disp_w, ctx->disp_h);
|
||||
} else {
|
||||
ESP_LOGW(TAG, "Image file not found for display: %s", ctx->saved_img_path);
|
||||
lv_image_set_src(ctx->img_page, LV_SYMBOL_IMAGE);
|
||||
lv_obj_clear_flag(ctx->img_page, LV_OBJ_FLAG_HIDDEN);
|
||||
}
|
||||
} else {
|
||||
if (photo_count==0) {
|
||||
lv_image_set_src(ctx->img_page, LV_SYMBOL_IMAGE);
|
||||
lv_obj_clear_flag(ctx->img_page, LV_OBJ_FLAG_HIDDEN);
|
||||
} else {
|
||||
// Keep previous image to avoid black flash, or hide if you prefer
|
||||
// lv_obj_add_flag(ctx->img_page, LV_OBJ_FLAG_HIDDEN);
|
||||
}
|
||||
}
|
||||
}
|
||||
tt_lvgl_unlock();
|
||||
}
|
||||
|
||||
static void thumb_task_fn(void* arg) {
|
||||
AppCtx* ctx = (AppCtx*)arg;
|
||||
if (!ctx) { vTaskDelete(NULL); return; }
|
||||
if (ctx->stop_requested) { ctx->thumb_task=NULL; ctx->thumb_in_progress=false; vTaskDelete(NULL); return; }
|
||||
if (!ctx->thumb_buf) {
|
||||
ctx->thumb_buf = heap_caps_malloc(THUMB_BUF_SIZE, MALLOC_CAP_SPIRAM | MALLOC_CAP_8BIT);
|
||||
if (!ctx->thumb_buf) ctx->thumb_buf = heap_caps_malloc(THUMB_BUF_SIZE, MALLOC_CAP_DEFAULT | MALLOC_CAP_8BIT);
|
||||
}
|
||||
if (!ctx->thumb_buf) { snprintf(ctx->status_text, sizeof(ctx->status_text), "OOM thumb buf"); ui_update_all(ctx); ctx->thumb_task=NULL; ctx->thumb_in_progress=false; vTaskDelete(NULL); return; }
|
||||
int idx = ctx->current_idx;
|
||||
if (idx<0 || idx>=photo_count) idx=0;
|
||||
Photo* p=&photos[idx];
|
||||
char cache_path[320]={0}; size_t cached_size=0;
|
||||
if (load_from_cache(ctx, p->id, cache_path, sizeof(cache_path), &cached_size)) {
|
||||
strncpy(ctx->saved_img_path, cache_path, sizeof(ctx->saved_img_path)-1);
|
||||
ctx->has_image=true; ctx->thumb_len=cached_size;
|
||||
snprintf(ctx->status_text, sizeof(ctx->status_text), "Cached PNG %d bytes %dx%d", (int)cached_size, ctx->img_w, ctx->img_h);
|
||||
ui_update_all(ctx); show_chrome(ctx);
|
||||
ctx->thumb_task=NULL; ctx->thumb_in_progress=false; vTaskDelete(NULL); return;
|
||||
}
|
||||
ESP_LOGI(TAG, "Downloading PNG idx %d id %s %dx%d", idx, p->id, ctx->img_w, ctx->img_h);
|
||||
snprintf(ctx->status_text, sizeof(ctx->status_text), "Fetching PNG %d/%d %dx%d", idx+1, photo_count, ctx->img_w, ctx->img_h);
|
||||
ui_update_all(ctx);
|
||||
if (ctx->stop_requested) { ctx->thumb_task=NULL; ctx->thumb_in_progress=false; vTaskDelete(NULL); return; }
|
||||
size_t out_len=0;
|
||||
int r = fetch_png_from_proxy(p->id, ctx->thumb_buf, THUMB_BUF_SIZE, &out_len, ctx->img_w, ctx->img_h);
|
||||
// Fallback to 320x240 if requested size fails
|
||||
if (r<=0 && (ctx->img_w != 320 || ctx->img_h != 240)) {
|
||||
ESP_LOGW(TAG, "Proxy fail %dx%d, trying fallback 320x240", ctx->img_w, ctx->img_h);
|
||||
r = fetch_png_from_proxy(p->id, ctx->thumb_buf, THUMB_BUF_SIZE, &out_len, 320, 240);
|
||||
}
|
||||
if (ctx->stop_requested) { ctx->thumb_task=NULL; ctx->thumb_in_progress=false; vTaskDelete(NULL); return; }
|
||||
if (r>0) {
|
||||
ctx->thumb_len=out_len;
|
||||
char saved_tmp[320]={0};
|
||||
const char* saved = save_buffer_to_cache(ctx, p->id, ctx->thumb_buf, out_len, saved_tmp, sizeof(saved_tmp));
|
||||
if (saved) { strncpy(ctx->saved_img_path, saved_tmp, sizeof(ctx->saved_img_path)-1); ctx->has_image=true; snprintf(ctx->status_text, sizeof(ctx->status_text), "PNG %d bytes %dx%d", (int)out_len, ctx->img_w, ctx->img_h); }
|
||||
else { ctx->has_image=false; snprintf(ctx->status_text, sizeof(ctx->status_text), "PNG %d save fail", (int)out_len); }
|
||||
} else {
|
||||
ctx->thumb_len=0; ctx->has_image=false; snprintf(ctx->status_text, sizeof(ctx->status_text), "Proxy PNG fail %d", r);
|
||||
}
|
||||
ui_update_all(ctx); show_chrome(ctx);
|
||||
ctx->thumb_task=NULL; ctx->thumb_in_progress=false; vTaskDelete(NULL);
|
||||
}
|
||||
|
||||
static void start_thumb_task(AppCtx* ctx) {
|
||||
if (!ctx || ctx->thumb_in_progress || ctx->stop_requested || photo_count==0) return;
|
||||
ctx->thumb_in_progress=true; ctx->thumb_len=0; ctx->has_image=false;
|
||||
if (xTaskCreate(thumb_task_fn, "thumb_fetch", 8192, ctx, 5, &ctx->thumb_task)!=pdPASS) { ctx->thumb_in_progress=false; ctx->thumb_task=NULL; }
|
||||
}
|
||||
|
||||
static void fetch_task_fn(void* arg) {
|
||||
AppCtx* ctx = (AppCtx*)arg;
|
||||
if (!ctx) { vTaskDelete(NULL); return; }
|
||||
if (ctx->stop_requested) { ctx->fetch_task=NULL; ctx->fetch_in_progress=false; vTaskDelete(NULL); return; }
|
||||
ctx->fetch_in_progress=true;
|
||||
const char* fetch_name = ctx->current_person_name[0] ? ctx->current_person_name : PERSON_NAME;
|
||||
const char* fetch_id = ctx->current_person_id[0] ? ctx->current_person_id : PERSONS[0].id;
|
||||
snprintf(ctx->status_text, sizeof(ctx->status_text), "Fetching %s photos...", fetch_name);
|
||||
ui_update_all(ctx);
|
||||
int n = fetch_person_assets(fetch_id);
|
||||
if (ctx->stop_requested) { ctx->fetch_task=NULL; ctx->fetch_in_progress=false; vTaskDelete(NULL); return; }
|
||||
if (n>0) {
|
||||
ctx->current_idx=0;
|
||||
snprintf(ctx->status_text, sizeof(ctx->status_text), "Loaded %d photos (random %dx%d)", n, ctx->img_w, ctx->img_h);
|
||||
ui_update_all(ctx);
|
||||
if (!ctx->thumb_buf) {
|
||||
ctx->thumb_buf = heap_caps_malloc(THUMB_BUF_SIZE, MALLOC_CAP_SPIRAM | MALLOC_CAP_8BIT);
|
||||
if (!ctx->thumb_buf) ctx->thumb_buf = heap_caps_malloc(THUMB_BUF_SIZE, MALLOC_CAP_DEFAULT | MALLOC_CAP_8BIT);
|
||||
}
|
||||
if (ctx->thumb_buf && !ctx->stop_requested) {
|
||||
Photo* p=&photos[0];
|
||||
char cache_path[320]={0}; size_t cached_size=0;
|
||||
if (load_from_cache(ctx, p->id, cache_path, sizeof(cache_path), &cached_size)) {
|
||||
strncpy(ctx->saved_img_path, cache_path, sizeof(ctx->saved_img_path)-1);
|
||||
ctx->has_image=true; ctx->thumb_len=cached_size;
|
||||
snprintf(ctx->status_text, sizeof(ctx->status_text), "Cached PNG %d random %dx%d", (int)cached_size, ctx->img_w, ctx->img_h);
|
||||
ui_update_all(ctx); show_chrome(ctx);
|
||||
} else {
|
||||
size_t ol=0;
|
||||
int r=fetch_png_from_proxy(p->id, ctx->thumb_buf, THUMB_BUF_SIZE, &ol, ctx->img_w, ctx->img_h);
|
||||
if (r<=0 && (ctx->img_w != 320 || ctx->img_h != 240)) {
|
||||
ESP_LOGW(TAG, "First image proxy fail %dx%d, trying 320x240", ctx->img_w, ctx->img_h);
|
||||
r=fetch_png_from_proxy(p->id, ctx->thumb_buf, THUMB_BUF_SIZE, &ol, 320, 240);
|
||||
}
|
||||
if (ctx->stop_requested) { ctx->fetch_task=NULL; ctx->fetch_in_progress=false; vTaskDelete(NULL); return; }
|
||||
if (r>0) {
|
||||
ctx->thumb_len=ol;
|
||||
char saved_tmp[320]={0};
|
||||
const char* saved=save_buffer_to_cache(ctx, p->id, ctx->thumb_buf, ol, saved_tmp, sizeof(saved_tmp));
|
||||
if (saved) { strncpy(ctx->saved_img_path, saved_tmp, sizeof(ctx->saved_img_path)-1); ctx->has_image=true; snprintf(ctx->status_text, sizeof(ctx->status_text), "PNG %d random %dx%d", (int)ol, ctx->img_w, ctx->img_h); }
|
||||
} else {
|
||||
snprintf(ctx->status_text, sizeof(ctx->status_text), "First PNG fail %d", r);
|
||||
}
|
||||
ui_update_all(ctx); show_chrome(ctx);
|
||||
}
|
||||
}
|
||||
} else {
|
||||
int cached = scan_cache_dir_for_photos(ctx);
|
||||
if (cached>0) {
|
||||
ctx->current_idx=0;
|
||||
snprintf(ctx->status_text, sizeof(ctx->status_text), "Offline: %d cached", cached);
|
||||
ui_update_all(ctx);
|
||||
Photo* p=&photos[0];
|
||||
char cp[320]={0}; size_t sz=0;
|
||||
if (load_from_cache(ctx, p->id, cp, sizeof(cp), &sz)) { strncpy(ctx->saved_img_path, cp, sizeof(ctx->saved_img_path)-1); ctx->has_image=true; ctx->thumb_len=sz; snprintf(ctx->status_text, sizeof(ctx->status_text), "Offline cached %d %dx%d", (int)sz, ctx->img_w, ctx->img_h); ui_update_all(ctx); show_chrome(ctx); }
|
||||
} else {
|
||||
snprintf(ctx->status_text, sizeof(ctx->status_text), "Fetch failed %s - no cache", IMMICH_BASE_URL);
|
||||
ui_update_all(ctx);
|
||||
}
|
||||
}
|
||||
ctx->fetch_task=NULL; ctx->fetch_in_progress=false; vTaskDelete(NULL);
|
||||
}
|
||||
|
||||
static void start_fetch_task(AppCtx* ctx) {
|
||||
if (!ctx || ctx->fetch_in_progress || ctx->stop_requested) return;
|
||||
ctx->fetch_in_progress=true;
|
||||
if (xTaskCreate(fetch_task_fn, "immich_fetch", 12288, ctx, 5, &ctx->fetch_task)!=pdPASS) { ctx->fetch_in_progress=false; ctx->fetch_task=NULL; }
|
||||
}
|
||||
|
||||
static void on_image_click(lv_event_t* e) {
|
||||
AppCtx* ctx = (AppCtx*)lv_event_get_user_data(e);
|
||||
if (!ctx) return;
|
||||
bool is_hidden = false;
|
||||
if (tt_lvgl_lock(pdMS_TO_TICKS(200))) {
|
||||
if (ctx->ctrl_bar) is_hidden = lv_obj_has_flag(ctx->ctrl_bar, LV_OBJ_FLAG_HIDDEN);
|
||||
else if (ctx->header_bar) is_hidden = lv_obj_has_flag(ctx->header_bar, LV_OBJ_FLAG_HIDDEN);
|
||||
tt_lvgl_unlock();
|
||||
}
|
||||
if (is_hidden) show_chrome(ctx); else hide_chrome(ctx);
|
||||
}
|
||||
static void prev_event_cb(lv_event_t* e) {
|
||||
AppCtx* ctx = (AppCtx*)lv_event_get_user_data(e);
|
||||
if (!ctx || photo_count==0 || ctx->thumb_in_progress || ctx->fetch_in_progress || ctx->stop_requested) return;
|
||||
ctx->current_idx--; if (ctx->current_idx<0) ctx->current_idx=photo_count-1;
|
||||
ctx->thumb_len=0; ctx->has_image=false;
|
||||
snprintf(ctx->status_text, sizeof(ctx->status_text), "Loading %d/%d %dx%d", ctx->current_idx+1, photo_count, ctx->img_w, ctx->img_h);
|
||||
ui_update_all(ctx); show_chrome(ctx); start_thumb_task(ctx);
|
||||
}
|
||||
static void next_event_cb(lv_event_t* e) {
|
||||
AppCtx* ctx = (AppCtx*)lv_event_get_user_data(e);
|
||||
if (!ctx || photo_count==0 || ctx->thumb_in_progress || ctx->fetch_in_progress || ctx->stop_requested) return;
|
||||
ctx->current_idx++; if (ctx->current_idx>=photo_count) ctx->current_idx=0;
|
||||
ctx->thumb_len=0; ctx->has_image=false;
|
||||
snprintf(ctx->status_text, sizeof(ctx->status_text), "Loading %d/%d %dx%d", ctx->current_idx+1, photo_count, ctx->img_w, ctx->img_h);
|
||||
ui_update_all(ctx); show_chrome(ctx); start_thumb_task(ctx);
|
||||
}
|
||||
static void reload_event_cb(lv_event_t* e) {
|
||||
AppCtx* ctx = (AppCtx*)lv_event_get_user_data(e);
|
||||
if (!ctx || ctx->fetch_in_progress || ctx->thumb_in_progress || ctx->stop_requested) return;
|
||||
photo_count=0; ctx->current_idx=0; ctx->thumb_len=0; ctx->has_image=false;
|
||||
snprintf(ctx->status_text, sizeof(ctx->status_text), "Reloading %dx%d...", ctx->img_w, ctx->img_h);
|
||||
ui_update_all(ctx); show_chrome(ctx); start_fetch_task(ctx);
|
||||
}
|
||||
static void show_picker(AppCtx* ctx);
|
||||
static void show_player(AppCtx* ctx);
|
||||
|
||||
static void back_event_cb(lv_event_t* e) {
|
||||
AppCtx* ctx = (AppCtx*)lv_event_get_user_data(e);
|
||||
if (!ctx) { tt_app_stop(); return; }
|
||||
// If in player, go back to person picker like Bible book selector
|
||||
if (ctx->picker_wrapper && ctx->player_wrapper) {
|
||||
if (!tt_lvgl_lock(pdMS_TO_TICKS(500))) { tt_app_stop(); return; }
|
||||
bool player_visible = !lv_obj_has_flag(ctx->player_wrapper, LV_OBJ_FLAG_HIDDEN);
|
||||
tt_lvgl_unlock();
|
||||
if (player_visible) {
|
||||
show_picker(ctx);
|
||||
return;
|
||||
}
|
||||
}
|
||||
tt_app_stop();
|
||||
}
|
||||
|
||||
static void* create_data(void) {
|
||||
AppCtx* ctx = calloc(1, sizeof(AppCtx));
|
||||
if (ctx) {
|
||||
ctx->current_idx=0; ctx->disp_w=FALLBACK_W; ctx->disp_h=FALLBACK_H; ctx->img_w=FALLBACK_W; ctx->img_h=FALLBACK_H;
|
||||
strncpy(ctx->current_person_id, PERSONS[0].id, sizeof(ctx->current_person_id)-1);
|
||||
strncpy(ctx->current_person_name, PERSONS[0].name, sizeof(ctx->current_person_name)-1);
|
||||
snprintf(ctx->status_text, sizeof(ctx->status_text), "Init %s proxy %s (random) person %s", IMMICH_BASE_URL, RESIZE_PROXY_URL, ctx->current_person_name);
|
||||
ctx->thumb_buf = heap_caps_malloc(THUMB_BUF_SIZE, MALLOC_CAP_SPIRAM | MALLOC_CAP_8BIT);
|
||||
if (!ctx->thumb_buf) ctx->thumb_buf = heap_caps_malloc(THUMB_BUF_SIZE, MALLOC_CAP_DEFAULT | MALLOC_CAP_8BIT);
|
||||
}
|
||||
return ctx;
|
||||
}
|
||||
static void destroy_data(void* data) {
|
||||
AppCtx* ctx = (AppCtx*)data; if (!ctx) return;
|
||||
if (ctx->fetch_task || ctx->thumb_task) {
|
||||
int waits=100;
|
||||
while ((ctx->fetch_task || ctx->thumb_task) && waits-- >0) vTaskDelay(pdMS_TO_TICKS(50));
|
||||
}
|
||||
if (ctx->fetch_task || ctx->thumb_task) return; // leak to avoid crash
|
||||
if (ctx->thumb_buf) { heap_caps_free(ctx->thumb_buf); ctx->thumb_buf=NULL; }
|
||||
free(ctx);
|
||||
}
|
||||
static void on_create(AppHandle app, void* data) { AppCtx* ctx=(AppCtx*)data; if(ctx) ctx->app=app; ESP_LOGI(TAG,"onCreate proxy=%s random %dx%d person %s", RESIZE_PROXY_URL, ctx?ctx->img_w:0, ctx?ctx->img_h:0, ctx?ctx->current_person_name:"?"); }
|
||||
static void on_destroy(AppHandle app, void* data) { (void)app;(void)data; }
|
||||
|
||||
static void person_selected_cb(lv_event_t* e) {
|
||||
AppCtx* ctx = (AppCtx*)lv_event_get_user_data(e);
|
||||
if (!ctx) return;
|
||||
int idx = (int)(intptr_t)lv_obj_get_user_data(lv_event_get_target(e));
|
||||
if (idx < 0 || idx >= PERSONS_COUNT) return;
|
||||
const PersonInfo* p = &PERSONS[idx];
|
||||
strncpy(ctx->current_person_id, p->id, sizeof(ctx->current_person_id)-1);
|
||||
strncpy(ctx->current_person_name, p->name, sizeof(ctx->current_person_name)-1);
|
||||
ESP_LOGI(TAG, "Person selected %s (%s)", p->name, p->id);
|
||||
photo_count = 0;
|
||||
ctx->current_idx = 0;
|
||||
ctx->thumb_len = 0;
|
||||
ctx->has_image = false;
|
||||
show_player(ctx);
|
||||
start_fetch_task(ctx);
|
||||
}
|
||||
|
||||
static void build_person_picker(AppCtx* ctx, lv_obj_t* parent) {
|
||||
ctx->picker_wrapper = lv_obj_create(parent);
|
||||
lv_obj_set_size(ctx->picker_wrapper, LV_PCT(100), LV_PCT(100));
|
||||
lv_obj_set_style_border_width(ctx->picker_wrapper, 0, 0);
|
||||
lv_obj_set_style_pad_all(ctx->picker_wrapper, 0, 0);
|
||||
lv_obj_set_style_bg_color(ctx->picker_wrapper, lv_color_hex(0x121212), LV_PART_MAIN);
|
||||
lv_obj_set_style_bg_opa(ctx->picker_wrapper, LV_OPA_COVER, LV_PART_MAIN);
|
||||
|
||||
lv_obj_t* toolbar = tt_lvgl_toolbar_create_for_app(ctx->picker_wrapper, ctx->app);
|
||||
|
||||
ctx->lst_persons = lv_list_create(ctx->picker_wrapper);
|
||||
lv_obj_set_width(ctx->lst_persons, LV_PCT(100));
|
||||
lv_obj_align_to(ctx->lst_persons, toolbar, LV_ALIGN_OUT_BOTTOM_MID, 0, 0);
|
||||
// Calculate height
|
||||
int32_t th = 44;
|
||||
// Use parent height minus toolbar
|
||||
lv_obj_set_height(ctx->lst_persons, LV_PCT(90));
|
||||
lv_obj_set_style_bg_color(ctx->lst_persons, lv_color_hex(0x121212), LV_PART_MAIN);
|
||||
|
||||
for (int i=0;i<PERSONS_COUNT;i++) {
|
||||
const PersonInfo* person = &PERSONS[i];
|
||||
char label[64];
|
||||
if (person->count_str[0]) snprintf(label, sizeof(label), "%s (%s)", person->name, person->count_str);
|
||||
else snprintf(label, sizeof(label), "%s", person->name);
|
||||
lv_obj_t* btn = lv_list_add_button(ctx->lst_persons, LV_SYMBOL_DIRECTORY, label);
|
||||
lv_obj_set_user_data(btn, (void*)(intptr_t)i);
|
||||
lv_obj_add_event_cb(btn, person_selected_cb, LV_EVENT_CLICKED, ctx);
|
||||
}
|
||||
}
|
||||
|
||||
static void show_picker(AppCtx* ctx) {
|
||||
if (!ctx) return;
|
||||
if (!tt_lvgl_lock(pdMS_TO_TICKS(500))) return;
|
||||
if (ctx->player_wrapper) lv_obj_add_flag(ctx->player_wrapper, LV_OBJ_FLAG_HIDDEN);
|
||||
if (ctx->picker_wrapper) lv_obj_remove_flag(ctx->picker_wrapper, LV_OBJ_FLAG_HIDDEN);
|
||||
tt_lvgl_unlock();
|
||||
}
|
||||
|
||||
static void show_player(AppCtx* ctx) {
|
||||
if (!ctx) return;
|
||||
if (!tt_lvgl_lock(pdMS_TO_TICKS(500))) return;
|
||||
if (ctx->picker_wrapper) lv_obj_add_flag(ctx->picker_wrapper, LV_OBJ_FLAG_HIDDEN);
|
||||
if (ctx->player_wrapper) lv_obj_remove_flag(ctx->player_wrapper, LV_OBJ_FLAG_HIDDEN);
|
||||
tt_lvgl_unlock();
|
||||
show_chrome(ctx);
|
||||
}
|
||||
|
||||
static void on_show(AppHandle app, void* data, lv_obj_t* parent) {
|
||||
AppCtx* ctx=(AppCtx*)data; if(!ctx) return;
|
||||
ctx->visible=true; ctx->stop_requested=false; ctx->app=app;
|
||||
detect_display_size(ctx, parent);
|
||||
lv_obj_remove_flag(parent, LV_OBJ_FLAG_SCROLLABLE);
|
||||
lv_obj_set_style_border_width(parent, 0, LV_PART_MAIN);
|
||||
lv_obj_set_style_pad_all(parent, 0, LV_PART_MAIN);
|
||||
lv_obj_set_style_pad_gap(parent, 0, LV_PART_MAIN);
|
||||
lv_obj_set_style_bg_color(parent, lv_color_hex(0x121212), LV_PART_MAIN);
|
||||
lv_obj_set_style_bg_opa(parent, LV_OPA_COVER, LV_PART_MAIN);
|
||||
|
||||
// Picker for persons (like Bible book selector)
|
||||
build_person_picker(ctx, parent);
|
||||
|
||||
// Fullscreen player like Bible app - image background, only bottom controls
|
||||
ctx->player_wrapper = lv_obj_create(parent);
|
||||
lv_obj_set_size(ctx->player_wrapper, LV_PCT(100), LV_PCT(100));
|
||||
lv_obj_set_style_border_width(ctx->player_wrapper, 0, 0);
|
||||
lv_obj_set_style_pad_all(ctx->player_wrapper, 0, 0);
|
||||
lv_obj_set_style_pad_gap(ctx->player_wrapper, 0, 0);
|
||||
lv_obj_set_style_bg_color(ctx->player_wrapper, lv_color_hex(0x121212), LV_PART_MAIN);
|
||||
lv_obj_set_style_bg_opa(ctx->player_wrapper, LV_OPA_COVER, LV_PART_MAIN);
|
||||
lv_obj_remove_flag(ctx->player_wrapper, LV_OBJ_FLAG_SCROLLABLE);
|
||||
lv_obj_add_flag(ctx->player_wrapper, LV_OBJ_FLAG_HIDDEN);
|
||||
|
||||
// Image covers entire background
|
||||
ctx->img_page = lv_image_create(ctx->player_wrapper);
|
||||
lv_obj_set_size(ctx->img_page, LV_PCT(100), LV_PCT(100));
|
||||
lv_obj_align(ctx->img_page, LV_ALIGN_CENTER, 0, 0);
|
||||
lv_obj_set_style_bg_opa(ctx->img_page, LV_OPA_TRANSP, 0);
|
||||
lv_obj_set_style_pad_all(ctx->img_page, 0, 0);
|
||||
lv_obj_set_style_border_width(ctx->img_page, 0, 0);
|
||||
lv_obj_remove_flag(ctx->img_page, LV_OBJ_FLAG_SCROLLABLE);
|
||||
lv_obj_add_flag(ctx->img_page, LV_OBJ_FLAG_CLICKABLE);
|
||||
lv_obj_add_event_cb(ctx->img_page, on_image_click, LV_EVENT_CLICKED, ctx);
|
||||
|
||||
ctx->header_bar = NULL;
|
||||
ctx->lbl_book_title = NULL;
|
||||
ctx->lbl_indicator = NULL;
|
||||
ctx->lbl_info = NULL;
|
||||
ctx->lbl_status = NULL;
|
||||
ctx->btn_back = NULL;
|
||||
|
||||
ctx->ctrl_bar = lv_obj_create(ctx->player_wrapper);
|
||||
lv_obj_set_size(ctx->ctrl_bar, LV_PCT(100), 44);
|
||||
lv_obj_align(ctx->ctrl_bar, LV_ALIGN_BOTTOM_MID, 0, 0);
|
||||
lv_obj_set_style_radius(ctx->ctrl_bar, 0, 0);
|
||||
lv_obj_set_style_bg_color(ctx->ctrl_bar, lv_color_hex(0x0E0E14), LV_PART_MAIN);
|
||||
lv_obj_set_style_bg_opa(ctx->ctrl_bar, LV_OPA_80, LV_PART_MAIN);
|
||||
lv_obj_set_style_border_width(ctx->ctrl_bar, 0, 0);
|
||||
lv_obj_set_style_pad_all(ctx->ctrl_bar, 6, 0);
|
||||
lv_obj_set_style_pad_gap(ctx->ctrl_bar, 8, 0);
|
||||
lv_obj_set_flex_flow(ctx->ctrl_bar, LV_FLEX_FLOW_ROW);
|
||||
lv_obj_set_flex_align(ctx->ctrl_bar, LV_FLEX_ALIGN_SPACE_BETWEEN, LV_FLEX_ALIGN_CENTER, LV_FLEX_ALIGN_CENTER);
|
||||
lv_obj_remove_flag(ctx->ctrl_bar, LV_OBJ_FLAG_SCROLLABLE);
|
||||
|
||||
ctx->btn_back = lv_btn_create(ctx->ctrl_bar);
|
||||
lv_obj_set_size(ctx->btn_back, 44, 32);
|
||||
lv_obj_set_style_radius(ctx->btn_back, 8, 0);
|
||||
lv_obj_set_style_bg_color(ctx->btn_back, lv_color_hex(0x252538), LV_PART_MAIN);
|
||||
lv_obj_t* lbl_back = lv_label_create(ctx->btn_back); lv_label_set_text(lbl_back, LV_SYMBOL_LEFT); lv_obj_center(lbl_back);
|
||||
lv_obj_add_event_cb(ctx->btn_back, back_event_cb, LV_EVENT_CLICKED, ctx);
|
||||
|
||||
ctx->btn_prev = lv_btn_create(ctx->ctrl_bar);
|
||||
lv_obj_set_size(ctx->btn_prev, 54, 32);
|
||||
lv_obj_set_style_radius(ctx->btn_prev, 16, 0);
|
||||
lv_obj_set_style_bg_color(ctx->btn_prev, lv_color_hex(0x313244), LV_PART_MAIN);
|
||||
lv_obj_t* lp = lv_label_create(ctx->btn_prev); lv_label_set_text(lp, LV_SYMBOL_PREV); lv_obj_center(lp);
|
||||
lv_obj_add_event_cb(ctx->btn_prev, prev_event_cb, LV_EVENT_CLICKED, ctx);
|
||||
|
||||
ctx->btn_reload = lv_btn_create(ctx->ctrl_bar);
|
||||
lv_obj_set_size(ctx->btn_reload, 44, 32);
|
||||
lv_obj_set_style_radius(ctx->btn_reload, 16, 0);
|
||||
lv_obj_set_style_bg_color(ctx->btn_reload, lv_color_hex(0x313244), LV_PART_MAIN);
|
||||
lv_obj_t* lr = lv_label_create(ctx->btn_reload); lv_label_set_text(lr, LV_SYMBOL_REFRESH); lv_obj_center(lr);
|
||||
lv_obj_add_event_cb(ctx->btn_reload, reload_event_cb, LV_EVENT_CLICKED, ctx);
|
||||
|
||||
ctx->btn_next = lv_btn_create(ctx->ctrl_bar);
|
||||
lv_obj_set_size(ctx->btn_next, 54, 32);
|
||||
lv_obj_set_style_radius(ctx->btn_next, 16, 0);
|
||||
lv_obj_set_style_bg_color(ctx->btn_next, lv_color_hex(0x89B4FA), LV_PART_MAIN);
|
||||
lv_obj_set_style_text_color(ctx->btn_next, lv_color_hex(0x11111B), LV_PART_MAIN);
|
||||
lv_obj_t* ln = lv_label_create(ctx->btn_next); lv_label_set_text(ln, LV_SYMBOL_NEXT); lv_obj_center(ln);
|
||||
lv_obj_add_event_cb(ctx->btn_next, next_event_cb, LV_EVENT_CLICKED, ctx);
|
||||
|
||||
if (tt_lvgl_lock(pdMS_TO_TICKS(500))) {
|
||||
if (ctx->slideshow_timer) { lv_timer_delete(ctx->slideshow_timer); ctx->slideshow_timer=NULL; }
|
||||
ctx->slideshow_timer = lv_timer_create(slideshow_timer_cb, 60000, ctx);
|
||||
if (ctx->autohide_timer) { lv_timer_delete(ctx->autohide_timer); ctx->autohide_timer=NULL; }
|
||||
ctx->autohide_timer = NULL;
|
||||
tt_lvgl_unlock();
|
||||
}
|
||||
|
||||
// Always show person picker first like Bible book selector
|
||||
show_picker(ctx);
|
||||
// If we already have photos for current person, user can still see them after selecting
|
||||
// Don't auto-start fetch here; wait for person selection to fetch fresh random page
|
||||
// But if we have cached photos from previous session for current person, we could show player directly
|
||||
// For now, just show picker and let user choose person
|
||||
if (photo_count>0 && ctx->current_person_id[0]!='\0') {
|
||||
// Optionally prefetch in background
|
||||
snprintf(ctx->status_text,sizeof(ctx->status_text),"Cached %d for %s - select person", photo_count, ctx->current_person_name);
|
||||
} else {
|
||||
snprintf(ctx->status_text,sizeof(ctx->status_text),"Select a person");
|
||||
}
|
||||
}
|
||||
|
||||
static void on_hide(AppHandle app, void* data) {
|
||||
(void)app; AppCtx* ctx=(AppCtx*)data; if(!ctx) return;
|
||||
ESP_LOGI(TAG, "onHide start, stopping tasks");
|
||||
ctx->stop_requested=true; ctx->visible=false;
|
||||
if (tt_lvgl_lock(pdMS_TO_TICKS(500))) {
|
||||
if (ctx->slideshow_timer) { lv_timer_delete(ctx->slideshow_timer); ctx->slideshow_timer=NULL; }
|
||||
if (ctx->autohide_timer) { lv_timer_delete(ctx->autohide_timer); ctx->autohide_timer=NULL; }
|
||||
tt_lvgl_unlock();
|
||||
}
|
||||
int timeout=200;
|
||||
while((ctx->fetch_task || ctx->thumb_task) && timeout>0){ vTaskDelay(pdMS_TO_TICKS(50)); timeout--; }
|
||||
if (ctx->fetch_task || ctx->thumb_task) ESP_LOGW(TAG, "onHide timeout fetch=%p thumb=%p", ctx->fetch_task, ctx->thumb_task);
|
||||
else ESP_LOGI(TAG, "onHide tasks stopped cleanly");
|
||||
}
|
||||
|
||||
int main(int argc, char* argv[]) {
|
||||
(void)argc;(void)argv;
|
||||
tt_app_register((AppRegistration){.createData=create_data,.destroyData=destroy_data,.onCreate=on_create,.onDestroy=on_destroy,.onShow=on_show,.onHide=on_hide});
|
||||
return 0;
|
||||
}
|
||||
@@ -0,0 +1,7 @@
|
||||
manifest.version=0.2
|
||||
target.sdk=0.8.0-dev
|
||||
target.platforms=esp32s3,esp32p4
|
||||
app.id=one.tactility.imichelias
|
||||
app.version.name=0.1.0
|
||||
app.version.code=1
|
||||
app.name=Immich Elias
|
||||
@@ -1,19 +0,0 @@
|
||||
# PipecatVoice — Tactility native voice companion
|
||||
|
||||
Native ESP32 ELF app (like VoiceRecorder/ReynaBot), not a web client.
|
||||
|
||||
- Uses `audio-stream`/`i2s_controller` APIs (see VoiceRecorder) for 16kHz PCM mic + speaker
|
||||
- Uses ReynaBot's `websocket.c/h` + `lwip` for WS transport
|
||||
- Default gateway: Hermes WS `ws://<mac>:8642/api/esp32/voice/ws` (voice profile, limited tools, imperfect STT)
|
||||
- Optional second target: Pipecat websocket transport `ws://<mac>:7861` (when server exposes `websocket` via `create_transport`)
|
||||
|
||||
Build:
|
||||
```
|
||||
unset PYTHONPATH; export IDF_PYTHON_ENV_PATH=~/.espressif/python_env/idf5.3_py3.9_env
|
||||
. ~/esp/esp-idf/export.sh
|
||||
export TACTILITY_SDK_PATH=~/Projects/electronics/tactility/tactility/Buildscripts/TactilitySDK # or firmware/release/TactilitySDK if built
|
||||
$IDF_PYTHON_ENV_PATH/bin/python tactility.py Apps/PipecatVoice build esp32s3 --local-sdk
|
||||
$IDF_PYTHON_ENV_PATH/bin/python tactility.py Apps/PipecatVoice install 192.168.68.112 esp32s3
|
||||
```
|
||||
|
||||
Adapted from ReynaBot (PTT + I2S + WS JSON events: ready/listening/transcript/thinking/response_text/audio_start/audio_end/done/error).
|
||||
@@ -1,830 +0,0 @@
|
||||
#include <tt_app.h>
|
||||
#include <tt_lvgl.h>
|
||||
#include <tt_lvgl_toolbar.h>
|
||||
#include <tt_wifi.h>
|
||||
|
||||
#include <tactility/device.h>
|
||||
#include <tactility/drivers/i2s_controller.h>
|
||||
|
||||
#include "websocket.h"
|
||||
#include <cJSON.h>
|
||||
|
||||
#include <string.h>
|
||||
#include <stdlib.h>
|
||||
#include <sys/stat.h>
|
||||
#include <errno.h>
|
||||
|
||||
#include "freertos/FreeRTOS.h"
|
||||
#include "freertos/task.h"
|
||||
#include "esp_log.h"
|
||||
|
||||
#define TAG "PipecatVoice"
|
||||
|
||||
typedef enum {
|
||||
STATE_IDLE,
|
||||
STATE_CONNECTING,
|
||||
STATE_LISTENING,
|
||||
STATE_THINKING,
|
||||
STATE_SPEAKING,
|
||||
STATE_ERROR
|
||||
} PipecatVoiceState;
|
||||
|
||||
typedef struct {
|
||||
AppHandle app;
|
||||
bool visible;
|
||||
PipecatVoiceState state;
|
||||
char server_url[128];
|
||||
char device_id[64];
|
||||
char api_key[128];
|
||||
|
||||
char last_transcript[256];
|
||||
char last_response[512];
|
||||
char error_message[128];
|
||||
|
||||
// UI components
|
||||
lv_obj_t* lbl_status;
|
||||
lv_obj_t* lbl_transcript;
|
||||
lv_obj_t* lbl_response;
|
||||
lv_obj_t* btn_ptt;
|
||||
lv_obj_t* btn_ptt_label;
|
||||
lv_obj_t* btn_grace;
|
||||
lv_obj_t* btn_elias;
|
||||
|
||||
bool speaker_selected;
|
||||
lv_obj_t* speaker_select_cont;
|
||||
lv_obj_t* app_main_cont;
|
||||
|
||||
// PTT control
|
||||
bool is_pressed;
|
||||
bool start_session;
|
||||
bool stop_session;
|
||||
bool cancel_session;
|
||||
|
||||
// FreeRTOS Tasks
|
||||
TaskHandle_t worker_task;
|
||||
TaskHandle_t rx_task;
|
||||
|
||||
// Hardware
|
||||
struct Device* i2s_dev;
|
||||
|
||||
// WebSocket
|
||||
int ws_fd;
|
||||
bool ws_connected;
|
||||
bool ws_done;
|
||||
|
||||
bool ui_update_pending;
|
||||
} PipecatVoiceCtx;
|
||||
|
||||
/* ─── Forward Declarations ─── */
|
||||
static void pipecatvoice_task(void* arg);
|
||||
static void pipecatvoice_rx_task(void* arg);
|
||||
static void update_ui(PipecatVoiceCtx* ctx);
|
||||
static void load_config(PipecatVoiceCtx* ctx);
|
||||
|
||||
/* ─── Helper for URL parsing ─── */
|
||||
static bool parse_ws_url(const char* url, char* host, int* port, char* path) {
|
||||
if (strncmp(url, "ws://", 5) != 0) return false;
|
||||
const char* p = url + 5;
|
||||
const char* colon = strchr(p, ':');
|
||||
const char* slash = strchr(p, '/');
|
||||
|
||||
if (slash == NULL) {
|
||||
strcpy(path, "/");
|
||||
} else {
|
||||
strcpy(path, slash);
|
||||
}
|
||||
|
||||
if (colon != NULL && (slash == NULL || colon < slash)) {
|
||||
int host_len = colon - p;
|
||||
memcpy(host, p, host_len);
|
||||
host[host_len] = '\0';
|
||||
*port = atoi(colon + 1);
|
||||
} else {
|
||||
int host_len = slash ? (slash - p) : strlen(p);
|
||||
memcpy(host, p, host_len);
|
||||
host[host_len] = '\0';
|
||||
*port = 80;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
/* ─── Config Loading/Saving ─── */
|
||||
static void load_config(PipecatVoiceCtx* ctx) {
|
||||
// Default fallback config
|
||||
snprintf(ctx->server_url, sizeof(ctx->server_url), "ws://192.168.68.102:8642/api/esp32/voice/ws");
|
||||
snprintf(ctx->device_id, sizeof(ctx->device_id), "pipecatvoice");
|
||||
snprintf(ctx->api_key, sizeof(ctx->api_key), "mcT1YA1vOr9wXSiHpCYalweEGGZKX-PIfZv2drp8BSg");
|
||||
|
||||
char path[256];
|
||||
size_t path_size = sizeof(path);
|
||||
tt_app_get_user_data_child_path(ctx->app, "config.json", path, &path_size);
|
||||
|
||||
FILE* file = fopen(path, "r");
|
||||
if (file == NULL) {
|
||||
// Create user data directory if it doesn't exist
|
||||
char dir_path[256];
|
||||
size_t dir_size = sizeof(dir_path);
|
||||
tt_app_get_user_data_path(ctx->app, dir_path, &dir_size);
|
||||
mkdir(dir_path, 0755);
|
||||
|
||||
// Write default config file
|
||||
file = fopen(path, "w");
|
||||
if (file != NULL) {
|
||||
fprintf(file, "{\n \"server_url\": \"ws://192.168.68.102:8642/api/esp32/voice/ws\",\n \"device_id\": \"pipecatvoice\",\n \"api_key\": \"mcT1YA1vOr9wXSiHpCYalweEGGZKX-PIfZv2drp8BSg\"\n}\n");
|
||||
fclose(file);
|
||||
}
|
||||
ESP_LOGI(TAG, "Created default config.json at %s", path);
|
||||
return;
|
||||
}
|
||||
|
||||
fseek(file, 0, SEEK_END);
|
||||
long size = ftell(file);
|
||||
fseek(file, 0, SEEK_SET);
|
||||
|
||||
if (size > 0 && size < 4096) {
|
||||
char* buf = malloc(size + 1);
|
||||
if (buf != NULL) {
|
||||
size_t read_bytes = fread(buf, 1, size, file);
|
||||
buf[read_bytes] = '\0';
|
||||
cJSON* json = cJSON_Parse(buf);
|
||||
if (json != NULL) {
|
||||
cJSON* url_item = cJSON_GetObjectItem(json, "server_url");
|
||||
if (url_item != NULL && url_item->valuestring != NULL) {
|
||||
strncpy(ctx->server_url, url_item->valuestring, sizeof(ctx->server_url) - 1);
|
||||
}
|
||||
cJSON* dev_item = cJSON_GetObjectItem(json, "device_id");
|
||||
if (dev_item != NULL && dev_item->valuestring != NULL) {
|
||||
strncpy(ctx->device_id, dev_item->valuestring, sizeof(ctx->device_id) - 1);
|
||||
}
|
||||
cJSON* key_item = cJSON_GetObjectItem(json, "api_key");
|
||||
if (key_item != NULL && key_item->valuestring != NULL) {
|
||||
strncpy(ctx->api_key, key_item->valuestring, sizeof(ctx->api_key) - 1);
|
||||
}
|
||||
cJSON_Delete(json);
|
||||
}
|
||||
free(buf);
|
||||
}
|
||||
}
|
||||
fclose(file);
|
||||
ESP_LOGI(TAG, "Loaded config: server=%s device=%s", ctx->server_url, ctx->device_id);
|
||||
}
|
||||
|
||||
/* ─── JSON Message Processing ─── */
|
||||
static void parse_json_message(PipecatVoiceCtx* ctx, const char* json_str) {
|
||||
cJSON* json = cJSON_Parse(json_str);
|
||||
if (json == NULL) return;
|
||||
|
||||
cJSON* evt_item = cJSON_GetObjectItem(json, "event");
|
||||
if (evt_item != NULL && evt_item->valuestring != NULL) {
|
||||
const char* evt = evt_item->valuestring;
|
||||
ESP_LOGI(TAG, "WS Event: %s", evt);
|
||||
|
||||
if (strcmp(evt, "ready") == 0) {
|
||||
ESP_LOGI(TAG, "Server ready");
|
||||
} else if (strcmp(evt, "listening") == 0) {
|
||||
ctx->state = STATE_LISTENING;
|
||||
ctx->ui_update_pending = true;
|
||||
} else if (strcmp(evt, "transcript") == 0) {
|
||||
cJSON* txt_item = cJSON_GetObjectItem(json, "text");
|
||||
if (txt_item != NULL && txt_item->valuestring != NULL) {
|
||||
strncpy(ctx->last_transcript, txt_item->valuestring, sizeof(ctx->last_transcript) - 1);
|
||||
ctx->ui_update_pending = true;
|
||||
}
|
||||
} else if (strcmp(evt, "thinking") == 0) {
|
||||
ctx->state = STATE_THINKING;
|
||||
ctx->ui_update_pending = true;
|
||||
} else if (strcmp(evt, "response_text") == 0) {
|
||||
cJSON* txt_item = cJSON_GetObjectItem(json, "text");
|
||||
if (txt_item != NULL && txt_item->valuestring != NULL) {
|
||||
strncpy(ctx->last_response, txt_item->valuestring, sizeof(ctx->last_response) - 1);
|
||||
ctx->ui_update_pending = true;
|
||||
}
|
||||
} else if (strcmp(evt, "audio_start") == 0) {
|
||||
ctx->state = STATE_SPEAKING;
|
||||
ctx->ui_update_pending = true;
|
||||
|
||||
// I2S is already configured globally at session startup
|
||||
} else if (strcmp(evt, "audio_end") == 0) {
|
||||
ESP_LOGI(TAG, "Audio response ended");
|
||||
} else if (strcmp(evt, "done") == 0) {
|
||||
ctx->ws_done = true;
|
||||
} else if (strcmp(evt, "error") == 0) {
|
||||
cJSON* msg_item = cJSON_GetObjectItem(json, "message");
|
||||
if (msg_item != NULL && msg_item->valuestring != NULL) {
|
||||
strncpy(ctx->error_message, msg_item->valuestring, sizeof(ctx->error_message) - 1);
|
||||
} else {
|
||||
snprintf(ctx->error_message, sizeof(ctx->error_message), "Unknown server error");
|
||||
}
|
||||
ctx->state = STATE_ERROR;
|
||||
ctx->ui_update_pending = true;
|
||||
}
|
||||
}
|
||||
cJSON_Delete(json);
|
||||
}
|
||||
|
||||
/* ─── WebSocket RX (Receive) Task ─── */
|
||||
static void pipecatvoice_rx_task(void* arg) {
|
||||
PipecatVoiceCtx* ctx = (PipecatVoiceCtx*)arg;
|
||||
uint8_t* rx_buf = malloc(8192);
|
||||
if (rx_buf == NULL) {
|
||||
ESP_LOGE(TAG, "Failed to allocate RX buffer");
|
||||
ctx->ws_done = true;
|
||||
ctx->rx_task = NULL;
|
||||
vTaskDelete(NULL);
|
||||
return;
|
||||
}
|
||||
int opcode;
|
||||
|
||||
ESP_LOGI(TAG, "WS Receive task started");
|
||||
|
||||
while (ctx->ws_fd >= 0) {
|
||||
int r = ws_recv(ctx->ws_fd, &opcode, rx_buf, 8191);
|
||||
if (r < 0) {
|
||||
if (r == -1) {
|
||||
ESP_LOGI(TAG, "WS connection closed or read error, errno=%d", errno);
|
||||
} else {
|
||||
ESP_LOGE(TAG, "WS rx buffer overflow");
|
||||
}
|
||||
ctx->ws_done = true;
|
||||
break;
|
||||
}
|
||||
|
||||
if (opcode == 0x01) { // Text frame (JSON)
|
||||
rx_buf[r] = '\0';
|
||||
parse_json_message(ctx, (char*)rx_buf);
|
||||
} else if (opcode == 0x02) { // Binary frame (Audio data)
|
||||
if (ctx->state == STATE_SPEAKING && ctx->i2s_dev != NULL) {
|
||||
const uint8_t* payload_ptr = rx_buf;
|
||||
size_t payload_len = r;
|
||||
|
||||
// Skip WAV header if present in the first chunk
|
||||
if (payload_len > 44 && memcmp(payload_ptr, "RIFF", 4) == 0) {
|
||||
payload_ptr += 44;
|
||||
payload_len -= 44;
|
||||
}
|
||||
|
||||
size_t written = 0;
|
||||
device_lock(ctx->i2s_dev);
|
||||
i2s_controller_write(ctx->i2s_dev, payload_ptr, payload_len, &written, pdMS_TO_TICKS(100));
|
||||
device_unlock(ctx->i2s_dev);
|
||||
}
|
||||
} else if (opcode == 0x09) { // PING frame
|
||||
ESP_LOGI(TAG, "WS PING received, sending PONG");
|
||||
ws_send_pong(ctx->ws_fd, rx_buf, r);
|
||||
}
|
||||
}
|
||||
|
||||
free(rx_buf);
|
||||
ctx->rx_task = NULL;
|
||||
vTaskDelete(NULL);
|
||||
}
|
||||
|
||||
/* ─── UI Speaker Selection Event Callback ─── */
|
||||
static void speaker_event_cb(lv_event_t* e) {
|
||||
PipecatVoiceCtx* ctx = (PipecatVoiceCtx*)lv_event_get_user_data(e);
|
||||
lv_obj_t* target = lv_event_get_target(e);
|
||||
lv_event_code_t code = lv_event_get_code(e);
|
||||
|
||||
if (code == LV_EVENT_CLICKED && !ctx->speaker_selected) {
|
||||
if (target == ctx->btn_grace) {
|
||||
strcpy(ctx->device_id, "grace-pipecat");
|
||||
} else if (target == ctx->btn_elias) {
|
||||
strcpy(ctx->device_id, "elias-pipecat");
|
||||
}
|
||||
|
||||
ctx->speaker_selected = true;
|
||||
|
||||
// Hide selection screen, show main screen
|
||||
lv_obj_add_flag(ctx->speaker_select_cont, LV_OBJ_FLAG_HIDDEN);
|
||||
lv_obj_remove_flag(ctx->app_main_cont, LV_OBJ_FLAG_HIDDEN);
|
||||
|
||||
// Trigger UI update to refresh status and PTT button
|
||||
ctx->ui_update_pending = true;
|
||||
}
|
||||
}
|
||||
|
||||
/* ─── UI Press-to-Talk Event Callback ─── */
|
||||
static void ptt_event_cb(lv_event_t* e) {
|
||||
PipecatVoiceCtx* ctx = (PipecatVoiceCtx*)lv_event_get_user_data(e);
|
||||
lv_event_code_t code = lv_event_get_code(e);
|
||||
|
||||
if (code == LV_EVENT_PRESSED) {
|
||||
ctx->is_pressed = true;
|
||||
if (ctx->state == STATE_IDLE || ctx->state == ERROR_NONE) {
|
||||
ctx->start_session = true;
|
||||
}
|
||||
} else if (code == LV_EVENT_RELEASED || code == LV_EVENT_PRESS_LOST) {
|
||||
ctx->is_pressed = false;
|
||||
if (ctx->state == STATE_LISTENING) {
|
||||
ctx->stop_session = true;
|
||||
} else if (ctx->state == STATE_CONNECTING) {
|
||||
ctx->cancel_session = true;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/* ─── Core UI Update Function ─── */
|
||||
static void update_ui(PipecatVoiceCtx* ctx) {
|
||||
if (!ctx->visible) return;
|
||||
|
||||
if (tt_lvgl_lock(pdMS_TO_TICKS(500))) {
|
||||
if (!ctx->visible) {
|
||||
tt_lvgl_unlock();
|
||||
return;
|
||||
}
|
||||
|
||||
if (!ctx->speaker_selected) {
|
||||
lv_label_set_text(ctx->lbl_status, "Choose Speaker");
|
||||
lv_obj_set_style_text_color(ctx->lbl_status, lv_color_hex(0x9E9E9E), LV_PART_MAIN);
|
||||
tt_lvgl_unlock();
|
||||
return;
|
||||
}
|
||||
|
||||
switch (ctx->state) {
|
||||
case STATE_IDLE:
|
||||
lv_label_set_text(ctx->lbl_status, "Ready");
|
||||
lv_obj_set_style_text_color(ctx->lbl_status, lv_color_hex(0x9E9E9E), LV_PART_MAIN);
|
||||
lv_label_set_text(ctx->btn_ptt_label, LV_SYMBOL_AUDIO " Hold to Talk");
|
||||
lv_obj_set_style_bg_color(ctx->btn_ptt, lv_color_hex(0x6200EE), LV_PART_MAIN);
|
||||
lv_obj_remove_flag(ctx->btn_ptt, LV_OBJ_FLAG_HIDDEN);
|
||||
break;
|
||||
case STATE_CONNECTING:
|
||||
lv_label_set_text(ctx->lbl_status, "Connecting...");
|
||||
lv_obj_set_style_text_color(ctx->lbl_status, lv_color_hex(0xFFC107), LV_PART_MAIN);
|
||||
lv_label_set_text(ctx->btn_ptt_label, "Connecting...");
|
||||
lv_obj_set_style_bg_color(ctx->btn_ptt, lv_color_hex(0x757575), LV_PART_MAIN);
|
||||
lv_obj_remove_flag(ctx->btn_ptt, LV_OBJ_FLAG_HIDDEN);
|
||||
break;
|
||||
case STATE_LISTENING:
|
||||
lv_label_set_text(ctx->lbl_status, "Listening...");
|
||||
lv_obj_set_style_text_color(ctx->lbl_status, lv_color_hex(0x00E676), LV_PART_MAIN);
|
||||
lv_label_set_text(ctx->btn_ptt_label, "Release to Stop");
|
||||
lv_obj_set_style_bg_color(ctx->btn_ptt, lv_color_hex(0xD50000), LV_PART_MAIN);
|
||||
lv_obj_remove_flag(ctx->btn_ptt, LV_OBJ_FLAG_HIDDEN);
|
||||
break;
|
||||
case STATE_THINKING:
|
||||
lv_label_set_text(ctx->lbl_status, "Thinking...");
|
||||
lv_obj_set_style_text_color(ctx->lbl_status, lv_color_hex(0x00B0FF), LV_PART_MAIN);
|
||||
lv_obj_add_flag(ctx->btn_ptt, LV_OBJ_FLAG_HIDDEN);
|
||||
break;
|
||||
case STATE_SPEAKING:
|
||||
lv_label_set_text(ctx->lbl_status, "Speaking...");
|
||||
lv_obj_set_style_text_color(ctx->lbl_status, lv_color_hex(0xAA00FF), LV_PART_MAIN);
|
||||
lv_obj_add_flag(ctx->btn_ptt, LV_OBJ_FLAG_HIDDEN);
|
||||
break;
|
||||
case STATE_ERROR:
|
||||
lv_label_set_text(ctx->lbl_status, "Error");
|
||||
lv_obj_set_style_text_color(ctx->lbl_status, lv_color_hex(0xFF1744), LV_PART_MAIN);
|
||||
lv_label_set_text(ctx->btn_ptt_label, LV_SYMBOL_REFRESH " Try Again");
|
||||
lv_obj_set_style_bg_color(ctx->btn_ptt, lv_color_hex(0xFF1744), LV_PART_MAIN);
|
||||
lv_obj_remove_flag(ctx->btn_ptt, LV_OBJ_FLAG_HIDDEN);
|
||||
|
||||
// Show the full error message in the bot's response bubble
|
||||
if (ctx->error_message[0] != '\0') {
|
||||
snprintf(ctx->last_response, sizeof(ctx->last_response), "Error: %s", ctx->error_message);
|
||||
}
|
||||
break;
|
||||
}
|
||||
|
||||
lv_label_set_text(ctx->lbl_transcript, ctx->last_transcript[0] != '\0' ? ctx->last_transcript : "(Your question will appear here)");
|
||||
lv_label_set_text(ctx->lbl_response, ctx->last_response[0] != '\0' ? ctx->last_response : "(Answer will appear here)");
|
||||
|
||||
tt_lvgl_unlock();
|
||||
}
|
||||
}
|
||||
|
||||
/* ─── Streaming & Main Worker Task ─── */
|
||||
static void pipecatvoice_task(void* arg) {
|
||||
PipecatVoiceCtx* ctx = (PipecatVoiceCtx*)arg;
|
||||
char host[64];
|
||||
char path[128];
|
||||
int port = 80;
|
||||
|
||||
ctx->ws_fd = -1;
|
||||
|
||||
|
||||
while (ctx->visible) {
|
||||
if (ctx->start_session) {
|
||||
ctx->start_session = false;
|
||||
ctx->stop_session = false;
|
||||
ctx->cancel_session = false;
|
||||
|
||||
ctx->state = STATE_CONNECTING;
|
||||
ctx->last_transcript[0] = '\0';
|
||||
ctx->last_response[0] = '\0';
|
||||
ctx->error_message[0] = '\0';
|
||||
ctx->ws_done = false;
|
||||
update_ui(ctx);
|
||||
|
||||
if (!parse_ws_url(ctx->server_url, host, &port, path)) {
|
||||
ctx->state = STATE_ERROR;
|
||||
snprintf(ctx->error_message, sizeof(ctx->error_message), "Invalid Server URL");
|
||||
update_ui(ctx);
|
||||
continue;
|
||||
}
|
||||
|
||||
ESP_LOGI(TAG, "Connecting: host=%s port=%d path=%s", host, port, path);
|
||||
int fd = ws_connect(host, port, path, ctx->device_id, ctx->api_key);
|
||||
if (fd < 0) {
|
||||
ctx->state = STATE_ERROR;
|
||||
snprintf(ctx->error_message, sizeof(ctx->error_message), "Connect failed");
|
||||
update_ui(ctx);
|
||||
continue;
|
||||
}
|
||||
|
||||
ctx->ws_fd = fd;
|
||||
ctx->ws_connected = true;
|
||||
|
||||
// Configure I2S on demand for this session (16 kHz, 16-bit, mono)
|
||||
struct I2sConfig session_cfg = {
|
||||
.communication_format = I2S_FORMAT_STAND_I2S,
|
||||
.sample_rate = 16000,
|
||||
.bits_per_sample = 16,
|
||||
.channel_left = 0,
|
||||
.channel_right = I2S_CHANNEL_NONE
|
||||
};
|
||||
if (ctx->i2s_dev != NULL) {
|
||||
device_lock(ctx->i2s_dev);
|
||||
i2s_controller_set_config(ctx->i2s_dev, &session_cfg);
|
||||
device_unlock(ctx->i2s_dev);
|
||||
}
|
||||
|
||||
// Spawn background RX task to read and parse events
|
||||
xTaskCreate(pipecatvoice_rx_task, "pipecat_rx", 4096, ctx, 6, &ctx->rx_task);
|
||||
|
||||
// Send start event handshake
|
||||
char start_json[256];
|
||||
snprintf(start_json, sizeof(start_json),
|
||||
"{\"event\":\"start\",\"device_id\":\"%s\",\"sample_rate\":16000,\"channels\":1,\"sample_width\":2,\"format\":\"pcm_s16le\"}",
|
||||
ctx->device_id);
|
||||
if (ws_send(ctx->ws_fd, (const uint8_t*)start_json, strlen(start_json), false) < 0) {
|
||||
ctx->state = STATE_ERROR;
|
||||
snprintf(ctx->error_message, sizeof(ctx->error_message), "Handshake send failed");
|
||||
ws_close(ctx->ws_fd);
|
||||
ctx->ws_fd = -1;
|
||||
ctx->ws_connected = false;
|
||||
update_ui(ctx);
|
||||
continue;
|
||||
}
|
||||
|
||||
// Wait for server to switch us to LISTENING
|
||||
int timeout_ms = 4000;
|
||||
while (ctx->state != STATE_LISTENING && timeout_ms > 0 && !ctx->cancel_session && !ctx->ws_done) {
|
||||
vTaskDelay(pdMS_TO_TICKS(50));
|
||||
timeout_ms -= 50;
|
||||
}
|
||||
|
||||
if (ctx->state != STATE_LISTENING) {
|
||||
if (ctx->state != STATE_ERROR) {
|
||||
ctx->state = STATE_ERROR;
|
||||
snprintf(ctx->error_message, sizeof(ctx->error_message), "Handshake timeout");
|
||||
}
|
||||
ws_close(ctx->ws_fd);
|
||||
ctx->ws_fd = -1;
|
||||
ctx->ws_connected = false;
|
||||
update_ui(ctx);
|
||||
continue;
|
||||
}
|
||||
|
||||
// I2S is already configured globally at session startup
|
||||
|
||||
uint8_t* buffer = malloc(1024);
|
||||
if (buffer == NULL) {
|
||||
ESP_LOGE(TAG, "Failed to allocate record buffer");
|
||||
ctx->state = STATE_ERROR;
|
||||
snprintf(ctx->error_message, sizeof(ctx->error_message), "Out of memory");
|
||||
ws_close(ctx->ws_fd);
|
||||
ctx->ws_fd = -1;
|
||||
ctx->ws_connected = false;
|
||||
update_ui(ctx);
|
||||
continue;
|
||||
}
|
||||
size_t total_sent_bytes = 0;
|
||||
|
||||
// Stream audio loop while PTT is held
|
||||
while (ctx->is_pressed && !ctx->stop_session && !ctx->cancel_session && !ctx->ws_done && total_sent_bytes < 320000) {
|
||||
size_t bytes_read = 0;
|
||||
error_t r = i2s_controller_read(ctx->i2s_dev, buffer, 1024, &bytes_read, pdMS_TO_TICKS(100));
|
||||
if (r == ERROR_NONE && bytes_read > 0) {
|
||||
if (ws_send(ctx->ws_fd, buffer, bytes_read, true) < 0) {
|
||||
ESP_LOGE(TAG, "Audio stream send failed");
|
||||
break;
|
||||
}
|
||||
total_sent_bytes += bytes_read;
|
||||
}
|
||||
}
|
||||
|
||||
free(buffer);
|
||||
|
||||
if (ctx->cancel_session || total_sent_bytes < 3200) {
|
||||
ESP_LOGI(TAG, "Cancelling audio session");
|
||||
const char* cancel_json = "{\"event\":\"cancel\"}";
|
||||
ws_send(ctx->ws_fd, (const uint8_t*)cancel_json, strlen(cancel_json), false);
|
||||
ctx->state = STATE_IDLE;
|
||||
} else {
|
||||
const char* stop_json = "{\"event\":\"stop\"}";
|
||||
ws_send(ctx->ws_fd, (const uint8_t*)stop_json, strlen(stop_json), false);
|
||||
ctx->state = STATE_THINKING;
|
||||
update_ui(ctx);
|
||||
|
||||
// Wait for response to finish (increase to 90s to match socket timeout)
|
||||
int wait_timeout_ms = 90000;
|
||||
while (!ctx->ws_done && ctx->state != STATE_ERROR && wait_timeout_ms > 0) {
|
||||
vTaskDelay(pdMS_TO_TICKS(100));
|
||||
wait_timeout_ms -= 100;
|
||||
if (ctx->ui_update_pending) {
|
||||
ctx->ui_update_pending = false;
|
||||
update_ui(ctx);
|
||||
}
|
||||
}
|
||||
|
||||
if (!ctx->ws_done && ctx->state != STATE_ERROR) {
|
||||
ctx->state = STATE_ERROR;
|
||||
snprintf(ctx->error_message, sizeof(ctx->error_message), "Response timeout");
|
||||
}
|
||||
}
|
||||
|
||||
// Clean up socket
|
||||
int fd_to_close = ctx->ws_fd;
|
||||
ctx->ws_fd = -1;
|
||||
ctx->ws_connected = false;
|
||||
ws_close(fd_to_close);
|
||||
|
||||
// Reset I2S controller to release DMA and stop white noise
|
||||
if (ctx->i2s_dev != NULL) {
|
||||
device_lock(ctx->i2s_dev);
|
||||
i2s_controller_reset(ctx->i2s_dev);
|
||||
device_unlock(ctx->i2s_dev);
|
||||
}
|
||||
|
||||
// Wait for receive task to exit
|
||||
int rx_timeout = 100;
|
||||
while (ctx->rx_task != NULL && rx_timeout > 0) {
|
||||
vTaskDelay(pdMS_TO_TICKS(10));
|
||||
rx_timeout--;
|
||||
}
|
||||
|
||||
if (ctx->state != STATE_ERROR) {
|
||||
ctx->state = STATE_IDLE;
|
||||
}
|
||||
update_ui(ctx);
|
||||
}
|
||||
|
||||
vTaskDelay(pdMS_TO_TICKS(100));
|
||||
if (ctx->ui_update_pending) {
|
||||
ctx->ui_update_pending = false;
|
||||
update_ui(ctx);
|
||||
}
|
||||
}
|
||||
|
||||
ctx->worker_task = NULL;
|
||||
vTaskDelete(NULL);
|
||||
}
|
||||
|
||||
/* ─── App Lifecycle ─── */
|
||||
|
||||
static void* create_data(void) {
|
||||
PipecatVoiceCtx* ctx = calloc(1, sizeof(PipecatVoiceCtx));
|
||||
if (ctx != NULL) {
|
||||
ctx->state = STATE_IDLE;
|
||||
ctx->ws_fd = -1;
|
||||
}
|
||||
return ctx;
|
||||
}
|
||||
|
||||
static void destroy_data(void* data) {
|
||||
free(data);
|
||||
}
|
||||
|
||||
static void on_create(AppHandle app, void* data) {
|
||||
PipecatVoiceCtx* ctx = (PipecatVoiceCtx*)data;
|
||||
ctx->app = app;
|
||||
}
|
||||
|
||||
static void on_destroy(AppHandle app, void* data) {
|
||||
// No-op
|
||||
}
|
||||
|
||||
static void on_show(AppHandle app, void* data, lv_obj_t* parent) {
|
||||
PipecatVoiceCtx* ctx = (PipecatVoiceCtx*)data;
|
||||
ctx->visible = true;
|
||||
|
||||
load_config(ctx);
|
||||
|
||||
// Find I2S controller
|
||||
ctx->i2s_dev = device_find_by_name("i2s0");
|
||||
if (ctx->i2s_dev == NULL) {
|
||||
ESP_LOGE(TAG, "I2S controller 'i2s0' not found!");
|
||||
}
|
||||
|
||||
// Style the parent screen
|
||||
lv_obj_set_flex_flow(parent, LV_FLEX_FLOW_COLUMN);
|
||||
lv_obj_set_flex_align(parent, LV_FLEX_ALIGN_START, LV_FLEX_ALIGN_CENTER, LV_FLEX_ALIGN_CENTER);
|
||||
lv_obj_set_style_pad_all(parent, 10, LV_PART_MAIN);
|
||||
lv_obj_set_style_pad_row(parent, 12, LV_PART_MAIN);
|
||||
lv_obj_set_style_bg_color(parent, lv_color_hex(0x0C0B12), LV_PART_MAIN); // Rich dark background
|
||||
|
||||
// Add App Toolbar
|
||||
lv_obj_t* toolbar = tt_lvgl_toolbar_create_for_app(parent, app);
|
||||
lv_obj_align(toolbar, LV_ALIGN_TOP_MID, 0, 0);
|
||||
|
||||
// Status Label inside toolbar (replaces the static "ReynaBot Voice" label)
|
||||
ctx->lbl_status = lv_label_create(toolbar);
|
||||
lv_label_set_text(ctx->lbl_status, "Ready");
|
||||
lv_obj_add_flag(ctx->lbl_status, LV_OBJ_FLAG_FLOATING);
|
||||
lv_obj_align(ctx->lbl_status, LV_ALIGN_RIGHT_MID, -10, 0);
|
||||
lv_obj_set_style_text_color(ctx->lbl_status, lv_color_hex(0x9E9E9E), LV_PART_MAIN);
|
||||
|
||||
// Initial state
|
||||
ctx->speaker_selected = false;
|
||||
|
||||
// Default to Grace if device_id is not already configured to elias-esp32
|
||||
if (strcmp(ctx->device_id, "elias-pipecat") != 0) {
|
||||
strcpy(ctx->device_id, "grace-pipecat");
|
||||
}
|
||||
|
||||
// 1. Speaker Selection Container (Startup Screen)
|
||||
ctx->speaker_select_cont = lv_obj_create(parent);
|
||||
lv_obj_set_width(ctx->speaker_select_cont, lv_pct(95));
|
||||
lv_obj_set_flex_grow(ctx->speaker_select_cont, 1);
|
||||
lv_obj_set_flex_flow(ctx->speaker_select_cont, LV_FLEX_FLOW_COLUMN);
|
||||
lv_obj_set_flex_align(ctx->speaker_select_cont, LV_FLEX_ALIGN_CENTER, LV_FLEX_ALIGN_CENTER, LV_FLEX_ALIGN_CENTER);
|
||||
lv_obj_set_style_pad_all(ctx->speaker_select_cont, 0, LV_PART_MAIN);
|
||||
lv_obj_set_style_bg_opa(ctx->speaker_select_cont, 0, LV_PART_MAIN);
|
||||
lv_obj_set_style_border_width(ctx->speaker_select_cont, 0, LV_PART_MAIN);
|
||||
lv_obj_set_style_pad_row(ctx->speaker_select_cont, 16, LV_PART_MAIN);
|
||||
|
||||
// Title Prompt
|
||||
lv_obj_t* prompt_lbl = lv_label_create(ctx->speaker_select_cont);
|
||||
lv_label_set_text(prompt_lbl, "Who is speaking?");
|
||||
lv_obj_set_style_text_color(prompt_lbl, lv_color_hex(0xFFFFFF), LV_PART_MAIN);
|
||||
|
||||
// Row of two selection buttons
|
||||
lv_obj_t* speaker_row = lv_obj_create(ctx->speaker_select_cont);
|
||||
lv_obj_set_size(speaker_row, lv_pct(100), 140);
|
||||
lv_obj_set_flex_flow(speaker_row, LV_FLEX_FLOW_ROW);
|
||||
lv_obj_set_flex_align(speaker_row, LV_FLEX_ALIGN_SPACE_BETWEEN, LV_FLEX_ALIGN_CENTER, LV_FLEX_ALIGN_CENTER);
|
||||
lv_obj_set_style_pad_all(speaker_row, 0, LV_PART_MAIN);
|
||||
lv_obj_set_style_bg_opa(speaker_row, 0, LV_PART_MAIN);
|
||||
lv_obj_set_style_border_width(speaker_row, 0, LV_PART_MAIN);
|
||||
|
||||
// Grace Button (Girl)
|
||||
ctx->btn_grace = lv_btn_create(speaker_row);
|
||||
lv_obj_set_size(ctx->btn_grace, lv_pct(46), 130);
|
||||
lv_obj_set_flex_flow(ctx->btn_grace, LV_FLEX_FLOW_COLUMN);
|
||||
lv_obj_set_flex_align(ctx->btn_grace, LV_FLEX_ALIGN_CENTER, LV_FLEX_ALIGN_CENTER, LV_FLEX_ALIGN_CENTER);
|
||||
lv_obj_set_style_pad_all(ctx->btn_grace, 8, LV_PART_MAIN);
|
||||
lv_obj_set_style_pad_row(ctx->btn_grace, 8, LV_PART_MAIN);
|
||||
lv_obj_set_style_radius(ctx->btn_grace, 12, LV_PART_MAIN);
|
||||
lv_obj_set_style_bg_color(ctx->btn_grace, lv_color_hex(0x1C1B22), LV_PART_MAIN);
|
||||
lv_obj_set_style_border_width(ctx->btn_grace, 1, LV_PART_MAIN);
|
||||
lv_obj_set_style_border_color(ctx->btn_grace, lv_color_hex(0x2D2B36), LV_PART_MAIN);
|
||||
lv_obj_add_event_cb(ctx->btn_grace, speaker_event_cb, LV_EVENT_CLICKED, ctx);
|
||||
|
||||
lv_obj_t* icon_grace = lv_image_create(ctx->btn_grace);
|
||||
lv_obj_set_size(icon_grace, 80, 80);
|
||||
char path_grace[256] = "A:";
|
||||
size_t sz_grace = sizeof(path_grace) - 2;
|
||||
tt_app_get_assets_child_path(ctx->app, "girl.png", path_grace + 2, &sz_grace);
|
||||
lv_image_set_src(icon_grace, path_grace);
|
||||
|
||||
lv_obj_t* lbl_grace = lv_label_create(ctx->btn_grace);
|
||||
lv_label_set_text(lbl_grace, "Grace");
|
||||
lv_obj_set_style_text_color(lbl_grace, lv_color_hex(0xFFFFFF), LV_PART_MAIN);
|
||||
|
||||
// Elias Button (Boy)
|
||||
ctx->btn_elias = lv_btn_create(speaker_row);
|
||||
lv_obj_set_size(ctx->btn_elias, lv_pct(46), 130);
|
||||
lv_obj_set_flex_flow(ctx->btn_elias, LV_FLEX_FLOW_COLUMN);
|
||||
lv_obj_set_flex_align(ctx->btn_elias, LV_FLEX_ALIGN_CENTER, LV_FLEX_ALIGN_CENTER, LV_FLEX_ALIGN_CENTER);
|
||||
lv_obj_set_style_pad_all(ctx->btn_elias, 8, LV_PART_MAIN);
|
||||
lv_obj_set_style_pad_row(ctx->btn_elias, 8, LV_PART_MAIN);
|
||||
lv_obj_set_style_radius(ctx->btn_elias, 12, LV_PART_MAIN);
|
||||
lv_obj_set_style_bg_color(ctx->btn_elias, lv_color_hex(0x1C1B22), LV_PART_MAIN);
|
||||
lv_obj_set_style_border_width(ctx->btn_elias, 1, LV_PART_MAIN);
|
||||
lv_obj_set_style_border_color(ctx->btn_elias, lv_color_hex(0x2D2B36), LV_PART_MAIN);
|
||||
lv_obj_add_event_cb(ctx->btn_elias, speaker_event_cb, LV_EVENT_CLICKED, ctx);
|
||||
|
||||
lv_obj_t* icon_elias = lv_image_create(ctx->btn_elias);
|
||||
lv_obj_set_size(icon_elias, 80, 80);
|
||||
char path_elias[256] = "A:";
|
||||
size_t sz_elias = sizeof(path_elias) - 2;
|
||||
tt_app_get_assets_child_path(ctx->app, "boy.png", path_elias + 2, &sz_elias);
|
||||
lv_image_set_src(icon_elias, path_elias);
|
||||
|
||||
lv_obj_t* lbl_elias = lv_label_create(ctx->btn_elias);
|
||||
lv_label_set_text(lbl_elias, "Elias");
|
||||
lv_obj_set_style_text_color(lbl_elias, lv_color_hex(0xFFFFFF), LV_PART_MAIN);
|
||||
|
||||
// 2. Main App Container (Usual App Screen)
|
||||
ctx->app_main_cont = lv_obj_create(parent);
|
||||
lv_obj_set_width(ctx->app_main_cont, lv_pct(95));
|
||||
lv_obj_set_flex_grow(ctx->app_main_cont, 1);
|
||||
lv_obj_set_flex_flow(ctx->app_main_cont, LV_FLEX_FLOW_COLUMN);
|
||||
lv_obj_set_flex_align(ctx->app_main_cont, LV_FLEX_ALIGN_START, LV_FLEX_ALIGN_CENTER, LV_FLEX_ALIGN_CENTER);
|
||||
lv_obj_set_style_pad_all(ctx->app_main_cont, 0, LV_PART_MAIN);
|
||||
lv_obj_set_style_pad_row(ctx->app_main_cont, 12, LV_PART_MAIN);
|
||||
lv_obj_set_style_bg_opa(ctx->app_main_cont, 0, LV_PART_MAIN);
|
||||
lv_obj_set_style_border_width(ctx->app_main_cont, 0, LV_PART_MAIN);
|
||||
lv_obj_add_flag(ctx->app_main_cont, LV_OBJ_FLAG_HIDDEN); // Hidden by default
|
||||
|
||||
// Conversation bubble container
|
||||
lv_obj_t* conv_card = lv_obj_create(ctx->app_main_cont);
|
||||
lv_obj_set_width(conv_card, lv_pct(100));
|
||||
lv_obj_set_flex_grow(conv_card, 1);
|
||||
lv_obj_set_flex_flow(conv_card, LV_FLEX_FLOW_COLUMN);
|
||||
lv_obj_set_flex_align(conv_card, LV_FLEX_ALIGN_START, LV_FLEX_ALIGN_START, LV_FLEX_ALIGN_START);
|
||||
lv_obj_set_style_pad_all(conv_card, 12, LV_PART_MAIN);
|
||||
lv_obj_set_style_pad_row(conv_card, 10, LV_PART_MAIN);
|
||||
lv_obj_set_style_bg_color(conv_card, lv_color_hex(0x13121A), LV_PART_MAIN);
|
||||
lv_obj_set_style_border_color(conv_card, lv_color_hex(0x23212C), LV_PART_MAIN);
|
||||
lv_obj_set_style_border_width(conv_card, 1, LV_PART_MAIN);
|
||||
lv_obj_set_style_radius(conv_card, 12, LV_PART_MAIN);
|
||||
|
||||
// User bubble
|
||||
lv_obj_t* user_label_title = lv_label_create(conv_card);
|
||||
lv_label_set_text(user_label_title, "👤 You:");
|
||||
lv_obj_set_style_text_color(user_label_title, lv_color_hex(0x00E676), LV_PART_MAIN);
|
||||
|
||||
ctx->lbl_transcript = lv_label_create(conv_card);
|
||||
lv_label_set_text(ctx->lbl_transcript, "(Your question will appear here)");
|
||||
lv_label_set_long_mode(ctx->lbl_transcript, LV_LABEL_LONG_WRAP);
|
||||
lv_obj_set_width(ctx->lbl_transcript, lv_pct(100));
|
||||
lv_obj_set_style_text_color(ctx->lbl_transcript, lv_color_hex(0xE0E0E0), LV_PART_MAIN);
|
||||
|
||||
// Separator line
|
||||
lv_obj_t* sep = lv_line_create(conv_card);
|
||||
static lv_point_precise_t line_points[] = { {0, 0}, {300, 0} };
|
||||
lv_line_set_points(sep, line_points, 2);
|
||||
lv_obj_set_style_line_color(sep, lv_color_hex(0x2D2B36), LV_PART_MAIN);
|
||||
lv_obj_set_style_line_width(sep, 1, LV_PART_MAIN);
|
||||
lv_obj_set_width(sep, lv_pct(100));
|
||||
|
||||
// Assistant bubble
|
||||
lv_obj_t* bot_label_title = lv_label_create(conv_card);
|
||||
lv_label_set_text(bot_label_title, "🤖 Pipecat:");
|
||||
lv_obj_set_style_text_color(bot_label_title, lv_color_hex(0xAA00FF), LV_PART_MAIN);
|
||||
|
||||
ctx->lbl_response = lv_label_create(conv_card);
|
||||
lv_label_set_text(ctx->lbl_response, "(Answer will appear here)");
|
||||
lv_label_set_long_mode(ctx->lbl_response, LV_LABEL_LONG_WRAP);
|
||||
lv_obj_set_width(ctx->lbl_response, lv_pct(100));
|
||||
lv_obj_set_style_text_color(ctx->lbl_response, lv_color_hex(0xE0E0E0), LV_PART_MAIN);
|
||||
|
||||
// Push-to-Talk Button
|
||||
ctx->btn_ptt = lv_btn_create(ctx->app_main_cont);
|
||||
lv_obj_set_size(ctx->btn_ptt, 180, 50);
|
||||
lv_obj_set_style_radius(ctx->btn_ptt, 25, LV_PART_MAIN);
|
||||
lv_obj_set_style_bg_color(ctx->btn_ptt, lv_color_hex(0x6200EE), LV_PART_MAIN);
|
||||
|
||||
ctx->btn_ptt_label = lv_label_create(ctx->btn_ptt);
|
||||
lv_label_set_text(ctx->btn_ptt_label, LV_SYMBOL_AUDIO " Hold to Talk");
|
||||
lv_obj_center(ctx->btn_ptt_label);
|
||||
|
||||
lv_obj_add_event_cb(ctx->btn_ptt, ptt_event_cb, LV_EVENT_ALL, ctx);
|
||||
|
||||
// Launch main worker task
|
||||
xTaskCreate(pipecatvoice_task, "pipecat_worker", 6144, ctx, 5, &ctx->worker_task);
|
||||
}
|
||||
|
||||
static void on_hide(AppHandle app, void* data) {
|
||||
PipecatVoiceCtx* ctx = (PipecatVoiceCtx*)data;
|
||||
if (ctx == NULL) return;
|
||||
|
||||
ESP_LOGI(TAG, "on_hide: cleaning up");
|
||||
ctx->visible = false;
|
||||
|
||||
if (ctx->ws_fd >= 0) {
|
||||
int fd_to_close = ctx->ws_fd;
|
||||
ctx->ws_fd = -1;
|
||||
ws_close(fd_to_close);
|
||||
}
|
||||
|
||||
// Reset I2S controller to stop DMA and looping noise
|
||||
if (ctx->i2s_dev != NULL) {
|
||||
device_lock(ctx->i2s_dev);
|
||||
i2s_controller_reset(ctx->i2s_dev);
|
||||
device_unlock(ctx->i2s_dev);
|
||||
}
|
||||
|
||||
// Wait briefly for tasks to exit
|
||||
int timeout = 100;
|
||||
while ((ctx->worker_task != NULL || ctx->rx_task != NULL) && timeout > 0) {
|
||||
vTaskDelay(pdMS_TO_TICKS(10));
|
||||
timeout--;
|
||||
}
|
||||
}
|
||||
|
||||
int main(int argc, char* argv[]) {
|
||||
(void)argc;
|
||||
(void)argv;
|
||||
tt_app_register((AppRegistration) {
|
||||
.createData = create_data,
|
||||
.destroyData = destroy_data,
|
||||
.onCreate = on_create,
|
||||
.onDestroy = on_destroy,
|
||||
.onShow = on_show,
|
||||
.onHide = on_hide
|
||||
});
|
||||
return 0;
|
||||
}
|
||||
@@ -1,240 +0,0 @@
|
||||
#include "websocket.h"
|
||||
|
||||
#include <string.h>
|
||||
#include <stdio.h>
|
||||
#include <stdlib.h>
|
||||
#include <unistd.h>
|
||||
|
||||
#include <lwip/sockets.h>
|
||||
#include <lwip/inet.h>
|
||||
#include <esp_log.h>
|
||||
|
||||
static int recv_all(int fd, void* buf, size_t len) {
|
||||
size_t total = 0;
|
||||
char* p = (char*)buf;
|
||||
while (total < len) {
|
||||
int r = lwip_recv(fd, p + total, len - total, 0);
|
||||
if (r <= 0) {
|
||||
return -1;
|
||||
}
|
||||
total += r;
|
||||
}
|
||||
return 0;
|
||||
}
|
||||
|
||||
static uint16_t my_htons(uint16_t val) {
|
||||
return (uint16_t)(((val & 0xff) << 8) | ((val & 0xff00) >> 8));
|
||||
}
|
||||
|
||||
int ws_connect(const char* host, int port, const char* path, const char* device_id, const char* auth_key) {
|
||||
int fd = lwip_socket(AF_INET, SOCK_STREAM, 0);
|
||||
if (fd < 0) return -1;
|
||||
|
||||
struct sockaddr_in addr;
|
||||
memset(&addr, 0, sizeof(addr));
|
||||
addr.sin_family = AF_INET;
|
||||
addr.sin_port = my_htons(port);
|
||||
addr.sin_addr.s_addr = ipaddr_addr(host);
|
||||
|
||||
if (lwip_connect(fd, (struct sockaddr*)&addr, sizeof(addr)) < 0) {
|
||||
close(fd);
|
||||
return -1;
|
||||
}
|
||||
|
||||
// Set socket receive timeout (e.g. 90 seconds) to prevent blocking indefinitely
|
||||
struct timeval tv;
|
||||
tv.tv_sec = 90;
|
||||
tv.tv_usec = 0;
|
||||
lwip_setsockopt(fd, SOL_SOCKET, SO_RCVTIMEO, &tv, sizeof(tv));
|
||||
|
||||
// Send HTTP upgrade handshake request
|
||||
char req[1024];
|
||||
snprintf(req, sizeof(req),
|
||||
"GET %s HTTP/1.1\r\n"
|
||||
"Host: %s:%d\r\n"
|
||||
"Upgrade: websocket\r\n"
|
||||
"Connection: Upgrade\r\n"
|
||||
"Sec-WebSocket-Key: dGhlIHNhbXBsZSBub25jZQ==\r\n"
|
||||
"Sec-WebSocket-Version: 13\r\n"
|
||||
"Authorization: Bearer %s\r\n"
|
||||
"X-Device-ID: %s\r\n"
|
||||
"\r\n",
|
||||
path, host, port, auth_key, device_id);
|
||||
|
||||
if (lwip_send(fd, req, strlen(req), 0) < 0) {
|
||||
close(fd);
|
||||
return -1;
|
||||
}
|
||||
|
||||
// Read HTTP response headers until we hit "\r\n\r\n"
|
||||
char header_buf[1024];
|
||||
size_t header_len = 0;
|
||||
while (header_len < sizeof(header_buf) - 1) {
|
||||
char c;
|
||||
int r = lwip_recv(fd, &c, 1, 0);
|
||||
if (r <= 0) {
|
||||
close(fd);
|
||||
return -1;
|
||||
}
|
||||
header_buf[header_len++] = c;
|
||||
header_buf[header_len] = '\0';
|
||||
|
||||
if (header_len >= 4 && strcmp(header_buf + header_len - 4, "\r\n\r\n") == 0) {
|
||||
break;
|
||||
}
|
||||
}
|
||||
|
||||
// Verify HTTP 101 Switching Protocols response status
|
||||
if (strstr(header_buf, "HTTP/1.1 101") == NULL && strstr(header_buf, "HTTP/1.0 101") == NULL) {
|
||||
close(fd);
|
||||
return -1;
|
||||
}
|
||||
|
||||
return fd;
|
||||
}
|
||||
|
||||
int ws_send(int fd, const uint8_t* data, size_t len, bool binary) {
|
||||
uint8_t header[10];
|
||||
size_t header_len = 0;
|
||||
|
||||
header[0] = binary ? 0x82 : 0x81;
|
||||
|
||||
if (len < 126) {
|
||||
header[1] = 0x80 | (uint8_t)len;
|
||||
header_len = 2;
|
||||
} else {
|
||||
header[1] = 0x80 | 126;
|
||||
header[2] = (uint8_t)((len >> 8) & 0xFF);
|
||||
header[3] = (uint8_t)(len & 0xFF);
|
||||
header_len = 4;
|
||||
}
|
||||
|
||||
// Use fixed client mask for performance: 0x12, 0x34, 0x56, 0x78
|
||||
uint8_t mask[4] = { 0x12, 0x34, 0x56, 0x78 };
|
||||
memcpy(header + header_len, mask, 4);
|
||||
header_len += 4;
|
||||
|
||||
// Send WebSocket frame header
|
||||
int sent = lwip_send(fd, header, header_len, 0);
|
||||
if (sent < 0) return -1;
|
||||
|
||||
// Mask the payload
|
||||
uint8_t* masked = malloc(len);
|
||||
if (masked == NULL) return -1;
|
||||
for (size_t i = 0; i < len; ++i) {
|
||||
masked[i] = data[i] ^ mask[i % 4];
|
||||
}
|
||||
|
||||
// Send masked payload
|
||||
sent = lwip_send(fd, masked, len, 0);
|
||||
free(masked);
|
||||
|
||||
return sent >= 0 ? 0 : -1;
|
||||
}
|
||||
|
||||
int ws_recv(int fd, int* out_opcode, uint8_t* payload, size_t max_len) {
|
||||
uint8_t header[2];
|
||||
if (recv_all(fd, header, 2) < 0) {
|
||||
return -1;
|
||||
}
|
||||
|
||||
int opcode = header[0] & 0x0F;
|
||||
if (out_opcode != NULL) {
|
||||
*out_opcode = opcode;
|
||||
}
|
||||
|
||||
int masked = (header[1] & 0x80) != 0;
|
||||
size_t len = header[1] & 0x7F;
|
||||
|
||||
if (len == 126) {
|
||||
uint8_t ext_len[2];
|
||||
if (recv_all(fd, ext_len, 2) < 0) return -1;
|
||||
len = ((size_t)ext_len[0] << 8) | ext_len[1];
|
||||
} else if (len == 127) {
|
||||
uint8_t ext_len[8];
|
||||
if (recv_all(fd, ext_len, 8) < 0) return -1;
|
||||
// Parse 64-bit length into size_t
|
||||
len = ((size_t)ext_len[4] << 24) | ((size_t)ext_len[5] << 16) | ((size_t)ext_len[6] << 8) | ext_len[7];
|
||||
}
|
||||
|
||||
if (masked) {
|
||||
uint8_t mask[4];
|
||||
if (recv_all(fd, mask, 4) < 0) return -1;
|
||||
|
||||
if (len > max_len) {
|
||||
ESP_LOGE("websocket", "ws_recv overflow (masked): len=%u, max_len=%u", (unsigned)len, (unsigned)max_len);
|
||||
// Buffer overflow, skip payload to align stream
|
||||
size_t to_discard = len;
|
||||
uint8_t discard_buf[256];
|
||||
while (to_discard > 0) {
|
||||
size_t chunk = to_discard < sizeof(discard_buf) ? to_discard : sizeof(discard_buf);
|
||||
if (recv_all(fd, discard_buf, chunk) < 0) return -1;
|
||||
to_discard -= chunk;
|
||||
}
|
||||
return -2;
|
||||
}
|
||||
|
||||
if (recv_all(fd, payload, len) < 0) return -1;
|
||||
for (size_t i = 0; i < len; ++i) {
|
||||
payload[i] ^= mask[i % 4];
|
||||
}
|
||||
} else {
|
||||
if (len > max_len) {
|
||||
ESP_LOGE("websocket", "ws_recv overflow (unmasked): len=%u, max_len=%u", (unsigned)len, (unsigned)max_len);
|
||||
size_t to_discard = len;
|
||||
uint8_t discard_buf[256];
|
||||
while (to_discard > 0) {
|
||||
size_t chunk = to_discard < sizeof(discard_buf) ? to_discard : sizeof(discard_buf);
|
||||
if (recv_all(fd, discard_buf, chunk) < 0) return -1;
|
||||
to_discard -= chunk;
|
||||
}
|
||||
return -2;
|
||||
}
|
||||
|
||||
if (recv_all(fd, payload, len) < 0) return -1;
|
||||
}
|
||||
|
||||
return (int)len;
|
||||
}
|
||||
|
||||
void ws_close(int fd) {
|
||||
if (fd >= 0) {
|
||||
close(fd);
|
||||
}
|
||||
}
|
||||
|
||||
int ws_send_pong(int fd, const uint8_t* payload, size_t len) {
|
||||
uint8_t header[10];
|
||||
size_t header_len = 0;
|
||||
|
||||
header[0] = 0x8A; // FIN | PONG (0x0A)
|
||||
|
||||
if (len < 126) {
|
||||
header[1] = 0x80 | (uint8_t)len;
|
||||
header_len = 2;
|
||||
} else {
|
||||
header[1] = 0x80 | 126;
|
||||
header[2] = (uint8_t)((len >> 8) & 0xFF);
|
||||
header[3] = (uint8_t)(len & 0xFF);
|
||||
header_len = 4;
|
||||
}
|
||||
|
||||
uint8_t mask[4] = { 0x12, 0x34, 0x56, 0x78 };
|
||||
memcpy(header + header_len, mask, 4);
|
||||
header_len += 4;
|
||||
|
||||
int sent = lwip_send(fd, header, header_len, 0);
|
||||
if (sent < 0) return -1;
|
||||
|
||||
if (len > 0 && payload != NULL) {
|
||||
uint8_t* masked = malloc(len);
|
||||
if (masked == NULL) return -1;
|
||||
for (size_t i = 0; i < len; ++i) {
|
||||
masked[i] = payload[i] ^ mask[i % 4];
|
||||
}
|
||||
sent = lwip_send(fd, masked, len, 0);
|
||||
free(masked);
|
||||
}
|
||||
|
||||
return sent >= 0 ? 0 : -1;
|
||||
}
|
||||
@@ -1,59 +0,0 @@
|
||||
#pragma once
|
||||
|
||||
#include <stdint.h>
|
||||
#include <stddef.h>
|
||||
#include <stdbool.h>
|
||||
|
||||
#ifdef __cplusplus
|
||||
extern "C" {
|
||||
#endif
|
||||
|
||||
/**
|
||||
* Connect to a WebSocket server.
|
||||
* @param host Server IP address (e.g. "192.168.68.126")
|
||||
* @param port Port number (e.g. 8642)
|
||||
* @param path WebSocket path (e.g. "/api/esp32/voice/ws")
|
||||
* @param device_id Unique device identifier
|
||||
* @param auth_key Hermes Bearer API key
|
||||
* @return Socket file descriptor on success, or -1 on failure
|
||||
*/
|
||||
int ws_connect(const char* host, int port, const char* path, const char* device_id, const char* auth_key);
|
||||
|
||||
/**
|
||||
* Send a WebSocket frame.
|
||||
* @param fd Socket file descriptor
|
||||
* @param data Data payload to send
|
||||
* @param len Length of the data payload
|
||||
* @param binary True for binary frame, false for text frame
|
||||
* @return 0 on success, or -1 on failure
|
||||
*/
|
||||
int ws_send(int fd, const uint8_t* data, size_t len, bool binary);
|
||||
|
||||
/**
|
||||
* Receive a WebSocket frame.
|
||||
* @param fd Socket file descriptor
|
||||
* @param out_opcode Pointer to store the received opcode (e.g. 0x01 text, 0x02 binary)
|
||||
* @param payload Buffer to store the received payload
|
||||
* @param max_len Maximum length of the payload buffer
|
||||
* @return Received payload length on success, -1 on connection failure, or -2 on buffer overflow
|
||||
*/
|
||||
int ws_recv(int fd, int* out_opcode, uint8_t* payload, size_t max_len);
|
||||
|
||||
/**
|
||||
* Close a WebSocket connection.
|
||||
* @param fd Socket file descriptor
|
||||
*/
|
||||
void ws_close(int fd);
|
||||
|
||||
/**
|
||||
* Send a WebSocket PONG frame.
|
||||
* @param fd Socket file descriptor
|
||||
* @param payload Payload to reflect
|
||||
* @param len Length of payload
|
||||
* @return 0 on success, or -1 on failure
|
||||
*/
|
||||
int ws_send_pong(int fd, const uint8_t* payload, size_t len);
|
||||
|
||||
#ifdef __cplusplus
|
||||
}
|
||||
#endif
|
||||
@@ -1,7 +0,0 @@
|
||||
manifest.version=0.2
|
||||
target.sdk=0.8.0-dev
|
||||
target.platforms=esp32s3
|
||||
app.id=one.tactility.pipecatvoice
|
||||
app.version.name=1.0.0
|
||||
app.version.code=1
|
||||
app.name=Pipecat Voice
|
||||
Reference in New Issue
Block a user