Self-hostable, end-to-end encrypted messaging
  • TypeScript 57.3%
  • Elixir 23%
  • JavaScript 10.1%
  • CSS 6.4%
  • Rust 1.7%
  • Other 1.5%
Find a file
Neon 81694df207
Decouple replies vs threads (#93)
## Summary

This PR fixes the long-standing conflation between **threads** and
**inline replies** in Vesper.

Previously, both behaviors were effectively modeled through the same
overloaded parent relationship:

- **threads** were meant to move side conversations off the main
timeline
- **inline replies** were meant to indicate which specific message a
user was responding to

That overloading led to two broken behaviors over time:

1. clicking **Reply** behaved like **Start Thread**
2. a later patch made thread replies render in the main timeline, which
made threads behave like replies

This PR decouples those concepts in the data model, transport layer,
cache/storage layers, and UI behavior.

## New model

Messages can now independently carry:

- `thread_root_message_id`
  - the message belongs to the thread rooted at this top-level message
- `reply_to_message_id`
  - the message is replying to this specific message

`parent_message_id` is retained as a **legacy compatibility field**
during the migration.

This allows all intended states:

- normal message
- inline reply in main feed
- thread message
- thread message replying to a specific message inside the thread

## User-visible behavior after this PR

- **Inline replies** stay in the main timeline and render a reply
preview
- **Thread messages** stay in the thread panel and do not leak into the
main timeline
- **Threads and replies are independent**
- **Replies inside threads now work properly** without collapsing back
to the thread root
- reply previews render inside thread view as well

## What changed

### Database / server model

- added `messages.thread_root_message_id`
- added `messages.reply_to_message_id`
- added indexes for thread and reply lookups
- kept `parent_message_id` and `is_reply` for compatibility during
transition

### Server behavior

- socket `new_message` handling now resolves thread membership and reply
targeting separately
- thread queries/counts now use `thread_root_message_id` with legacy
fallback
- thread endpoint resolves thread roots using the new model first
- message JSON/sync payloads now include:
  - `thread_root_message_id`
  - `reply_to_message_id`
- room/thread relation projection now tracks thread membership instead
of treating all parents as thread links

### Client behavior

- main feed filtering now excludes thread messages based on thread
membership, not overloaded parent semantics
- thread panel membership uses `thread_root_message_id`
- reply preview rendering uses `reply_to_message_id`
- thread composer can now send:
  - ordinary thread messages
- thread messages that explicitly reply to another message inside the
thread
- optimistic state, renderer state, and hydration all understand the new
fields

### Cache / storage

updated all relevant cache/storage paths to persist the new relations:

- Electron sqlite cache
- preload/main bridge
- renderer store hydration
- SDK crypto storage runtime
- IndexedDB adapter
- file/memory SDK storage adapters

### Backfill tooling

added:

- `mix vesper.backfill_thread_reply_fields`
- `mix vesper.backfill_thread_reply_fields --dry-run`

This backfills historical rows from the legacy parent model:

- legacy inline replies -> `reply_to_message_id`
- legacy thread messages -> `thread_root_message_id`

## Migration / deployment notes

This PR is designed for a staged deployment rather than a forced wipe.

### Safe rollout

1. deploy image with new migration
2. let app auto-run migrations
3. optionally run backfill:

```bash
cd server
MIX_ENV=prod mix vesper.backfill_thread_reply_fields
```

Dry-run:

```bash
cd server
MIX_ENV=prod mix vesper.backfill_thread_reply_fields --dry-run
```

The app still dual-reads legacy fields during the transition, so
immediate backfill is not required for correctness, but it is
recommended.

## Testing

### Passed

- `cd client && npm run typecheck`
- targeted server test in Docker:
  - `mix test test/vesper/chat/thread_reply_separation_test.exs`

### Notes

- `npm run check:web` still fails on a pre-existing web build issue
unrelated to this PR:
  - unresolved `slate-react` import in the web build path

## Added test coverage

- server test covering:
  - inline replies excluded from thread lists/counts
- thread-internal replies with separate reply targets still remain in
the thread

## Why this approach

This PR moves Vesper away from an implementation-led model where one
parent pointer tried to mean two different things.

Threads and replies are different user intents:

- **thread** = conversation partitioning
- **reply** = message targeting

Treating them as separate first-class relations makes the behavior
clearer, the UI more predictable, and future work like notifications,
search, thread unread counts, and jump-to-message behavior much easier
to reason about.

---------

Co-authored-by: Monika <monika@neosynth.net>
2026-03-30 23:46:42 +02:00
.githooks chore: enforce web build checks before push 2026-03-14 21:05:51 -07:00
.github ci: pre-pull Postgres image before test runs (#70) 2026-03-21 18:26:46 +01:00
client Decouple replies vs threads (#93) 2026-03-30 23:46:42 +02:00
config feat: polish chat composer and markdown rendering (#43) 2026-03-17 18:29:25 -07:00
doc Decouple replies vs threads (#93) 2026-03-30 23:46:42 +02:00
landing feat: ship ui overhaul and e2ee-safe search sync 2026-03-12 14:15:14 -07:00
prototype OpenMLS Migration (#81) 2026-03-25 09:45:06 -07:00
scripts ci: overhaul change detection, add SDK/E2E/Docker test workflows (#59) 2026-03-20 23:36:24 +01:00
sdk Decouple replies vs threads (#93) 2026-03-30 23:46:42 +02:00
server Decouple replies vs threads (#93) 2026-03-30 23:46:42 +02:00
.dockerignore fix: include SDK in Docker web build context (#46) 2026-03-20 09:48:18 -07:00
.env.example feat: move client logic into sdk (#45) 2026-03-20 09:29:04 -07:00
.gitignore feat: self-hosted twemoji, message load perf, and cache headers (#89) 2026-03-27 16:43:35 -07:00
CONTRIBUTING.md test: fix flaky sync event assertions in SDK offline mutation test (#78) 2026-03-21 23:41:39 +01:00
docker-compose.yml feat: web push notifications + video call UI (#85) 2026-03-25 17:03:46 -07:00
flake.lock Add Nix flake for server and web builds 2026-03-14 19:33:05 -07:00
flake.nix Add Nix flake for server and web builds 2026-03-14 19:33:05 -07:00
LICENSE Initial commit — Vesper v0.1.0 2026-03-03 15:58:52 -05:00
package-lock.json Feat/slate composer (#92) 2026-03-27 23:36:23 -07:00
package.json feat: move client logic into sdk (#45) 2026-03-20 09:29:04 -07:00
README.md Decouple replies vs threads (#93) 2026-03-30 23:46:42 +02:00
turnserver.conf Fix TURN auth: remove lt-cred-mech, pass secret via CLI 2026-03-03 17:14:13 -05:00

Vesper

Self-hostable, end-to-end encrypted messaging.

Features

  • End-to-end encryption — MLS protocol (RFC 9420), all encryption/decryption happens client-side
  • Voice calls — WebRTC with SFU architecture, supports DM and channel calls
  • Servers & channels — create communities with text and voice channels
  • Direct messages — private 1-on-1 conversations
  • File sharing — encrypted file uploads with previews
  • Threads & replies — side threads stay off the main timeline, inline replies target specific messages
  • Mentions — @user and @everyone notifications
  • Emoji reactions — react to messages
  • Message pinning — pin important messages in channels
  • Invite links — shareable invite codes for servers
  • Docker deployment — one-command self-hosting with Docker Compose

Running the Server

Pre-built multi-arch images (linux/amd64, linux/arm64) are published to GHCR:

Image Description
ghcr.io/alderban107/vesper-app Phoenix API server
ghcr.io/alderban107/vesper-web Web client (nginx)
  1. Copy the example environment file:

    cp .env.example .env
    
  2. Edit .env and configure — see Environment Variables for the full reference. At minimum, set:

    • SECRET_KEY_BASE — generate with mix phx.gen.secret or openssl rand -base64 48
    • POSTGRES_PASSWORD — database password
    • TURN_PASSWORD — password for the TURN server (voice relay)
  3. Start the stack:

    docker compose pull && docker compose up -d
    

This starts the Phoenix server, PostgreSQL, and a coturn TURN server for voice relay. No source checkout needed — images are pulled from GHCR.

From source

Prerequisites: Elixir 1.15+, PostgreSQL

cd server
mix setup        # install deps, create DB, run migrations
mix phx.server   # start on localhost:4000

Dev database defaults: postgres:postgres@localhost/vesper_dev

To point local Phoenix dev at a custom Postgres instance, set: DEV_DB_HOST, DEV_DB_PORT, DEV_DB_USER, DEV_DB_PASS, and optionally DEV_DB_NAME.

The repo pre-commit hook now defaults its test database settings to the SDK local DB helper (localhost:55432, user/password vesper_sdk) unless TEST_DB_* is already set.

For a shared local source setup, put these in the repo root .env so Phoenix dev, the SDK live tests, and the Playwright harness all point at the same local Postgres instance:

DEV_DB_HOST, DEV_DB_PORT, DEV_DB_USER, DEV_DB_PASS, DEV_DB_NAME, TEST_DB_HOST, TEST_DB_PORT, TEST_DB_USER, TEST_DB_PASS, VESPER_SDK_TEST_DB_HOST, VESPER_SDK_TEST_DB_PORT, VESPER_SDK_TEST_DB_USER, and VESPER_SDK_TEST_DB_PASS.

Downloading the Client

Pre-built releases

Download from Releases — available for Linux (AppImage, deb), macOS (DMG), and Windows (installer, portable).

Web client (Docker)

A browser-based client is available as a Docker image — no download required. Add the web service to your Docker Compose stack:

docker compose up -d web

This serves the web client on port 8080 (configurable via WEB_PORT in .env). Users can access it at http://your-host:8080. The web client has full feature parity with the desktop app, including E2EE messaging, voice calls, and file sharing — all running in the browser via IndexedDB and the Web Notification API.

Build from source

Prerequisites: Node 20+

cd client
npm install
npm run dev          # Electron dev with hot reload
npm run dev:web      # web client dev server
npm run check:web    # typecheck + production web build
npm run build:web    # production web build (outputs dist-web/)
npm run test:e2e:p0  # Playwright smoke run
npm run test:sdk:e2e # SDK live suite
npm run dist:linux   # build AppImage + deb

To run the same verification used by the git pre-commit hook from the repo root:

./scripts/setup-git-hooks.sh
./scripts/pre-commit-checks.sh
./scripts/pre-push-checks.sh

The dev server connects to http://localhost:4000 by default.

Connecting

The client connects to a Vesper server URL. In development, this defaults to localhost:4000. For self-hosted instances, enter your server's URL when registering or logging in.

Tech Stack

Component Technology
Backend Elixir / Phoenix
Frontend Electron + React + TypeScript
Database PostgreSQL
E2EE MLS via OpenMLS (Rust/WASM)
Voice WebRTC via ex_webrtc (SFU)
Auth Argon2 + JWT
State Zustand (client), ETS + PubSub (server)
Styling Tailwind CSS
Jobs Oban

Project Structure

The root package.json defines an npm workspaces monorepo linking client/ and sdk/.

server/                  Elixir/Phoenix backend (API + WebSocket)
  lib/vesper/              domain logic (accounts, chat, encryption)
  lib/vesper_web/          controllers, channels, router
  config/test.exs          test DB config (supports TEST_DB_* env overrides)
  priv/repo/migrations/    database migrations
  test/
    test_helper.exs        ExUnit bootstrap
    support/
      data_case.ex         base test case (Ecto sandbox)
      factory.ex           test data factories
    vesper/chat/           domain-level tests
      message_deletion_test.exs

client/                  Electron + React frontend
  src/main/                Electron main process (encrypted SQLite, IPC)
  src/preload/             context bridge (IPC between main ↔ renderer)
  src/renderer/src/
    sdk/                   renderer bootstrap for the SDK and E2EE test hooks
    stores/                Zustand app state wired to SDK-owned E2EE flows
    components/            React UI components
  e2e/                     Playwright E2E test suite
    tests/                   spec files (p0-smoke, p1-core, p2-edge)
    fixtures/                test data and attachments
    harness/                 test orchestration helpers
    REQUIREMENTS.md          full test plan

sdk/                     TypeScript SDK (@vesper/sdk npm workspace)
  src/                     SDK source (auth, crypto, client, transport, voice)
  test/                    live integration tests (boots Phoenix)
  examples/                example apps (CLI client, bots, OpenTUI)
  scripts/                 local Postgres helper and chaos/load test runners

scripts/                 repo-level tooling
  setup-git-hooks.sh       configure git hooks from .githooks/
  pre-commit-checks.sh     pre-commit gate (server precommit + client web check)
  pre-push-checks.sh       pre-push gate (client web check)
  load-test-env.sh         source repo .env and normalize test DB vars (shell)
  load-repo-env.mjs        same as above for Node (used by SDK test harness)

.github/workflows/
  test-server.yml          server CI — mix test + PostgreSQL 17
  test-client.yml          client CI — typecheck + production build
  docker-server.yml        build & push server Docker image
  docker-web.yml           build & push web client Docker image
  release.yml              build desktop installers
  nightly.yml              daily nightly release (Docker + desktop)
.github/CI.md             CI/CD pipeline documentation

doc/
  DESIGN.md                architecture overview
  PROTOCOL.md              HTTP + WebSocket protocol reference
  E2EE-CORRECTNESS-PLAN.md RCA + remaining durability/scale follow-up
  sdk/                     SDK developer guides (quickstart, auth, messaging, etc.)
  e2ee/                    end-to-end encryption documentation
    README.md                     current E2EE doc index
    E2EE-IMPLEMENTATION.md        developer guide
    MLS-BOOTSTRAP-AND-EVICTION.md large-room topology notes
    SDK-CHAOS-SCALE-PLAN.md       chaos + scale validation plan

docker-compose.yml       full stack (PostgreSQL, Phoenix, web client, coturn)
turnserver.conf          coturn configuration for voice relay

Environment Variables

All variables are set in .env (loaded by Docker Compose) or exported in the shell when running from source. Copy .env.example as a starting point.

Full environment variable reference

Database

Variable Default Required Description
POSTGRES_USER vesper No PostgreSQL username
POSTGRES_PASSWORD Yes PostgreSQL password
POSTGRES_DB vesper_prod No PostgreSQL database name
DATABASE_URL Yes (prod) Full Ecto connection string, e.g. ecto://user:pass@host/db. Only used in production; dev/test use compiled config.
POOL_SIZE 10 No Database connection pool size
ECTO_IPV6 No Set to true or 1 to connect to PostgreSQL over IPv6
DEV_DB_HOST localhost No Phoenix dev PostgreSQL host when running from source
DEV_DB_PORT 5432 No Phoenix dev PostgreSQL port when running from source
DEV_DB_USER postgres No Phoenix dev PostgreSQL username when running from source
DEV_DB_PASS postgres No Phoenix dev PostgreSQL password when running from source
DEV_DB_NAME vesper_dev No Phoenix dev PostgreSQL database name when running from source

Server

Variable Default Required Description
SECRET_KEY_BASE Yes (prod) Secret for signing cookies and tokens. Generate with mix phx.gen.secret or openssl rand -base64 48.
PHX_HOST localhost No Hostname for URL generation (e.g. vesper.yourdomain.com)
APP_PORT 4000 No External port the API server listens on
PHX_SERVER No Set to true to start the HTTP server (set automatically in Docker)
JWT_SECRET same as SECRET_KEY_BASE No Separate secret for JWT signing, if desired
DNS_CLUSTER_QUERY No DNS query for clustering in multi-node deployments

CORS & Origins

Variable Default Required Description
CORS_ORIGIN * (prod) No Allowed origin for CORS and WebSocket connections. Set to your frontend URL in production (e.g. https://app.example.com). Use a comma-separated list for multiple origins. When unset, CORS allows all origins and a warning is logged.

Voice / WebRTC

Variable Default Required Description
TURN_PASSWORD Yes Shared secret for the TURN relay server
TURN_SERVER_URL turn:coturn:3478 No TURN server URL. For proxied web deployments, use turns:your-host:443?transport=tcp.
TURN_USERNAME vesper No TURN username
VOICE_ICE_TRANSPORT_POLICY relay if TURN is set, else all No ICE transport policy: all (STUN + TURN) or relay (TURN only)

File Storage

Variable Default Required Description
FILE_EXPIRY_DAYS 30 No Number of days uploaded files are retained before cleanup

Web Client (Docker)

These apply to the web service in Docker Compose.

Variable Default Required Description
API_URL No Full URL to the API server (e.g. https://vesper.yourdomain.com). Injected into the web client at container startup. When empty, the client connects to the same host it's served from.
WEB_PORT 8080 No External port the web client is served on

Development & Testing

These are not needed for production deployments.

Variable Default Description
TEST_DB_HOST localhost PostgreSQL host for test database
TEST_DB_PORT 5432 PostgreSQL port for test database
TEST_DB_USER postgres PostgreSQL user for test database
TEST_DB_PASS postgres PostgreSQL password for test database
VESPER_SDK_TEST_DB_HOST 127.0.0.1 Host used by the SDK local Postgres helper
VESPER_SDK_TEST_DB_PORT 55432 Port used by the SDK local Postgres helper
VESPER_SDK_TEST_DB_USER vesper_sdk User used by the SDK local Postgres helper
VESPER_SDK_TEST_DB_PASS vesper_sdk Password used by the SDK local Postgres helper
VESPER_E2E Set to 1 to run the server in E2E test mode (real connection pool instead of Ecto sandbox)
MIX_TEST_PARTITION Appended to test database name for parallel test runs
ELECTRON_RENDERER_URL Dev server URL for Electron hot reload

File Upload Limits

The maximum upload size is 50 MiB, hardcoded in two places:

Location What it controls
server/lib/vesper_web/endpoint.exPlug.Parsers :length Maximum HTTP request body the server will accept. Requests exceeding this are rejected with a 413 before any application code runs.
server/lib/vesper/chat/file_storage.exmax_upload_size/0 Application-level limit checked by AttachmentController. Returns a descriptive error to the client.

To change the limit, update both values. They must match — if Plug.Parsers is lower than max_upload_size, uploads between the two values will fail silently with a 413 and no CORS headers. The server must be rebuilt after changing either value.

Contributing

See CONTRIBUTING.md for development setup, testing, and code conventions.

License

AGPL-3.0