lucasrguerra/LambdaMail
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases 1 Wiki Activity

docs: sessions, accounts, trusted senders, conversations, themes and the new reader

Browse Source 
README: what the webmail now does, the architecture roles, a short guide with the
shortcuts, testing with a real database, and an honest list of what is not
finished. New docs/WEBMAIL.md as the reference for each feature and for the
colour system. OPERATIONS: sessions and trusted senders. DEPLOYMENT: the
two-step release as API calls, and what to expect when upgrading. MAILBOX-CORE:
the conversation invariants and migrations 0015 to 0018.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
Lucas RayanandClaude Sonnet 5.5 committed 2026-10-23 06:24:36 -03:00
1 parent c00134f2eb
commit 58d6f8e9e2
5 files changed
+471 -17
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

docs/DEPLOYMENT.md
+49 -1
View File
@@ -218,6 +218,19 @@ IMAGE_TAG=sha-$(git rev-parse HEAD)
# 2. then deploy
```
On Coolify the two steps are two API calls, and the order matters: set the
variable, then deploy, and only after the **Release workflow has finished green**
(a deploy while it is still running pulls the previous images):
```bash
# 1. point the tag at the commit that was released
curl -X PATCH "$COOLIFY_URL/api/v1/applications/$APP_UUID/envs" \
-H "Authorization: Bearer $TOKEN" -H 'Content-Type: application/json' \
-d "{\"key\":\"IMAGE_TAG\",\"value\":\"sha-$(git rev-parse HEAD)\"}"
# 2. then deploy (a POST; a GET answers 405)
curl -X POST "$COOLIFY_URL/api/v1/deploy?uuid=$APP_UUID" -H "Authorization: Bearer $TOKEN"
```
Confirm which one is actually serving before believing a green deployment:
```bash
@@ -225,7 +238,42 @@ curl -s https://<WEBMAIL_HOST>/user/login | grep -o '/_next/static/css/[a-z0-9]*
```
The asset fingerprint changes with every build. If it did not move, the old
image is still running whatever the deployment log said.
image is still running whatever the deployment log said. A route that exists
only in the new version is a surer check: an unauthenticated request to
`/api/v1/mail/threads` answers `401` on a version that has conversations and
`404` on one that does not.
### Upgrading to sessions, accounts, trusted senders and conversations
Three migrations run on the first start (the `migrate` service runs them before
anything that needs them):
| | Adds |
| :--- | :--- |
| `0016` | `trusted_senders` |
| `0017` | The SQL subject helpers and a one-time grouping of older mail into conversations, by subject and only where a reply exists; an index for the conversation list |
| `0018` | `mailboxes.conversation_view` (on by default) |
They are additive and idempotent in effect; on a large mailbox `0017`'s grouping
reads the whole `email_messages` table once. Nothing needs configuring.
What people will notice the first time:
- Sessions opened before the upgrade keep their old 8-hour expiry and are
replaced by 30-day ones at the next sign-in. A session that began before
accounts existed is added to the account switcher automatically.
- Trusted-sender choices a person made in a browser are moved to the server the
first time the webmail loads there.
- Right after a deploy, while the proxy is swapping containers, one page load
can meet a script from the other version. This was seen once, as a React
hydration error in the browser console on the sign-in page, and did not come
back on the next loads. If a page misbehaves in the first minute, reload it.
Nothing about the upgrade needs new environment variables or ports. The webmail
now serves `/manifest.webmanifest`, `/icon.svg`, `/favicon.ico` and
`/icons/*`; they are public, like the sign-in page, and need no proxy rule
beyond the one already sending the host to the web service. Installing the
webmail as an app needs HTTPS, which the deployment already has.
## Troubleshooting
docs/MAILBOX-CORE.md
+32
View File
@@ -12,6 +12,9 @@ the rules and the data code; none of them has its own copy.
| Credentials (password, app password, scopes) | `usecase/credentials.go` |
| Sending, and the copy filed in Sent | `ProcessOutboundEmail.Submit` (`FileSentCopy`) |
| Default folders of a mailbox | SQL function `provision_default_folders` (migration 0014) |
| Which conversation a message joins | `resolveThread`, called by `persistOne` (`conversation_thread.go`), at delivery |
| Conversation lists, answered state | `conversations.go` (read side only) |
| Whether a trusted sender's mail skips Junk | `usecase/trusted_delivery.go`; the list is `trusted_senders` |
The IMAP repositories, the webmail repository and the TS auth service are thin
adapters: they translate their protocol and call the code above.
@@ -45,12 +48,41 @@ operation must follow the same order.
2. Make the adapters call it.
3. Add it to the contract test and run it through both paths.
## Conversations
Every message filed through `persistOne` - delivery, the Sent copy, a draft -
gets a `thread_id` and an `in_reply_to`. The order that decides it is in
[docs/WEBMAIL.md](WEBMAIL.md#conversations). The invariants:
- `thread_id` NULL means "a conversation of its own" and is read as
`COALESCE(thread_id, id)`. Nothing may assume it is set.
- A **copy** keeps its source's conversation (`copyOne` copies
`COALESCE(thread_id, id)`), or the same message in two folders would be two
conversations.
- Delivery computes the subject key in Go and does **not** call the SQL helpers
from migration 0017: the end-to-end tests apply only migration 0001, and
delivery must work on a schema that has not been migrated yet.
`TestGoAndSQLAgreeOnTheSubjectKey` keeps the Go and SQL spellings equal.
- Reading a conversation is scoped by mailbox in the SQL, and another mailbox's
conversation id answers "not found" exactly like one that does not exist.
`conversation_thread_test.go` runs each of these against Postgres.
## Migration 0014
Creates `provision_default_folders`, creates the missing default folders of
existing mailboxes, and recomputes folder counters and `used_bytes` from the
messages. It is idempotent and runs with the other migrations at deploy.
## Migrations after 0014
| | |
| :--- | :--- |
| 0015 | BIMI certificate |
| 0016 | `trusted_senders` |
| 0017 | Conversation grouping: `lm_subject_key`, `lm_has_reply_prefix`, a one-time backfill by subject, and the index the list uses |
| 0018 | `mailboxes.conversation_view` |
## Not done
IMAP `APPEND` is still a stub. When it is implemented, drop a client-appended
docs/OPERATIONS.md
+50 -4
View File
@@ -132,6 +132,47 @@ DELETE FROM mfa_totp WHERE mailbox_id = (SELECT id FROM mailboxes WHERE email_ad
DELETE FROM mfa_recovery_codes WHERE mailbox_id = (SELECT id FROM mailboxes WHERE email_address = 'them@example.com');
```
### Sessions
A webmail session lasts 30 days and is renewed once it is a day old; the
console's lasts 8 hours and is never renewed. The sessions a person holds are
listed in Settings, where each can be signed out; a password change or reset
revokes every other one, and a reset revokes all of them. To end every session
of an account from the database:
```sql
UPDATE web_sessions SET revoked_at = NOW()
WHERE mailbox_id = (SELECT id FROM mailboxes WHERE email_address = 'user@example.com')
AND revoked_at IS NULL;
```
Renewal retires the old token after a two-minute grace rather than at once
(its `revoked_at` is briefly in the future), so a session list can show two rows
for a moment after a renewal. Rotating `JWT_SECRET` signs everyone out, webmail
and console alike.
Each account signed in on a browser also has a `lm_acct_<mailbox id>` cookie.
They are only the browser's own copy of its sessions: revoking a session on the
server makes that account drop out of the switcher the next time it is listed.
### Trusted senders
`trusted_senders (mailbox_id, address)` holds what each person has trusted. It
is part of the database backup and nothing else needs backing up for it. To see
or remove a mailbox's entries:
```sql
SELECT address, created_at FROM trusted_senders
WHERE mailbox_id = (SELECT id FROM mailboxes WHERE email_address = 'user@example.com');
DELETE FROM trusted_senders WHERE mailbox_id = '<id>' AND address = 'x@example.org';
```
A trusted sender's mail is rescued from Junk only when it proves where it came
from (DMARC passed, or SPF passed for the From domain); see
[docs/WEBMAIL.md](WEBMAIL.md#trusted-senders). If a trusted sender's mail still
lands in Junk, look at the `Authentication-Results` header on it: a message
without that proof is deliberately left in Junk.
## Delivery
The outbound queue lives in Postgres, so it survives a restart of anything. The
@@ -175,10 +216,15 @@ disagree — most often because a record was edited by hand.
Rspamd holds its state in Redis and its thresholds in `system_config`, editable
from the admin console. The defaults reject at 15, tag at 6 and greylist at 4.
A message the filter routes to `Junk` is filed there; one over the reject
threshold is refused during the SMTP session, so the sender is told. Nothing is
accepted and silently discarded — a silent discard loses legitimate mail with
nobody aware of it.
A message the filter routes to `Junk` is filed there, unless the mailbox trusts
the sender and the message proves it came from them (see
[Trusted senders](#trusted-senders)); one over the reject threshold is refused
during the SMTP session, so the sender is told. Nothing is accepted and
silently discarded — a silent discard loses legitimate mail with nobody aware
of it.
"Not spam" in the webmail trusts the sender; it does not train Rspamd. To teach
the filter, use Rspamd's own learning (`rspamc learn_ham`) or its web interface.
## Rotating a DKIM key
docs/WEBMAIL.md
+272
View File
@@ -0,0 +1,272 @@
# The webmail
What the webmail at `/user/*` does, how each part behaves, and where it lives in
the code. The README has the overview; this is the reference.
- [Signing in, and staying signed in](#signing-in-and-staying-signed-in)
- [Several accounts at once](#several-accounts-at-once)
- [Trusted senders](#trusted-senders)
- [Conversations](#conversations)
- [Reading a message](#reading-a-message)
- [Writing a message](#writing-a-message)
- [Appearance, the command menu and shortcuts](#appearance-the-command-menu-and-shortcuts)
- [For developers](#for-developers)
## Signing in, and staying signed in
A webmail session lasts **30 days and is renewed while it is used**: once a
session token is a day old, the next request swaps it for a fresh one, so a
mailbox opened every few days never asks for the password again. The admin
console is deliberately different: **8 hours, fixed**, with a step-up second
factor for destructive actions. The two surfaces keep separate cookies
(`lm_user_session`, `lm_admin_session`) and separate token audiences.
The lifetimes are constants in `services/auth/src/accounts.ts`
(`USER_SESSION_TTL_SECONDS`, `ADMIN_SESSION_TTL_SECONDS`, `RENEW_AFTER_SECONDS`).
They are not environment variables on purpose: a lifetime that can be set to
"never" is a setting that will be.
How renewal works, because it explains what you see in the sessions list:
- A token carries its own expiry and cannot be extended, so renewal issues a
**new token and a new row** in `web_sessions`.
- The old token is retired after a **two-minute grace** (`revoked_at` set in the
future), so a request already in flight with it does not come back `401`.
- Only the cookie is renewed. A bearer token has nowhere to receive a new one.
- Signing out, changing the password or resetting it revokes immediately, with
no grace. "Sign out other devices" in Settings revokes every other session.
Sessions that began before 30-day sessions existed keep their 8-hour expiry;
they are replaced by 30-day ones the next time the person signs in.
## Several accounts at once
Each webmail sign-in is also filed under its own cookie,
`lm_acct_<mailbox id>`, next to `lm_user_session`. **`lm_user_session` is always
the active account**, which is why the Go service, the Next middleware and every
route that existed before accounts needed no change.
| | |
| :--- | :--- |
| `GET /api/v1/auth/accounts` | The accounts signed in on this browser whose sessions are still good, and which one is active. A session that predates account cookies is listed too, and filed on the spot. |
| `POST /api/v1/auth/switch` `{ "id" }` | Makes one of them active. Refuses an id that is not signed in on this browser. |
| `POST /api/v1/auth/logout` `{ "all"?: true }` | Signs out the **active** account only; the next one becomes active. `all: true` signs out everything on this browser. |
Add an account from the menu at the foot of the sidebar ("Add another
account"): it opens the normal sign-in, and the new account joins the others.
Switching reloads the page, so nothing from the previous account stays in
memory. Each account adds about half a kilobyte of cookie; a handful is fine,
and a reverse proxy with a very small header limit is the only thing that would
notice more.
## Trusted senders
One decision, two effects. Trusting a sender means:
1. **Their mail is not filed as spam.** If Rspamd's verdict is Junk, delivery
files the message in the inbox instead.
2. **Their remote images load without asking.**
The list is kept **on the server**, per mailbox, in `trusted_senders`
(migration 0016): it follows the account to every browser. What an earlier
version remembered in one browser's storage is moved up the first time the
webmail loads, and then cleared.
How to use it: the shield pill beside the sender's name trusts or untrusts, and
in the Spam folder a banner offers **Not spam**, which moves the message to the
inbox and trusts the sender in one step.
**The rescue has a condition, and it is the point.** A From header is the
easiest field to forge, so trusting an address would otherwise hand a spammer a
way in by writing a contact's address on their mail. Delivery only rescues a
message when it proves where it came from:
- DMARC passed, **or**
- SPF passed for the very domain written in From (the envelope sender's domain
equals the From domain).
Anything else stays in Junk. Also:
- Only the **Junk** verdict is rescued. A message the filter *rejects*, a virus
and a greylist deferral are not affected.
- The mailbox's own Sieve rules run after the rescue and have the last word.
- A failed lookup leaves the message in Junk.
- The list holds at most 5000 addresses per mailbox.
- Rspamd itself does **not** learn from "Not spam": the sender stops being
condemned, but similar mail from strangers is still judged as before.
API: `GET`, `PUT` and `DELETE /api/v1/user/trusted-senders`.
Code: `services/protocols/internal/application/usecase/trusted_delivery.go`.
## Conversations
On by default; one switch in **Settings → Language** turns it off for the account
(`conversation_view` in preferences).
The list shows **one row per conversation**: who wrote, in order and with "me"
for you, the newest message's subject and preview, the number of messages, an
unread dot, and a reply arrow when you have answered. Opening one stacks its
messages; the newest and any unread ones start open, the rest are a line each.
The reply is written **under the conversation, in the same pane**, so the
messages stay in view.
**How messages are grouped**, in this order, at delivery (`resolveThread` in
`conversation_thread.go`):
1. A message already in the mailbox with the same `Message-ID` (your Sent copy
and a copy delivered back to you are one conversation).
2. The message it answers, through `In-Reply-To`, then `References` (nearest
ancestor first).
3. A subject beginning with `Re:`, `Fwd:` or a translation (`Res`, `Enc`, `RV`,
`AW`, `WG`, `TR`, `Antw`), when the headers found nothing.
4. Otherwise a conversation of its own.
`thread_id` NULL means "its own conversation" (`COALESCE(thread_id, id)`), so
mail from before threading needs no rewrite and a reply simply adopts it.
Migration 0017 grouped older mail **by subject, only where a reply is in the
mailbox**; two unrelated messages that merely share a subject stay apart. The
subject key exists twice, in Go for delivery and in SQL (`lm_subject_key`) for
the backfill, and a test holds the two equal.
**What counts as "answered":** the last message someone else wrote has the
`\Answered` flag (set when you reply from the webmail), **or** a message of
yours came after it - which is how a reply written in another mail program
shows up, since it leaves no flag.
A conversation seen from one folder includes its messages in the others, but
**not Trash, Junk or Drafts** (a reply still being written, or one deleted, is
not part of the thread you are reading) unless that is the folder you are in.
Delete and Move on a conversation act on its messages that sit in the open
folder.
API: `GET /api/v1/mail/threads?folder=&q=&limit=&offset=` and
`GET /api/v1/mail/thread/{id}?folder=`. A server without conversations answers
`501` and the webmail falls back to the plain list.
## Reading a message
The body is drawn in a **sandboxed frame** (`allow-scripts`, no
`allow-same-origin`): an opaque origin that cannot read a cookie or touch the
page, running only the few lines that report its height, open links in a new
tab and fold quoted history. The HTML is sanitised first (DOMPurify; no script,
form, iframe, `link`, `base`; `<style>` kept but stripped of `@import` and the
legacy script-in-CSS forms).
**Two ways of drawing it**, chosen by `looksDesigned` in `src/lib/emailBody.ts`:
| | Looks like | Drawn |
| :--- | :--- | :--- |
| **Written** | A letter, a reply, a notice: no coloured backgrounds, no `bgcolor`, fewer than three tables. Plain white is not design (Outlook paints it everywhere). | In the app's own colours, on the page: light text on dark, dark on light, links in the accent. Text too dark for a dark ground is lifted and white backdrops are cleared. "View as sent" shows the original. |
| **Designed** | A newsletter, a receipt: any coloured or image background, or a layout of tables. | On the white sheet it was drawn for, with rounded corners and a hairline. |
A reply's quoted history is folded behind a **···** button, as in Gmail, and
opens in a quieter tone.
**Remote images are blocked by default**: a unique URL per recipient tells the
sender the message was opened, when and from roughly where. A bar offers to load
them for this message; trusting the sender loads them always. Images embedded in
the message (`cid:` parts) are shown, because they make no request.
Attachments are listed with a type tag and downloaded from
`/api/v1/mail/message/{uid}/attachment/{name}`.
## Writing a message
One composer, used twice: the full page at `/user/compose` and the inline reply
under a conversation (`src/components/Composer.tsx`).
- **Drafts autosave** 2.5 seconds after you stop typing and are replaced, not
duplicated. **Discard** removes the stored draft for good (it does not go to
Trash) and stops a save still in flight.
- **Undo send:** after Send the message waits ten seconds; **Undo** cancels it.
Leaving the screen with a send pending sends it once rather than losing it.
- The signature from Settings is inserted into a new message.
- **Attachments are not sent yet.** The picker lists the names and says so; the
server has no upload endpoint.
## Appearance, the command menu and shortcuts
**Two themes, dark by default.** Choose **Dark, Light or System** from the menu
at the foot of the sidebar, in the admin console's sidebar, or on either
sign-in page. The choice is stored **per browser** (a laptop in a bright office
and a phone in bed are different answers). A script in `<head>` applies it
before the first paint, so there is no flash of the wrong theme.
**Command menu: `Ctrl K` / `Cmd K`.** Folders, Compose, Settings, the admin
console, switching accounts, adding one, appearance and sign out, all by typing.
Text that matches no command offers to search the mail for it, so Enter is never
a dead key. It is also reachable from the "Command menu" row in the sidebar.
| Key | |
| :--- | :--- |
| `Ctrl K` / `Cmd K` | Open or close the command menu |
| `c` | Write a new message |
| `/` | Go to the search field |
| `Up` / `Down`, `Enter`, `Esc` | Move in the menu, run, close |
`c` and `/` do nothing while you are typing in a field.
**Install as an app.** The webmail serves a web manifest and icons, so a browser
can install it into its own window (over HTTPS, from the browser's install
control). The tab icon, home-screen icon and install icons are all the same mark.
Fonts (Geist) are **served by the app itself**: opening your mail makes no
request to a font host.
## For developers
### Where things are
| | |
| :--- | :--- |
| Sessions, accounts, trusted-senders API | `services/auth/src/router.ts`, `accounts.ts`, `repository.ts` |
| Conversation reads, answered logic | `services/protocols/.../infrastructure/postgres/conversations.go` |
| Conversation assignment at delivery | `.../postgres/conversation_thread.go` |
| Spam rescue for trusted senders | `.../application/usecase/trusted_delivery.go` |
| Reader, thread reader, composer | `services/webmail/src/components/MessageReader.tsx`, `ThreadReader.tsx`, `Composer.tsx` |
| Message document and mode | `services/webmail/src/lib/emailBody.ts`, `renderMessage.ts`, `sanitizer.ts` |
| Command menu | `services/webmail/src/components/CommandPalette.tsx` |
| Theme | `services/webmail/src/lib/theme.ts`, `src/app/globals.css`, `tailwind.config.js` |
### The colour system
Every colour is a CSS variable defined **once per theme** in `globals.css`
(`[data-theme='dark']` and `[data-theme='light']`). The Tailwind scales the
screens already use - `slate` (neutrals), `indigo` (the accent), `amber`
(warning), `rose` (danger), `emerald` (done, trusted, answered) - read those
variables, so a theme is a change of values and never a change of classes.
The ramps run the same direction in both themes: **100 is the text you read,
950 the ground it sits on, 400 the quieter text, 700 a border.** That is why
`text-slate-400` is correct in light and in dark. Three more tokens:
- `ink` is the colour that sits on the ground (white on dark, near-black on
light): hover tints and hairlines are `ink` at low opacity, never `white/..`.
- `fill` is the solid fill of the one main action on a screen (`bg-fill`).
- `rail` is the sidebar's ground.
To add a colour, add the variable in **both** blocks. `theme-integrity.test.ts`
fails if the two blocks define different variables, if a component tints with
`white/..`, or if the boot script stops defaulting to dark.
### The mark
A lambda built as two square-cut strokes of one weight on a flat ultramarine
tile with continuous corners. The geometry and the PNG/ICO renderer are in
[`scripts/brand/`](../scripts/brand/README.md); `Logo.tsx` carries the same path
inline. Change the number in `mark.py`, never the SVG.
### Adding text
Strings live in `services/webmail/messages/{en,pt-BR,es}.json` and are read with
`t("section.key")`. A test requires `pt-BR.json` and `es.json` to have exactly
the keys of `en.json`, so a string added in one language and forgotten in
another fails the build.
### Testing it
`services/webmail` has no browser test runner; the suites that guard this are
file-level and logic-level (`vitest`), and the screens were checked in a real
browser. To look at the reader or a theme change locally without the mail
services, point `AUTH_SERVICE_URL` and `PROTOCOLS_SERVICE_URL` at a small stub
that serves the JSON shapes above and run `next dev`.