Lucas RayanandClaude Sonnet 5.5
58d6f8e9e2
README: what the webmail now does, the architecture roles, a short guide with the shortcuts, testing with a real database, and an honest list of what is not finished. New docs/WEBMAIL.md as the reference for each feature and for the colour system. OPERATIONS: sessions and trusted senders. DEPLOYMENT: the two-step release as API calls, and what to expect when upgrading. MAILBOX-CORE: the conversation invariants and migrations 0015 to 0018. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
90 lines
3.9 KiB
Markdown
90 lines
3.9 KiB
Markdown
# Mailbox core
|
|
|
|
The webmail, IMAP, POP3 and SMTP are four ways in to one mailbox. They share
|
|
the rules and the data code; none of them has its own copy.
|
|
|
|
## Where each rule lives
|
|
|
|
| Rule | One place |
|
|
|------|-----------|
|
|
| Flags, expunge, move, copy, folder delete, counters, quota | `MailStore` (`services/protocols/internal/infrastructure/postgres/mail_store.go`) |
|
|
| Folder and message reads | `mail_store_read.go` (same store) |
|
|
| Credentials (password, app password, scopes) | `usecase/credentials.go` |
|
|
| Sending, and the copy filed in Sent | `ProcessOutboundEmail.Submit` (`FileSentCopy`) |
|
|
| Default folders of a mailbox | SQL function `provision_default_folders` (migration 0014) |
|
|
| Which conversation a message joins | `resolveThread`, called by `persistOne` (`conversation_thread.go`), at delivery |
|
|
| Conversation lists, answered state | `conversations.go` (read side only) |
|
|
| Whether a trusted sender's mail skips Junk | `usecase/trusted_delivery.go`; the list is `trusted_senders` |
|
|
|
|
The IMAP repositories, the webmail repository and the TS auth service are thin
|
|
adapters: they translate their protocol and call the code above.
|
|
|
|
## Invariants
|
|
|
|
After any operation, through any port:
|
|
|
|
- `folders.total_count` and `unread_count` equal the live messages and the ones
|
|
without `\Seen`.
|
|
- `folders.highest_modseq` rises only when a message really changed.
|
|
- `mailboxes.used_bytes` equals the size of the live messages.
|
|
- Deleting twice, or from two ports at once, counts once.
|
|
|
|
`mail_state_contract_test.go` checks all of this after every operation through
|
|
the IMAP path and the webmail path, and under concurrency.
|
|
`e2e/one_mailbox_everywhere_e2e_test.go` does the same over real SMTP, IMAP and
|
|
HTTP on one mailbox.
|
|
|
|
## Lock order
|
|
|
|
Every operation locks **folders first, lowest id first, then the message**
|
|
(`lockFolders`). Two moves in opposite directions, or a move racing a flag
|
|
change, used to deadlock (SQLSTATE 40P01) because they locked in different
|
|
orders. `TestContract_OppositeMovesAndFlagChangesNeverDeadlock` covers it. A new
|
|
operation must follow the same order.
|
|
|
|
## Adding an operation
|
|
|
|
1. Put it in `MailStore`, in one transaction, with `lockFolders`.
|
|
2. Make the adapters call it.
|
|
3. Add it to the contract test and run it through both paths.
|
|
|
|
## Conversations
|
|
|
|
Every message filed through `persistOne` - delivery, the Sent copy, a draft -
|
|
gets a `thread_id` and an `in_reply_to`. The order that decides it is in
|
|
[docs/WEBMAIL.md](WEBMAIL.md#conversations). The invariants:
|
|
|
|
- `thread_id` NULL means "a conversation of its own" and is read as
|
|
`COALESCE(thread_id, id)`. Nothing may assume it is set.
|
|
- A **copy** keeps its source's conversation (`copyOne` copies
|
|
`COALESCE(thread_id, id)`), or the same message in two folders would be two
|
|
conversations.
|
|
- Delivery computes the subject key in Go and does **not** call the SQL helpers
|
|
from migration 0017: the end-to-end tests apply only migration 0001, and
|
|
delivery must work on a schema that has not been migrated yet.
|
|
`TestGoAndSQLAgreeOnTheSubjectKey` keeps the Go and SQL spellings equal.
|
|
- Reading a conversation is scoped by mailbox in the SQL, and another mailbox's
|
|
conversation id answers "not found" exactly like one that does not exist.
|
|
|
|
`conversation_thread_test.go` runs each of these against Postgres.
|
|
|
|
## Migration 0014
|
|
|
|
Creates `provision_default_folders`, creates the missing default folders of
|
|
existing mailboxes, and recomputes folder counters and `used_bytes` from the
|
|
messages. It is idempotent and runs with the other migrations at deploy.
|
|
|
|
## Migrations after 0014
|
|
|
|
| | |
|
|
| :--- | :--- |
|
|
| 0015 | BIMI certificate |
|
|
| 0016 | `trusted_senders` |
|
|
| 0017 | Conversation grouping: `lm_subject_key`, `lm_has_reply_prefix`, a one-time backfill by subject, and the index the list uses |
|
|
| 0018 | `mailboxes.conversation_view` |
|
|
|
|
## Not done
|
|
|
|
IMAP `APPEND` is still a stub. When it is implemented, drop a client-appended
|
|
Sent copy that matches the server's one by `Message-ID`.
|