- Implemented tests for ingesting TLS RPT reports from emails and ensuring they are stored correctly in the database.
- Added tests to verify that the same report can be ingested multiple times without duplication.
- Created tests for backfilling reports that were previously delivered to the inbox.
- Developed WebSocket event handling tests to ensure events are correctly routed to subscribers based on mailbox ownership.
- Included tests for the mail API to validate session handling, mailbox isolation, and error mapping.
- Ensured that the API rejects unauthorized requests and handles malformed JSON gracefully.
- Added `MoveToTrash` method in `WebmailRepository` to move messages to the Trash folder instead of deleting them permanently.
- Introduced `handleDelete` method in `mailAPI` to handle message deletion requests from the client.
- Updated router to include a new route for message deletion.
- Enhanced the compose page to manage draft integrity, ensuring drafts are correctly associated with sent messages.
- Implemented UI changes to allow users to delete messages from the mail folder view.
- Added tests to verify the correct behavior of message deletion and draft handling.
- Updated database migration scripts to reconcile folder counters and ensure accurate unread counts.
- Added tests for TLS-RPT report parsing and webmail repository functionalities.
Three things the console and the mail client were getting wrong.
**Sent stayed empty and drafts would not save.** Both write through the same
persistence path, and both failed on the same constraint: spf_result,
dkim_result and dmarc_result are restricted to the RFC verdict vocabulary and
are nullable, so "not evaluated" has to be NULL. The compose path left them at
Go's zero value, which sends an empty string - satisfying neither the CHECK nor
the meaning - and the insert was rejected. Filing the Sent copy is deliberately
non-fatal, so it failed silently; saving a draft reported the error. Inbound
mail was unaffected because it always supplies a real verdict, which is also
why the existing test never caught it.
**DNS reconciliation could not verify anything.** The console posted to the
auth service, which has no resolver, so the endpoint recorded the request and
re-read the status already in the database. The machinery to do it properly
already existed - the background reconciler resolves every expected record
through public resolvers - it just was not reachable from the console.
Adds GET /api/v1/admin/dns/verify to the protocols service, which owns both the
resolver and the record spec, and wires the domains screen to it: every
expected record, its expected value, and whether it actually resolves. A
proxied record is reported as present rather than matching, because behind a
proxy the published answer is the provider's address and comparing values would
always disagree. Reconciliation still runs afterwards so the stored status and
the audit trail stay in step.
The spec is read, never provisioned: opening a screen must not create DKIM keys
as a side effect.
Verified: the persistence test fails without the fix with the exact constraint
violation (email_messages_dkim_result_check), 22 Go packages and 36 webmail
tests green.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The autosave I had just added appended a new message on every debounce, so a
few minutes of typing would have left the Drafts folder holding a dozen copies
of the same unfinished mail. Caught before deploying.
The composer now returns the stored UID and sends it back as replace_uid on
the next save; the use case expunges the superseded draft, but only after the
replacement has committed, so a failure leaves the older draft in place rather
than losing both. Expunge is a soft delete matching the expunged_at column the
schema already uses, and it locates the message by mailbox as well as UID so a
caller cannot delete someone else's mail by guessing a number.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Four faults, all in the path between the compose screen and the mailbox.
**Every message sent from the webmail had an empty body.** The HTTP handler
read a field named "body"; the composer has always sent "html". The two never
matched, so the payload handed to the queue contained headers and nothing
else. The handler now reads html and text, and still accepts body so an older
client is not broken. Sending was otherwise working, which is why this went
unnoticed: the mail left, it just said nothing.
**Bcc was discarded.** The field was absent from the request struct and from
ComposeInput, so a blind copy typed into the webmail reached nobody. It now
joins the envelope recipients and is deliberately kept out of the headers,
which is what makes it blind.
**HTML was labelled text/plain.** Even once the body was carried, the composer
produces markup and the message declared text/plain, so a recipient would see
raw tags. Messages with HTML now go out as multipart/alternative with a
plain-text part derived from the markup and ordered first per RFC 2046 5.1.4.
An empty or missing text alternative is itself a spam signal.
**Sent mail left no trace.** Nothing wrote to the Sent folder, so a delivered
message vanished from the sender's view entirely. A copy is now filed after
the message is durably queued - not before, or a send refused by the rate
limit would still leave a copy - reusing the inbound persistence path so it
gets a real IMAP UID and appears over IMAP too. A failure to file is logged
and swallowed: the mail is already on its way, and reporting failure would
invite the user to send it twice.
**Drafts were never saved.** There was no endpoint at all. The composer showed
"draft saved automatically" from a setTimeout with no request behind it, so
closing the tab lost the message the screen had just promised was safe. Adds
POST /api/v1/mail/draft, storing the draft as a whole RFC 5322 message in the
Drafts folder, and debounces the composer onto it.
The composer also navigated to Sent whether or not the send succeeded, which
is how a rejected message disappears silently. It now reports the error and
stays put.
**Delay notifications were being filed as spam.** A DSN this server writes is
injected through the same inbound path as mail from the internet, but without
a client IP - so SPF can only be "none" - and its bounce-shaped content scores
highly. The warning telling you your mail was delayed landed in your own Junk
folder. Messages the server generates itself now skip content scanning; the
exemption is narrow, and a test covers that ordinary mail is still scanned so
this cannot become a way past the filter. The DSN also declared
Auto-Submitted: auto-replied, which belongs to vacation responders; a delivery
report is auto-generated (RFC 3834 section 5), and filters read that header.
Not fixed here, because it is not a code fault: outbound port 25 is blocked by
the provider, which is what the delay notification was reporting
("i/o timeout" to Google's MX - a timeout, not a refusal, is the signature of
a block). Receiving is unaffected and queued mail keeps retrying. Documented
with the two ways out, including the relay path, which is verified working
code rather than another stub.
Verified: 22 Go packages pass, 5 new compose tests and 3 for the DSN handling.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
DKIM rotation broke the domain it was run on. It retired the live key and
inserted a placeholder as ACTIVE - private_key_enc was the literal string
"dummy-encrypted-key" - so FindActiveKeys then failed for the whole
domain and outbound mail stopped being signed, with the real key already
gone. Reproduced against a database before changing anything.
The root cause runs deeper: the auth service derives its encryption key
with a plain SHA-256 of the master key while the protocols service uses
HKDF with its own info string, so nothing the auth service seals can ever
be opened by the signer. Rotation therefore moves to the protocols
service, which owns that vault and the real key generator. It retires the
previous key with a seven-day overlap in the same transaction rather than
deleting it, because mail already sent under it is still being verified.
Domain onboarding had three defects in one function. It built the alias
destination by interpolating the domain name into the SQL text, which was
injectable; it reused one placeholder for a CITEXT and a VARCHAR column,
which made the statement fail and happened to hide the injection; and it
ignored its scope argument entirely, so any DOMAIN_ADMIN could create
domains. It also provisioned postmaster, abuse and security, missing the
dmarc and tlsrpt aliases that the published records point at - PLAN.md
section 7.4b is explicit that reports bounce without them.
The TLS panel returned fixed values: acme_status "VALID" and 72 days to
expiry, whatever the process actually held. It now reads the certificate
watcher, and a compile-time assertion keeps the wiring from silently
regressing into "disabled". The audit log and the readiness scorecard were
fixtures too, and their field names did not match the API they now read.
Two tests failed and a third asserted a fabricated constant: one depended
on test order through a shared fixture that had a second factor enrolled,
one used a fixed domain name and so passed once per database, and one
expected "records: 13" from a function that never touches DNS. All three
are fixed, and the suite now passes twice in a row against one database.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
F4 lists real-time updates and the interface had none: an open inbox only
changed when the user reloaded it.
Events come from the transactional outbox rather than from an in-process
callback, which is what the outbox is for (PLAN.md section 9.4): the row
is written in the same transaction as the message, so an event can never
describe a delivery that rolled back, and a crash between commit and
publish loses nothing because the row is still there on restart. The relay
claims rows with SKIP LOCKED so a second instance can help without
publishing anything twice.
The hub fans events out per mailbox, keyed by the mailbox the session
resolved to, so one account's arrivals are never visible to another even
though they share the process. A subscriber that stops reading is dropped
rather than allowed to stall the relay; it reconciles on reconnect.
The client reconnects with capped exponential backoff and treats every
successful connect as a resync, refetching the folder. Without that, a
socket that dropped for a minute would leave the list quietly stale, which
is worse than having no real-time updates at all.
The upgrade goes through a Next rewrite, not the API route handler: a
route handler sees a finished request and cannot upgrade a connection.
Verified against the running stack - the handshake returns 101 through the
Next server, and a message delivered on port 25 arrives as a frame on an
already-open socket without the page asking for it.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The mail screens had no backend at all: they rendered a hardcoded array,
and the storage service that ADR-001 assigned this to is still a health
stub. The API lands here instead, because the folder, UID, MODSEQ, flag
and blob logic already exists and is tested in this service for IMAP.
Reimplementing it in TypeScript would mean two implementations of the
same semantics and two chances to disagree about what a UID means.
Reads use a projected read model rather than hydrated aggregates, which
is the CQRS exception PLAN.md section 2 carves out for webmail queries.
Every query is scoped to the mailbox resolved from the verified session,
never to an identifier taken from the request, so one account cannot read
another's mail by guessing a UID - covered by a test that tries exactly
that and expects 404.
Sending reuses ProcessOutboundEmailUseCase, so mail written in the browser
gets the same treatment as mail submitted over 587: send limits, DKIM
signing, and the durable queue. The From address comes from the session
and never from the request body; letting the body choose it is how a
webmail becomes a relay for spoofed mail.
MIME is parsed server-side because this service already depends on a MIME
library. A message that will not parse still returns its raw text - an
unreadable message beats an inbox that refuses to open one.
Marking a message read updates the flag and the folder counter in one
transaction, and only on a real transition, so re-opening a message cannot
drive unread_count negative.
Verified against the running stack: a message delivered over port 25
appears in the list with its decoded subject and sender, opens, marks
itself read, is found by search, and a message composed in the webmail
reaches outbound_jobs with the session's address as the envelope sender.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Two problems the mail screens exposed, both in the delivery path.
The subject, snippet, sender name and message-id columns have existed
since the first migration, but nothing ever wrote them: every row went in
with those fields NULL. A message list built on the database would have
shown blank subjects for every message. Delivery now parses them once per
message and stores them, decoding RFC 2047 encoded words so an accented
subject is readable rather than raw. Parsing failures are non-fatal: a
message with unreadable headers is still delivered, just with less
metadata, because refusing it would lose mail over a malformed Subject.
Worse, a message the spam filter routed to Junk failed delivery outright
when the mailbox had no Junk folder - the sender got 451 and retried
forever against a mailbox that would never grow one on its own. Delivery
now falls back to INBOX and says so in the log. Found by sending a real
message through port 25 against the running stack.
The SMTP handler also swallowed the cause of any unexpected failure: the
peer saw "temporary error" and the operator got nothing. It is logged now,
which is how the missing folder was diagnosed at all.
Also adds the JWT verifier the mail API needs. The tokens are minted by
the auth service and verified here under the same secret, so the test
fixtures are real tokens produced by that service: if the two
implementations drift, this fails before the webmail does. Challenge
tokens, wrong secrets, tampered payloads, expired tokens and the other
surface's audience are all rejected.
The dev seed gained the standard folder set, which is what a provisioned
mailbox has and what the fallback above should not have to compensate for.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Every fix here is a case where the code and the plan disagreed, and the
code was wrong.
Certificates (section 8.2, 8.4). The parser documented that it skipped
expired entries but never checked, so an expired certificate would be
served until every handshake failed. LastReload() only advanced when
acme.json changed - and it does not change for sixty days - which made
the timestamp risk R2 monitors indistinguishable from a watcher gone
blind. Expiry is now checked on every pass, LastReload records the check
and LastChange the swap, and the <7d/<24h alerts of section 8.4 exist.
ARC (section 5). buildArcSealer resolved the DKIM key at startup, before
runDnsReconciler had provisioned one. On a first boot ARC was disabled
until someone restarted the process, which the log even admitted while
offering no way for it to start. A LazySealer binds on first use instead.
Health (section 8.4). There was no degraded state at all. /health now
reports one - still 200, because restarting cannot fix a certificate
Traefik never issued.
Bounces. DSNs were sent From postmaster@lambdamail.local, a domain that
does not resolve, so the notification was itself undeliverable. They now
carry the configured mail host, a Message-ID, and Auto-Submitted so a
vacation responder cannot loop against them.
Fan-out atomicity (section 6.1). Persist ran one transaction per
recipient, so a failure halfway through an alias fan-out left earlier
recipients with a copy while the sender was told to retry - and the retry
delivered to them twice. PersistAll commits the whole delivery or none of
it. Verified against Postgres by reverting the repository to the old
behaviour and confirming the new test fails on it.
SMTP replies. The use case encoded replies in error text ("550 5.7.1 ...")
which the adapter re-parsed by prefix, so rewording a message could turn a
permanent refusal into a retry. SmtpRejection carries the decision and the
adapter renders it.
Repository tests. These only run with TEST_DATABASE_URL, which CI does
set - so the job was red: copy and expunge seeded a 68-character digest
into a CHAR(64) column, and three others used fixed digests that collide
with the UNIQUE index on any re-run. A shared testBlobDigest() helper
fixes both; the suite now passes twice in a row against one database.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Lucas Rayan