Fixing the recovery-code OOM removed one way to exhaust the container's
memory; it did not remove the reason that was possible. Argon2id is configured
for 64 MiB, and every password hash or verify reserves that for its duration.
Nothing limited how many ran at once, and verification is reached by an
unauthenticated login - so roughly twenty simultaneous sign-in attempts
reserve 1.28 GiB against a 512 MiB container and the process is killed. No
credentials needed, and account lockout does not help: it is per account, and
the requests need not share one.
Hashing now passes through a four-slot gate. Callers queue rather than being
refused, so every request stays correct and only gets slower under load, and
peak memory is four hashes regardless of traffic - about 256 MiB, comfortably
inside the limit.
This is the answer to "why not just remove the memory limits". The limit was
never the fault; unbounded per-request memory was. Removing it would move the
kill decision to the kernel, which picks by size and would take Postgres -
losing queued mail rather than one request.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Confirming a second factor returned "Upstream service is unreachable" - the
Next proxy's message for a backend it cannot reach. The backend was not
unreachable, it was dead: the process had just been OOM-killed.
generateRecoveryCodes hashed all ten codes with Promise.all. Argon2id here is
configured for 64 MiB per hash, so ten concurrently reserve 640 MiB against a
container limited to 512 MiB. Hashing is now sequential, peaking at 64 MiB and
taking a few seconds - the right trade for something that happens once per
enrolment. A timing-based test covers it, against the real hash rather than a
mock, because the cost being guarded is memory held concurrently and only the
real implementation reserves it.
The worse half is what the crash left behind. The enrolment row was marked
CONFIRMED before the codes were generated, so the account came out of this with
a second factor enabled, no recovery codes, and nobody having seen any: locked
behind an authenticator entry with no way back. That is how an account ends up
"assuming" it has a factor its owner cannot use.
The order is now verify, store the codes, then confirm. Anything that fails in
between leaves the enrolment pending and retryable, which is the recoverable
end of that failure.
Both bugs sat behind the one fixed in #28 - the code check failed first, so the
crash was never reached.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
An enrolment can end up confirmed against a secret the authenticator no longer
holds - the account is then locked behind codes nobody can produce, and the
only documented recovery was a DELETE against mfa_totp. That SQL was also
incomplete: it left the recovery codes valid and the sessions live.
reset-mfa removes the authenticator secrets and the recovery codes, and signs
the account's sessions out, because a live session would keep the console open
without the factor that was just removed. The password is untouched, so the
command alone grants nobody access.
The alternative people reach for - delete the mailbox and create it again -
takes its mail, folders and aliases with it, and none of those are the problem.
The docs now say so.
Also corrects the advice printed after create-admin, which still told the
operator to enrol from the webmail settings before the console would open. The
console has enrolled inline since #23.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Enrolling a second factor could reach a state where the authenticator showed
perfectly valid codes and every one of them was rejected, with the only
feedback being "TOTP verification failed".
The browser holds the enrolment secret in component state alone. Reloading the
settings page - or leaving and coming back - brought the "enable" button back,
and pressing it called the enrol endpoint again, which deleted the pending row
and stored a brand new secret. The entry already saved in the authenticator
then belonged to a secret the server no longer had. Nothing about that is
visible from either side: the app generates codes correctly, the server
compares them correctly, and they are for different keys.
Enrolment now resumes. The endpoint returns the pending secret if there is
one, so the QR shown after a reload is the same one already scanned and
rescanning is harmless. Starting over is still possible - ?reset=1, exposed in
the settings screen - because someone who deleted the entry from their app
needs a secret they can actually generate codes for. A pending row that can no
longer be decrypted is replaced rather than resumed, since nobody could ever
confirm it.
Also: the submitted code now has every non-digit stripped rather than just
surrounding whitespace. Authenticators display the code grouped as "123 456",
and a value typed or pasted with that space failed the six-character length
check before it was ever compared - which, to someone reading the right number
off their phone, looks like the server being wrong. The inputs sanitise as you
type for the same reason.
Ruled out first, rather than guessed at: the algorithm passes all six RFC 6238
reference vectors, base32 round-trips, and the deployment's clock is exactly in
sync with real time, so neither the implementation nor drift explains it.
Verified: the resume test fails without the change and passes with it; 47 auth
tests green against a real Postgres.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The interface was reporting things that were not true. Several screens were
mockups wired to nothing, and a reader with the console in Portuguese was
served a mixture of two languages.
Fabricated data, removed:
- The outbound queue was two invented jobs held in component state. Retry and
Freeze only edited that local array, so the screen showed the same two rows
on every deployment and reported success for actions that never happened.
"Freeze" named an operation the backend has never implemented. It now reads
/api/v1/admin/queue, which was already serving the real thing, and offers
retry and cancel, which exist.
- Every dashboard KPI fell back to an invented constant (1420 received, 890
sent, 3 queued, 18.4% disk), so an unreachable API produced a dashboard of
plausible traffic for a server that might have delivered nothing. Two of
those fields were never returned by the endpoint at all, so the fake disk
figure was what everyone always saw. DMARC did the same with 1250/98%/97%.
- The domains screen showed thirteen DNS records for example.com, all marked
VERIFIED, and posted {domain_id: "default-domain"} to endpoints expecting a
real id. DKIM rotation sends "domain", not "domain_id", so its 400 was
swallowed and a made-up TXT record displayed as though a key had been
issued: rotation had never once worked from this screen.
- The TLS panel asserted a Let's Encrypt RSA 4096 certificate and verified
DANE from static markup, on a deployment currently serving a self-signed
one. It now reads the cert watcher via /api/v1/admin/tls.
- The readiness scorecard printed "10 / 10 all checks passed" regardless of
what preflight reported, including failures.
- A message trace that found nothing invented a five-step delivery - TLS 1.3
accepted, SPF pass, spam score 0.8 - for a message that does not exist.
- Both sidebars showed a fixed admin@lambdamail.local with a hardcoded
"SUPER_ADMIN (2FA)", claiming a second factor whether or not one existed.
- Settings seeded a "[Boletim]" Sieve rule. Saving posts the whole list, so
the first edit wrote that invented filter into the account's real script.
The reconcile endpoint itself answered {records_verified: 13, status:
"VERIFIED"} for any domain id, including one that does not exist, without
resolving a name. It now reports what it does, which is record the request.
Surfaces:
The root was a chooser listing webmail and the console side by side, which
asked visitors a question they should not have to answer and advertised cookie
names and audiences to anonymous readers. It now redirects into the mail
client. Accounts that may administer the server get a link to the console from
inside the app - a link to /admin/login, so crossing over still costs the
second factor the admin audience requires - and the console gets a plain link
back, since the webmail session was never given up to get there.
Enrolment:
Both enrolment screens printed the base32 secret as text and left the operator
to type 32 characters into their phone. The enrol endpoint already returned an
otpauth:// URI; it simply was not drawn. Added a shared component that renders
it as a QR locally - the seed is already in the page, and sending it to an
image service would put it in another request log for nothing.
The recovery-code step rendered its own "sign in" button while the form's
submit button stayed visible below it: two identical buttons, and no way to
copy the codes it insisted you save. It is now terminal, with copy and
continue.
Language:
translate() had no interpolation, so "Undo Send ({seconds}s remaining)" showed
the placeholder and a second hardcoded countdown was appended after it. It now
substitutes, leaving an unmatched token visible rather than blanking it.
The pt-BR bundle had lost its diacritics - "Destinatario", "Ultimo erro",
"Acoes" - from the ASCII-source rule being applied to the translations, which
are data. Restored, and the three bundles are back in key parity at 270 each,
with roughly 120 strings that were hardcoded in the source now keyed.
Healthchecks:
web had none, despite being the only service users reach directly: a wedged
Next process kept its container running and Traefik kept routing to it. Added,
using node's own fetch since the image ships no curl. rspamd and clamav gained
the timeouts and start_periods whose absence would fail them during startup.
migrate stays without one and says why: it is a one-shot job, and a probe
would report unhealthy exactly when it has succeeded.
Verified: 36 webmail tests and 41 auth tests against a real Postgres, both new
admin/me cases confirmed to fail without the route (404). lang-lint clean,
production build clean, 0 vulnerabilities, compose validates.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The console demanded a second factor and refused to open without one, but
enrolment lived on the other surface - so an operator whose only account
was the administrator had no way in. The password was already verified at
that point, so the refusal now carries a short-lived grant that permits
enrolling a factor and nothing else, and the admin login screen walks
through it: secret, confirmation, recovery codes, sign in. Two tests cover
it, including that the grant opens no other endpoint.
The rewrite also undid work that was already in place:
- The postcss override was dropped and the advisory came back, along
with a critical one in vitest. postcss is a direct dependency now, so
the fix is the version plus an override scoped to Next's own copy;
vitest moves to 3.2.7. The lockfile needed regenerating for either to
take effect. Zero advisories now.
- 100 language-lint violations: the admin console had been rewritten
with its strings in Portuguese, hardcoded. That fails CI, and it also
puts those strings beyond the reach of the translation bundles - an
English or Spanish user would have read Portuguese. They are English
in the source again.
- The mail screen put a fixed 320px list beside the reader with no
breakpoint, so on a phone the viewport overflowed and the message was
a few pixels wide. One pane at a time below md, with a way back.
- The icon-only navigation toggles had nothing for a screen reader to
announce.
The layouts' own mobile navigation and every previously fixed behaviour
survived the rewrite - checked rather than assumed.
Also fixes a marker collision I introduced: the E2E helper locates the
repository by a migrations directory, and services/migrations satisfied
that one directory too early. It now requires the SQL to actually be
there.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The deployment sets NODE_ENV=production, and npm omits devDependencies
when it is set - so "npm ci" in the build stage installed no typescript
and "npm run build" failed with exit 127, command not found. The first
Coolify deployment died there.
The build stage now asks for them explicitly rather than depending on the
environment it happens to run in. Reproduced by building all three images
with NODE_ENV=production before and after.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
DKIM rotation broke the domain it was run on. It retired the live key and
inserted a placeholder as ACTIVE - private_key_enc was the literal string
"dummy-encrypted-key" - so FindActiveKeys then failed for the whole
domain and outbound mail stopped being signed, with the real key already
gone. Reproduced against a database before changing anything.
The root cause runs deeper: the auth service derives its encryption key
with a plain SHA-256 of the master key while the protocols service uses
HKDF with its own info string, so nothing the auth service seals can ever
be opened by the signer. Rotation therefore moves to the protocols
service, which owns that vault and the real key generator. It retires the
previous key with a seven-day overlap in the same transaction rather than
deleting it, because mail already sent under it is still being verified.
Domain onboarding had three defects in one function. It built the alias
destination by interpolating the domain name into the SQL text, which was
injectable; it reused one placeholder for a CITEXT and a VARCHAR column,
which made the statement fail and happened to hide the injection; and it
ignored its scope argument entirely, so any DOMAIN_ADMIN could create
domains. It also provisioned postmaster, abuse and security, missing the
dmarc and tlsrpt aliases that the published records point at - PLAN.md
section 7.4b is explicit that reports bounce without them.
The TLS panel returned fixed values: acme_status "VALID" and 72 days to
expiry, whatever the process actually held. It now reads the certificate
watcher, and a compile-time assertion keeps the wiring from silently
regressing into "disabled". The audit log and the readiness scorecard were
fixtures too, and their field names did not match the API they now read.
Two tests failed and a third asserted a fabricated constant: one depended
on test order through a shared fixture that had a second factor enrolled,
one used a fixed domain name and so passed once per database, and one
expected "records: 13" from a function that never touches DNS. All three
are fixed, and the suite now passes twice in a row against one database.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sessions and password changes were the two things PLAN.md section 14.2
asks of the account area that were still missing, and the admin console
had no audit trail at all despite section 14.3 requiring one for every
mutation.
Session revocation is enforced on every request, not just recorded. A JWT
carries its own validity, so revoking a row without checking it would have
made "sign out this device" a button that did nothing until the token
expired eight hours later. Changing a password revokes every other session
for the same reason: a password change that leaves the attacker's session
alive has not locked anyone out.
Admin queries are scoped by role in the SQL rather than in the UI, as
section 14.3 requires: a DOMAIN_ADMIN calling the API directly sees only
its own domain, and a mailbox outside that scope answers 404 rather than
403 so the scope cannot be used to enumerate what exists elsewhere.
The two mutations that exist so far - enabling or disabling a mailbox, and
retrying or cancelling a queued delivery - write actor, IP and target to
audit_log, and there is an endpoint to read it back.
Covered by tests against a real database: a revoked token stops working
mid-session, a password change needs the current password and takes the
other sessions down with it, and the minimum length is enforced.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The F4 auth service issued a valid session for any email and any password,
and accepted any non-empty string as a second factor. Reproduced against
the router before touching it: an invented password plus the code "000000"
returned a SUPER_ADMIN token that opened /api/v1/admin/dashboard.
Passwords. Hashing moves from PBKDF2 to Argon2id via hash-wasm, which is
what PLAN.md section 9 and the Go service already use - the previous
scheme meant a password set through the mail stack could never verify
here. Interoperability was checked in both directions before adopting the
library: a hash written by alexedwards/argon2id verifies in Node and one
written here verifies in Go. hash-wasm is pure WASM, so the multi-arch
Alpine images need no native build.
verifyPassword no longer falls back to comparing the password against the
stored hash. That fallback turned every stored hash into a working
credential: anyone with a database dump could authenticate by sending the
hash back. Anything that is not an argon2id PHC string is now refused.
Logins are decided against the database, with the failure counter and the
temporary lockout of PLAN.md section 5.2. An unknown address and a wrong
password give the same answer, and the unknown case still pays for a hash
so response time does not enumerate valid addresses.
Second factor. The TOTP secret is generated server-side, stored encrypted
and never accepted back from the client - the confirm step used to take
whichever secret the caller sent, so anyone could confirm with their own.
Recovery codes are Argon2id, single-use, and consumed under a guard so two
concurrent requests cannot spend the same one. App passwords are issued
and revoked through real endpoints instead of browser state.
Sessions gain a "purpose" claim. Without it the challenge token - handed
out after the password but before the second factor - satisfied every
check a real session did, which skipped MFA entirely.
The dev seed carried a hand-written password_hash whose digest was 24
bytes instead of 32, so it matched no password at all. Nothing noticed
while passwords went unverified; it is a real hash now.
Verified with 30 tests, 13 of them against a real Postgres: wrong password,
unknown account, pass-the-hash, wrong TOTP code, challenge token replay,
lockout, single-use recovery codes, and the surface isolation that F4's
acceptance criteria require.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
CI gains a security job (Trivy fs: vuln, secret and misconfig) and an
image job that builds each of the four services and scans the result. The
secret scanner earns its place here: a DKIM key or Cloudflare token
committed by accident is a full compromise of every domain served.
release.yml publishes on a merge into main and on v* tags. Four services
on two architectures is eight build jobs pushing by digest, then four
publish jobs joining each pair into one manifest list, tagged latest, sha
and semver. github.repository is lowercased first because GHCR rejects an
uppercase path and this repository is lucasrguerra/LambdaMail.
Both scanners failed on first run, so this commit also fixes what they
found rather than shipping a workflow that is red on arrival:
- All four images ran as root (DS-0002), which section 17 risk R5
explicitly argues against. They now run unprivileged; protocols keeps
its eight ports below 1024 through cap_net_bind_service on the binary
rather than through being root.
- A HIGH advisory in postcss, reached only through Next. "npm audit fix
--force" would have resolved it by downgrading Next from 15 to 9, so
it is pinned with an override instead.
- The Node images copied the build stage's node_modules wholesale,
shipping typescript, eslint and vitest into production, and the base
image's own bundled npm accounted for the rest of the CVEs. Runtime
dependencies now come from npm ci --omit=dev and npm is removed from
the final layer.
- compose-up could never have passed: the proxy network is external
because Coolify owns it on the deploy host, and nothing creates it in
CI, so compose refused to start the stack at all.
Verified locally: actionlint clean, all ten action refs confirmed to
exist, Trivy fs and all four image scans exit 0, the full stack reaches
Healthy on every container, and the containers run as uid 10001/1000 with
all eight privileged ports bound and SMTP answering on 25. The arm64
build itself is unverified - this machine has no buildx - and will first
be exercised by release.yml.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Stands up the LambdaMail monorepo per PLAN.md's F0 milestone: local dev
scaffolding (.env.example, Makefile, .gitignore excluding PLAN.md/docs/
.superpowers/), the full initial Postgres schema as a golang-migrate
migration, the complete Go domain layer for services/protocols (value
objects, entities, domain events - zero external imports beyond
github.com/google/uuid) with unit tests, health-checked service skeletons
for auth/storage (TypeScript) and webmail (Next.js), a local
docker-compose.yaml wiring all services together, a language-enforcement
lint script, and a GitHub Actions CI workflow.
Built via 11 sequential tasks under subagent-driven development, each with
an independent implementer + reviewer cycle. Fixes applied during review:
DomainName.IsPunycode() added to satisfy the promised interface; ioredis
default-import typing under NodeNext; next.js bumped 15.0.3->15.5.21 to
resolve high/critical CVEs; lang-lint.mjs corrected to not flag its own
banned-terms list; web service healthcheck added so compose --wait
actually waits on it; webmail Dockerfile's missing public/ directory:
protocols go.mod version aligned with CI's Go 1.22 pin.
Known follow-ups for later phases (not blocking F0): go-arch-lint isn't
wired into CI yet; live docker compose up and migration apply have not
been exercised in this dev environment (no Docker/psql available here)
and will get their first real run in CI, which does have Docker.
