Duas coisas apanhadas ao correr o reconcile contra a zona real depois de a
rota passar a funcionar.
1) Nao era idempotente: 0 criados, 4 corrigidos, 11 iguais -- e outra vez 4
corrigidos na passagem seguinte, para sempre, com o temporizador a fazer o
mesmo em ciclo. Os 4 eram os SRV. A Cloudflare devolve o content de um SRV
SEM a prioridade ("1 993 host" para um registo guardado como "0 1 993
host") e guarda-a em data. O adaptador so recorria a data quando o content
vinha vazio, portanto lia sempre um valor diferente do da spec.
Para SRV e TLSA a data passa a ser a autoridade.
2) A spec de cada dominio incluia o A e o AAAA do mail host. Um segundo
dominio servido pelo mesmo servidor aponta o MX para um host que nao lhe
pertence, por isso reconciliar o cienciaembarcada.com.br tentava escrever
mail.lucasrguerra.dev.br -- zona alheia. A Cloudflare respondia "an
identical record already exists" e a consola mostrava dois erros para um
dominio que estava correcto. O mesmo valia para o TLSA, que tambem tem o
nome do mail host.
Os registos de endereco passam a ser publicados so pela zona que contem o
host; o MX e os SRV, que apontam para ele, continuam a ser de cada dominio.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
O lang-lint exige que a fonte deste repositório seja ASCII, e os testes
que escrevi usavam acentos em texto de exemplo. Isto teria feito o CI
vermelho.
Onde o acento era só texto de exemplo, foi substituído. Nos dois testes
onde o acento é o que está a ser testado - que um nome de pasta acentuado
sobrevive ao parser, e que uma pasta pode chamar-se "Relatorios de
Marco" - fica escrito como escape Unicode, que mantém o teste a
verificar exactamente a mesma coisa sem pôr o caractere na fonte.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Encontrado a correr o script guardado na caixa real antes de o pôr no
ar. O Google envia os relatórios DMARC de
noreply-dmarc-support@google.com, e a verificação só reconhecia a
palavra exacta ou um sufixo com "+": a caixa teria respondido
automaticamente a todos os relatórios diários que recebe.
O prefixo passa a ser comparado com os separadores que os remetentes
usam de facto - "+", "-", "." e "_" - e a lista cresce com os nomes
correntes deste tipo de endereço. Continua a responder a uma pessoa
cujo endereço apenas começa por uma dessas palavras, o que também tem
teste: "bouncer" é gente, "bounces-123" não é.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Regras. A tela pedia um tipo de correspondência do Sieve pelo nome -
:contains, :is, :matches (regex) - e colava o valor escrito
diretamente num script. Punha uma linguagem de padrões à frente de
quem só queria dizer "o que vem do banco vai para Faturas", e bastava
uma aspa no valor para o script deixar de fazer parse, desativando
silenciosamente todas as regras de uma vez.
O vocabulário passa a ser o de quem escreve: um campo, uma condição em
português, um valor e uma ação. Os curingas são gerados a partir de
"começa com" e "termina com" - ninguém os escreve, e um asterisco
escrito por acaso continua a ser um asterisco. A pasta de destino é
escolhida de uma lista das pastas que existem mesmo, em vez de ser
escrita à mão: antes uma regra podia nomear uma pasta inexistente e
nunca disparar, sem dizer porquê.
A ordem passa a ser visível e alterável, porque a primeira regra que
arquiva uma mensagem decide para onde ela vai.
Um script que esta tela não consegue exprimir - escrito à mão, ou por
um cliente de secretária sobre ManageSieve - é mostrado tal como está,
em vez de ser reescrito na coisa mais parecida que este formulário sabe
dizer, o que mudaria em silêncio o que o correio de alguém faz.
Pastas próprias. Existiam na base de dados e as regras podiam arquivar
nelas, mas a barra lateral listava só as seis fixas, portanto não havia
como ver uma - nem como criar. Criar fica onde a vontade aparece, na
própria barra; apagar avisa que leva as mensagens junto, porque leva.
Encontrado a escrever isto: o motor tratava "\\*" como barra seguida de
curinga em vez de um asterisco literal. Como é esta tela que gera os
curingas, uma regra sobre um assunto com "50% *" teria correspondido a
todos os assuntos começados por "50% ". Corrigido no motor, com testes
dos dois lados.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Primeira metade do que falta para a ausência e os filtros funcionarem:
o motor em si, como domínio puro e testável, ainda sem estar ligado à
entrega.
É um subconjunto deliberado do RFC 5228 mais o vacation do RFC 5230:
os testes e as ações que a interface gera e que um script escrito à
mão provavelmente usa - header com :is, :contains e :matches, anyof,
allof, not, if/elsif/else, fileinto, discard, addflag, keep, stop e
vacation. O que não entende é recusado em vez de ignorado: uma regra
que o utilizador escreveu e que silenciosamente não faz nada é pior do
que uma que diz que não consegue correr.
:matches é correspondência com curingas, não expressão regular. Em
Sieve todos os outros caracteres são literais, portanto "a.b@x.test"
tem de casar apenas com um ponto verdadeiro; compilá-lo como regex
faria casar também "axb@x.test" e entregaria a um autor de scripts toda
a gramática das expressões regulares, incluindo padrões que levam tempo
exponencial numa linha de assunto construída para isso. O algoritmo faz
backtracking iterativo, linear no caso comum, e há um teste com limite
de tempo que falha se voltar a ser exponencial.
A parte da ausência que realmente importa é quando não responder. O
RFC 3834 é a regra: responde-se à mensagem de uma pessoa e a mais nada.
Remetente de envelope vazio, auto-submitted, precedence bulk, qualquer
cabeçalho de lista, x-auto-response-suppress, endereços reservados como
mailer-daemon, e a própria caixa - nenhum recebe resposta. É assim que
uma caixa evita responder a uma lista à frente de toda a gente, ou
entrar num ciclo com outro respondedor automático.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
- Implemented tests for ingesting TLS RPT reports from emails and ensuring they are stored correctly in the database.
- Added tests to verify that the same report can be ingested multiple times without duplication.
- Created tests for backfilling reports that were previously delivered to the inbox.
- Developed WebSocket event handling tests to ensure events are correctly routed to subscribers based on mailbox ownership.
- Included tests for the mail API to validate session handling, mailbox isolation, and error mapping.
- Ensured that the API rejects unauthorized requests and handles malformed JSON gracefully.
- Added `MoveToTrash` method in `WebmailRepository` to move messages to the Trash folder instead of deleting them permanently.
- Introduced `handleDelete` method in `mailAPI` to handle message deletion requests from the client.
- Updated router to include a new route for message deletion.
- Enhanced the compose page to manage draft integrity, ensuring drafts are correctly associated with sent messages.
- Implemented UI changes to allow users to delete messages from the mail folder view.
- Added tests to verify the correct behavior of message deletion and draft handling.
- Updated database migration scripts to reconcile folder counters and ensure accurate unread counts.
- Added tests for TLS-RPT report parsing and webmail repository functionalities.
A fresh deployment published "v=DKIM1; k=rsa; p=" for its own domain. An empty
p= is not an absent key, it is a revoked one (RFC 6376 section 3.6.1): every
verifier reading it treats all signatures from that selector as invalid. On a
domain whose DMARC policy is p=quarantine with strict alignment - which this
reconciler generates - that quarantines all of its own outbound mail. Worse
than publishing nothing.
The chain: DkimRepository.Insert is an INSERT ... SELECT conditional on the
domain existing in the domains table, so on an empty database it inserts no
rows and returns no error. FindPublicKey then finds nothing, ensure() returns
an empty string with no error, and the reconciler faithfully published the
empty value. Every field along that path was behaving as written; nothing
checked that the result meant anything.
Two changes:
- The record builder omits a DKIM record when its key is empty, so the
revocation cannot be published regardless of why the key is missing.
- Provisioning logs when it produced no key, naming the likely cause. The
insert being conditional on an onboarded domain is exactly the case a new
operator hits, and "no key, domain probably not onboarded" is the difference
between a diagnosis and unsigned mail with no explanation. It stays
non-fatal: erroring here would abort the whole reconcile and leave a new
install with no A, MX or SPF records either.
Found while migrating to a new host: the fresh database had no domain row, so
the first reconcile revoked the domain's DKIM. The two published records have
been deleted from the zone.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Every message was resolved through MX and delivered over SMTP, including one
addressed to a mailbox on this very host. Two users on the same server could
not write to each other without a working outbound port 25 - which the
provider blocks, so on this deployment they could not write to each other at
all. It also made internal mail take a pointless round trip through the public
internet.
The worker now checks whether the recipient resolves to a local mailbox and,
if so, hands the message to the inbound path directly. The check runs ahead of
the relay branch on purpose: sending local mail out to a smarthost only to
have it delivered back is slower and has more ways to fail.
The shortcut is narrow. It relies on ResolveDeliveryTargets, which only
resolves addresses on an active local domain, and a test covers that a remote
recipient still goes out over SMTP - otherwise local delivery would swallow
mail meant for the internet. A resolver error is treated as retry rather than
as not-local: falling through to SMTP on an unknown answer would send to the
internet a message that may belong here. Local delivery is still
content-scanned; a virus between two accounts on one host is no less a virus.
This also surfaced a permissive test double: fakeAliasFanOutRepository claimed
every address was local, which was harmless while nothing consulted it while
routing and wrong the moment something did. It now hosts one address, as the
real repository does, and the DSN fan-out test names it explicitly.
Verified: the local-delivery test fails without the change (the recipient
triggers an MX lookup) and passes with it; 22 Go packages green.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Four faults, all in the path between the compose screen and the mailbox.
**Every message sent from the webmail had an empty body.** The HTTP handler
read a field named "body"; the composer has always sent "html". The two never
matched, so the payload handed to the queue contained headers and nothing
else. The handler now reads html and text, and still accepts body so an older
client is not broken. Sending was otherwise working, which is why this went
unnoticed: the mail left, it just said nothing.
**Bcc was discarded.** The field was absent from the request struct and from
ComposeInput, so a blind copy typed into the webmail reached nobody. It now
joins the envelope recipients and is deliberately kept out of the headers,
which is what makes it blind.
**HTML was labelled text/plain.** Even once the body was carried, the composer
produces markup and the message declared text/plain, so a recipient would see
raw tags. Messages with HTML now go out as multipart/alternative with a
plain-text part derived from the markup and ordered first per RFC 2046 5.1.4.
An empty or missing text alternative is itself a spam signal.
**Sent mail left no trace.** Nothing wrote to the Sent folder, so a delivered
message vanished from the sender's view entirely. A copy is now filed after
the message is durably queued - not before, or a send refused by the rate
limit would still leave a copy - reusing the inbound persistence path so it
gets a real IMAP UID and appears over IMAP too. A failure to file is logged
and swallowed: the mail is already on its way, and reporting failure would
invite the user to send it twice.
**Drafts were never saved.** There was no endpoint at all. The composer showed
"draft saved automatically" from a setTimeout with no request behind it, so
closing the tab lost the message the screen had just promised was safe. Adds
POST /api/v1/mail/draft, storing the draft as a whole RFC 5322 message in the
Drafts folder, and debounces the composer onto it.
The composer also navigated to Sent whether or not the send succeeded, which
is how a rejected message disappears silently. It now reports the error and
stays put.
**Delay notifications were being filed as spam.** A DSN this server writes is
injected through the same inbound path as mail from the internet, but without
a client IP - so SPF can only be "none" - and its bounce-shaped content scores
highly. The warning telling you your mail was delayed landed in your own Junk
folder. Messages the server generates itself now skip content scanning; the
exemption is narrow, and a test covers that ordinary mail is still scanned so
this cannot become a way past the filter. The DSN also declared
Auto-Submitted: auto-replied, which belongs to vacation responders; a delivery
report is auto-generated (RFC 3834 section 5), and filters read that header.
Not fixed here, because it is not a code fault: outbound port 25 is blocked by
the provider, which is what the delay notification was reporting
("i/o timeout" to Google's MX - a timeout, not a refusal, is the signature of
a block). Receiving is unaffected and queued mail keeps retrying. Documented
with the two ways out, including the relay path, which is verified working
code rather than another stub.
Verified: 22 Go packages pass, 5 new compose tests and 3 for the DSN handling.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Every fix here is a case where the code and the plan disagreed, and the
code was wrong.
Certificates (section 8.2, 8.4). The parser documented that it skipped
expired entries but never checked, so an expired certificate would be
served until every handshake failed. LastReload() only advanced when
acme.json changed - and it does not change for sixty days - which made
the timestamp risk R2 monitors indistinguishable from a watcher gone
blind. Expiry is now checked on every pass, LastReload records the check
and LastChange the swap, and the <7d/<24h alerts of section 8.4 exist.
ARC (section 5). buildArcSealer resolved the DKIM key at startup, before
runDnsReconciler had provisioned one. On a first boot ARC was disabled
until someone restarted the process, which the log even admitted while
offering no way for it to start. A LazySealer binds on first use instead.
Health (section 8.4). There was no degraded state at all. /health now
reports one - still 200, because restarting cannot fix a certificate
Traefik never issued.
Bounces. DSNs were sent From postmaster@lambdamail.local, a domain that
does not resolve, so the notification was itself undeliverable. They now
carry the configured mail host, a Message-ID, and Auto-Submitted so a
vacation responder cannot loop against them.
Fan-out atomicity (section 6.1). Persist ran one transaction per
recipient, so a failure halfway through an alias fan-out left earlier
recipients with a copy while the sender was told to retry - and the retry
delivered to them twice. PersistAll commits the whole delivery or none of
it. Verified against Postgres by reverting the repository to the old
behaviour and confirming the new test fails on it.
SMTP replies. The use case encoded replies in error text ("550 5.7.1 ...")
which the adapter re-parsed by prefix, so rewording a message could turn a
permanent refusal into a retry. SmtpRejection carries the decision and the
adapter renders it.
Repository tests. These only run with TEST_DATABASE_URL, which CI does
set - so the job was red: copy and expunge seeded a 68-character digest
into a CHAR(64) column, and three others used fixed digests that collide
with the UNIQUE index on any re-run. A shared testBlobDigest() helper
fixes both; the suite now passes twice in a row against one database.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Stands up the LambdaMail monorepo per PLAN.md's F0 milestone: local dev
scaffolding (.env.example, Makefile, .gitignore excluding PLAN.md/docs/
.superpowers/), the full initial Postgres schema as a golang-migrate
migration, the complete Go domain layer for services/protocols (value
objects, entities, domain events - zero external imports beyond
github.com/google/uuid) with unit tests, health-checked service skeletons
for auth/storage (TypeScript) and webmail (Next.js), a local
docker-compose.yaml wiring all services together, a language-enforcement
lint script, and a GitHub Actions CI workflow.
Built via 11 sequential tasks under subagent-driven development, each with
an independent implementer + reviewer cycle. Fixes applied during review:
DomainName.IsPunycode() added to satisfy the promised interface; ioredis
default-import typing under NodeNext; next.js bumped 15.0.3->15.5.21 to
resolve high/critical CVEs; lang-lint.mjs corrected to not flag its own
banned-terms list; web service healthcheck added so compose --wait
actually waits on it; webmail Dockerfile's missing public/ directory:
protocols go.mod version aligned with CI's Go 1.22 pin.
Known follow-ups for later phases (not blocking F0): go-arch-lint isn't
wired into CI yet; live docker compose up and migration apply have not
been exercised in this dev environment (no Docker/psql available here)
and will get their first real run in CI, which does have Docker.
