Lucas RayanandClaude Sonnet 5
c64bbbf775
Pillar 1 v1.1 (shipped earlier) flagged benign browsing bursts as beaconing: on short captures, grid periods longer than the observed span never complete a cycle, so significance saturates to 1 and the score sticks at exactly 0.5. v1.2 caps the grid at a third of the observed span, requires a minimum number of completed cycles, and excludes protocols that are periodic by nature (NTP, DNS, mDNS, ICMP). Validated against 300 seeds per scenario: false positives on bursty traffic drop from the 290-300/300 that v1.1 produced to 0/300, with every beacon-detection scenario scoring unchanged. Add Pillar 2 (persistent/long-lived sessions, with Zeek's fragmented TCP sessions rejoined after a 5-minute gap), Pillar 3 (DNS tunneling via Shannon-entropy information volume per subdomain per hour; the Jensen-Shannon divergence was measured and does not separate tunnels from benign CDN subdomains, so it stays in the evidence at zero weight instead of in the score), and Pillar 4 (deterministic noisy-OR fusion across the three pillars). The fusion formula was validated against 8 labeled real-traffic vectors (IoT-23, CTU-Normal) with tolerance <= 1e-4. A known limitation is kept, not hidden: a benign Philips HUE bulb's strict periodic polling and a 21.9h session score it above three real C2 hosts. No weight or threshold was tuned to mask this; it is documented in the architecture docs instead. Also fixes real bugs found along the way: ingest_dir's partition count summed file names instead of distinct dates, DuckDB errors were counted as bad rows, /api/alerts always returned an empty list regardless of the last analysis, and the dashboard still claimed every pillar was unimplemented. Restores four design corrections that were missing from docs/architecture.md (KL to Jensen-Shannon, the persistent-session detector, the JA4 licensing boundary, and deterministic reduction order), and catalogs the public datasets used for validation (IoT-23, CTU-Normal, Mendeley DTQBC) with their licenses and SHA-256 checksums under research/validation/. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VAyr8d7HuAdUGB6aA5xmDD
2.5 KiB
2.5 KiB
Roadmap
English · Português
This roadmap separates what exists from what is planned. The four detection pillars are implemented in their initial version (v1) and validated against public traffic.
Done
- Rust workspace with
acaua-coreandacaua-cli. - CLI commands (
analyze,serve,doctor,update), Axum API and embedded web dashboard. - 100% Docker containerization, compose deployment, non-root hardened profile.
- Prebuilt DuckDB linking and daily Hive-partitioned Parquet storage with sandbox isolation.
- Real Zeek log parser for
conn.loganddns.logmapping columns by name. - Pillar 1 (v1.2): Rayleigh/Schuster periodogram with bounded grid (
T \le \text{span}/3), regularity gating, and infrastructure exclusions. - Pillar 2 (v1.0): Persistent TCP sessions with inactivity timeout chaining (gaps $\le 3600$s without SYN), log persistence score, and PCR/failed attempts in evidence.
- Pillar 3 (v1.0): DNS information capacity (Shannon entropy
\timesunique subdomain chars per UTC hour) to detect tunneling, with JSD in evidence. - Pillar 4 (v1.0): Per-host weighted noisy-OR fusion (
w_1=w_2=w_3=0.75) with deterministic JSON evidence and priority ranking (hosts:output). - In-memory alert reporting in
/api/alertsand updated web interface. - English/Portuguese documentation, CI/release workflows and
scripts/check.sh. - Deterministic test suite and validation scripts against IoT-23 and CTU-Normal captures.
Next
- Continuous ingestion: automated log rotation and directory watching in daemon mode.
- Persistent alerts: SQLite/DuckDB persistence of alerts across daemon restarts.
- Live long connection tracking: sensor-side long connection logging before TCP teardown.
- Public Suffix List (PSL): replace base domain heuristic with official PSL.
- Allowlisting & baselining: reduce benign cloud IoT polling (e.g. Philips HUE) false positives.
Later
- UDP sessions and additional protocol parsers (
ssl.log,smb_mapping.log,dce_rpc.log,rdp.log,ssh.log,kerberos.log,ntlm.log). - Identity tracking, lateral movement detection, and pivot graphs.
- NetFlow/IPFIX as a documented degraded mode and Suricata EVE JSON as an optional secondary input.
- Signed release metadata, SBOM and reproducible builds.
Non-goals
- Opaque AI scoring that cannot explain an alert.
- A required cloud service or telemetry pipeline.
- Python in the shipped runtime.
- Treating flow records as equivalent to full Zeek protocol logs.
- Native desktop shell or binary self-mutation.