acaua/docs/architecture.md

Architecture

English · Português

Acauã is a single Rust product: one daemon/CLI, one shared core, and one embedded web dashboard. The shipped runtime has no Python and no external database. This document records the structural decisions and the six corrections applied to the original blueprint.

Stack

Layer	Choice	Why
Core, CLI and daemon	Rust 2021 (MSRV 1.88)	One toolchain, memory safety, no runtime dependency
HTTP API	Axum + Tokio + Tower HTTP	Small async surface, easy to embed
Web Dashboard	Vanilla HTML/CSS/JS	No frontend build step, assets embedded in the binary via rust-embed
Analytical store	DuckDB + Parquet	Mandatory embedded columnar store, linked to official prebuilt libduckdb.so
Deployment	Docker / docker-compose	Collective SOC daemon, unprivileged non-root user, read-only rootfs
Scientific validation	Python in research/ only	astropy/scipy/networkx for fixtures, never shipped

The workspace has two crates: acaua-core and acaua-cli. Both use version.workspace = true.

Components

Zeek logs ──► acaua-core::zeek ──► normalized events ──► storage
                                      │
                                      ├──► pillar1 (beaconing)
                                      ├──► pillar2 (persistent sessions)
                                      ├──► pillar3 (protocol anomalies)
                                      └──► pillar4 (fusion) ──► alerts + evidence
                                                                    │
acaua-cli (serve) ──► axum API ──► embedded web dashboard ──────────┘

• zeek: bounded parser and path confinement. It must never follow a symlink outside the dataset root.
• storage: DuckDB in-memory engine over daily partitioned Parquet files with a seven-day TTL.
• pillar1–pillar4: pure functions over normalized events. They return scores plus evidence, never only a float.
• fusion: fixed-order, bounded combination. It must be symmetric and reproducible.
• doctor: local environment checks; reports prebuilt DuckDB linkage and containerized Zeek status.
• update: checks for updates against the Gitea Releases API.

The six corrections

1. Rayleigh/Schuster, not pure Lomb–Scargle

Event timestamps are a one-dimensional point process: there is no y value to fit. The primary periodogram is therefore Rayleigh/Schuster:

P(T) = (2 / N) * [ (sum cos(2*pi*t_i / T))^2 + (sum sin(2*pi*t_i / T))^2 ]

Generalized Lomb–Scargle is reserved for the future extension that evaluates a real response attribute (such as payload size or duration) against time.

2. Guard the zero-dispersion case in MADM

A synthetic or unjittered beacon produces equal intervals. The median absolute deviation from the median (MADM) is then 0.0. Dividing by 0.0 or treating it as a missing value is wrong: zero dispersion is the most suspicious outcome. The implementation guards this case with an explicit score assignment (1.0) and a documented sentinel.

3. Replace Pearson skewness with Bowley skewness

Sample skewness based on the third central moment is sensitive to single outliers and can diverge. Bowley skewness uses quartiles:

B = (Q3 + Q1 - 2*Q2) / (Q3 - Q1)

It is bounded in [-1.0, 1.0] by definition and resistant to extreme outliers. When Q3 == Q1 (concentrated data), the implementation assigns 0.0 with a sentinel flag.

4. Jensen-Shannon divergence instead of Kullback-Leibler

Kullback-Leibler divergence diverges to infinity when Q(x) = 0 on non-empty P(x). Acauã adopts the Jensen-Shannon Divergence (JSD): bounded in [0, 1], symmetric, and strictly finite even on disjoint supports. Measured in real DNS tunneling and CDN traffic, JSD alone does not separate benign CDNs from tunnels; hence in v1 it is recorded in alert evidence with weight 0.0 rather than driving the score.

5. Dedicated persistent session detector (Pillar 2)

A long-lived interactive session or steady C2 channel does not generate frequent reconnections (Δt), making it invisible to point-process periodicity analysis. Acauã implements a dedicated persistent session detector with TCP session chaining that re-aggregates sessions split by Zeek's 5-minute inactivity timeouts.

6. JA4 licensing boundary

Only the original TLS client fingerprint JA4 is BSD-3 licensed. Other modules (JA4S, JA4H, JA4X, JA4SSH, JA4L) use the proprietary FoxIO License 1.1 (non-OSI). Acauã does not bundle or depend on FoxIO-licensed modules by default in an open-source architecture.

7. Determinism and fixed reduction order

Parallel reductions (e.g. via rayon) reorder floating-point additions and multiplications across threads, breaking bit-level reproducibility. All reductions, periodogram summations, and host evidence combinations are computed in a fixed, deterministic sequential order, producing byte-for-byte identical JSON evidence.

8. Isolate the Python research workspace

research/ has its own virtual environment and dependencies. It is excluded from the Cargo workspace, ignored by Docker builds and CI releases, and tested only when research validation scripts are invoked directly.

9. Documented degraded mode for NetFlow/IPFIX

NetFlow v9 and IPFIX do not carry DNS query names, TLS SNI, certificate chains or HTTP headers. Any analysis that depends on protocol features is unavailable. Acauã documents this as a degraded mode and never claims parity between flow records and full Zeek logs.

10. Single HTTP and RPC boundary

The daemon exposes an Axum API on loopback by default. The embedded web UI communicates with the daemon exclusively via standard HTTP calls.

Detection (v1)

Pillar 1: Low-and-slow C2 Beaconing (v1.2)

Evaluates session starts per (orig_h, resp_h, resp_p, proto) using a Rayleigh/Schuster periodogram over an effective grid bounded by span / 3 (requiring at least 3 full cycles observed). Significance is gated by dispersion regularity (gate = min(1, regularity / 0.25)), preventing false positives on bursty benign browsing. Alert threshold: score >= 0.40. Non-TCP/UDP protocols, multicast/broadcast addresses, and infrastructure service ports (e.g., NTP 123, DNS 53, mDNS 5353, NetBIOS, DHCP) are audited and excluded.

Pillar 2: Persistent Sessions (v1.0)

Chains TCP connection segments across Zeek inactivity timeouts (gap <= 3600s with no S SYN flag in Zeek history). Persistence score is logarithmic: persistence = clamp(ln(L / 3600) / ln(24), 0, 1). Sessions with L >= 7200s (score >= 0.2181) trigger an alert. Producer-Consumer Ratio (PCR), byte transfer rates, and failed connection attempts are captured in evidence.

Pillar 3: DNS Subdomain Information Capacity (v1.0)

Measures the information payload capacity pushed across DNS queries by aggregating unique subdomains per (orig_h, base_domain, utc_hour). Normalizes queries to lowercase, strips trailing dots, and excludes reverse lookups (.in-addr.arpa, .ip6.arpa), .local, and discovery ports (5353, 5355, 137):

• Information capacity: I = H \times C bits, where H = -\sum p_c \log_2 p_c (Shannon entropy per char) and C is the sum of characters across unique subdomains.
• Evaluated at peak hour: I_{\max} = \max_h I. Requires U_{\text{peak}} \ge 10 unique subdomains; otherwise status is insufficient_data with score 0.0.
• Scoring: \text{score} = \text{clamp}(\log_2(I_{\max} / 8192) / \log_2(32), 0, 1) (8\text{ kbit} \to 0.0, 32\text{ kbit} \to 0.40, 256\text{ kbit} \to 1.0).
• Alert threshold: \text{score} \ge 0.40 (\iff I_{\max} \ge 32768 bits/h).
• Jensen-Shannon Divergence (JSD) against the mixture of other active domains in the window is stored in evidence with weight 0.0 (tunnels measured 0.157–0.193 vs. benign CDNs 0.226–0.227, inverting rank).

Pillar 4: Per-Host Evidence Fusion (v1.0)

Combines alerts per orig_h using a weighted noisy-OR model across pillars 1, 2, and 3:

• For each host, m_i is the maximum alert score from pillar i (0.0 if no alert exists).
• Fixed weights w_1 = w_2 = w_3 = 0.75.
• Score formula: S = 1 - \prod_{i=1}^3 (1 - w_i m_i) evaluated in fixed sequential order [1, 2, 3].
• Properties: S \in [0, 1], monotonic in every m_i, single pillar capped at 0.75, two pillars capped at 0.9375, three pillars capped at 0.984375. Invariant under alert input permutations.
• Fusion ranks hosts in AnalysisReport.hosts by (S \text{ desc}, orig\_h \text{ asc}); it provides investigation prioritization without fabricating synthetic alert classes.

Known Limitations

1. In live captures, open sessions remain unobserved until Zeek flushes them on closure or timeout.
2. Benign cloud polling and persistent services (e.g., smart home devices like Philips HUE, websockets, push notifications) exhibit genuine periodicity and persistence, ranking high in fused scores without network-wide context.
3. Pillar 3 tunnel detection sensitivity is approximately 100 unique queries per hour.
4. Base domain extraction uses an effective heuristic without a full embedded Public Suffix List (PSL).

Validation Datasets

Public datasets used for empirical verification (detailed in research/validation/datasets.tsv):

• IoT-23 (captures 34-1, 20-1, 8-1, 42-1, Honeypot 4-1): CC BY 2.0 ground truth for C2 beaconing, persistence, and benign IoT controls.
• CTU-Normal (captures 20, 18, 26): public captures representing benign browsing and Facebook traffic.
• Mendeley dtqbc-m-test (Y. Bubnov): CC BY 4.0 dataset with real DNS tunneling payloads (dns2tcp, dnscapy, tuns).

API

The HTTP API is served by acaua serve:

Method	Path	Meaning
GET	/api/health	Liveness and version
GET	/api/status	Uptime, data directory, dataset summary
GET	/api/alerts	Paginated alerts with evidence
POST	/api/analyze	Ingest and analyze a dataset directory
GET	/api/update	Check for newer releases
GET	/logo.png	Embedded brand asset
GET	/*	Embedded UI fallback (rust-embed)

serve binds to 127.0.0.1:8787 by default. Exposing it on an external network requires an authenticating reverse proxy.

Storage

DuckDB is a mandatory dependency linked dynamically to the official prebuilt libduckdb.so (version 1.5.6 pinned with SHA-256 in scripts/fetch-libduckdb.sh). Parquet, JSON and ICU extensions are statically built into the official library. The schema consists of daily partitioned Parquet files under the configured data directory, with in-memory DuckDB querying and a seven-day TTL.

Updates

Updates for Acauã are delivered via container images (docker compose pull && docker compose up -d). The acaua update command queries the Gitea Releases API for ciberfauna/acaua and notifies operators when a newer version tag is available. Binary self-overwrite (update --apply) is removed: in a collective SOC daemon running under container orchestration, updates must be controlled by image pulls rather than self-mutating processes.

Rejected alternatives

1. Python service with FastAPI/ClickHouse: Rejected. A Python runtime introduces heavy container footprints and unnecessary memory boundaries; ClickHouse adds an external server operational burden for datasets that fit an embedded columnar store. The product is 100% Rust.
2. Desktop application (Tauri / AppImage / NSIS): Rejected. Unlike Guará (local raw disk forensics) and Sabiá (Wi-Fi monitor-mode hardware), Acauã has no local hardware dependencies: it is a multi-user SOC daemon consuming Zeek logs. A desktop wrapper adds maintenance and security surface without operational benefits.

Deployment

• Docker Compose (Official): Multi-arch container image (git.lucasrguerra.dev.br/ciberfauna/acaua), running as unprivileged user acaua (UID 10001), read-only root filesystem, dropped capabilities.
• Native compilation: scripts/fetch-libduckdb.sh ~/.cache/acaua/duckdb followed by cargo build --release -p acaua-cli.

Failure behavior

Malformed input is skipped with a counted, bounded diagnostic. A missing log is not fatal. A corrupt store is reported by doctor and never silently recreated over existing data.

Versioning and compatibility

SemVer 2.0.0. Evidence JSON carries a schema version. Any change to a score or to the evidence shape is a breaking change for 0.x minor versions and must be recorded in the changelog.