acaua/docs/architecture.md

# Architecture

**English** · [Português](pt-BR/arquitetura.md)

Acauã is a single Rust product: one daemon/CLI, one shared core, and one
embedded web dashboard. The shipped runtime has no Python and no external
database. This document records the structural decisions and the six
corrections applied to the original blueprint.

## Stack

| Layer | Choice | Why |
|---|---|---|
| Core, CLI and daemon | Rust 2021 (MSRV 1.88) | One toolchain, memory safety, no runtime dependency |
| HTTP API | Axum + Tokio + Tower HTTP | Small async surface, easy to embed |
| Web Dashboard | Vanilla HTML/CSS/JS | No frontend build step, assets embedded in the binary via `rust-embed` |
| Analytical store | DuckDB + Parquet | Mandatory embedded columnar store, linked to official prebuilt `libduckdb.so` |
| Deployment | Docker / docker-compose | Collective SOC daemon, unprivileged non-root user, read-only rootfs |
| Scientific validation | Python in `research/` only | astropy/scipy/networkx for fixtures, never shipped |

The workspace has two crates: `acaua-core` and `acaua-cli`. Both use `version.workspace = true`.

## Components

```
Zeek logs ──► acaua-core::zeek ──► normalized events ──► storage
                                      │
                                      ├──► pillar1 (beaconing)
                                      ├──► pillar2 (persistent sessions)
                                      ├──► pillar3 (protocol anomalies)
                                      └──► pillar4 (fusion) ──► alerts + evidence
                                                                    │
acaua-cli (serve) ──► axum API ──► embedded web dashboard ──────────┘
```

- `zeek`: bounded parser and path confinement. It must never follow a symlink
  outside the dataset root.
- `storage`: DuckDB in-memory engine over daily partitioned Parquet files with a
  seven-day TTL.
- `pillar1`–`pillar4`: pure functions over normalized events. They return
  scores plus evidence, never only a float.
- `fusion`: fixed-order, bounded combination. It must be symmetric and
  reproducible.
- `doctor`: local environment checks; reports prebuilt DuckDB linkage and
  containerized Zeek status.
- `update`: checks for updates against the Gitea Releases API.

## The six corrections

### 1. Rayleigh/Schuster, not pure Lomb–Scargle

Event timestamps are a one-dimensional point process: there is no `y` value to
fit. The primary periodogram is therefore Rayleigh/Schuster:

```text
P(T) = (2 / N) * [ (sum cos(2*pi*t_i / T))^2 + (sum sin(2*pi*t_i / T))^2 ]
```

Generalized Lomb–Scargle is reserved for the future extension that evaluates a
real response attribute (such as payload size or duration) against time.

### 2. Guard the zero-dispersion case in MADM

A synthetic or unjittered beacon produces equal intervals. The median absolute
deviation from the median (MADM) is then `0.0`. Dividing by `0.0` or treating it
as a missing value is wrong: zero dispersion is the most suspicious outcome. The
implementation guards this case with an explicit score assignment (`1.0`) and a
documented sentinel.

### 3. Replace Pearson skewness with Bowley skewness

Sample skewness based on the third central moment is sensitive to single
outliers and can diverge. Bowley skewness uses quartiles:

```text
B = (Q3 + Q1 - 2*Q2) / (Q3 - Q1)
```

It is bounded in `[-1.0, 1.0]` by definition and resistant to extreme outliers.
When `Q3 == Q1` (concentrated data), the implementation assigns `0.0` with a
sentinel flag.

### 4. Jensen-Shannon divergence instead of Kullback-Leibler

Kullback-Leibler divergence diverges to infinity when `Q(x) = 0` on non-empty `P(x)`.
Acauã adopts the Jensen-Shannon Divergence (JSD): bounded in `[0, 1]`, symmetric,
and strictly finite even on disjoint supports. Measured in real DNS tunneling and
CDN traffic, JSD alone does not separate benign CDNs from tunnels; hence in v1 it
is recorded in alert evidence with weight 0.0 rather than driving the score.

### 5. Dedicated persistent session detector (Pillar 2)

A long-lived interactive session or steady C2 channel does not generate frequent
reconnections (`Δt`), making it invisible to point-process periodicity analysis.
Acauã implements a dedicated persistent session detector with TCP session chaining
that re-aggregates sessions split by Zeek's 5-minute inactivity timeouts.

### 6. JA4 licensing boundary

Only the original TLS client fingerprint JA4 is BSD-3 licensed. Other modules
(JA4S, JA4H, JA4X, JA4SSH, JA4L) use the proprietary FoxIO License 1.1 (non-OSI).
Acauã does not bundle or depend on FoxIO-licensed modules by default in an
open-source architecture.

### 7. Determinism and fixed reduction order

Parallel reductions (e.g. via `rayon`) reorder floating-point additions and
multiplications across threads, breaking bit-level reproducibility. All reductions,
periodogram summations, and host evidence combinations are computed in a fixed,
deterministic sequential order, producing byte-for-byte identical JSON evidence.

### 8. Isolate the Python research workspace

`research/` has its own virtual environment and dependencies. It is excluded
from the Cargo workspace, ignored by Docker builds and CI releases, and tested
only when research validation scripts are invoked directly.

### 9. Documented degraded mode for NetFlow/IPFIX

NetFlow v9 and IPFIX do not carry DNS query names, TLS SNI, certificate chains
or HTTP headers. Any analysis that depends on protocol features is unavailable.
Acauã documents this as a degraded mode and never claims parity between flow
records and full Zeek logs.

### 10. Single HTTP and RPC boundary

The daemon exposes an Axum API on loopback by default. The embedded web UI
communicates with the daemon exclusively via standard HTTP calls.

## Detection (v1)

### Pillar 1: Low-and-slow C2 Beaconing (v1.2)
Evaluates session starts per `(orig_h, resp_h, resp_p, proto)` using a Rayleigh/Schuster
periodogram over an effective grid bounded by `span / 3` (requiring at least 3 full cycles
observed). Significance is gated by dispersion regularity (`gate = min(1, regularity / 0.25)`),
preventing false positives on bursty benign browsing. Alert threshold: `score >= 0.40`.
Non-TCP/UDP protocols, multicast/broadcast addresses, and infrastructure service ports
(e.g., NTP 123, DNS 53, mDNS 5353, NetBIOS, DHCP) are audited and excluded.

### Pillar 2: Persistent Sessions (v1.0)
Chains TCP connection segments across Zeek inactivity timeouts (gap <= 3600s with no `S`
SYN flag in Zeek history). Persistence score is logarithmic:
`persistence = clamp(ln(L / 3600) / ln(24), 0, 1)`. Sessions with `L >= 7200s`
(`score >= 0.2181`) trigger an alert. Producer-Consumer Ratio (PCR), byte transfer rates,
and failed connection attempts are captured in evidence.

### Pillar 3: DNS Subdomain Information Capacity (v1.0)
Measures the information payload capacity pushed across DNS queries by aggregating unique subdomains
per `(orig_h, base_domain, utc_hour)`. Normalizes queries to lowercase, strips trailing dots, and excludes
reverse lookups (`.in-addr.arpa`, `.ip6.arpa`), `.local`, and discovery ports (5353, 5355, 137):
- Information capacity: $I = H \times C$ bits, where $H = -\sum p_c \log_2 p_c$ (Shannon entropy per char)
  and $C$ is the sum of characters across unique subdomains.
- Evaluated at peak hour: $I_{\max} = \max_h I$. Requires $U_{\text{peak}} \ge 10$ unique subdomains;
  otherwise status is `insufficient_data` with score 0.0.
- Scoring: $\text{score} = \text{clamp}(\log_2(I_{\max} / 8192) / \log_2(32), 0, 1)$ ($8\text{ kbit} \to 0.0$,
  $32\text{ kbit} \to 0.40$, $256\text{ kbit} \to 1.0$).
- Alert threshold: $\text{score} \ge 0.40$ ($\iff I_{\max} \ge 32768$ bits/h).
- Jensen-Shannon Divergence (JSD) against the mixture of other active domains in the window is stored in
  evidence with weight 0.0 (tunnels measured 0.157–0.193 vs. benign CDNs 0.226–0.227, inverting rank).

### Pillar 4: Per-Host Evidence Fusion (v1.0)
Combines alerts per `orig_h` using a weighted noisy-OR model across pillars 1, 2, and 3:
- For each host, $m_i$ is the maximum alert score from pillar $i$ ($0.0$ if no alert exists).
- Fixed weights $w_1 = w_2 = w_3 = 0.75$.
- Score formula: $S = 1 - \prod_{i=1}^3 (1 - w_i m_i)$ evaluated in fixed sequential order $[1, 2, 3]$.
- Properties: $S \in [0, 1]$, monotonic in every $m_i$, single pillar capped at $0.75$, two pillars capped at
  $0.9375$, three pillars capped at $0.984375$. Invariant under alert input permutations.
- Fusion ranks hosts in `AnalysisReport.hosts` by $(S \text{ desc}, orig\_h \text{ asc})$; it provides investigation
  prioritization without fabricating synthetic alert classes.

### Known Limitations
1. In live captures, open sessions remain unobserved until Zeek flushes them on closure or timeout.
2. Benign cloud polling and persistent services (e.g., smart home devices like Philips HUE, websockets, push notifications) exhibit genuine periodicity and persistence, ranking high in fused scores without network-wide context.
3. Pillar 3 tunnel detection sensitivity is approximately 100 unique queries per hour.
4. Base domain extraction uses an effective heuristic without a full embedded Public Suffix List (PSL).

### Validation Datasets
Public datasets used for empirical verification (detailed in `research/validation/datasets.tsv`):
- **IoT-23** (captures 34-1, 20-1, 8-1, 42-1, Honeypot 4-1): CC BY 2.0 ground truth for C2 beaconing, persistence, and benign IoT controls.
- **CTU-Normal** (captures 20, 18, 26): public captures representing benign browsing and Facebook traffic.
- **Mendeley dtqbc-m-test** (Y. Bubnov): CC BY 4.0 dataset with real DNS tunneling payloads (dns2tcp, dnscapy, tuns).

## API

The HTTP API is served by `acaua serve`:

| Method | Path | Meaning |
|---|---|---|
| GET | `/api/health` | Liveness and version |
| GET | `/api/status` | Uptime, data directory, dataset summary |
| GET | `/api/alerts` | Paginated alerts with evidence |
| POST | `/api/analyze` | Ingest and analyze a dataset directory |
| GET | `/api/update` | Check for newer releases |
| GET | `/logo.png` | Embedded brand asset |
| GET | `/*` | Embedded UI fallback (`rust-embed`) |

`serve` binds to `127.0.0.1:8787` by default. Exposing it on an external network
requires an authenticating reverse proxy.

## Storage

DuckDB is a mandatory dependency linked dynamically to the official prebuilt
`libduckdb.so` (version 1.5.6 pinned with SHA-256 in `scripts/fetch-libduckdb.sh`).
Parquet, JSON and ICU extensions are statically built into the official library.
The schema consists of daily partitioned Parquet files under the configured
data directory, with in-memory DuckDB querying and a seven-day TTL.

## Updates

Updates for Acauã are delivered via container images (`docker compose pull && docker compose up -d`).
The `acaua update` command queries the Gitea Releases API for `ciberfauna/acaua` and
notifies operators when a newer version tag is available. Binary self-overwrite
(`update --apply`) is removed: in a collective SOC daemon running under container
orchestration, updates must be controlled by image pulls rather than self-mutating
processes.

## Rejected alternatives

1. **Python service with FastAPI/ClickHouse:** Rejected. A Python runtime introduces
   heavy container footprints and unnecessary memory boundaries; ClickHouse adds an
   external server operational burden for datasets that fit an embedded columnar store.
   The product is 100% Rust.
2. **Desktop application (Tauri / AppImage / NSIS):** Rejected. Unlike Guará (local
   raw disk forensics) and Sabiá (Wi-Fi monitor-mode hardware), Acauã has no local
   hardware dependencies: it is a multi-user SOC daemon consuming Zeek logs. A desktop
   wrapper adds maintenance and security surface without operational benefits.

## Deployment

- **Docker Compose (Official):** Multi-arch container image (`git.lucasrguerra.dev.br/ciberfauna/acaua`),
  running as unprivileged user `acaua` (UID 10001), read-only root filesystem, dropped capabilities.
- **Native compilation:** `scripts/fetch-libduckdb.sh ~/.cache/acaua/duckdb` followed by `cargo build --release -p acaua-cli`.

## Failure behavior

Malformed input is skipped with a counted, bounded diagnostic. A missing log is
not fatal. A corrupt store is reported by `doctor` and never silently recreated
over existing data.

## Versioning and compatibility

SemVer 2.0.0. Evidence JSON carries a schema version. Any change to a score or
to the evidence shape is a breaking change for `0.x` minor versions and must be
recorded in the changelog.