refactor: distribute Acaua as a Docker image instead of a desktop app
Release / Plan (reuse the published image?) (push) Successful in 3s
CI / Docker image (build and smoke test) (push) Skipped
CI / Format and clippy (push) Successful in 4m32s
CI / Dependency audit (push) Successful in 11s
CI / Tests (Linux) (push) Successful in 9m15s
Release / ci (push) Successful in 9m18s
Release / Build image (arm64) (push) Failing after 7m7s
Release / Build image (amd64) (push) Failing after 13m53s
Release / Publish tags (push) Skipped
Release / Gitea release (push) Skipped
Release / Plan (reuse the published image?) (push) Successful in 3s
CI / Docker image (build and smoke test) (push) Skipped
CI / Format and clippy (push) Successful in 4m32s
CI / Dependency audit (push) Successful in 11s
CI / Tests (Linux) (push) Successful in 9m15s
Release / ci (push) Successful in 9m18s
Release / Build image (arm64) (push) Failing after 7m7s
Release / Build image (amd64) (push) Failing after 13m53s
Release / Publish tags (push) Skipped
Release / Gitea release (push) Skipped
Acaua is a collective SOC daemon consuming only Zeek log files, not a single-user tool needing local hardware like its siblings. Drop the Tauri desktop shell, installers and self-update entirely; the server and its web dashboard ship as a container, updated by pulling a new image tag. Link against the official prebuilt libduckdb instead of compiling it from source via the bundled feature, cutting image build time from 15+ minutes to about 3. Fix a real bug found along the way: as PID 1 in a container, the daemon ignored SIGTERM and was always killed after the stop timeout; it now shuts down gracefully. Also corrects the Pillar 1 beaconing score: the period estimate had a harmonic alias (a perfectly periodic 4h series resolved to 600s), and the old score formula flagged pure random traffic as suspicious (p95 = 0.839 on Poisson noise). The new formula was validated against 300 samples per scenario. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VAyr8d7HuAdUGB6aA5xmDD
This commit is contained in:
1 parent
1b974bb90d
commit
8a80504c45
72 files changed
+2524
-6384
No files matched your search
+75
-90
@@ -2,8 +2,8 @@
|
||||
|
||||
**English** · [Português](pt-BR/arquitetura.md)
|
||||
|
||||
Acauã is a single Rust product: one daemon/CLI, one shared core, one embedded
|
||||
web UI and one Tauri shell. The shipped runtime has no Python and no external
|
||||
Acauã is a single Rust product: one daemon/CLI, one shared core, and one
|
||||
embedded web dashboard. The shipped runtime has no Python and no external
|
||||
database. This document records the structural decisions and the six
|
||||
corrections applied to the original blueprint.
|
||||
|
||||
@@ -11,19 +11,14 @@ corrections applied to the original blueprint.
|
||||
|
||||
| Layer | Choice | Why |
|
||||
|---|---|---|
|
||||
| Core, CLI and daemon | Rust 2021 | One toolchain, memory safety, no runtime dependency |
|
||||
| Core, CLI and daemon | Rust 2021 (MSRV 1.88) | One toolchain, memory safety, no runtime dependency |
|
||||
| HTTP API | Axum + Tokio + Tower HTTP | Small async surface, easy to embed |
|
||||
| UI | Vanilla HTML/CSS/JS | No frontend build step, assets embedded in the binary |
|
||||
| Desktop | Tauri 2 | Native window over the same UI |
|
||||
| Analytical store | DuckDB + Parquet (optional `duckdb` feature) | Embedded, columnar, local-first |
|
||||
| Web Dashboard | Vanilla HTML/CSS/JS | No frontend build step, assets embedded in the binary via `rust-embed` |
|
||||
| Analytical store | DuckDB + Parquet | Mandatory embedded columnar store, linked to official prebuilt `libduckdb.so` |
|
||||
| Deployment | Docker / docker-compose | Collective SOC daemon, unprivileged non-root user, read-only rootfs |
|
||||
| Scientific validation | Python in `research/` only | astropy/scipy/networkx for fixtures, never shipped |
|
||||
|
||||
The workspace has two crates: `acaua-core` and `acaua-cli`. The GUI lives in
|
||||
`apps/gui`, **outside** the Cargo workspace, as in Guará and Sabiá. The spec
|
||||
mentioned `version.workspace = true` for the desktop manifest; keeping the GUI
|
||||
outside the workspace makes that impossible, so `check.sh` enforces SemVer
|
||||
equality across `Cargo.toml`, `acaua-cli/Cargo.toml`, the desktop manifest and
|
||||
`tauri.conf.json` instead. This is a deliberate deviation, not an oversight.
|
||||
The workspace has two crates: `acaua-core` and `acaua-cli`. Both use `version.workspace = true`.
|
||||
|
||||
## Components
|
||||
|
||||
@@ -35,20 +30,20 @@ Zeek logs ──► acaua-core::zeek ──► normalized events ──► stora
|
||||
├──► pillar3 (protocol anomalies)
|
||||
└──► pillar4 (fusion) ──► alerts + evidence
|
||||
│
|
||||
acaua-cli ──► axum API ──► embedded UI ◄── Tauri shell ─────────────┘
|
||||
acaua-cli (serve) ──► axum API ──► embedded web dashboard ──────────┘
|
||||
```
|
||||
|
||||
- `zeek`: bounded parser and path confinement. It must never follow a symlink
|
||||
outside the dataset root.
|
||||
- `storage`: DuckDB/Parquet writer with a seven-day TTL. Disabled by default so
|
||||
the skeleton does not require a C++ build.
|
||||
- `storage`: DuckDB in-memory engine over daily partitioned Parquet files with a
|
||||
seven-day TTL.
|
||||
- `pillar1`–`pillar4`: pure functions over normalized events. They return
|
||||
scores plus evidence, never only a float.
|
||||
- `fusion`: fixed-order, bounded combination. It must be symmetric and
|
||||
reproducible.
|
||||
- `doctor`: local environment checks; no network calls except an explicit
|
||||
update check.
|
||||
- `update`: Gitea Releases client for the CLI only.
|
||||
- `doctor`: local environment checks; reports prebuilt DuckDB linkage and
|
||||
containerized Zeek status.
|
||||
- `update`: checks for updates against the Gitea Releases API.
|
||||
|
||||
## The six corrections
|
||||
|
||||
@@ -58,117 +53,107 @@ Event timestamps are a one-dimensional point process: there is no `y` value to
|
||||
fit. The primary periodogram is therefore Rayleigh/Schuster:
|
||||
|
||||
```text
|
||||
P(f) = |sum_k exp(2*pi*i*f*t_k)|^2 / N
|
||||
P(T) = (2 / N) * [ (sum cos(2*pi*t_i / T))^2 + (sum sin(2*pi*t_i / T))^2 ]
|
||||
```
|
||||
|
||||
Generalized Lomb–Scargle is used only when a real `y` exists (for example bytes per bucket).
|
||||
This avoids inventing a dependent variable and keeps the null distribution
|
||||
honest.
|
||||
Generalized Lomb–Scargle is reserved for the future extension that evaluates a
|
||||
real response attribute (such as payload size or duration) against time.
|
||||
|
||||
### 2. Bowley skew with an explicit sentinel
|
||||
### 2. Guard the zero-dispersion case in MADM
|
||||
|
||||
The Bowley coefficient uses quartiles. When `Q3 == Q1`, the denominator is zero;
|
||||
the implementation returns a documented sentinel (`0.0` with an
|
||||
`degenerate_quartiles` evidence flag) instead of `NaN`, infinity or a panic. The
|
||||
sentinel is part of the alert record, not a silent fallback.
|
||||
A synthetic or unjittered beacon produces equal intervals. The median absolute
|
||||
deviation from the median (MADM) is then `0.0`. Dividing by `0.0` or treating it
|
||||
as a missing value is wrong: zero dispersion is the most suspicious outcome. The
|
||||
implementation guards this case with an explicit score assignment (`1.0`) and a
|
||||
documented sentinel.
|
||||
|
||||
### 3. Jensen–Shannon, not raw KL
|
||||
### 3. Replace Pearson skewness with Bowley skewness
|
||||
|
||||
KL diverges when the reference distribution has a zero where the observed
|
||||
distribution does not. Fusion uses Jensen–Shannon, which is symmetric, bounded
|
||||
and defined for zero-probability bins with a small documented smoothing floor.
|
||||
This makes the score comparable across pillars and safe for sparse protocol
|
||||
histograms.
|
||||
Sample skewness based on the third central moment is sensitive to single
|
||||
outliers and can diverge. Bowley skewness uses quartiles:
|
||||
|
||||
### 4. Persistent sessions get their own detector
|
||||
```text
|
||||
B = (Q3 + Q1 - 2*Q2) / (Q3 - Q1)
|
||||
```
|
||||
|
||||
A connection that stays open for hours never produces a short inter-arrival
|
||||
time, so a beaconing detector can miss it entirely. Pillar 2 scores duration,
|
||||
idle gaps and repeated long-lived flows directly. The two pillars may both fire;
|
||||
fusion treats that as separate evidence, not double counting of the same signal.
|
||||
It is bounded in `[-1.0, 1.0]` by definition and resistant to extreme outliers.
|
||||
When `Q3 == Q1` (concentrated data), the implementation assigns `0.0` with a
|
||||
sentinel flag.
|
||||
|
||||
### 5. JA4 license caution
|
||||
### 4. Isolate the Python research workspace
|
||||
|
||||
The client TLS JA4 fingerprint is usable. JA4S, JA4H, JA4X, JA4SSH and JA4L
|
||||
carry a non-OSI license and are **not** enabled by default. Any future use needs
|
||||
an explicit licensing decision and a documented opt-in. The default feature set
|
||||
must remain redistributable under MIT/Apache-2.0.
|
||||
`research/` has its own virtual environment and dependencies. It is excluded
|
||||
from the Cargo workspace, ignored by Docker builds and CI releases, and tested
|
||||
only when research validation scripts are invoked directly.
|
||||
|
||||
### 6. Determinism and per-alert JSON evidence
|
||||
### 5. Documented degraded mode for NetFlow/IPFIX
|
||||
|
||||
Every alert carries a JSON record: dataset hash, code version, pillar, inputs,
|
||||
intermediate values, sentinels, weights and final score. Reductions run in a
|
||||
fixed order; no hash-map iteration or thread scheduling may change a score.
|
||||
Given the same input and version, the evidence must be byte-identical.
|
||||
NetFlow v9 and IPFIX do not carry DNS query names, TLS SNI, certificate chains
|
||||
or HTTP headers. Any analysis that depends on protocol features is unavailable.
|
||||
Acauã documents this as a degraded mode and never claims parity between flow
|
||||
records and full Zeek logs.
|
||||
|
||||
## Data flow
|
||||
### 6. Single HTTP and RPC boundary
|
||||
|
||||
1. `acaua analyze <dir>` validates the directory, reads supported logs and
|
||||
normalizes events. In this skeleton it validates and returns a stub report.
|
||||
2. Normalized events are written to the local store (when the `duckdb` feature
|
||||
is enabled) with a seven-day TTL.
|
||||
3. Each pillar runs over a deterministic window and returns candidate scores
|
||||
with evidence.
|
||||
4. Fusion combines candidates, applies thresholds and writes alerts.
|
||||
5. `serve` exposes the API and the embedded UI; the Tauri shell can open the same
|
||||
UI against a local or remote daemon.
|
||||
The daemon exposes an Axum API on loopback by default. The embedded web UI
|
||||
communicates with the daemon exclusively via standard HTTP calls.
|
||||
|
||||
## API
|
||||
|
||||
| Method | Path | Purpose |
|
||||
The HTTP API is served by `acaua serve`:
|
||||
|
||||
| Method | Path | Meaning |
|
||||
|---|---|---|
|
||||
| GET | `/api/health` | Liveness and version |
|
||||
| GET | `/api/status` | Uptime, data directory, pillar state |
|
||||
| GET | `/api/status` | Uptime, data directory, dataset summary |
|
||||
| GET | `/api/alerts` | Paginated alerts with evidence |
|
||||
| POST | `/api/analyze` | Analyze a dataset path |
|
||||
| POST | `/api/update` | Check for a CLI update |
|
||||
| POST | `/api/analyze` | Ingest and analyze a dataset directory |
|
||||
| GET | `/api/update` | Check for newer releases |
|
||||
| GET | `/logo.png` | Embedded brand asset |
|
||||
| GET | `/*` | Embedded UI fallback |
|
||||
| GET | `/*` | Embedded UI fallback (`rust-embed`) |
|
||||
|
||||
`serve` binds to `127.0.0.1:8787` by default. There is no authentication layer in
|
||||
the skeleton; exposing it on a network interface requires an authenticating
|
||||
reverse proxy.
|
||||
`serve` binds to `127.0.0.1:8787` by default. Exposing it on an external network
|
||||
requires an authenticating reverse proxy.
|
||||
|
||||
## Storage
|
||||
|
||||
DuckDB is an **optional** dependency (`features = ["duckdb"]`, `bundled`). The
|
||||
default skeleton does not compile the C++ backend. When enabled, the schema is
|
||||
append-only normalized events plus an alerts table, with Parquet export and a
|
||||
seven-day retention job. The database file lives under the configured data
|
||||
directory and never outside it.
|
||||
DuckDB is a mandatory dependency linked dynamically to the official prebuilt
|
||||
`libduckdb.so` (version 1.5.6 pinned with SHA-256 in `scripts/fetch-libduckdb.sh`).
|
||||
Parquet, JSON and ICU extensions are statically built into the official library.
|
||||
The schema consists of daily partitioned Parquet files under the configured
|
||||
data directory, with in-memory DuckDB querying and a seven-day TTL.
|
||||
|
||||
## Updates
|
||||
|
||||
`update.rs` queries the Gitea Releases API for the `acaua` repository, selects
|
||||
the asset for the current platform, downloads over HTTPS with `curl`, verifies
|
||||
SHA-256 and replaces the CLI binary. It refuses to update a running GUI
|
||||
automatically; the desktop only reports that a new version exists. This mirrors
|
||||
the Sabiá updater and avoids a second update mechanism.
|
||||
|
||||
The Tauri updater plugin, its `latest.json` manifest, `updater.pubkey` and
|
||||
`createUpdaterArtifacts` are intentionally not used: they would add a second
|
||||
signing and distribution path, and the desktop must not self-update. The single
|
||||
CLI path is easier to audit and keeps release assets to tarballs plus SHA-256.
|
||||
Updates for Acauã are delivered via container images (`docker compose pull && docker compose up -d`).
|
||||
The `acaua update` command queries the Gitea Releases API for `ciberfauna/acaua` and
|
||||
notifies operators when a newer version tag is available. Binary self-overwrite
|
||||
(`update --apply`) is removed: in a collective SOC daemon running under container
|
||||
orchestration, updates must be controlled by image pulls rather than self-mutating
|
||||
processes.
|
||||
|
||||
## Rejected alternatives
|
||||
|
||||
An earlier blueprint proposed a Python service with FastAPI and ClickHouse.
|
||||
Both are rejected. A Python service would add a second runtime, packaging burden
|
||||
and a network boundary inside a local-first tool; ClickHouse would add an
|
||||
external server and operational cost for datasets that fit an embedded columnar
|
||||
store. The product is 100% Rust, with Python confined to `research/`.
|
||||
1. **Python service with FastAPI/ClickHouse:** Rejected. A Python runtime introduces
|
||||
heavy container footprints and unnecessary memory boundaries; ClickHouse adds an
|
||||
external server operational burden for datasets that fit an embedded columnar store.
|
||||
The product is 100% Rust.
|
||||
2. **Desktop application (Tauri / AppImage / NSIS):** Rejected. Unlike Guará (local
|
||||
raw disk forensics) and Sabiá (Wi-Fi monitor-mode hardware), Acauã has no local
|
||||
hardware dependencies: it is a multi-user SOC daemon consuming Zeek logs. A desktop
|
||||
wrapper adds maintenance and security surface without operational benefits.
|
||||
|
||||
## Deployment
|
||||
|
||||
- **Native:** `cargo build --release -p acaua-cli`, then `acaua serve`.
|
||||
- **Desktop:** Tauri bundles for Linux and Windows, built in CI.
|
||||
- **Docker:** a slim runtime image for lab use; see [docker.md](docker.md).
|
||||
- **Docker Compose (Official):** Multi-arch container image (`git.lucasrguerra.dev.br/ciberfauna/acaua`),
|
||||
running as unprivileged user `acaua` (UID 10001), read-only root filesystem, dropped capabilities.
|
||||
- **Native compilation:** `scripts/fetch-libduckdb.sh ~/.cache/acaua/duckdb` followed by `cargo build --release -p acaua-cli`.
|
||||
|
||||
## Failure behavior
|
||||
|
||||
Malformed input is skipped with a counted, bounded diagnostic. A missing log is
|
||||
not fatal. A corrupt store is reported by `doctor` and never silently recreated
|
||||
over existing data. A failed update leaves the current binary in place.
|
||||
over existing data.
|
||||
|
||||
## Versioning and compatibility
|
||||
|
||||
|
||||
Reference in new issue
Block a user