Lucas RayanandClaude Sonnet 5
c64bbbf775
Pillar 1 v1.1 (shipped earlier) flagged benign browsing bursts as beaconing: on short captures, grid periods longer than the observed span never complete a cycle, so significance saturates to 1 and the score sticks at exactly 0.5. v1.2 caps the grid at a third of the observed span, requires a minimum number of completed cycles, and excludes protocols that are periodic by nature (NTP, DNS, mDNS, ICMP). Validated against 300 seeds per scenario: false positives on bursty traffic drop from the 290-300/300 that v1.1 produced to 0/300, with every beacon-detection scenario scoring unchanged. Add Pillar 2 (persistent/long-lived sessions, with Zeek's fragmented TCP sessions rejoined after a 5-minute gap), Pillar 3 (DNS tunneling via Shannon-entropy information volume per subdomain per hour; the Jensen-Shannon divergence was measured and does not separate tunnels from benign CDN subdomains, so it stays in the evidence at zero weight instead of in the score), and Pillar 4 (deterministic noisy-OR fusion across the three pillars). The fusion formula was validated against 8 labeled real-traffic vectors (IoT-23, CTU-Normal) with tolerance <= 1e-4. A known limitation is kept, not hidden: a benign Philips HUE bulb's strict periodic polling and a 21.9h session score it above three real C2 hosts. No weight or threshold was tuned to mask this; it is documented in the architecture docs instead. Also fixes real bugs found along the way: ingest_dir's partition count summed file names instead of distinct dates, DuckDB errors were counted as bad rows, /api/alerts always returned an empty list regardless of the last analysis, and the dashboard still claimed every pillar was unimplemented. Restores four design corrections that were missing from docs/architecture.md (KL to Jensen-Shannon, the persistent-session detector, the JA4 licensing boundary, and deterministic reduction order), and catalogs the public datasets used for validation (IoT-23, CTU-Normal, Mendeley DTQBC) with their licenses and SHA-256 checksums under research/validation/. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VAyr8d7HuAdUGB6aA5xmDD
146 lines
6.2 KiB
Markdown
146 lines
6.2 KiB
Markdown
<p align="center"><img src="logo.png" width="300" alt="Acauã"></p>
|
||
<h1 align="center">Acauã</h1>
|
||
<p align="center">Deterministic network threat hunting.<br>Zeek-first NDR research · local-first · authorized use.</p>
|
||
|
||
<p align="center"><a href="docs/cli.md">Terminal</a> · <a href="docs/interface.md">Interface</a> · <a href="docs/architecture.md">Architecture</a> · <a href="docs/docker.md">Docker</a> · <a href="docs/support.md">Support</a> · <a href="ROADMAP.md">Roadmap</a> · <a href="CHANGELOG.md">Changelog</a></p>
|
||
|
||
<p align="center"><b>English</b> · <a href="README.pt-BR.md">Português</a></p>
|
||
|
||
---
|
||
|
||
**Status: pre-release (initial version).** The project structure, documentation, CI,
|
||
Zeek ingestion for `conn.log` and `dns.log`, DuckDB/Parquet analytical storage, and
|
||
initial versions of detection pillars 1 to 4 are implemented and validated against
|
||
public traffic. Automatic continuous ingestion and cross-restart alert persistence are
|
||
planned for upcoming releases.
|
||
|
||
Acauã is a network threat hunting engine built around deterministic evidence,
|
||
not an opaque model. The goal is to answer not only "this looks suspicious" but
|
||
"these exact inputs, with these intermediate values, produced this score".
|
||
|
||
- **Deterministic by design:** fixed-order reductions, documented sentinels and
|
||
a JSON evidence record for every future alert.
|
||
- **Local-first:** the analytical store is embedded in the process; no external
|
||
database and no telemetry by default.
|
||
- **Zeek-first:** `dns.log`, `ssl.log`, `conn.log` and the protocol logs needed
|
||
for identity and lateral-movement analysis. NetFlow/IPFIX is a documented
|
||
degraded mode, never presented as equivalent. Suricata EVE JSON is an optional
|
||
secondary input.
|
||
- **Server daemon and web dashboard:** `acaua serve` runs continuous ingestion,
|
||
scheduling, the API and the embedded web UI, distributed 100% via Docker.
|
||
- **Research stays outside the product:** `research/` is an isolated Python
|
||
workspace for scientific validation and fixtures. It is never compiled,
|
||
packaged or released.
|
||
|
||
## What exists today
|
||
|
||
| Area | State |
|
||
|---|---|
|
||
| Rust workspace and CLI (`acaua`) | ✅ functional |
|
||
| Axum API with health, status, alerts and analyze routes | ✅ functional |
|
||
| Embedded web dashboard | ✅ UI served by daemon |
|
||
| Documentation and CI/release workflows | ✅ |
|
||
| Docker multi-arch image and compose deployment | ✅ |
|
||
| DuckDB analytical store | ✅ official prebuilt library linked |
|
||
| Four detection pillars (P1–P4) | ✅ initial version |
|
||
| Zeek log parser (`conn.log`, `dns.log`) | ✅ functional |
|
||
| Alert evidence and scoring | ✅ deterministic JSON + noisy-OR fusion |
|
||
| Published releases | ❌ none yet |
|
||
|
||
## The four pillars
|
||
|
||
All four pillars are implemented in their **initial version (v1)** and validated against synthetic fixtures and public captures:
|
||
|
||
1. **Beaconing (Pilar 1 v1.2):** periodicity on event timestamps with a Rayleigh/Schuster
|
||
periodogram bounded by the observed span, regularity significance gating, and infrastructure exclusions.
|
||
2. **Persistent sessions (Pilar 2 v1.0):** long TCP connections chained across Zeek inactivity timeouts (up to 3600s gap without new SYN), log-scale persistence score, and PCR/failed attempt tracking.
|
||
3. **Protocol anomalies (Pilar 3 v1.0):** deterministic information capacity (Shannon entropy $\times$ unique subdomain characters per UTC hour) in DNS queries to detect tunneling.
|
||
4. **Evidence fusion (Pilar 4 v1.0):** per-host weighted noisy-OR combination ($w_1=w_2=w_3=0.75$) with deterministic JSON evidence and ranking.
|
||
|
||
The design decisions behind these pillars, including the six corrections to the
|
||
original blueprint, are in **[docs/architecture.md](docs/architecture.md)**.
|
||
|
||
## Authorized use
|
||
|
||
Acauã is defensive software for networks you own or are explicitly authorized
|
||
to monitor. Capturing or analyzing third-party traffic without permission may
|
||
be illegal. The operator is responsible for scope, privacy and retention.
|
||
|
||
The daemon binds to `127.0.0.1` by default. The Docker Compose deployment
|
||
isolates the daemon in an unprivileged, read-only container with dropped capabilities.
|
||
|
||
## Install
|
||
|
||
Run via Docker Compose (recommended for operators):
|
||
|
||
```bash
|
||
cp .env.example .env
|
||
docker compose pull && docker compose up -d
|
||
```
|
||
|
||
Or build and run locally with Docker:
|
||
|
||
```bash
|
||
docker compose up -d --build
|
||
```
|
||
|
||
To compile from source natively (Rust 1.88+ required):
|
||
|
||
```bash
|
||
scripts/fetch-libduckdb.sh ~/.cache/acaua/duckdb
|
||
export DUCKDB_LIB_DIR=~/.cache/acaua/duckdb LD_LIBRARY_PATH=~/.cache/acaua/duckdb
|
||
cargo build --release -p acaua-cli
|
||
./target/release/acaua --help
|
||
```
|
||
|
||
Supported platform matrix for published container images: `linux/amd64` and `linux/arm64`.
|
||
|
||
## Getting started
|
||
|
||
```bash
|
||
acaua doctor
|
||
acaua serve --bind 127.0.0.1:8787 --data-dir .acaua
|
||
acaua analyze /var/log/zeek/current --json
|
||
acaua update
|
||
```
|
||
|
||
`analyze` currently validates the directory and returns a skeleton report; it
|
||
does not parse Zeek logs yet. `serve` exposes the stub API and the embedded web UI.
|
||
`update` checks for new versions and guides container image updates.
|
||
|
||
## Development
|
||
|
||
```bash
|
||
scripts/fetch-libduckdb.sh ~/.cache/acaua/duckdb
|
||
export DUCKDB_LIB_DIR=~/.cache/acaua/duckdb LD_LIBRARY_PATH=~/.cache/acaua/duckdb
|
||
cargo test --workspace
|
||
cargo clippy --workspace --all-targets
|
||
node scripts/check-ui-i18n.mjs
|
||
scripts/check.sh --full
|
||
```
|
||
|
||
See **[CONTRIBUTING.md](CONTRIBUTING.md)** and **[docs/architecture.md](docs/architecture.md)**.
|
||
|
||
## Known limits
|
||
|
||
- The detection engine, Zeek parser and storage layer are stubs.
|
||
- There is no published release image yet.
|
||
- NetFlow/IPFIX is a degraded input mode by design; it cannot replace the Zeek
|
||
protocol logs for every pillar.
|
||
- The project targets Linux container environments (`linux/amd64`, `linux/arm64`).
|
||
|
||
## Where it's going
|
||
|
||
Summary of the [ROADMAP](ROADMAP.md):
|
||
|
||
| | |
|
||
|---|---|
|
||
| **Done** | Workspace, CLI/API stubs, web dashboard, Docker containerization, CI/release workflows |
|
||
| **Next** | Pillar 1 and 2 implementations with deterministic fixtures; real Zeek parser |
|
||
| **Later** | Full Parquet partitioning, TTL pruning, pillar 3 and 4, evidence fusion |
|
||
| **Not planned** | Opaque AI scoring, a required cloud service, Python in the shipped runtime |
|
||
|
||
## License
|
||
|
||
[MIT](LICENSE-MIT) or [Apache-2.0](LICENSE-APACHE), your choice.
|