feat: implement Pillars 2-4 and fix a production false-positive in Pillar 1

Pillar 1 v1.1 (shipped earlier) flagged benign browsing bursts as
beaconing: on short captures, grid periods longer than the observed
span never complete a cycle, so significance saturates to 1 and the
score sticks at exactly 0.5. v1.2 caps the grid at a third of the
observed span, requires a minimum number of completed cycles, and
excludes protocols that are periodic by nature (NTP, DNS, mDNS,
ICMP). Validated against 300 seeds per scenario: false positives on
bursty traffic drop from the 290-300/300 that v1.1 produced to 0/300,
with every beacon-detection scenario scoring unchanged.

Add Pillar 2 (persistent/long-lived sessions, with Zeek's fragmented
TCP sessions rejoined after a 5-minute gap), Pillar 3 (DNS tunneling
via Shannon-entropy information volume per subdomain per hour; the
Jensen-Shannon divergence was measured and does not separate tunnels
from benign CDN subdomains, so it stays in the evidence at zero
weight instead of in the score), and Pillar 4 (deterministic
noisy-OR fusion across the three pillars).

The fusion formula was validated against 8 labeled real-traffic
vectors (IoT-23, CTU-Normal) with tolerance <= 1e-4. A known
limitation is kept, not hidden: a benign Philips HUE bulb's strict
periodic polling and a 21.9h session score it above three real C2
hosts. No weight or threshold was tuned to mask this; it is
documented in the architecture docs instead.

Also fixes real bugs found along the way: ingest_dir's partition
count summed file names instead of distinct dates, DuckDB errors
were counted as bad rows, /api/alerts always returned an empty list
regardless of the last analysis, and the dashboard still claimed
every pillar was unimplemented.

Restores four design corrections that were missing from
docs/architecture.md (KL to Jensen-Shannon, the persistent-session
detector, the JA4 licensing boundary, and deterministic reduction
order), and catalogs the public datasets used for validation
(IoT-23, CTU-Normal, Mendeley DTQBC) with their licenses and SHA-256
checksums under research/validation/.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VAyr8d7HuAdUGB6aA5xmDD
This commit is contained in:
Lucas RayanandClaude Sonnet 5 committed 2026-11-01 00:28:54 -03:00
1 parent 8a80504c45
commit c64bbbf775
41 files changed
+4034 -348

No files matched your search

+27
View File
@@ -65,6 +65,33 @@ async fn run(cli: Cli) -> acaua_core::Result<()> {
for alert in &report.alerts {
println!(" [{:.3}] {}", alert.score, alert.summary);
}
if !report.hosts.is_empty() {
println!("hosts:");
for host in report.hosts.iter().take(10) {
let m1 = host
.terms
.iter()
.find(|t| t.pillar == 1)
.map(|t| t.m)
.unwrap_or(0.0);
let m2 = host
.terms
.iter()
.find(|t| t.pillar == 2)
.map(|t| t.m)
.unwrap_or(0.0);
let m3 = host
.terms
.iter()
.find(|t| t.pillar == 3)
.map(|t| t.m)
.unwrap_or(0.0);
println!(
" [{:.3}] {:<18} {:.3}/{:.3}/{:.3}",
host.score, host.orig_h, m1, m2, m3
);
}
}
println!("estado: {}", report.status);
}
Ok(())
+15 -3
View File
@@ -27,6 +27,7 @@ struct Assets;
#[derive(Clone)]
struct AppState {
storage: Arc<Storage>,
last_report: Arc<std::sync::Mutex<Option<AnalysisReport>>>,
}
#[derive(Serialize)]
@@ -70,8 +71,13 @@ async fn status(State(state): State<AppState>) -> Json<Status> {
})
}
async fn alerts() -> Json<Vec<Alert>> {
Json(Vec::new())
async fn alerts(State(state): State<AppState>) -> Json<Vec<Alert>> {
let report_lock = state.last_report.lock().unwrap();
let list = report_lock
.as_ref()
.map(|r| r.alerts.clone())
.unwrap_or_default();
Json(list)
}
async fn analyze(
@@ -79,7 +85,12 @@ async fn analyze(
Json(request): Json<AnalyzeRequest>,
) -> std::result::Result<Json<AnalysisReport>, (StatusCode, Json<ErrorBody>)> {
zeek::analyze_dir(&request.path, state.storage.root())
.map(Json)
.map(|report| {
if let Ok(mut lock) = state.last_report.lock() {
*lock = Some(report.clone());
}
Json(report)
})
.map_err(|error| {
(
StatusCode::BAD_REQUEST,
@@ -148,6 +159,7 @@ pub async fn serve(bind: &str, data_dir: PathBuf) -> Result<()> {
std::fs::create_dir_all(&data_dir)?;
let state = AppState {
storage: Arc::new(Storage::open(&data_dir)?),
last_report: Arc::new(std::sync::Mutex::new(None)),
};
let app = Router::new()
.route("/api/health", get(health))