feat: implement Pillars 2-4 and fix a production false-positive in Pillar 1
Pillar 1 v1.1 (shipped earlier) flagged benign browsing bursts as beaconing: on short captures, grid periods longer than the observed span never complete a cycle, so significance saturates to 1 and the score sticks at exactly 0.5. v1.2 caps the grid at a third of the observed span, requires a minimum number of completed cycles, and excludes protocols that are periodic by nature (NTP, DNS, mDNS, ICMP). Validated against 300 seeds per scenario: false positives on bursty traffic drop from the 290-300/300 that v1.1 produced to 0/300, with every beacon-detection scenario scoring unchanged. Add Pillar 2 (persistent/long-lived sessions, with Zeek's fragmented TCP sessions rejoined after a 5-minute gap), Pillar 3 (DNS tunneling via Shannon-entropy information volume per subdomain per hour; the Jensen-Shannon divergence was measured and does not separate tunnels from benign CDN subdomains, so it stays in the evidence at zero weight instead of in the score), and Pillar 4 (deterministic noisy-OR fusion across the three pillars). The fusion formula was validated against 8 labeled real-traffic vectors (IoT-23, CTU-Normal) with tolerance <= 1e-4. A known limitation is kept, not hidden: a benign Philips HUE bulb's strict periodic polling and a 21.9h session score it above three real C2 hosts. No weight or threshold was tuned to mask this; it is documented in the architecture docs instead. Also fixes real bugs found along the way: ingest_dir's partition count summed file names instead of distinct dates, DuckDB errors were counted as bad rows, /api/alerts always returned an empty list regardless of the last analysis, and the dashboard still claimed every pillar was unimplemented. Restores four design corrections that were missing from docs/architecture.md (KL to Jensen-Shannon, the persistent-session detector, the JA4 licensing boundary, and deterministic reduction order), and catalogs the public datasets used for validation (IoT-23, CTU-Normal, Mendeley DTQBC) with their licenses and SHA-256 checksums under research/validation/. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VAyr8d7HuAdUGB6aA5xmDD
This commit is contained in:
1 parent
8a80504c45
commit
c64bbbf775
41 files changed
+4034
-348
No files matched your search
@@ -65,6 +65,33 @@ async fn run(cli: Cli) -> acaua_core::Result<()> {
|
||||
for alert in &report.alerts {
|
||||
println!(" [{:.3}] {}", alert.score, alert.summary);
|
||||
}
|
||||
if !report.hosts.is_empty() {
|
||||
println!("hosts:");
|
||||
for host in report.hosts.iter().take(10) {
|
||||
let m1 = host
|
||||
.terms
|
||||
.iter()
|
||||
.find(|t| t.pillar == 1)
|
||||
.map(|t| t.m)
|
||||
.unwrap_or(0.0);
|
||||
let m2 = host
|
||||
.terms
|
||||
.iter()
|
||||
.find(|t| t.pillar == 2)
|
||||
.map(|t| t.m)
|
||||
.unwrap_or(0.0);
|
||||
let m3 = host
|
||||
.terms
|
||||
.iter()
|
||||
.find(|t| t.pillar == 3)
|
||||
.map(|t| t.m)
|
||||
.unwrap_or(0.0);
|
||||
println!(
|
||||
" [{:.3}] {:<18} {:.3}/{:.3}/{:.3}",
|
||||
host.score, host.orig_h, m1, m2, m3
|
||||
);
|
||||
}
|
||||
}
|
||||
println!("estado: {}", report.status);
|
||||
}
|
||||
Ok(())
|
||||
|
||||
@@ -27,6 +27,7 @@ struct Assets;
|
||||
#[derive(Clone)]
|
||||
struct AppState {
|
||||
storage: Arc<Storage>,
|
||||
last_report: Arc<std::sync::Mutex<Option<AnalysisReport>>>,
|
||||
}
|
||||
|
||||
#[derive(Serialize)]
|
||||
@@ -70,8 +71,13 @@ async fn status(State(state): State<AppState>) -> Json<Status> {
|
||||
})
|
||||
}
|
||||
|
||||
async fn alerts() -> Json<Vec<Alert>> {
|
||||
Json(Vec::new())
|
||||
async fn alerts(State(state): State<AppState>) -> Json<Vec<Alert>> {
|
||||
let report_lock = state.last_report.lock().unwrap();
|
||||
let list = report_lock
|
||||
.as_ref()
|
||||
.map(|r| r.alerts.clone())
|
||||
.unwrap_or_default();
|
||||
Json(list)
|
||||
}
|
||||
|
||||
async fn analyze(
|
||||
@@ -79,7 +85,12 @@ async fn analyze(
|
||||
Json(request): Json<AnalyzeRequest>,
|
||||
) -> std::result::Result<Json<AnalysisReport>, (StatusCode, Json<ErrorBody>)> {
|
||||
zeek::analyze_dir(&request.path, state.storage.root())
|
||||
.map(Json)
|
||||
.map(|report| {
|
||||
if let Ok(mut lock) = state.last_report.lock() {
|
||||
*lock = Some(report.clone());
|
||||
}
|
||||
Json(report)
|
||||
})
|
||||
.map_err(|error| {
|
||||
(
|
||||
StatusCode::BAD_REQUEST,
|
||||
@@ -148,6 +159,7 @@ pub async fn serve(bind: &str, data_dir: PathBuf) -> Result<()> {
|
||||
std::fs::create_dir_all(&data_dir)?;
|
||||
let state = AppState {
|
||||
storage: Arc::new(Storage::open(&data_dir)?),
|
||||
last_report: Arc::new(std::sync::Mutex::new(None)),
|
||||
};
|
||||
let app = Router::new()
|
||||
.route("/api/health", get(health))
|
||||
|
||||
Reference in new issue
Block a user