Lucas RayanandClaude Sonnet 5
c64bbbf775
Pillar 1 v1.1 (shipped earlier) flagged benign browsing bursts as beaconing: on short captures, grid periods longer than the observed span never complete a cycle, so significance saturates to 1 and the score sticks at exactly 0.5. v1.2 caps the grid at a third of the observed span, requires a minimum number of completed cycles, and excludes protocols that are periodic by nature (NTP, DNS, mDNS, ICMP). Validated against 300 seeds per scenario: false positives on bursty traffic drop from the 290-300/300 that v1.1 produced to 0/300, with every beacon-detection scenario scoring unchanged. Add Pillar 2 (persistent/long-lived sessions, with Zeek's fragmented TCP sessions rejoined after a 5-minute gap), Pillar 3 (DNS tunneling via Shannon-entropy information volume per subdomain per hour; the Jensen-Shannon divergence was measured and does not separate tunnels from benign CDN subdomains, so it stays in the evidence at zero weight instead of in the score), and Pillar 4 (deterministic noisy-OR fusion across the three pillars). The fusion formula was validated against 8 labeled real-traffic vectors (IoT-23, CTU-Normal) with tolerance <= 1e-4. A known limitation is kept, not hidden: a benign Philips HUE bulb's strict periodic polling and a 21.9h session score it above three real C2 hosts. No weight or threshold was tuned to mask this; it is documented in the architecture docs instead. Also fixes real bugs found along the way: ingest_dir's partition count summed file names instead of distinct dates, DuckDB errors were counted as bad rows, /api/alerts always returned an empty list regardless of the last analysis, and the dashboard still claimed every pillar was unimplemented. Restores four design corrections that were missing from docs/architecture.md (KL to Jensen-Shannon, the persistent-session detector, the JA4 licensing boundary, and deterministic reduction order), and catalogs the public datasets used for validation (IoT-23, CTU-Normal, Mendeley DTQBC) with their licenses and SHA-256 checksums under research/validation/. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VAyr8d7HuAdUGB6aA5xmDD
36 lines
1.7 KiB
Python
36 lines
1.7 KiB
Python
# Writes a pcap: one TCP session 10.0.0.5:40000 -> 203.0.113.9:443 with a 600 s idle gap
|
|
# (longer than Zeek's tcp_inactivity_timeout = 5 min), then a fresh connection from a new port.
|
|
import struct, sys
|
|
out = open(sys.argv[1], 'wb')
|
|
out.write(struct.pack('<IHHiIII', 0xa1b2c3d4, 2, 4, 0, 0, 65535, 1))
|
|
def ip(a): return bytes(int(x) for x in a.split('.'))
|
|
def pkt(t, src, sp, dst, dp, flags, seq, ack, payload=b''):
|
|
tcp = struct.pack('!HHIIBBHHH', sp, dp, seq, ack, 5 << 4, flags, 65535, 0, 0) + payload
|
|
iph = struct.pack('!BBHHHBBH4s4s', 0x45, 0, 20 + len(tcp), 1, 0, 64, 6, 0, ip(src), ip(dst))
|
|
frame = b'\x00\x11\x22\x33\x44\x55' + b'\x66\x77\x88\x99\xaa\xbb' + b'\x08\x00' + iph + tcp
|
|
sec = int(t); usec = int(round((t - sec) * 1e6))
|
|
out.write(struct.pack('<IIII', sec, usec, len(frame), len(frame)) + frame)
|
|
SYN, ACK, PSH, FIN = 0x02, 0x10, 0x08, 0x01
|
|
C, S = '10.0.0.5', '203.0.113.9'
|
|
t0 = 1793400000.0
|
|
cs, ss = 1000, 5000
|
|
pkt(t0, C, 40000, S, 443, SYN, cs, 0); cs += 1
|
|
pkt(t0+0.05, S, 443, C, 40000, SYN|ACK, ss, cs); ss += 1
|
|
pkt(t0+0.10, C, 40000, S, 443, ACK, cs, ss)
|
|
# heartbeat every 60 s for 30 min (stays alive)
|
|
t = t0 + 1
|
|
for i in range(30):
|
|
pkt(t, C, 40000, S, 443, PSH|ACK, cs, ss, b'x'*20); cs += 20
|
|
pkt(t+0.05, S, 443, C, 40000, PSH|ACK, ss, cs, b'y'*40); ss += 40
|
|
t += 60
|
|
# idle 600 s, then resume heartbeats for 30 min more, then FIN
|
|
t += 600
|
|
for i in range(30):
|
|
pkt(t, C, 40000, S, 443, PSH|ACK, cs, ss, b'x'*20); cs += 20
|
|
pkt(t+0.05, S, 443, C, 40000, PSH|ACK, ss, cs, b'y'*40); ss += 40
|
|
t += 60
|
|
pkt(t, C, 40000, S, 443, FIN|ACK, cs, ss); cs += 1
|
|
pkt(t+0.05, S, 443, C, 40000, FIN|ACK, ss, cs); ss += 1
|
|
pkt(t+0.10, C, 40000, S, 443, ACK, cs, ss)
|
|
out.close()
|