lucasrguerra 4143bf599e
CI / Format and vet (push) Successful in 1m21s
CI / Dependency audit (push) Successful in 47s
CI / Tests (Linux, race detector) (push) Successful in 2m26s
Release / Plan (reuse the published image?) (push) Successful in 4s
Release / ci (push) Skipped
Release / Build hub image (amd64) (push) Skipped
Release / Build probe image (amd64) (push) Skipped
Release / Build hub image (arm64) (push) Skipped
Release / Build probe image (arm64) (push) Skipped
Release / Publish tags (probe) (push) Successful in 21s
Release / Publish tags (hub) (push) Successful in 21s
Release / Gitea release (push) Successful in 4s
Merge pull request 'fix(egress): container map refreshed on start; no new_geo for anycast CDNs (2.2.2)' (#4) from fix/egress-attribution-anycast into main
2026-11-19 18:43:51 +00:00

Urutau

Urutau

Lightweight, always-on threat intelligence and privacy-friendly web analytics for self-hosted servers.
Go, single static binary · local-first · authorized use.

Terminal · Interface · Architecture · Configuration · Docker · Support · Migrating to 2.0 · Roadmap · Changelog

English · Português


Urutau overview page: headline sentence, KPI cards, activity by category timeline
The Overview page: hostile IPs, attack rate and activity by category (demo data).

Urutau live page on the 3D globe: animated arcs from attacker cities to the servers edge-1 and edge-2
Live: attacks as animated arcs on a 3D globe (demo data, reserved addresses only).

Urutau IP analysis page: verdict, score, critical exposure and recommended actions for one attacker
IP analysis: verdict, evidence and recommended actions for one address.

Urutau defense page with the second firewall section
Defense: the second firewall in observe mode, next to the host fail2ban.

Urutau outbound traffic page: verdict, beacons, scan bursts and mining from containers
Outbound traffic: scan bursts, beacons and mining from containers.

Urutau incidents page: tickets for outbound-traffic findings
Incidents: every outbound finding becomes a ticket you triage.

Urutau overview on a phone Urutau defense page on a phone
The phone layout: a tab bar replaces the side rail and tables become cards.

Status: stable, v2.0.0. See CHANGELOG.md for exactly what shipped.

Urutau is a small hub plus one ultra-light probe per server. The probe (a container of a few MB, no open port) reads the logs the server already writes and sends them to the hub, which answers one question for the whole fleet: who is attacking me right now? It classifies hostile activity, scores every source IP, keeps 30 days on disk, serves an embedded dashboard, a JSON API and an exportable blocklist, and can decide which attackers to block (a second firewall), watch what your own containers send out (outbound traffic analysis) and turn what it finds into tickets and an HTML report.

  • Reads what already exists, through probes: on each server the probe follows the Traefik access log (JSON, or the default text format with fewer fields), /var/log/auth.log (sshd) and /var/log/fail2ban.log. Two optional sources come from the host kit in deploy/host: the network probe log (new inbound connections to ports nobody listens on, each a port-scan event) and the outbound connection log. No agent on the network path, no packet capture, no telemetry.
  • Classifies: scanner, exploit, bruteforce, ssh-fail, ssh-probe, ban, port-scan, plus the benign ok, notfound and ssh-ok. Browsing a public code forge (/owner/repo/src/commit/<sha>/..., raw, blame, blob, tree paths) is not an attack; real probes such as /.env or /.git/config still are.
  • Scores IPs: every event adds weight to its source IP. A score of 10 or more is suspicious; hostile needs 50 or more and a strong signal (an attack tool, two or more attack categories, a ban, or a clear exploit attempt that was refused). Scanner and exploit probes that got a 2xx or 3xx answer weigh only 0.5. An IP with a successful SSH login is never hostile and never reaches the blocklist. Bans of the urutau jail (Urutau's own decisions) and of recidive do not count as evidence, so the system does not feed itself.
  • Second firewall (optional): URUTAU_ENFORCE=off|observe|on. The hub decides which attacking addresses to block and sends each decision to the probe of the host, which appends it to a file; the host fail2ban (jail urutau) applies them, so Urutau needs no privileges. observe shows what would be blocked. See Second firewall.
  • Outbound traffic analysis (optional): new outbound connections of the host and containers are checked for beaconing, mining, IRC and SMTP ports, contact with attackers or Tor relays, DNS outside the allowed resolvers, unauthorised scans and new countries. See Outbound traffic and incidents.
  • Incident management: every outbound finding becomes a ticket (INC-<n>) with status, severity, assignee, tags, verdict, comments and history.
  • HTML report: a printable A4 report (/api/v1/report?format=html): cover, executive summary in plain language, rule-based recommendations, threats, geography, errors and health, visitors, defense, second firewall and outbound traffic. The Reports page previews, downloads or prints it.
  • Tor: the official exit and relay lists are downloaded every 6 hours (URUTAU_TOR_AUTO, on by default); requests from Tor exits are marked tor, and a workload talking to a Tor relay is a high outbound finding. --tor-exits adds your own file.
  • Geolocation, offline: country, city and ASN for every IP from the free DB-IP Lite databases, downloaded and refreshed by Urutau itself, read by a built-in MMDB reader with no dependency. Nothing is looked up online.
  • Understands User-Agents: browser, operating system, device, bots and attack tools. Googlebot, Bingbot and Applebot are recognized by forward-confirmed reverse DNS whatever the User-Agent says.
  • Web analytics for legitimate traffic: unique visitors, pages, referrers, countries, browsers, devices, latency (p50 and p95) and bytes, with a world map, replacing your proxy's analytics. Separate Threats and Traffic pages.
  • Lightweight: Go standard library only, one static binary, a soft memory limit of 256 MiB, state in a single snapshot file. No database.
  • Live dashboard: embedded vanilla UI (English and Portuguese, light and dark, with a phone layout), updated through Server-Sent Events, with 23 pages in five groups (see Interface).
  • Blocklist: hostile, globally routable IPs as plain text or JSON, ready for fail2ban, ipset, nginx or a firewall. IPv6 is aggregated by /64 and exported as x:x:x:x::/64.
  • Probes and hub: one hub holds the data and the dashboard; each server runs urutau probe (image urutau-probe, under 15 MB, about 20 MB of memory), which dials out over HTTPS, spools to disk if the hub is away and loses nothing. The token of each probe is stored on the hub only as a hash, and the Sources page creates, edits and revokes probes. Coming from 1.x: docs/migration-2.0.md.

How it works

 on each server: urutau probe (no open port)
  Traefik access.log   auth.log (sshd)   fail2ban.log   probe.log*   egress.log*
          └────────────────────┬───────────────────────────────────────┘
                               ▼
           tail (rotation aware) ──► batches ──► disk spool ──► HTTPS push ─┐
                               ▲                                            │
          decisions.log ◄── validated ban/unban lines ◄── long-poll ◄─────┐ │
                │                                                         │ │
       host fail2ban (jail urutau)                                        │ │
                                                                          │ ▼
 on the hub: urutau serve                                                 │ one node per probe
         parse + classify ──► score per IP ──► state in memory ──► snapshot (/data)
                          │                            │            │
                          ▼                            │            ▼
                 second firewall* ─────────────────────┘      outbound analysis ──► incident tickets
                    ┌────────────────────────┼─────────────────────┐
                    ▼                        ▼                     ▼
              dashboard (SSE)         /api/v1/* JSON      blocklist.txt / .json
              (all servers)           HTML report

   * optional, fed by the host kit in deploy/host

Investigation

Beyond live numbers, Urutau remembers and explains. It keeps 30 days of state and an event registry on disk (<data-dir>/events/, capped by --events-max, 512 MB by default), so you can pick any period (twelve named ranges or a custom from/to), replay it, and open an IP analysis page with a verdict and its reasons, attack families, MITRE ATT&CK techniques, network type, cadence, sessions, campaigns and recommended actions, with a STIX 2.1 export. The registry holds attackers only: never the address of a legitimate visitor. See docs/architecture.md.

Second firewall

Optional (URUTAU_ENFORCE, default off; observe decides in memory only, on also sends the decisions to the probes, which write them to their URUTAU_ENFORCE_LOG). The host kit in deploy/host (install.sh --probe --enforce) installs the fail2ban filter, action and jail urutau, which apply the decisions; the ban only drops connections the address starts, so replies to the server's own connections keep flowing. A decision has a reason, among them:

  • a burst of attack events in 10 minutes (--enforce-threshold, default 5);
  • known-attacker: an address already hostile (or suspicious with score 100 or more) is banned on its first new malicious event;
  • slow attackers: 10 SSH failures (ssh-probe counts half) or 10 to 20 web scan/exploit events in 24 h or 7 days without a legitimate visit;
  • port-scan: 10 distinct unserved ports in 10 minutes;
  • neighbor-campaign: 3 or more distinct addresses of a /24 (IPv4) or /64 (IPv6) banned in the last 24 h make one attack event of another address of the prefix enough;
  • a sweep every 10 minutes bans addresses active in the last 24 h that nothing blocks yet, at most --enforce-max-per-min (default 60) decisions per minute.

It never blocks trusted addresses, private ranges, CDN (Cloudflare) and public resolver ranges, probes, the server itself, legitimate visitors, crawlers verified by forward-confirmed reverse DNS (Googlebot, Bingbot, Applebot) or addresses you released. The Defense page shows mode, recent decisions, coverage and releases; the Unban button (IPs, IP analysis and Defense) asks the host, through unban.log next to --enforce-log, to lift the block and can keep the address from being banned again for 1 h, 24 h or 7 days. With URUTAU_ENFORCE_SHARE=on a hub and its peers exchange decisions, applied only when the receiving server also saw the address attack. Coverage counts an attacker as protected when it is listed or banned by any jail.

Outbound traffic and incidents

Optional (URUTAU_EGRESS=auto|on|off; auto turns it on for a probe that sends the outbound log). The host kit logs only new TCP connection attempts through nftables, and a container map names the workloads. Urutau checks them for beaconing (regular conversations with one destination), mining, IRC and SMTP ports, contact with addresses that attacked the server or with Tor relays, DNS outside the allowed resolvers (URUTAU_EGRESS_RESOLVERS), unauthorised outbound scans and new countries. Workloads are defined by network or container name (URUTAU_EGRESS_WORKLOADS); a scanner workload (an authorised surface scanner) has only its scan-shaped traffic summarised, everything else is judged like any other workload. The servers themselves, peers and the host's own provider never raise findings, cloud metadata services polled by the host are infra, and regular connections to large CDNs and clouds are low severity, and they never raise a new country (their database country means nothing for anycast).

Every finding above info severity becomes a ticket INC-<n> (list, ticket page, manual tickets, CSV export, /api/v1/incidents). A ticket marked false_positive or benign can suppress findings with exactly its key; a resolved ticket reopens if the finding comes back.

Interface

Pages are grouped in a side rail (a tab bar on phones), with the view kept in the URL hash. A "?" button next to metrics and categories explains what they mean and what to do.

  • Monitor: Overview (headline numbers, timeline, top IPs; a period selector applies everywhere), Live (attacks as animated arcs on a flat map or a 3D globe, from every probe; each probe's location is set on the Sources page), Services (SLO, error budget, burn rate and Apdex per service).
  • Investigate: IPs (the scored table and the IP analysis page, #/ip/<ip>), Sessions, Campaigns, Replay (a period played back on the map), Event search, Incidents (tickets for outbound findings) and Outbound traffic.
  • Analyze: Threats, MITRE ATT&CK, Geography (country, state, city, #/geo/BR/Pernambuco/Recife), Evolution, Concentration, Survival (time to ban), Traffic (privacy-friendly analytics for legitimate visitors), Where errors come from (app, proxy, blocked, attack, bot, client, unknown) and Protocols and TLS.
  • Operate: Defense (ban and blocklist coverage, second firewall, Unban), Sources and hosts (probes, their log sources and the blocklist) and Integrity (health of sources, registry and snapshot).
  • Reports: the HTML report of a period.

On phones, plain data tables become stacked cards, so nothing scrolls sideways. See docs/interface.md for every page with screenshots and the HTTP routes.

Install

Two images from one Dockerfile: the hub (urutau) once, and the probe (urutau-probe) on every server you want to watch. With Docker Compose (recommended), first the hub:

cp .env.example .env
# set URUTAU_TOKEN (32+ characters, 128+ bits of entropy; `openssl rand -base64 48` works):
sed -i "s|^URUTAU_TOKEN=.*|URUTAU_TOKEN=$(openssl rand -base64 48 | tr -d '\n')|" .env
docker compose pull && docker compose up -d
docker compose exec urutau urutau probes create web1    # prints the probe token ONCE

Then on each monitored server (after sudo sh deploy/host/install.sh --probe, see docs/docker.md):

cd deploy/probe && cp .env.example .env
# set URUTAU_HUB=https://urutau.example.com  URUTAU_PROBE_NAME=web1  URUTAU_PROBE_TOKEN=urp_...
docker compose up -d

The dashboard answers at http://127.0.0.1:8790 and asks for the token; the Sources page shows the probe connected. The images are public: docker pull git.lucasrguerra.dev.br/ciberfauna/urutau:latest and .../urutau-probe:latest. Or build locally with docker compose up -d --build.

To compile from source (Go 1.26.11 or newer, no dependency to fetch):

go build -o urutau ./cmd/urutau
URUTAU_TOKEN=$(openssl rand -base64 48) ./urutau serve --data-dir ./data

Supported platform matrix for container images: linux/amd64 and linux/arm64. See docs/docker.md for a Coolify deployment.

Getting started

# Check the data directory and the probes
urutau doctor

# Run the hub (dashboard, API, probe ingestion)
urutau serve --bind 127.0.0.1:8790

# Register a probe and run it on the server to watch
urutau probes create web1
URUTAU_HUB=https://urutau.example.com URUTAU_PROBE_NAME=web1 URUTAU_PROBE_TOKEN=urp_... urutau probe

# Bring a 1.x data directory in, losing nothing
urutau import --from /backup/data --host old-server

See docs/cli.md for the command reference and docs/configuration.md for every URUTAU_* variable.

Noise and monitoring

Your own infrastructure produces events too. Two filters keep it out of the picture:

  • --trusted / URUTAU_TRUSTED: CIDRs of trusted origins (your office, your monitors, your other servers). Their events are dropped entirely. --ignore remains as an alias.
  • --monitor-ua / URUTAU_MONITOR_UA: a regular expression matching monitoring User-Agents (the default includes Uptime-Kuma). A request that matches and is benign is not counted as traffic; it feeds a separate "monitoring" counter. If the same request is a scanner, exploit or bruteforce, it counts as an attack, because a User-Agent is trivial to forge.
  • --monitor-from / URUTAU_MONITOR_FROM: CIDRs the monitors really come from. When set, the monitoring User-Agent is honored only from those networks. Without it, only ok answers are filtered (a notfound from a monitor User-Agent still counts), since anyone can forge the header.
  • --domains / URUTAU_DOMAINS: an allowlist of domains for the Traffic page (exact names or *.example.com). A domain only gets a slot after a 2xx answer, and inactive domains are pruned, so random Host headers cannot fill the analytics.

Geolocation and attribution

Urutau shows country, city and ASN for every IP using the free DB-IP Lite databases. It downloads them by itself into --geo-dir and keeps them updated (--geo-update); lookups are local, so no IP is ever sent to a third party.

IP geolocation data by DB-IP, licensed under CC BY 4.0.

The attribution is also shown in the dashboard. The state and city boundaries drawn on the Geography page come from Natural Earth (public domain). If you redistribute the databases or anything derived from them, keep it.

Privacy

The Traffic tab counts legitimate visitors without tracking them. A unique visitor is a hash computed with a salt that changes every day, so it cannot follow a person from one day to the next, and the IP address of a legitimate visitor is never persisted: not in the snapshot, not in a log, not in an export. IP addresses are kept only for activity classified as hostile, which is what a security tool exists to record. Without cookies, fingerprinting or third-party requests, this is designed to need no consent banner, but whether your use is lawful under the LGPD or the GDPR remains your decision as the controller. See SECURITY.md.

Using the blocklist

The blocklist needs the access token as a bearer token. By default it lists IPs with a score of 50 or more in the last 7 days.

curl -fsS -H "Authorization: Bearer $URUTAU_TOKEN" \
  "https://urutau.example/api/v1/blocklist.txt?min_score=50&range=7d"

fail2ban (ban every listed IP, refreshed hourly from cron):

curl -fsS -H "Authorization: Bearer $URUTAU_TOKEN" https://urutau.example/api/v1/blocklist.txt \
  | while read -r ip; do fail2ban-client set sshd banip "$ip" >/dev/null; done

ipset (loaded atomically through a temporary set; family=4 because hash:ip holds no IPv6 prefixes, use a hash:net family inet6 set for family=6):

ipset create urutau-new hash:ip -exist
ipset flush urutau-new
curl -fsS -H "Authorization: Bearer $URUTAU_TOKEN" https://urutau.example/api/v1/blocklist.txt?family=4 \
  | while read -r ip; do ipset add urutau-new "$ip" -exist; done
ipset create urutau hash:ip -exist
ipset swap urutau-new urutau
iptables -C INPUT -m set --match-set urutau src -j DROP 2>/dev/null \
  || iptables -I INPUT -m set --match-set urutau src -j DROP

nginx (a deny list, then nginx -s reload):

curl -fsS -H "Authorization: Bearer $URUTAU_TOKEN" https://urutau.example/api/v1/blocklist.txt \
  | sed 's/^/deny /; s/$/;/' > /etc/nginx/conf.d/urutau-deny.conf
nginx -t && nginx -s reload

The list only holds globally routable addresses, never anything in --ignore or --trusted, and an IPv6 attacker is listed as its /64 prefix (2001:db8:1:2::/64), which ipset hash:net, nginx deny and fail2ban accept. Review the list before automating it: a score is a heuristic, and an IP behind a shared NAT can be innocent.

Security

Urutau handles hostile text by design. Every field it reads from a log (paths, User-Agents, user names, hosts) was written by an attacker: it is length-limited, never executed and always escaped before it reaches the page. The dashboard and the API require a token (URUTAU_TOKEN, at least 32 characters, e.g. openssl rand -base64 48), the session cookie is HttpOnly and SameSite=Strict, failed logins are rate limited per IP, each probe token (stored only as a hash on the hub) opens only the three probe routes, and the probe itself opens no port and writes only lines that pass a strict format check. Put it behind a reverse proxy with TLS before exposing it. See SECURITY.md.

Authorized use

Urutau is defensive software. It only reads logs on servers you own or administer, and it never sends traffic to the IPs it lists. Use the blocklist responsibly, on infrastructure you are authorized to protect.

Development

go test -race ./...
go vet ./...
scripts/check.sh

See CONTRIBUTING.md and docs/architecture.md.

Known limits

  • Sources (read by the probes): Traefik access log (JSON or default text format), sshd lines from auth.log (ISO timestamps), fail2ban.log, and the optional port-probe and outbound connection logs written by the host kit.
  • The probe needs the hub reachable over HTTPS (plain HTTP only for private addresses); it spools up to URUTAU_PROBE_SPOOL_MAX (64 MiB) while the hub is away and drops the oldest batches beyond that, counting them.
  • State is kept for 30 days at most (--retention), with up to 20,000 tracked IPs; the event registry is capped by --events-max; incidents hold up to 5000 tickets.
  • The dashboard login is a single shared token; there are no user accounts, and a ticket author is a free-text name.
  • Scores and second firewall decisions are heuristics, not proof of malicious intent. Start with observe.
  • Outbound analysis and the second firewall need the host kit (nftables, rsyslog, fail2ban); without it those pages stay empty.

Where it's going

Summary of the ROADMAP:

Done (1.0.0) Log collection, classification and scoring, offline geolocation, User-Agent parsing, privacy-friendly web analytics, snapshot persistence, dashboard with SSE, token login, blocklist, federation, Docker image and CI/CD
Done (1.2.0) 30-day retention, any period, on-disk event registry, replay, IP analysis with verdict, MITRE ATT&CK, campaigns, STIX 2.1 and stable metrics
Done (1.4.0 to 1.6.0) Second firewall with Unban and decision sharing, outbound traffic analysis, incident management, HTML report, Tor lists, network probes, text-format Traefik logs, verified crawlers and forge browsing recognized, phone layout
Done (2.0.0) Hub and ultra-light probes (own image), HTTPS ingestion with disk spool, per-probe tokens, enforce decisions through the probes, urutau import for 1.x data
Next More log sources, webhook alerts, Prometheus metrics
Later IP reputation enrichment
Not planned Active scanning or counterattack, mandatory cloud services, telemetry

License

MIT or Apache-2.0, your choice.

S
Description
Lightweight, always-on threat intelligence for self-hosted servers
Readme
50 MiB
0 Stars 1 Watchers 0 Forks
Urutau 2.2.2
Latest
2026-11-19 18:50:56 +00:00
Languages
Go 55%
JavaScript 35.7%
CSS 7.8%
Shell 1.1%
HTML 0.2%
Other 0.1%