Security Log Collectors

Reference collectors · Canonical schema · JetStream

Overview

Collectors turn raw logs into canonical SecurityEvent messages. They publish to the embedded NATS JetStream bus (sec.events.>), where 1SEC modules subscribe only to the event types they care about.

This gives you an “adapter layer” that works in environments where you can’t deploy kernel agents, and it keeps ingestion auditable: collectors are simple, deterministic transformations.

Supported Collectors

nginx

HTTP access logs

Parse access logs into http_request events with route, status, latency, and user agent.

auth

SSH/PAM/auth logs

Parse auth.log style lines into auth_success/auth_failure and privilege_change events.

pfsense

Firewall logs

Parse filterlog into network_connection events with src/dst IP, port, and protocol.

json

CloudTrail / k8s audit

Read JSON-line sources and normalize into canonical events (with strict key mapping).

If you need a new collector, the “right” approach is to map it into the canonical schema instead of inventing a new per-source data model.

How It Works

Pipeline

1. Tail or read a log source (file, syslog stream, JSON lines)

2. Parse a record and extract minimal fields (IPs, username, path, status)

3. Classify into a canonical event type

4. Emit a SecurityEvent to sec.events.>

5. Modules subscribe and evaluate detections; alerts flow to enforcement

Canonical schema details live in Docs.

Examples

CLI: start collectors

# Tail nginx access logs
1sec collect nginx --log-path /var/log/nginx/access.log

# Tail SSH/PAM auth logs
1sec collect auth --log-path /var/log/auth.log

# Tail pfSense/OPNsense filterlog
1sec collect pfsense --log-path /var/log/filterlog.log

# Tail JSON-line logs (CloudTrail, k8s audit)
1sec collect json --log-path /var/log/cloudtrail.json

Sitemap-friendly internal links

/docs#collectors
/docs#event-schema
/event-schema
/syslog

Best Practices

Normalize first. Detect second. Keep collectors deterministic.
Emit stable event types; avoid “stringly-typed” Details keys.
Prefer syslog or append-only log files over ad-hoc HTTP ingestion.
Start with archive enabled if you need long retention.