How Cernity works: from packet to investigation

Suricata sees traffic and writes observations. A shipper sends those observations to Cernity over a Kafka-compatible bus. Central services normalize them, detect on them, and produce findings — which export to your SIEM and surface for analysts in Cernity Vantage. Each step is a separate service with a separate responsibility.

You do not point the SIEM at a packet capture interface. You do not need Zeek on every sensor for the core Cernity pipeline. And installing Cernity centrally does not automatically collect the remote sensor’s files.

Follow one connection through the system

MONITORED NETWORK
A client connects to a server
        │ copied traffic from a TAP / SPAN / virtual mirror
        ▼
SENSOR HOST
Suricata: captures packets, writes local EVE JSON
Rolling packet buffer: bounded, budgeted pre-trigger capture
        │ eve-alerts.json + eve-nsm.json   +   sensor-health.v1 heartbeats
        ▼
Fluent Bit: tails the files, parses JSON, adds sensor identity
        │ outgoing Kafka-protocol connection (SASL/SCRAM + TLS, TCP 19092)
        ▼
CENTRAL CERNITY HOST
Redpanda: holds events in topics for consumers
        ▼
Normalizer: one canonical observation.v1 view over typed records
        ▼
Detectors: compare observations, produce candidate findings
        ▼
finding-service: lifecycle, consolidation, evidence, delivery decisions
        │                                   ▲
        ▼                                   │ disposition.v1 (tuning loop)
findings-forwarder ─────► YOUR SIEM         │
        │                                   │
        └──────────► Cernity Vantage ───────┘
                     queue · Overview · fleet-health · evidence pivots · cases

The arrows from Suricata to Fluent Bit are local file reads. The next arrow is a network connection initiated by the sensor. Delivery to the SIEM is initiated by Cernity. Vantage reads the finding and evidence stores. These are not one shared connection.

1. Packet visibility comes first

Suricata can analyze only the packets it receives. A mirror on an internet uplink can see routed egress but miss two internal machines talking through a switch. An internal fan-out detector cannot reconstruct traffic that never reached the sensor.

Encrypted traffic often still exposes addresses, ports, sizes, and some handshake metadata. It does not automatically expose HTTP paths or transferred files. TLS logging is not decryption.

A monitoring sensor typically has a capture interface for mirrored traffic and a management interface with a routable address. The management path carries telemetry to Cernity. See sensor dependencies before selecting a capture point.

2. Suricata creates the observations

EVE is Suricata’s JSON logging format. One line can be a flow summary, an alert, a DNS transaction, or another observation. The supplied shipper reads two configured files:

• eve-alerts.json contains alert events.
• eve-nsm.json contains flow, protocol, file metadata, anomaly, and statistics events you enable.

Splitting is a convention used by the supplied configuration, not a requirement of all Suricata installations. Existing deployments may have one eve.json; the logger and shipper paths must agree. Merely changing a filename does not enable a protocol logger.

3. The sensor also reports its own health

Alongside EVE, each sensor emits sensor-health.v1 heartbeats — CPU and memory, capture drops, EVE rates, ingest backlog, disk, connectivity, running versions, and clock offset. The central sensor registry records each sensor’s identity, tenant, site, tags, groups, and capabilities, and a read-only fleet API reports last-seen state and flags clock skew. This is how the SOC knows a sensor is healthy — or going blind — before it trusts what the sensor reported. See fleet management.

4. Fluent Bit moves the data off the sensor

Fluent Bit runs on the host with access to the Suricata log directory. It opens the EVE files read-only, follows appended lines, parses each line as JSON, and applies route.lua.

The routing function stamps tenant and sensor_id from configured environment variables, chooses a topic based on event_type, and uses the source IP as the Kafka message key when available. A topic is a named stream of messages. A partition is an ordered lane within that stream. Ordering within a partition is not global ordering across all sensors or topics.

The shipper connects to CENTRAL_HOST:19092. Suricata itself is not making this Kafka connection. The central host does not SSH into the sensor to pull files. The sensor transport guide traces the exact paths, credentials, and troubleshooting checks.

5. Redpanda separates collection from analysis

Redpanda is the Kafka-compatible message broker. It stores messages so producers and consumers do not have to run in lockstep. Consumers track offsets, meaning their position in each stream.

Notice that alerts go to suricata.raw.v1, not an assumed suricata.alert.v1. The file name and the destination topic are different concepts.

6. The normalizer builds the evidence plane

A normalizer turns the per-source events into one canonical observation.v1 record, exposed as a unified view over typed records — connection, DNS, TLS, HTTP. Each record carries capability flags describing exactly what its source could observe, so an evidence-poor source never fabricates a field. The result is a single, queryable evidence plane that a finding can point back into. See the evidence plane.

7. Detectors ask questions across observations

A detector may ask whether one host repeatedly connects at regular intervals, whether a source reaches many internal targets, or whether outbound volume departs from an entity’s baseline and its peer cohort. It emits a candidate finding, not a verdict from a human investigator. The native anomaly detector attaches its full explanation — features, both baselines, cohort, model and version — to every anomaly candidate.

One detector is the threat-intel service, which matches active indicators against observed traffic live — on IP, domain, URL, JA3, JA4, and certificate — and attaches the matched indicator and its provenance to the finding. The same managed indicator lifecycle also supports retrospective hunting, replaying indicators against the stored observation history. See threat intelligence.

The core services also promote existing Suricata signature alerts. That path adds lifecycle and consolidation; it must not be described as a new detection if the signature already identified the activity.

8. Findings have a lifecycle

finding-service receives candidates on ndr.finding.candidate.v1. Depending on the finding, it can score, suppress, finalize, or request extra evidence — including a retroactive packet capture from the sensor’s rolling buffer. Deliverable findings reach ndr.finding.final.v1.

A finding can be final for delivery while enrichment remains pending. A later revision may report a timeout. Neither state erases the original observation. Consolidation is scoped to finding identity and revision; a repeated behavior can still produce multiple identities or findings from different detectors, as the beacon and fan-out examples show.

9. The forwarder delivers to the SIEM

The forwarder consumes final findings, excludes suppressed findings from analyst delivery, and applies the selected adapter. JSON-based sinks preserve a richer document than the compact CEF mapping. Cernity does not build your SIEM dashboards, configure its index mappings, or retain every raw flow there. See SIEM delivery and record formats.

10. Analysts work in Vantage, and their decisions tune the pipeline

Cernity Vantage is the native console — a queue with triage tiers, a benign allowlist, an Overview health dashboard, fleet-health, and evidence pivots from a finding into its normalized observations. Analysts also work multi-finding investigations as cases (ownership, a new → investigating → resolved → closed status, notes, and linked findings/entities, fully audited), and Vantage surfaces ATT&CK coverage and production detection-quality views computed from analyst dispositions. Those dispositions and allowlist decisions are published as disposition.v1 and consumed by a feedback-service that maintains anomaly ground-truth, false-positive suppression, and an ignore-list. Every entry is owner-attributed, justified, expiring, audited, and applied as a suggestion — never a silent automatic change. See the analyst workflow.

Where Zeek fits

The Suricata-derived baseline plus central derivations is the default. Continuous Zeek is an optional mode, not a prerequisite. The optional packet-forensics path requires capture orchestration centrally, an agent able to access the sensor’s capture output, reachable object storage, and a central Zeek worker. Treat the overlay as a separate integration project, not a prerequisite for understanding basic findings.

What the basic deployment does not install

The central core includes Redpanda, the normalizer, the detector services, finding-service, and findings-forwarder. Source availability does not make every optional capability active. A service must be running, subscribed to the appropriate input, supplied with the fields it needs, and able to deliver its output.

Implementation references

This walkthrough was checked against Cernity revision 18174b8. Inspect the central Compose file, sensor bundle, and routing function.

← All documentation