The Open Information Security Foundation (OISF) released Suricata 1.0 on 19 July 2010, an intrusion detection engine written in C with a multi-threaded architecture and compatible with the Snort rule format. It is the first reasonably widespread open source IDS designed from the start to spread analysis across multiple cores, after more than two years of development and contributions from over twenty-five people.

Context

For open source intrusion detection the reference is Snort, which has been running since the late 1990s. Its engine performs capture, decoding, stream reassembly and signature matching in a single thread. On a single-core machine this was no limitation; on hardware with four or eight cores, a single-threaded process saturates one core and leaves the others idle while traffic grows.

The OISF is a non-profit set up to build a next-generation IDS/IPS engine. Initial funding comes from the US Department of Homeland Security, from the US Navy’s Space and Naval Warfare Systems Command (SPAWAR), and from a few private companies in the OISF consortium, among them Endace, NitroSecurity and Everis. The project is part of the DHS Homeland Open Security Technology (HOST) programme. A first beta had come out over the turn of 31 December 2009 to 1 January 2010; 1.0 is the first release declared stable.

The code ships under the GPLv2 licence, and so does the HTP library the engine uses to analyse HTTP traffic.

Architecture

The basic idea is to break the analysis pipeline into modules that run as separate threads and communicate through queues. The same packet passes through a sequence of stages:

  • capture of packets from the network interface;
  • decoding of the lower layers of the stack;
  • the stream engine, which reassembles TCP segments into flows;
  • detection, where rule matching happens;
  • output, which writes alerts and logs.

Since there is more than one detection thread, different flows are inspected in parallel on different cores. Capture, decoding and matching no longer contend for the same thread, and the sustainable throughput on commodity multi-core hardware rises accordingly. How the modules are spread across threads is configurable, because the sweet spot depends on the core count and the traffic profile.

The second element of the architecture is automatic application-protocol recognition. Suricata inspects the start of a flow to work out which protocol it carries, regardless of the numeric port it travels on. An HTTP session on a non-standard port is recognised as HTTP and analysed with HTTP rules, which a system tied to port 80 would not do. This recognition feeds the application-layer parsers, in particular the HTTP analysis handled by the HTP library written by Ivan Ristic, author of ModSecurity.

The critical point

Compatibility with the Snort rule format is the choice that makes Suricata adoptable without rewriting the existing body of signatures. Suricata interprets Snort VRT rule syntax, handles the rule-language options and unified-format logging. Anyone who already has a calibrated signature set — including the public Emerging Threats collections — runs it on the new engine and compares the behaviour, without throwing away the tuning work.

On the same ground, compatibility is also the main constraint. Adopting Snort’s language means inheriting its expressivity limits: a rule is almost always a byte pattern, with little ability to describe relationships between separate events over time. Suricata begins to push past that boundary with the flowbits keyword family, which leaves it to one signature to set a state on the flow and to another to read it, building multi-step detection. It is a first move towards rules that express logic and not just string matches, yet it stays tied to a syntax born for another engine.

Implications

Port-independent protocol recognition changes how signatures are written and reasoned about. As long as detection stays tied to the port, an attacker who moves a service onto an unconventional port slips past rules that take the port-protocol correspondence for granted. By moving the decision from the port number to the content of the flow, Suricata reduces this class of evasions and makes possible rules that reason about protocol semantics rather than placement.

A dedicated HTTP parser, which normalises the flow before matching, has the same effect on application evasions: obfuscation techniques that break a pattern through encodings or fragmentation are brought back to a canonical form before the comparison against the rules. Normalisation is where an IDS gains or loses robustness, and isolating it in a separate, maintained HTTP library is an engineering choice worth following over time.

Operationally, the multi-threaded architecture moves the bottleneck. With a single-threaded engine the practical limit is the clock speed of one core; by distributing across threads the limit approaches the overall capacity of the machine and the quality of balancing across the detection threads. That is why an assessment on real hardware is worth running before drawing conclusions: the gain depends on how well the traffic spreads across flows.

Limits

Suricata 1.0 is a first stable release, and should be measured as such. Compatibility with Snort is stated for the rule format and unified logging, but rule-by-rule behavioural parity is to be verified on one’s own set, not taken for granted. Protocol recognition covers the protocols implemented in the parsers of this version, not an arbitrary set. And a multi-threaded architecture brings a coordination cost — queues, synchronisation, contention — that on modest loads may not pay off against a simpler engine. The right way to approach this release is to install it next to an existing Snort sensor, give it the same traffic and the same signatures, and read the numbers.

Cover image: Screenshot of a Snort IDS console in an Internet Explorer browser: tables listing alert counts by signature, network sources,… — logo by CJS ULE~commonswiki, CC BY-SA 3.0 — https://commons.wikimedia.org/wiki/File:Snort_ids_console.gif