A detection rule written for a Security Information and Event Management (SIEM) platform is not portable: logic in Splunk Search Processing Language does not run on an Elasticsearch cluster, and an Ariel Query Language query for QRadar has to be rewritten for ArcSight. The same detection idea — “a whoami process spawned by a system service” — exists in as many implementations as there are products it has to run on. Sigma, published on GitHub by Florian Roth and Thomas Patzke, proposes writing that logic once in YAML and leaving the translation to whichever search engine is in play to a compiler.

Context and problem

Detection content outlives the product that runs it. An organisation swaps SIEM for cost or contractual reasons, a Managed Security Service Provider (MSSP) has to serve clients on different stacks, a public threat report describes behaviour observable on any Windows endpoint. In all of these the investigative knowledge — which fields to look at, which values to correlate — stays the same, yet it remains trapped in the syntax of a single engine.

The authors explain it by analogy with tools already in place: Snort describes network traffic, YARA describes files, and an equivalent for log events was missing. Sigma fills that slot with a textual, readable, version-controllable format that describes what to look for and leaves the how of translating it to another layer.

Format architecture

A Sigma rule is a YAML document with few sections. The identifying part holds title, a description, the author and references to the public material the rule derives from. The status field flags whether the rule is experimental or stable.

The real work sits in two sections: logsource and detection.

logsource declares the source: which product (product: windows), which service (service: security) or event category (category: process_creation). It is an abstract taxonomy, not the field names of the final SIEM.

detection defines one or more selection blocks and a boolean condition that combines them. A minimal example:

logsource:
    category: process_creation
    product: windows
detection:
    selection:
        Image|endswith: '\whoami.exe'
        ParentImage|endswith: '\services.exe'
    condition: selection

The field names (Image, ParentImage) are those of the normalised telemetry — for Windows, the record model produced by Sysmon. The rule knows nothing of how that field is named in the target SIEM’s schema.

The sigmac converter

Translation is done by sigmac, the Python script bundled with the repository. It takes a rule, a target backend and a configuration that bridges the rule’s abstract names to the real names in the installation:

sigmac -t splunk -c splunk-windows rule.yml

The configuration file (splunk-windows in the example) holds the field mapping: it states that Image, in the Splunk-Windows context, corresponds to a given indexed field name, and so on for the rest. It is the point where an abstract format meets the reality of a concrete deployment, and what determines whether the output will be usable at all.

The backends available today cover the more widespread engines: Splunk, Elasticsearch (both query strings and Query DSL), Kibana, Logpoint, QRadar, ArcSight, plus utility targets such as grep for quick testing. Each backend is a Python class that renders the condition structure into native syntax.

Critical point: the quality lives in the mapping

The YAML format is the easy, visible part. Whether Sigma actually works in production is decided by the mapping layer between the abstract logsource and the real schema. A correct rule translated with the wrong configuration produces a syntactically valid query that matches nothing — or, worse, one that matches everything.

Two dependencies make the problem concrete. The first is upstream log normalisation: if the telemetry does not collect the fields the rule assumes (for Windows, a Sysmon configuration logging process_creation with the full command line), the rule has nothing to act on. The second is the correctness of the field mapping for that SIEM: every deployment indexes fields under its own names, and the mapping has to be kept aligned with the actual schema.

In practice Sigma shifts the effort from rewriting logic to maintaining backend configurations. It pays off when there are many rules and few environments; it pays off less when each environment has a log schema of its own and nobody maintains the mapping.

Operational implications

For anyone managing detection content, the format enables a couple of concrete practices. Rules become text artefacts: they live in a Git repository, get reviewed through a pull request, and are versioned. A public report on a campaign can ship with a Sigma rule that anyone applies to their own stack, whatever the product.

Reference to catalogued attack techniques — the MITRE ATT&CK model, public since 2015 — can be annotated in the rule’s metadata, and gives a common vocabulary to reason about what a rule intends to cover, beyond the details of the log.

Limits

Sigma describes conditions on single events and simple correlations, expressed as boolean combinations of selections. Detection logic that needs state, extended temporal sequences or statistical aggregation stays outside the model: it has to be expressed natively in the target engine, outside Sigma.

The project is young. Backend coverage is uneven — some engines translate the full range of conditions, others only a subset — and mapping configurations for real environments fall largely on whoever adopts the tool. The value of Sigma grows with the size of the shared rule library and with the care put into the configuration files; and neither grows on its own.

Cover image: Two operators seated in front of a wall of monitors in a Security Operations Center, with network dashboards and charts on the screens — photo by UMD-Eskin, public domain — https://commons.wikimedia.org/wiki/File:SOC_Security_Monitors.jpg