Detection Engineering
Log Pipeline & Normalisation
Raw log data from different sources cannot be correlated without normalisation.
What it is
A SIEM ingesting logs from Windows, Linux, cloud platforms, firewalls, and identity providers receives logs in dozens of different formats. Without normalisation — mapping source-specific field names to a common schema — correlation rules cannot operate across sources. A login event from Azure AD and a login event from a VPN appliance have different field names, different timestamp formats, and different event structures. Detection rules that require correlation across both will not fire without a normalisation layer.
Attack techniques
- Inconsistent field naming across log sources
- Timestamp timezone mismatches breaking correlation
- Missing enrichment (hostname to IP resolution)
- Raw log ingest without schema mapping
- Parser failures silently dropping log lines
Business impact
Correlation rules fail silently when log normalisation is broken. The SIEM appears to be working but cross-source detections never fire.