Normalization

Engines live

Normalization is the step that makes everything else possible. Its job is to take every encoding an attacker can hide behind and turn it into one canonical form, so a detection engine only ever has to recognize an attack once.

The problem it solves

These are all the same attack:

UNION SELECT
uNiOn SeLeCt
UN/**/ION SELECT
%55NION %53ELECT
ＵＮＩＯＮ (fullwidth Unicode)
UNION SELECT (HTML entity)

A signature WAF needs a rule for each disguise. Heimdall normalizes them to a single value first, so the SQL engine matches one thing.

Multi-pass convergence

Each Param's raw value (and the path) is normalized by a loop that keeps applying decoders until the output stops changing (bounded to a few passes so a decoding bomb cannot spin). Within each pass it applies, in order:

1. Percent-decoding — %XX, with optional + → space for query/form sources.
2. HTML entity decoding — the full named-entity table (~2,200 references), plus numeric and hex entities, two-rune entities, no-semicolon short entities, and the Windows-1252 remap.
3. Unicode folding (only if the value has non-ASCII) — NFKC normalization followed by a curated homoglyph / confusable fold that maps Cyrillic and Greek look-alikes back to ASCII (а → a).

After the loop it collapses whitespace and lowercases ASCII. The result is Param.Norm; the pre-normalized bytes remain in Param.Raw.

The loop is what defeats double-encoding: %2555 → %55 → U across passes. Folding happens inside the loop so that, e.g., a homoglyph that decodes into a fresh percent-escape gets decoded on the next pass too.

Engine-local normalization

Some transforms are attack-specific and would cause false positives if applied globally, so an engine asks for them itself. The clearest example is SQL-comment stripping (/* */ and --), which the SQL engine applies to defeat UN/**/ION without corrupting unrelated inputs.

Norm → Raw offset mapping

Detection runs in normalized space, but the Attack Explorer needs to highlight the attacker's original bytes. The normalizer can therefore run in a tracked mode that records a source-offset map, so a hit on a canonical span can be mapped back to the exact raw offset and length for evidence capture. This only runs for flagged requests, keeping the clean path allocation-free.

Guarantees (proved by fuzzing)

• Convergence — the multi-pass loop always reaches a fixed point within the pass budget.
• Idempotence — normalizing an already-normal value changes nothing.
• Bomb-resistance — nested/expanding encodings cannot cause runaway work or allocation.

Status

Normalization is the most complete engine in the system: percent/double-decode, case fold, NFKC + homoglyph, the full HTML5 entity table, SQL-comment stripping, whitespace collapse — all zero-allocation and fuzzed. Base64 detection is deliberately deferred (it is high false-positive as a global transform and is better applied per-endpoint later).