NormalizerMatch

Defined in: packages/engine/src/normalizer/NormalizerRule.ts:60

Result of a successful rule match attempt against the token stream.

When a NormalizerRule.match function finds a pattern at the current position, it returns a NormalizerMatch describing how many tokens to consume and what to replace them with.

Example

// The phrase "to the power of" (5 tokens) becomes a single CARET token
const match: NormalizerMatch = {
  consumed: 5,
  replacement: [caretToken],
};

Properties

consumed

consumed: number;

Defined in: packages/engine/src/normalizer/NormalizerRule.ts:65

Number of tokens consumed from the stream at the match position. Must be ≥ 1, a match always advances the cursor.

---

replacement

replacement: Token[];

Defined in: packages/engine/src/normalizer/NormalizerRule.ts:72

Replacement tokens to insert at the match position. May be empty (deletion), a single token (fusion), or multiple tokens (expansion/splitting).

---

ruleName?

optional ruleName?: string;

Defined in: packages/engine/src/normalizer/NormalizerRule.ts:80

Human-readable rule name for diagnostic fusion tracking. When set, the normalizer uses this instead of the rule’s name in TokenFusion records. Used by PhraseTrie to report which specific phrase matched (e.g., “phrase:to the power of”).