NormalizerRule

Defined in: packages/engine/src/normalizer/NormalizerRule.ts:125

A pluggable normalization rule registered with the TokenNormalizer.

Each rule has a name, priority, and match function. The match function receives the current token stream and a position, and returns a NormalizerMatch on success or null on failure.

Priority ordering

Higher priority rules are tried first at each position. This allows long phrases (priority 100, e.g. “to the power of”) to match before shorter fragments (priority 80, e.g. “power of”).

Match contract

• Must be pure (no side effects, no mutation of input tokens)
• Must return null for any position that doesn’t match
• Consumed tokens must be consecutive starting at pos
• Replacement tokens must be valid for downstream parsing

Example

// A phrase fusion rule that converts "to the power of" into CARET
const phraseRule: NormalizerRule = {
  name: 'phrase:to the power of',
  priority: 100,
  match: (tokens, pos) => {
    if (pos + 4 > tokens.length) return null;
    const phrase = tokens.slice(pos, pos + 5)
      .map(t => t.value.toLowerCase()).join(' ');
    if (phrase === 'to the power of') {
      return {
        consumed: 5,
        replacement: [createFusedToken('CARET', 'to the power of', tokens.slice(pos, pos + 5))],
      };
    }
    return null;
  },
};

Properties

name

readonly name: string;

Defined in: packages/engine/src/normalizer/NormalizerRule.ts:130

Human-readable name for debugging and diagnostic display. Convention: "category:description", e.g. "phrase:to the power of".

---

priority

readonly priority: number;

Defined in: packages/engine/src/normalizer/NormalizerRule.ts:140

Priority for ordering rules. Higher values are tried first. Recommended ranges:

• 100: Long multi-word phrase fusion (e.g., “to the power of”)
• 80: Short phrase fusion (e.g., “power of”, “times by”)
• 50: Implicit operator insertion (e.g., implicit multiply)
• 20: Domain-specific transformations

Methods

match()

match(tokens, pos):
  | NormalizerMatch
  | null;

Defined in: packages/engine/src/normalizer/NormalizerRule.ts:149

Attempt to match a pattern starting at position pos in the token stream.

Parameters

Parameter	Type	Description
tokens	Token[]	The current token stream (may be partially normalized from prior passes)
pos	number	The current position to attempt matching from

Returns

| NormalizerMatch | null

A NormalizerMatch if the pattern is found, or null if no match