createFusedToken
function createFusedToken( type, text, sourceTokens): Token;Defined in: packages/engine/src/normalizer/TokenNormalizer.ts:104
Creates a new normalized token from fused source tokens.
The fused token inherits position information (offset, line, column) from the first source token, which preserves source-map accuracy for error messages and diagnostic highlighting.
It also records where the source text ENDS, on sourceEnd. The start alone
is not enough to describe the span a fusion covers, because text is the
replacement rather than the original: 10 frames fuses into a FRAME_COUNT
whose text is 10, and a timecode fuses into a token whose text is a
comma-separated tuple that appears nowhere in the line. Anything painting the
line needs both ends, and only this function is in a position to know them.
Parameters
Section titled “Parameters”| Parameter | Type | Description |
|---|---|---|
type | string | The new token type (e.g., “CARET”, “TIMES_BY”) |
text | string | The combined text representation (e.g., “to the power of”) |
sourceTokens | Token[] | The original tokens being fused (at least 2) |
Returns
Section titled “Returns”A new LexerToken with the fused type and combined text