LanguageService

Defined in: packages/engine/src/language/LanguageService.ts:212

Editor-agnostic “language server” for solve expressions: turns a line of text into semantic token ranges, using the exact same lexer real evaluation uses (so it only ever classifies what the engine’s grammar actually recognizes, never a separate/duplicated tokenizer). No knowledge of CSS, CodeMirror, VS Code, or any other rendering concept lives here. See language/adapters/ for that.

Classification happens at the LEXER stage, before the normalizer runs (normalization, phrase fusion, implicit multiply, and package-specific rules, happens later, only on the real evaluation path). A package’s lexer-level custom token types (e.g. a custom keyword) are recognized here exactly as evaluation would see them. A package’s normalizer-fused synthetic tokens (e.g. OSRS’s GAME_ITEM, built by fusing several consecutive IDENT tokens against an item-name trie) are NOT. This service still shows the pre-fusion IDENT tokens individually for those. IEnginePackage.tokenCategories entries for normalizer-only token types are still valid, correct registrations (queryable via getTokenCategory) , they just won’t currently be reachable through this lexer-only classification path. Folding normalization in would require running it per keystroke on the highlighting path too, which needs its own careful design (span recomputation for fused multi-token ranges, in particular) rather than a quick addition here.

Lexing alone is NOT sufficient to decide “recognized”, though: a run of plain-English words (“My name is ron”) lexes into a sequence of individually-valid IDENT tokens with no grammar tying them together every word “recognized” at the token level, but the line as a whole is not something the engine would ever accept as an expression. Surfacing per-token colors for that case looks like the editor mistook prose for code. So a line’s tokens are only surfaced once the line as a whole parses successfully (via ExpressionEngine.tryCompileExpression, the same parse pipeline, and the same bytecode cache, real evaluation uses; no separate/duplicated grammar check). That check is read-only: a line is compiled, never run, so highlighting cannot change what the document holds. A single bare word (“hello”, a valid variable reference) or a keyword-only line (“pi”) still parses and still highlights, only genuinely ungrammatical text is suppressed, unless it’s a known variable elsewhere in the document (see variableNameSource).

getCompletions() is the other half of this “language server”: unlike getSemanticTokens(), it’s explicitly for incomplete, mid-typing text , it deliberately does NOT gate on parse validity (a half-typed expression almost never parses), using simple prefix matching instead.

Must be constructed with an already-configured ExpressionEngine (one with all currently-relevant packages registered) rather than a bare lexer, reusing an existing engine is both the fast path (no throwaway lexer construction) and the correct one: a highlighting-only lexer built independently of the evaluation engine would silently fail to recognize plugin-contributed tokens (e.g. a package’s custom keywords) unless it happened to have the identical packages registered.

Constructors

Constructor

new LanguageService(engine?, options?): LanguageService;

Defined in: packages/engine/src/language/LanguageService.ts:298

Parameters

Parameter	Type
engine?	| ExpressionEngine | null
options?	LanguageServiceOptions

Returns

LanguageService

Methods

findReferences()

findReferences(text, position): VariableReference[];

Defined in: packages/engine/src/language/LanguageService.ts:757

Every place the variable at position is named, its definitions and its reads, in document order. A word in prose is never included, since only a line that parses holds variables. Empty when position is not on one.

Parameters

Parameter	Type	Description
text	string	The whole document.
position	DocumentPosition	A one-based line and a zero-based character on it.

Returns

VariableReference[]

---

getCompletions()

getCompletions(lineText, cursorOffset): CompletionItem[];

Defined in: packages/engine/src/language/LanguageService.ts:546

Completion candidates for the identifier prefix immediately before cursorOffset on lineText. Deliberately simple prefix matching, not parser-driven “what’s grammatically valid here” prediction, a half-typed expression almost never parses, so gating on parse validity (the way getSemanticTokens does) would suppress completions almost always. This is the safest, fastest option that still delivers real value.

Candidates come from these sources. Static per engine configuration, and cached lazily: lexer keywords that have a highlight category (which include function names, see ExpressionLexer.getKeywords()), the call words packages declare with callFusions (sha256), registered phrases (average of, net present value of), units, and package-contributed items (IEnginePackage.completionItems). Read fresh on every call, since they change on every edit: variable names from variableNameSource(), and the units the document defines (engine.userUnitNames()).

A phrase is matched by its opening words, from the word under the cursor or across the words typed before it (net pres), and such a match carries replaceLength. It is not continued part-way through: after net present the service offers what starts with present, not the rest of the phrase from the grammar.

Parameters

Parameter	Type
lineText	string
cursorOffset	number

Returns

CompletionItem[]

---

getDefinition()

getDefinition(text, position):
  | VariableReference
  | null;

Defined in: packages/engine/src/language/LanguageService.ts:770

Go to definition: the definition the variable at position reads, which is the last one above it (or the occurrence itself, where it defines the name). Null when position is not on a variable or nothing above defines it, which is when the engine reports the name as undefined.

Parameters

Parameter	Type	Description
text	string	The whole document.
position	DocumentPosition	A one-based line and a zero-based character on it.

Returns

| VariableReference | null

---

getHover()

getHover(
   text,
   position,
   results?
):
  | VariableHover
  | null;

Defined in: packages/engine/src/language/LanguageService.ts:785

What to show when the pointer rests on a variable: the occurrence, the definition it reads with that line’s text, and the value, read from the results the host already has. Nothing is evaluated here. Null when position is not on a variable.

Parameters

Parameter	Type	Description
text	string	The whole document.
position	DocumentPosition	A one-based line and a zero-based character on it.
results?	LineResults	The document’s results (parseDocument’s return value, or a function from a line number to its value), for the hover’s value.

Returns

| VariableHover | null

---

getSemanticTokens()

getSemanticTokens(lineText, lineNumber): SemanticToken[];

Defined in: packages/engine/src/language/LanguageService.ts:411

Classify every recognized token on one line.

Parameters

Parameter	Type	Description
lineText	string	The raw line text (may be a markdown-structural line the engine’s classifier skips, that’s handled by the underlying lexer, which returns no tokens for those).
lineNumber	number	1-based line number, used purely as a cache key.

Returns

SemanticToken[]

---

getTokenCategory()

getTokenCategory(tokenType):
  | TokenCategory
  | undefined;

Defined in: packages/engine/src/language/LanguageService.ts:294

A token type’s highlight category, as this service’s engine reads it: a category a registered package declared (IEnginePackage.tokenCategories), then the built-in table. Without an engine, the built-in table alone.

Parameters

Parameter	Type	Description
tokenType	string	The token’s type, as a token or a highlight span carries it.

Returns

| TokenCategory | undefined

The category, or undefined for a type that renders unstyled.

---

invalidateCache()

invalidateCache(): void;

Defined in: packages/engine/src/language/LanguageService.ts:895

Full cache clear. Reserved for cases with no meaningful “which lines changed” (e.g. the document was swapped wholesale, or a package was registered/unregistered mid-session, changing what categories exist). Prefer invalidateLines for ordinary edits. Also rebuilds the lazily-cached keyword/unit/package-item completion candidates on next use, the only thing that can change that list mid-session.

Returns

void

---

invalidateLines()

invalidateLines(lineNumbers): void;

Defined in: packages/engine/src/language/LanguageService.ts:881

Evict specific lines (e.g. the lines actually touched by a CodeMirror change set) instead of the whole cache, the surgical counterpart to invalidateCache, letting a single-line edit stay cheap even in a large document: every other cached line is untouched and still hits on the next call.

Parameters

Parameter	Type
lineNumbers	Iterable<number>

Returns

void

---

rename()

rename(
   text,
   position,
   newName
): RenameResult;

Defined in: packages/engine/src/language/LanguageService.ts:799

Rename the variable at position, editing only the places it is named. Returns the edits, or a named refusal: not a variable, a global, a newName that is a keyword, a unit or not a name, a newName the document already uses, or a rename that would change how a line reads.

Parameters

Parameter	Type	Description
text	string	The whole document.
position	DocumentPosition	A one-based line and a zero-based character on it.
newName	string	The name to give the variable, without a : sigil.

Returns

RenameResult

---

shiftLineReferences()

shiftLineReferences(text, change): LineShiftResult;

Defined in: packages/engine/src/language/LanguageService.ts:816

The edits that keep absolute line N references on the lines they meant after lines are inserted or deleted, as a spreadsheet keeps a reference on its row. A reference into a deleted line becomes line deleted, which answers with a named error, and is listed in the result.

Parameters

Parameter	Type	Description
text	string	The whole document, after the change.
change	LineShift	Which lines were inserted or deleted.

Returns

LineShiftResult