This site describes solve-engine as it is on main: 2.43.0, which npm does not have yet. npm installs 2.40.0, so a page may show an answer that version does not give yet.
Core concepts
Five ideas cover almost everything.
The unit of evaluation is a line, not a file and not a cell. Each line produces at most one result. Lines can refer to each other, which is why the engine wants line numbers rather than only text.
Values
Section titled “Values”Everything the engine produces is a value: a type, a payload, and sometimes a unit. The types include numbers, but also dates, durations, matrices, ranges, booleans, big integers, symbolic expressions, errors, and a pending state.
That last pair matters more than it sounds. An error is a value, so it flows through arithmetic and arrives at the top with its cause intact rather than silently becoming a confident and wrong number. Pending means the answer depends on data that has not arrived, which is not the same as zero.
Packages
Section titled “Packages”Almost every piece of syntax comes from a package, including the word forms of
arithmetic (5 and 3). A package can
contribute vocabulary to the lexer, parsing rules, functions the virtual machine
can call, normalisation rules, and conversion targets.
createEngine() registers all 47 built-in packages; three
more (stocks, crypto and knowledge) are opt-in because they need you to supply
how to fetch the data. A bare new ExpressionEngine() registers none, so you
pass exactly the packages you want. The operators themselves (+, -, *,
/, ^, %) are read by the parser’s own core rather than a package, so a
bare engine still answers 2 + 2 while refusing units, functions and money;
Upgrading to 2.0 shows what that looks like.
The pipeline
Section titled “The pipeline”Five stages, in order:
- Lex. Text becomes tokens.
- Normalise. Multi-word phrases fuse into single tokens and implicit
operators are made explicit. This is where
next fridaybecomes one thing. - Parse. Tokens become a structure, using precedence climbing.
- Compile. That structure becomes bytecode.
- Execute. The bytecode runs on a stack virtual machine.
Compiled bytecode is cached per expression, so re-evaluating an unchanged line skips the first four stages entirely.
Incremental evaluation
Section titled “Incremental evaluation”Evaluating a document again after an edit does not have to mean running every line again, but whether it does depends on which entry point a host calls.
parseDocument, the batch pass Embedding teaches, runs every line on every call. The bytecode cache above still lets an unchanged line skip the first four stages, so a second pass costs less than the first, but every line executes.evaluateDocumentstands up a fresh incremental evaluator for one pass and discards it, so it too runs every line. It exists for the forms that need to re-run a line, such as goal seek, not for speed.- A long-lived
ThreeTierEvaluator, the object an editor keeps for the life of a document and tells about each edit and about which lines are on screen, is the incremental one. A dependency graph records which lines read which variables, and after an edit the evaluator runs the edited line, the lines that transitively depend on it, and the visible lines, down to the bottom of the screen. An unchanged line above the screen is left alone, and the lines below it wait until they are scrolled to. Performance has the measured difference, and driving a live editor the walkthrough.
The explainer below walks that last case through one keystroke:
The boundary: the saving comes from the lines that are off screen. Asked for
every line of the document after an edit, a ThreeTierEvaluator runs every
line, because a variable’s value depends on where in the note it is read, and
running the lines in order from the top is what keeps it right.