DocumentModel

Defined in: packages/engine/src/engine/DocumentModel.ts:186

Persistent document model with O(log N) line lookups and structural edits.

Design:

• Each line has an immutable lineId (monotonically increasing counter).
• LineState objects are stored in a Map<lineId, LineState> for O(1) access.
• Line ordering is maintained in a SegmentTree (order-statistic Treap) that supports O(log N) insert, delete, and get-at-index operations.
• A lazy position cache (Map<lineId, number>) provides O(1) position lookups after the first getLinePosition() call and is invalidated on structural edits.

Key invariant: line IDs never change, only their positions in the order tree. This means cached bytecode, dependency graph entries, and VM checkpoints keyed by lineId remain valid across all structural edits.

Constructors

Constructor

new DocumentModel(maxLines?): DocumentModel;

Defined in: packages/engine/src/engine/DocumentModel.ts:313

Parameters

Parameter	Type	Default value	Description
maxLines	number	DEFAULT_CONFIG.performance.maxDocumentLines	Ceiling on the line count, defaulting to the engine’s configured one. Every line costs a LineState with six arrays in it whatever the line says, so the cost of a document is its line count and nothing else bounds it: two hundred thousand lines of 1 + 1 exhausted the heap here, before a single expression had been looked at.

Returns

DocumentModel

Accessors

dirtyCount

Get Signature

get dirtyCount(): number;

Defined in: packages/engine/src/engine/DocumentModel.ts:932

Number of lines currently marked dirty. For diagnostics/tests.

Returns

number

---

isEmpty

Get Signature

get isEmpty(): boolean;

Defined in: packages/engine/src/engine/DocumentModel.ts:1112

Returns

boolean

---

layoutRevision

Get Signature

get layoutRevision(): number;

Defined in: packages/engine/src/engine/DocumentModel.ts:223

A count bumped by every change to which line sits at which position (a new document, an insert or a delete), and by nothing else: an edit that changes a line’s text in place leaves it alone, where it moves revision. For a caller keeping something ordered by position.

Returns

number

---

lineCount

Get Signature

get lineCount(): number;

Defined in: packages/engine/src/engine/DocumentModel.ts:1108

Returns

number

---

revision

Get Signature

get revision(): number;

Defined in: packages/engine/src/engine/DocumentModel.ts:210

A count bumped by every change to the document’s text or line order, so a caller can keep something it derived from the text for as long as the text is unchanged: the evaluator keeps a table block’s answer this way (#616).

Returns

number

Methods

[iterator]()

iterator: IterableIterator<LineState>;

Defined in: packages/engine/src/engine/DocumentModel.ts:1119

Iterator over LineState in document order.

Returns

IterableIterator<LineState>

---

applyChanges()

applyChanges(changes): ApplyChangesResult;

Defined in: packages/engine/src/engine/DocumentModel.ts:404

Apply one or more line-level changes to the document.

Precondition: Changes must be non-overlapping in their line ranges. If two changes target the same or adjacent lines, the reverse-order processing may produce incorrect results because the first-applied change shifts the line numbers that the second change references.

Changes are applied in reverse order (highest startLine first) so that earlier changes in the document don’t shift the indices of later changes during processing.

Returns both the newly inserted line IDs and the removed line IDs. Callers should use removed to clean up the dependency graph and other data structures keyed by lineId.

Parameters

Parameter	Type
changes	LineChange[]

Returns

ApplyChangesResult

---

assertChangesFit()

assertChangesFit(changes): void;

Defined in: packages/engine/src/engine/DocumentModel.ts:495

Refuse changes that would leave the model holding more than maxLines, before any of them is applied, as setDocument refuses a document. A paste of a hundred thousand lines into one line grew the model past its ceiling, since only a whole document was counted.

The count is the one the changes leave: each inserts its lines and deletes as many of its lines as the document has from its first line on.

Parameters

Parameter	Type	Description
changes	readonly LineChange[]	The changes, with their texts already split at line breaks.

Returns

void

Throws

DOCUMENT_TOO_LARGE when they do not fit. Recoverable: nothing has changed yet.

---

clear()

clear(): void;

Defined in: packages/engine/src/engine/DocumentModel.ts:1128

Returns

void

---

deleteLines()

deleteLines(startLine, endLine): number[];

Defined in: packages/engine/src/engine/DocumentModel.ts:587

Delete lines in the given 1-based range [startLine, endLine] inclusive. Convenience wrapper around applyChanges.

Parameters

Parameter	Type
startLine	number
endLine	number

Returns

number[]

---

editLine()

editLine(lineNumber, newText): boolean;

Defined in: packages/engine/src/engine/DocumentModel.ts:604

Update the text of a single line in place. If the text hash differs, marks the line dirty and clears its bytecode/result so it gets re-evaluated.

Returns true if the text actually changed.

Parameters

Parameter	Type
lineNumber	number
newText	string

Returns

boolean

---

forgetResult()

forgetResult(lineId): void;

Defined in: packages/engine/src/engine/DocumentModel.ts:979

Forget what a line answered, without forgetting the line.

For a line whose answer was computed about a document that no longer exists. Marking it dirty says it must run again; this says that until it does, it has nothing to tell anyone who asks, which is the state a pass over the text from scratch would be in.

The bytecode is left alone. It is compiled from the line’s own text, which a structural edit does not change, and dropping it would recompile the document for nothing.

Parameters

Parameter	Type	Description
lineId	number	Persistent line identifier.

Returns

void

---

getAllLines()

getAllLines(): LineState[];

Defined in: packages/engine/src/engine/DocumentModel.ts:831

Get all LineState entries in order. Useful for batch processing.

Returns

LineState[]

---

getDirtyLines()

getDirtyLines(): LineState[];

Defined in: packages/engine/src/engine/DocumentModel.ts:845

Get all lines that are marked dirty.

Returns

LineState[]

---

getLineAt()

getLineAt(position):
  | LineState
  | undefined;

Defined in: packages/engine/src/engine/DocumentModel.ts:744

Get the LineState at the given 1-based line position. O(1).

Parameters

Parameter	Type
position	number

Returns

| LineState | undefined

---

getLineById()

getLineById(lineId):
  | LineState
  | undefined;

Defined in: packages/engine/src/engine/DocumentModel.ts:838

Get a LineState by its persistent line ID. O(1).

Parameters

Parameter	Type
lineId	number

Returns

| LineState | undefined

---

getLinePosition()

getLinePosition(lineId): number;

Defined in: packages/engine/src/engine/DocumentModel.ts:779

Get the 1-based position of a line by its persistent ID. Returns -1 if the line ID is not in the document.

Uses a lazy position cache: O(N) on first call after structural edit, O(1) on subsequent calls. The cache is invalidated by any structural edit.

Parameters

Parameter	Type
lineId	number

Returns

number

---

getLineStatesInRange()

getLineStatesInRange(startLine, endLine): (
  | LineState
  | undefined)[];

Defined in: packages/engine/src/engine/DocumentModel.ts:821

The lines from startLine to endLine, one entry per position.

The same lines getVisibleLines returns, except that a position whose line is missing keeps its place as undefined rather than closing the gap, so the caller can index by position.

Resolving the ids here rather than handing them back is deliberate and measured: the loop below is monomorphic over this class’s own map, and asking getLineById once per position instead was 20% slower on a twenty-thousand-line document.

One in-order walk of the order tree, O(span + log N), against the O(log N) descent per position that asking getLineAt in a loop costs. The evaluator walks from line 1 to the end of the viewport on every pass, so that descent was a third of the cost of an edit on a long document scrolled near the bottom.

Parameters

Parameter	Type	Description
startLine	number	First position, 1-based, inclusive.
endLine	number	Last position, 1-based, inclusive.

Returns

( | LineState | undefined)[]

One entry per position in the span, in document order.

---

getStructuralEditor()

getStructuralEditor(): ((changes) => void) | null;

Defined in: packages/engine/src/engine/DocumentModel.ts:294

The editor setStructuralEditor installed, or null; for an evaluator taking itself off.

Returns

((changes) => void) | null

---

getVisibleLines()

getVisibleLines(startLine, endLine): LineState[];

Defined in: packages/engine/src/engine/DocumentModel.ts:789

Get all LineState entries within the given viewport range (1-based, inclusive). Uses SegmentTree.getRange() for O(viewport + log N) collection instead of O(viewport × log N) per-line lookups.

Parameters

Parameter	Type
startLine	number
endLine	number

Returns

LineState[]

---

hasAnyDirtyLineBefore()

hasAnyDirtyLineBefore(position): boolean;

Defined in: packages/engine/src/engine/DocumentModel.ts:865

Whether any line before position (1-based, exclusive) is dirty.

Used by ThreeTierEvaluator.setViewport() to decide whether cached checkpoint state might be stale and a full evaluate() (from line 1) is needed instead of the cheap viewport-only path.

O(d log N) where d = current dirty line count via dirtyLineIds, not O(N log N), a document that’s mostly clean (the steady state after initial load) answers this in the cost of resolving a handful of lineIds to positions, not walking every line up to position.

Parameters

Parameter	Type
position	number

Returns

boolean

---

hasAnyDirtyVariableDefLineBefore()

hasAnyDirtyVariableDefLineBefore(position): boolean;

Defined in: packages/engine/src/engine/DocumentModel.ts:896

Whether any variable-definition line before position (1-based, exclusive) is dirty.

Narrower than hasAnyDirtyLineBefore: VMCheckpointer.snapshot() only ever records state for lines with writes.length > 0 (see VMCheckpoints.ts), so a dirty plain-expression line before the viewport cannot have invalidated any checkpoint, there’s no checkpoint entry for it to invalidate. Only a dirty variable-def line can mean the VM state a checkpoint would restore is stale.

This distinction matters because PageManager.evictPageBytecode() marks evicted non-variable-def lines dirty (so they get Tier 1 if scrolled back into view), and Tier 3’s compile-only path never clears dirty for non-variable-def lines by design. Using the broader hasAnyDirtyLineBefore here meant scrolling far into a large, variable-def-free document would trip setViewport()’s fallback to evaluate() on every single call, evaluate() reprocesses the evicted lines via Tier 3, which recompiles their bytecode without clearing dirty, so the very next maintainAfterEval() re-evicts and re-dirties the same lines, forever re-triggering the fallback on an otherwise unchanged viewport.

Parameters

Parameter	Type
position	number

Returns

boolean

---

hasAnyUncompiledDirtyLineBefore()

hasAnyUncompiledDirtyLineBefore(position): boolean;

Defined in: packages/engine/src/engine/DocumentModel.ts:921

Whether any line before position (1-based, exclusive) is dirty with text that has never been compiled: a line just loaded or just edited, which may define a name or be read by position and has no answer yet.

The evaluator’s viewport-only path asks this beside hasAnyDirtyVariableDefLineBefore, which only knows a line is a definition once it has been compiled: a viewport set on a document that has never been evaluated found no dirty definition above it, ran the lines in view alone, and prev + 1 at its top reported the line above as not evaluated. A line whose compiled program was evicted keeps its expressions, so it does not count here, and blank lines never do.

O(d log N) in the dirty lines, like hasAnyDirtyLineBefore.

Parameters

Parameter	Type
position	number

Returns

boolean

---

insertLines()

insertLines(atLine, texts): number[];

Defined in: packages/engine/src/engine/DocumentModel.ts:573

Insert new lines at the given 1-based position. Convenience wrapper around applyChanges.

Parameters

Parameter	Type
atLine	number
texts	string[]

Returns

number[]

---

invalidateAll()

invalidateAll(): void;

Defined in: packages/engine/src/engine/DocumentModel.ts:1021

Mark all lines as dirty (e.g., after plugin register/unregister).

Returns

void

---

isBytecodeValid()

isBytecodeValid(
   lineId,
   compiledAgainstHash,
   compiledAgainstText?
): boolean;

Defined in: packages/engine/src/engine/DocumentModel.ts:953

Verify that bytecode compiled by a worker is still valid for this line.

When Phase 5.2h sends compilation to a worker, the worker posts back {lineId, bytecode, reads, writes, compiledAgainstHash}. Between dispatch and response, the user may have edited the line. This method lets the main thread check whether the bytecode is still applicable.

Parameters

Parameter	Type	Description
lineId	number	The line the bytecode was compiled for.
compiledAgainstHash	number	The line’s text hash when the compile was dispatched.
compiledAgainstText?	string	The line’s text then. Pass it: two texts can share a hash (ab and bA do), so the hash alone can accept bytecode compiled for text the line no longer has (#664).

Returns

boolean

true if the line still exists and its text is the one compiled.

---

linesCarryingTag()

linesCarryingTag(tag): readonly number[];

Defined in: packages/engine/src/engine/DocumentModel.ts:673

The 1-based positions of the lines carrying #tag, in document order.

What total of #tag reads. Case-insensitive, matching how a tag is read on a line, and it names membership only: a line that merely asks about the group (total of #tag) is not in it, and neither is a #heading.

Ascending, because an aggregate reports the first line it cannot read and the line it names has to be the first one in the document, not whichever happened to be indexed first.

Read from tagGroups, so every aggregate over the same tag in one pass shares one sorted list rather than each sorting its own.

Parameters

Parameter	Type	Description
tag	string	The tag name without its #.

Returns

readonly number[]

Its lines’ positions, ascending; empty when no line carries it.

---

markClean()

markClean(lineId): void;

Defined in: packages/engine/src/engine/DocumentModel.ts:987

Parameters

Parameter	Type
lineId	number

Returns

void

---

markDirty()

markDirty(lineId): void;

Defined in: packages/engine/src/engine/DocumentModel.ts:1010

Mark a line as dirty (needs re-evaluation).

Parameters

Parameter	Type
lineId	number

Returns

void

---

markDirtyByLineNumber()

markDirtyByLineNumber(lineNumber): void;

Defined in: packages/engine/src/engine/DocumentModel.ts:999

Mark a line as dirty (needs re-evaluation) by its 1-based position. Convenience for callers that have line numbers instead of line IDs.

Parameters

Parameter	Type
lineNumber	number

Returns

void

---

setDocument()

setDocument(text): void;

Defined in: packages/engine/src/engine/DocumentModel.ts:331

Initialize or replace the entire document from a text blob. Clears all existing state and assigns new persistent line IDs.

A line ends at a CRLF pair, a line feed or a lone carriage return, the breaks parseDocument reads, and the break is not part of the line’s text.

Parameters

Parameter	Type
text	string

Returns

void

Throws

DOCUMENT_TOO_LARGE for a document past maxLines, before any of it is stored. Recoverable: nothing has been replaced yet, so the model still holds whatever it held.

---

setStructuralEditor()

setStructuralEditor(editor): void;

Defined in: packages/engine/src/engine/DocumentModel.ts:289

Who applies an edit that editLine finds changes the line count, because its text holds a line break: null for applyChanges itself. An evaluator over this model installs its own applyTransaction, which applies the change and keeps its dependency graph and checkpoint chain in step with the lines that moved; a direct applyChanges would leave both describing the old positions. It takes itself off again when disposed.

Parameters

Parameter	Type	Description
editor	((changes) => void) | null	The function, or null for the model’s own.

Returns

void

---

tagGroups()

tagGroups(): ReadonlyMap<string, readonly number[]>;

Defined in: packages/engine/src/engine/DocumentModel.ts:688

Every category tag the document’s lines carry, lower-cased, each with the 1-based positions of its lines in ascending order.

What total by tag reads to find its groups, instead of reading the tags off every line of the note on every call (#734). The answer is the same object until the text or the line order changes (see revision), so a caller may key work of its own on it.

Returns

ReadonlyMap<string, readonly number[]>

The groups; empty when no line carries a tag.

---

toJSON()

toJSON(): object;

Defined in: packages/engine/src/engine/DocumentModel.ts:1142

Serialize the document model to a plain object for debugging.

Returns

object

---

updateLineCompiled()

updateLineCompiled(
   lineId,
   expressions,
   bytecodes,
   reads,
   writes,
   isVariableDef,
   inlineSolveCount?
): void;

Defined in: packages/engine/src/engine/DocumentModel.ts:1086

Update a line’s compile-only state (Tier 3: background compilation).

Stores expressions, bytecodes, reads, and writes. Does NOT set results and does NOT mark the line clean, it still needs execution (Tier 1 or Tier 2) to produce results. This distinction allows the three-tier evaluation strategy: compile invisible lines in the background without executing them, then execute from cached bytecode when scrolled into view.

Parameters

Parameter	Type	Default value	Description
lineId	number	undefined	Persistent line identifier.
expressions	string[]	undefined	Extracted expression strings (in order).
bytecodes	BytecodeProgram[]	undefined	Compiled bytecode for each expression (in order).
reads	string[]	undefined	Aggregated read variables across all expressions.
writes	string[]	undefined	Aggregated write variables across all expressions.
isVariableDef	boolean	undefined	True if any expression defines a variable.
inlineSolveCount	number	0	Number of inline solves (0 for full-line).

Returns

void

---

updateLineResult()

updateLineResult(
   lineId,
   results,
   bytecodes,
   expressions,
   reads,
   writes,
   isVariableDef,
   inlineSolveCount?
): void;

Defined in: packages/engine/src/engine/DocumentModel.ts:1043

Update a line’s evaluation state after successful execution (Tier 1 / Tier 2).

Sets results, bytecodes, reads, writes, and marks the line clean. Supports multi-expression lines (inline solves) via parallel arrays.

Parameters

Parameter	Type	Default value	Description
lineId	number	undefined	Persistent line identifier.
results	Value[][]	undefined	Evaluation result groups for each expression (in order). Each element is a Value[].
bytecodes	BytecodeProgram[]	undefined	Compiled bytecode for each expression (in order).
expressions	string[]	undefined	Extracted expression strings (in order).
reads	string[]	undefined	Aggregated read variables across all expressions.
writes	string[]	undefined	Aggregated write variables across all expressions.
isVariableDef	boolean	undefined	True if any expression defines a variable.
inlineSolveCount	number	0	Number of inline solves (0 for full-line).

Returns

void