9. Tolerant reading and recovery

A design file is edited by the editor, by ud, by agents writing lines by hand, by people in a text editor and by git merges. A reader therefore repairs what it can and reports it, instead of refusing the file (NFR-6).

9.1 The tree#

Problem	Repair
tree missing or not an object	An empty document with one page named Page 1, warning.
document.id missing	A new doc- id is made and written with the next change; warning (libraries referring to the old id will not find the file by id).
No pages	A page named Page 1 is added; nodes whose page is missing move to it.
A missing or mistyped property	Its default, as if absent. A mistyped property is reported once per property name: <n> node(s) had an invalid <property> and use the default.
A member the reader does not know	Kept and written back (PLT-4).
A map entry whose value is not an object	Dropped: Dropped a <node or page or style or …> entry <id> that is not an object.
A node without type	Dropped with its descendants (they become orphans, 6.5).
A node with an unknown type	Kept, rendered as a placeholder box with its stored geometry, children kept: Node "<name>" has the unknown type <type>.

9.2 The WAL and events#

Problem	Repair	Warning
wal missing or not an array	An empty list.	The event list was missing or not a list.
An entry that is not an object	Dropped.	Dropped a WAL entry that is not an object.
An entry without op	Dropped.	Dropped a WAL entry without an operation.
seq missing or not a number	Read as the previous event’s seq + 1 so that reports can name the event; the event is applied at its place in the file; the line is not rewritten.	One for the file: <n> WAL event(s) had no sequence number.
seq not greater than the previous event’s	Not a problem: it is a merge, and the start of a new run (6.4).	none; conflicts are reported by 6.4
at missing or malformed	The previous event’s at.	none
by missing	Unknown.	none
An unknown op	Skipped, kept in the list (5.12).	Skipped event <seq> (<op>): this build does not know the operation.
A target that does not exist, an id that is taken	Event skipped (5.11).	Skipped event <seq> (<op>): <reason>.
A line that appears twice	Two events; the second create is skipped because its id exists, the second update is harmless.	as above
styledRuns beyond the end of characters	Cut to the text; empty runs dropped.	none
set that names type, parent or index	Those members ignored.	Event <seq> (node.update) tried to change <member>; use node.move.

Repairs happen in memory and never rewrite a line: the tree is written in its repaired form only by consolidation, and a WAL line is written back exactly as it was read, missing seq included. New events appended to a repaired file take max(seq) + 1 as always (5.1).

9.3 Recovery of a file that is not valid JSON#

A bad hand edit or a merge without merge=union can leave the file unparseable. Because every event is one line, most of it can still be read. When JSON.parse fails, the reader tries in order:

1. Strip git conflict markers, in the WAL only. Lines that start with <<<<<<<, |||||||, ======= or >>>>>>> and stand after the line "wal": [ are removed. In the WAL this gives exactly the union of both sides, which is what merge=union would have produced. Parse again. A conflict marker before that line is a conflict in the tree (both sides consolidated, or a hand edit): it is not stripped, because the union of two trees is not a tree, and the file is handled by 9.4.
2. Read line by line. The text up to the line "wal": [ is parsed as the head of the object (closed with } if needed). Each later line is parsed on its own, after removing four spaces of indentation, a leading comma and the closing ] and }. Lines that do not parse are dropped.

If either step gives a document, the editor opens it with the warning This file was not valid JSON and was recovered: <n> line(s) could not be read., the list of dropped lines (with their line numbers) in the output channel, and the document marked as changed. Saving writes the recovered file in the canonical layout; until then the file on disk is untouched. ud validate --recover does the same from the command line and writes the result to <name>.recovered.design.

What is not recovered. A tree that does not parse (both branches consolidated, a hand edit in the tree), a top level that is not an object, and a newer major format version (section 10). The editor shows the reason, offers “Open as Text” (PLT-6) and leaves the file as it is.

9.4 Files that are not folded#

Folding a file whose tree and WAL do not belong together gives a wrong document silently: a union merge of two consolidated branches parses, with the last of each duplicate key winning, and a branch that never consolidated re-applies its events on top of a tree that already holds them. A reader therefore refuses to fold when it finds any of these, and does not guess:

Finding	What happened	Message
Two or more wal.base lines	A branch was merged across a consolidation: one side’s tree, both sides’ markers, the other side’s events.	This file has <n> event list markers (epochs <ep-…>, <ep-…>): it was merged across a consolidation.
wal.base.treeHash differs from the hash of the parsed tree (6.9)	The tree was changed by something other than a consolidation: a hand edit, or a merge that mixed two trees without a duplicate key.	The tree does not match its marker (expected <hash>, found <hash>): it was changed outside a consolidation.
A duplicate key in tree (the tree is parsed with duplicate keys detected; WAL lines are parsed as ordinary JSON)	A union merge of two trees.	The tree has duplicate entries (<n>, first: nodes/<id>): two consolidated trees were merged.

The editor shows the message and offers two ways out: ud rebase-branch, or Open as Text (PLT-6). It does not open the file as a design and does not write to it. ud validate reports the same with exit 2.

ud rebase-branch (CLI-001) rebuilds the file from the two sides of the merge, which git still has: it takes the tree and the marker of one side (--ours by default, --theirs otherwise), finds the events the other side appended to its marker — the marker’s epoch says which tree they were written against, and the commit that wrote that marker is in the history — and appends them after the kept marker, in their order and with their seq values unchanged. Events that no longer apply are skipped with the usual warnings (5.10), and conflicts are reported as after any merge (6.4). The result is written in the canonical layout and the file is read as usual. Like consolidation, this rewrites more than one line, so it is committed on its own.