How a code-owned template declares numbered versions, and how publishing a version moves every one of its processes, live and finished, onto it in one transaction (issue #103).
The invariant: every process of a versioned template is on the template's latest version. Always. Publishing a version is a transaction: the application publishes version N together with its migration from N−1, and the platform applies it to every process of the template. If every process migrates, N becomes latest and every process is on it. If anything fails, nothing changes: N is not published, no process is touched, and every blocking process is named with its reason. A version number is taken only when a publication commits. Mixed versions would make every later operation over a process (bulk actions, reports, history, the next migration) branch on version, so there are none: no "keep on version", no "new instances only", no process left behind.
Templates without a lineage are unversioned: their processes stay pinned to the execution they started on, as before.
templateVersion, distinct from the authoring document's version, the aggregate's revision and schemaVersion. A version can compile to several digests: up to v5, spell-request's execution-date dropdown listed the next cycles when the module loaded, so a 14-day window roll plus a restart built a new digest of the same version. From v6 that list is a recipe (choice lists): the digest covers the rule, not the dates, so a version compiles to one digest whatever the day. Another build of the version already published is still published at startup as before (no process moves, and its number is recorded with it).templateVersion(key, executionId), write-once). A process records its execution's number in its binding (binding.templateVersion) when it starts or is migrated. The platform stores an opaque positive integer supplied by the host and never interprets it; the number never enters the compiled ExecutionDefinition, so no digest changes.version-conflict, unknown-version). The completeness and target checks use the process's actual stored execution.executionIds lists its legacy digests, reconstructed from git with templates:lineage. An execution the map does not place fails the publication (unknown-digest): fail safe.applications/spell-review/src/versions.ts, exported as @processos/spell-review/versions. The declaration types (TemplateLineage, MigrationOp) are in @processos/app-authoring. Main collects lineages in applications/main/src/templates/versions.ts.migration from the previous one. migrate: "new-instances-only" is gone.The application owns its lineage and the meaning of each op: the planner (applications/main/src/services/template-migrations.ts, step-level ops in applications/main/src/templates/declarative/migrations.ts) turns each process into a migration request. The platform (platform/runtime/src/application-service.ts) owns the transaction and its rules: publishTemplateVersion re-checks the version order and that every process of the template is either already on the version or migrated by this publication, runs every process's migration, enforces the required-value rule, and has the store commit the registration, the version record and every process together (commitTemplateVersion), or nothing. previewTemplateVersion is the same without the commit. New processes are created only on the latest (createOnLatest; see creation at the commit instant). The platform never depends on the application.
Migrations operate on a process's step-level state, never on compiled continuation paths:
| Part | Meaning |
|---|---|
currentStep | The step a live process waits at (the active visit, the innermost named frame and the pending input form must agree); null for a finished process. |
data | The per-step memory buckets, memory[stepKey][field]. |
completed | The steps with a completed visit. |
cursor: { input: <step key>, path: "root" } with no frames and no pending effect. A finished process (completed or failed) is migrated data-only: data ops and the new binding and stamp; no re-entry, no step moves; its status and outcome are unchanged and it keeps its last-updated time. Both get a migration audit entry.close(when, reason) finishes a live process for which when is true, as superseded by this version (completed, outcome { superseded: { reason, templateVersion } }, its open visit abandoned). It is for a process whose process changed too much to continue. Later ops and versions still migrate its data, as for any finished process.Anything else fails the publication.
| Op | Effect on a process |
|---|---|
addStep(step) | None; declares that the step is new. |
renameStep(from, to) | Moves data[from], the current step and completion, and rewrites the step's visit history, field audit and state audit keys. The originals stay in the migration audit. |
removeStep(step, moveTo?) | Drops the bucket (kept in the audit). A live process at step needs moveTo, or the publication fails. |
addField(step, field, default?) | For a completed step (live and finished processes alike) whose field has no value, applies the default. "No value" is the platform's completion rule (hasRequiredValue); the not-recorded marker counts as a value. The step a live process waits at is never pre-filled. A mandatory field with no default fails the publication: NOT NULL without a default. A default that evaluates to null means no default for that process. |
renameField(step, from, to) | Moves the value, and rewrites the field's key in the step's field audit (originals in the migration audit). |
removeField(step, field) | Drops the value (kept in the audit). |
changeField(step, field, existing) | The field's type or dropdown choices changed: its fixed values, its choice recipe, or one kind of list to the other. existing: "validate" fails a process whose stored value fails the new contract; "keep" leaves stored values as the platform does for a compatible input. "keep" is accepted only when the choices changed, and is the only option when the new choices are a recipe (a recipe is checked when a value is written, never against stored values). |
moveCurrentStep(from, to) or moveCurrentStep(expression) | Repositions a live process. The expression returns a step key, or null to stay. |
keepCurrentStep(at, despite) | None; records the decision that processes waiting at at stay there although the new step despite was inserted immediately before it. |
close(when, reason) | Finishes a matching live process as superseded (see above). Never applies to a finished one. |
transform(expression) | Escape hatch. A pure closed expression from {currentStep, data, completed} to the same shape. It may not finish a live process (that is close) or revive a finished one. |
Defaults and expressions are closed platform expressions evaluated over the step-level state.
Completeness is checked, not trusted. Before anything is applied, each migration on the path is compared with the real structural difference of the two executions: steps added or gone, fields added or gone, fields that became mandatory, fields whose type or choices changed (a recipe is compared by its digest, never by its output), and new steps inserted immediately before an existing step. An undeclared difference fails the publication with migration-incomplete, naming what is missing. The verify gate applies the same check to the current version against its frozen previous shape.
An insertion before an existing step is a decision. When a new step leads straight into a step that already existed, processes already waiting at the existing step would never run the new one. Each such pair needs moveCurrentStep({ from: existing, to }) or keepCurrentStep({ at: existing, despite: new }).
Migrated data must satisfy the target's input contracts. Values the migration wrote or moved, fields declared changeField(..., "validate"), and every stored value of a strict interaction are checked with the rule a strict submission uses (fieldValueProblem). A completed step must not lose a mandatory value it had. Failures are target-invalid, naming every bad field.
Auto-generation. generateMigration emits the unambiguous ops (added steps, added optional fields) and reports everything else for the author to decide. It never guesses.
The migration decides; the platform has one rule. After a migration, every mandatory, visible input field of every step the process has completed (other than the step it now waits at) has a value, or the publication fails (required-value-missing, naming each step.field). This is checked by the platform for every process (requiredValueProblems), whatever the application's planner did.
For each such field the author chooses an addField default: a value, an expression over the process's data, or the platform's not-recorded marker, default: { notRecorded: true }. The marker is a tool, not a policy:
{ "processos:not-recorded": "predates-field" } (@processos/contracts/not-recorded): never a string, number, bool, list or file, so never confusable with a real answer and never one of a dropdown's choices;renameStep and renameField rewrite the process's history so every later reader sees one shape: visit keys, field audit step keys and field keys, and state audit keys (a state audit entry's before/after values stay the raw snapshots they recorded). The migration audit entry keeps every original it rewrote (history: the renames, and the original key of each visit, each field audit entry and each state audit key). A renamed value is the same value: the migration records no data change for it.
The release step (npm run release, which runs templates:publish) runs before a new deployment takes traffic. For each versioned template:
quiesce). While it is open, no new work starts on the template's processes: new processes and every person command (edit, complete, reopen, abandon, sharing, delete) are refused with 503 template-upgrading, a spawned child of the template is not created (its parent's spawn is not dispatched, and the step worker retries it later), and the step worker does not wake a process sleeping on a timer. Work already in progress (a running continuation, an effect being delivered) continues to its next rest point.TEMPLATE_PUBLISH_SETTLE_MS (default 60 s, at most 150 s; one budget for the whole step, stragglers after a commit included) for in-flight work to settle. A process that cannot settle fails the publication at once, named: held for reconciliation, delivery in doubt, a parent with child receipts, a spawned child, or a timer in a step that is not safe to restart. One still in flight when the wait ends fails it too ("did not settle").A process can be created at the very moment a publication commits: a start that passed the upgrade-window check just before the window opened, or a spawned child. Each one reads the latest, builds the process on it, and then asks the store to create it with createOnLatest, which writes only if that execution is still the template's latest, atomically with commitTemplateVersion:
latest-moved), and the start or child re-binds to the new latest, exactly like a fresh start (up to five attempts with a short pause between them, then 503 template-upgrading; a child's withdrawn reservation is replaced by one for the new latest). Like a fresh start, the re-bound one checks the upgrade window again, so while the release step still holds it open it is usually refused with 503 template-upgrading instead (a child: its spawn is retried later);Either way, no process is ever created on a version a publication has just replaced. The stores make the check and the write atomic with the commit in their own way: the memory store does both without yielding; the file store holds the template's lock, as the commit does; the Mongo store writes a per-template head document (template-heads-v2) in the creation's transaction, as every publication transaction does. MongoDB transactions are snapshot-isolated and a read never conflicts, so without that common write a creation that read the old latest and a publication that did not see the new process could both commit; with it, whichever writes second gets a write conflict and is retried on a snapshot that sees the other. A concurrency test races starts against publication commits on every store (MongoDB as a real single-node replica set) and checks that every process ends on the new version; removing either head write, or the check, makes it fail.
Only commits that can move processes take part. A startup publish of another build of the published version (a registration-only compare-and-set of the latest pointer, with no transaction) does not: a process created as it lands may be bound to the previous build, which is the same version, so the invariant holds. Unversioned templates keep their rule (processes stay on the execution they started on); their starts still create only on the latest they read, re-binding if it moved.
A process sleeping on a timer is at rest when the step it sleeps in is safe to restart (restartSafeStep): the compiled step contains no effect other than local computation and state, time, reads, and HTTP declared repeat: "safe". A polling watcher (spell-request's s13, s19 and s20 watchers) qualifies: re-entry restarts the watch (and its window) from the start.
Each template is its own transaction: with several versioned templates, one can commit while another fails. The step then exits non-zero (the deployment fails) with the first template published; the old build keeps serving it, as versions only move forward.
npm run release && npm start (not a pre-deploy command: Railway does not mount volumes there, and the release step reads the access policy from the volume). A non-zero exit means the server never starts and never passes /health, so the deployment fails and the old deployment keeps serving the old version. No other service runs it. See deployment for every service's settings.Each command reads MONGO_URL (or the file store) exactly as the app does.
npm run templates:publish -- --dry-run # the release step's plan and the platform's preview; writes nothing npm run templates:publish # the release step (normally run at the start of the process-platform start command) npm run migrations:status # lineages, recorded versions, processes per execution, unknown digests npm run migrations:dry-run -- --template spell-request # same plan and preview as --dry-run, one template npm run migrations:dry-run -- --id <processId> --json # one process's plan and preview npm run migrations:backfill -- --template spell-request [--yes] [--actor <name>]
migrations:apply was removed: a migration is applied only by publishing its version, to every process at once. templates:seed --force no longer overwrites a published versioned template.
Executions published before version numbers were recorded carry no number and no timestamp.
npm run templates:lineage -- --template spell-request --digests d1,d2,... # or --from-store npm run templates:lineage -- --template spell-request --digests ... --diff d1,d2,d3
The tool walks every commit that touched the template source (following renames) and evaluates the file as it was, with the clock pinned at each day from the commit onward. It reports exact digest matches with their commits and everything else as unmatched. --diff prints the auto-generated migration between consecutive digests.
applications/main/src/templates/versions.test.ts) recompiles each versioned template with the clock pinned to its current fingerprint and fails if the execution changed.fingerprint and any legacy executionIds). Add a new current entry with fingerprint: { at, executionId }.migration, starting from templates:lineage -- --diff or generateMigration. Resolve every required item: each insertion before an existing step (moveCurrentStep or keepCurrentStep), each new or newly mandatory field on a completed step (a value, an expression, or { notRecorded: true }), and any process to close. Remember that finished processes migrate too.npm run templates:lineage -- --template <key> --digests <outgoing digest> --emit-shape <outgoing digest> --out <file> (spell-request: applications/spell-review/src/versions-previous.ts), and set it as the new entry's previous.npm run templates:publish -- --dry-run against a copy of real data and review every process before deploying. The deploy's release step then publishes it.The 12 past-cycle items without pre-risk verdicts (the nine Sept 24 items waiting at s19 and the three Sept 10 items waiting at s10, all CC-tracker backfills): the v2→v3 migration gives s07a_pre_risk_review_write.verdict, s07b_pre_risk_review_confirm.verdict and s07b_pre_risk_review_confirm.notes the not-recorded marker where a completed step has no value (approved by Adam, 2026-10-01). They migrate with every other process; the nine Sept 24 items also move back to the vote watcher (the v5 decision below).
spell-request v4 defaults (approved by Adam, 2026-10-01): items that had already filed s01 get the Summary "Not recorded: this item was filed before the intake asked for a summary.", and the item type moves from s04 with "Prime Action" → "Prime Spell" and "Sky Core" → "Sky Core Spell"; an item with no s04 answer gets "Prime Spell".
spell-request v5 insertions (approved by Adam, 2026-10-01): items waiting at s19_executive_vote move back to s19_watch_executive_vote; items waiting at s13_publish_technical_scope and s20_execute_spell stay there (keepCurrentStep).
The two leftover test items, closed by the v6 migration (#104), the first version published through this transaction (approved by Adam, 2026-10-01). Each close matches conservatively on fields taken from the 2026-09-30 snapshot; if any was edited since, nothing closes and the item simply migrates:
Before deploying v6, run npm run templates:publish -- --dry-run against a fresh copy of production data and confirm exactly two close outcomes (these two items), every other process migrating, and nothing failing. Done on a read-only export of 2026-10-02T00:46Z (63 spell-request processes, all on v5 df4703d2): 34 migrated live, 27 migrated data-only (finished), exactly these 2 closed, 0 failing; committing that local copy left all 63 on v6 42aa2625.
spell-request v6 (#104): the s01 execution-date dropdown's choices become a recipe (EXEC_DATE_RECIPE in applications/spell-review/src/template.ts: the next 10 cadence Thursdays from 2026-07-30, every 14 days, at least 22 days out, skipping 2026-12-31). Declared changeField(s01_deal_registration, execDate, keep): every stored date stays as it is. New dates are checked against the recipe when written. The Stars list stays a fixed list: no suitable data source exists yet (choice lists).
Finished spell-request items (27 in the 2026-09-30 snapshot) migrate data-only through v1→v5; every migration on the lineage is valid for them (dry run of that snapshot: 36 live processes migrated, 27 finished processes migrated data-only, none closed, none failing).
child-receipts, spawned-child). No versioned template spawns children today.compareAndSet (migrations apply, the backfill) is not checked against the latest: such tools bind processes on purpose. One written during a release step is converged by its post-commit re-plan, or else by the next release step.templates:seed --force and the platform's internal register replace the latest with a plain publish, with no check and no serialization. They are operator and test paths; templates:seed --force already leaves a published versioned template alone.loadExtraTemplates) replacing a versioned key declares no version: that service never becomes ready.upcomingSpellDates(), a TypeScript twin of the v6 execution-date recipe; a test pins the two to the same dates for every day and hour across two years, so a change to one must be made in both.notes; the s07b verdict defaults to the s07a verdict). They run only when a migrated past-cycle item is reopened to those steps; the person must then answer them (a live completion never accepts the marker), and the s07b verdict's default may not render cleanly. Changing them changes the template, so it waits for its next version.version-conflict; give the code a distinct execution or declare the version it really is.choices on execDate), and registration validation refuses unknown properties (invalid-registration). Once the release step commits v6, the previous build still serving refuses spell-request starts, and a restart of it fails to initialize, until the new build takes traffic (the release step runs in the new service's start command, so this lasts about as long as its start-up). Measured locally (2026-10-01): a running pre-#104 build refused every spell-request start after the commit, and could not start again. Rolling the code back below #104 after v6 is published therefore needs a release that accepts the field; deploy v6 when a short spell-review interruption is acceptable.