feat(spec): a flow step log declares the warnings and droppedFields the run history already serves

.changeset/33020-step-log-advisory-members.md

@@ -0,0 +1,13 @@
+---
+'@objectstack/spec': minor
+'@objectstack/service-automation': patch
+---
+
+feat(spec): a flow run's step log declares the `warnings` and `droppedFields` it already carried
+
+Clause-②: yes (widening) — `ExecutionStepLogSchema` (`@objectstack/spec/automation`) gains two optional members, `warnings` (strings) and `droppedFields` (an array of the data engine's own `DroppedFieldsEventSchema` event). Both reach every published schema that embeds a step log: `ExecutionLogSchema.steps`, and through it `GetRunResponseSchema` (`GET /api/v1/automation/:name/runs/:runId`) and `ListRunsResponseSchema` (the runs list). Nothing that parsed before is refused.
+Metadata-protocol: added
+
+The engine has written both members onto a step that ran (`warnings` for a write whose requested fields the data engine legally stripped, `droppedFields` for the structured events behind those warnings), and the run-history routes serve the step as persisted. The published step-log schema named neither, so a reader that parsed a run through it stripped both and could not tell a step that partially did not land from a clean one. `droppedFields` reuses the data engine's event, so there is no second reason vocabulary.
+
+`@objectstack/service-automation`: the engine now copies the two members onto a step only in that declared shape. From a host-registered executor's result, a `droppedFields` element that is not a well-typed drop event is skipped (the same filter the run's terminal `droppedFields` answer already applied), and so is a warning that is not a string; a member with nothing well-typed left is omitted. Before, any value with a truthy `length` was persisted and served as it came. The in-tree write nodes produce well-typed values, so their steps are unchanged.

content/docs/references/automation/execution.mdx

@@ -147,6 +147,8 @@ const result = CheckpointSchema.parse(data);
 | **regionKind** | `string` | optional | Region kind the step ran in: loop-body \| parallel-branch \| try \| catch. Stays `try` / `catch` for a step inside a try/catch region nested in a loop body; the loop is reported through `iteration`. For `parallel-branch` the branch index is reported through `branch`, and the enclosing loop iteration — when the parallel node sits inside a loop body — through `iteration`. |
 | **metrics** | `{ selected?: integer; acted?: integer; unmeasuredEffect?: boolean; failures?: integer }` | optional | Records this step selected / acted on — and, for a step that delegated to a child run (`subflow`, a `map` item), the failures that child contained — as reported by the node executor |
 | **skippedBy** | `{ nodeId: string; edgeId?: string; label?: string }` | optional | The gate that closed, when `status` is `skipped` |
+| **warnings** | `string[]` | optional | Advisory warnings from the node executor. A `success` step may carry them — e.g. a write whose requested fields the data engine legally stripped — so a step that partially did not land never reads as a clean success. Absent when the executor raised none |
+| **droppedFields** | `{ object: string; fields: string[]; reason: Enum<'readonly' \| 'readonly_when' \| 'primary_key' \| 'computed' \| 'stored_expression'> }[]` | optional | The data engine's own drop events behind those warnings: which caller-supplied fields of which object the write did not store, and why. Absent when the step dropped nothing |
 
 ### Nested Shape: `ExecutionLog.summary`
 
@@ -205,6 +207,8 @@ const result = CheckpointSchema.parse(data);
 | **regionKind** | `string` | optional | Region kind the step ran in: loop-body \| parallel-branch \| try \| catch. Stays `try` / `catch` for a step inside a try/catch region nested in a loop body; the loop is reported through `iteration`. For `parallel-branch` the branch index is reported through `branch`, and the enclosing loop iteration — when the parallel node sits inside a loop body — through `iteration`. |
 | **metrics** | `{ selected?: integer; acted?: integer; unmeasuredEffect?: boolean; failures?: integer }` | optional | Records this step selected / acted on — and, for a step that delegated to a child run (`subflow`, a `map` item), the failures that child contained — as reported by the node executor |
 | **skippedBy** | `{ nodeId: string; edgeId?: string; label?: string }` | optional | The gate that closed, when `status` is `skipped` |
+| **warnings** | `string[]` | optional | Advisory warnings from the node executor. A `success` step may carry them — e.g. a write whose requested fields the data engine legally stripped — so a step that partially did not land never reads as a clean success. Absent when the executor raised none |
+| **droppedFields** | `{ object: string; fields: string[]; reason: Enum<'readonly' \| 'readonly_when' \| 'primary_key' \| 'computed' \| 'stored_expression'> }[]` | optional | The data engine's own drop events behind those warnings: which caller-supplied fields of which object the write did not store, and why. Absent when the step dropped nothing |
 
 ### Nested Shape: `ExecutionStepLog.error`
 
@@ -231,6 +235,16 @@ const result = CheckpointSchema.parse(data);
 | **edgeId** | `string` | optional | Edge whose condition evaluated false |
 | **label** | `string` | optional | Edge label, when the flow names its branches |
 
+### Nested Shape: `ExecutionStepLog.droppedFields[number]`
+
+A write-path strip event: caller-supplied fields legally dropped from the payload, or written with the engine's computed value in place of the caller's
+
+| Property | Type | Required | Description |
+| :--- | :--- | :--- | :--- |
+| **object** | `string` | ✅ | Object the write targeted (resolved object name) |
+| **fields** | `string[]` | ✅ | Caller-supplied field names whose value the engine did not store: removed from the write payload, or (`stored_expression`) written with the computed value in place of the caller's |
+| **reason** | `Enum<'readonly' \| 'readonly_when' \| 'primary_key' \| 'computed' \| 'stored_expression'>` | ✅ | Why the fields were dropped: static readonly, a TRUE readonlyWhen predicate, the primary-key strip of a payload id the engine ruled is not an identifier, a computed (formula) field no driver has a column for, or a stored field whose declared expression the engine computes on every write, so the caller's different value was replaced by the computed one |
+
 
 ---
 

packages/services/service-automation/src/dag-traversal.ts

@@ -41,6 +41,7 @@ import {
     type StepLogEntry,
 } from './engine.js';
 import { createEvalUser, type EvalUserParsed } from '@objectstack/spec/identity';
+import { DroppedFieldsEventSchema, type DroppedFieldsEvent } from '@objectstack/spec/data';
 import { structuralConditionRefusal } from '@objectstack/spec/automation';
 // [#15137] The `value`-role half of the ledger. Both halves of "is this envelope
 // well-formed?" are IMPORTED, never re-spelled here: the shape rule is
@@ -98,6 +99,41 @@ export function runUserOf(context: AutomationContext | undefined): EvalUserParse
     });
 }
 
+/**
+ * The step-log copy of an executor's advisory members, in exactly the shape
+ * `ExecutionStepLogSchema` declares for them. The step is persisted with the
+ * run and served verbatim by the run-history routes, so what reaches it is
+ * what a reader of the run is handed: a host-registered executor's malformed
+ * value must not be copied onto it as-is (a truthy `length` is not a list of
+ * drop events).
+ *
+ * `droppedFields` keeps each element that is a well-typed
+ * `DroppedFieldsEventSchema` event and skips the rest: the same per-element
+ * filter the run's terminal answer applies when it folds the step log
+ * (`foldDropEvents`), so the step and the answer can never disagree about
+ * which events were real. `warnings` keeps its string entries. A member with
+ * nothing well-typed left is omitted, never `[]`. Total, never throwing: it
+ * runs inside the step's `try`, where a throw would read as a node failure.
+ */
+function stepAdvisoriesOf(result: NodeExecutionResult): { warnings?: string[]; droppedFields?: DroppedFieldsEvent[] } {
+    const out: { warnings?: string[]; droppedFields?: DroppedFieldsEvent[] } = {};
+    const rawWarnings: unknown = result.warnings;
+    if (Array.isArray(rawWarnings)) {
+        const warnings = rawWarnings.filter((w): w is string => typeof w === 'string');
+        if (warnings.length > 0) out.warnings = warnings;
+    }
+    const rawDrops: unknown = result.droppedFields;
+    if (Array.isArray(rawDrops)) {
+        const droppedFields: DroppedFieldsEvent[] = [];
+        for (const raw of rawDrops) {
+            const parsed = DroppedFieldsEventSchema.safeParse(raw);
+            if (parsed.success) droppedFields.push(parsed.data);
+        }
+        if (droppedFields.length > 0) out.droppedFields = droppedFields;
+    }
+    return out;
+}
+
 /**
  * Does this `decision` take EVERY out-edge whose condition holds (#15429)?
  * Only an explicit `config.mode: 'inclusive'` says so; an omitted `mode` and
@@ -414,8 +450,8 @@ export async function executeNode(
                 completedAt: new Date().toISOString(),
                 durationMs: Date.now() - stepStart,
                 error: { code: 'NODE_FAILURE', message: errMsg },
-                ...(result.warnings?.length ? { warnings: result.warnings } : {}),
-                ...(result.droppedFields?.length ? { droppedFields: result.droppedFields } : {}),
+                // In the declared step-log shape only (`stepAdvisoriesOf`).
+                ...stepAdvisoriesOf(result),
                 // #4354 — a node that failed PART WAY may still have written
                 // rows; dropping its counts would understate what the run did.
                 ...(result.metrics ? { metrics: result.metrics } : {}),
@@ -495,10 +531,11 @@ export async function executeNode(
             startedAt: stepStartedAt,
             completedAt: new Date().toISOString(),
             durationMs: Date.now() - stepStart,
-            ...(result.warnings?.length ? { warnings: result.warnings } : {}),
-            // #32861 — the structured events behind those warnings, folded
-            // onto the run's terminal answer (`dropEventsOf`).
-            ...(result.droppedFields?.length ? { droppedFields: result.droppedFields } : {}),
+            // #32861 — `droppedFields`, the structured events behind those
+            // warnings, is folded onto the run's terminal answer
+            // (`dropEventsOf`). Both members are copied in the declared
+            // step-log shape only (`stepAdvisoriesOf`).
+            ...stepAdvisoriesOf(result),
             ...(result.metrics ? { metrics: result.metrics } : {}),
         });
 

packages/services/service-automation/src/run-dropped-fields.test.ts

@@ -15,7 +15,7 @@
  */
 import { describe, it, expect } from 'vitest';
 import type { AutomationContext } from '@objectstack/spec/contracts';
-import { defineActionDescriptor } from '@objectstack/spec/automation';
+import { defineActionDescriptor, ExecutionStepLogSchema } from '@objectstack/spec/automation';
 import { AutomationEngine } from './engine.js';
 import type { NodeExecutor, StepLogEntry } from './engine.js';
 import { InMemorySuspendedRunStore } from './suspended-run-store.js';
@@ -473,6 +473,97 @@ describe('delegated writes roll their drops up onto the parent\'s answer (#33019
     });
 });
 
+// The step itself is persisted with the run and served verbatim by the
+// run-history routes, so it must carry `warnings` / `droppedFields` only in the
+// shape `ExecutionStepLogSchema` declares — the same per-element filter the
+// answer's fold applies, so step and answer agree about which events were real.
+describe('the step log carries the advisory members only in the declared shape', () => {
+    const SLOPPY_DROPS = [
+        null,
+        'status',
+        { object: 'expense_report', fields: 'status', reason: 'readonly' },
+        { object: 'expense_report', fields: ['status'], reason: 'no_such_reason' },
+        { object: 'expense_line', fields: ['amount'], reason: 'readonly' },
+    ];
+    const stepOf = async (engine: AutomationEngine, flow: string, nodeId: string) => {
+        const [run] = await engine.listRuns(flow);
+        return run.steps.find((s) => s.nodeId === nodeId)!;
+    };
+
+    it('a host executor\'s malformed elements are not copied onto its success step; the well-typed ones are, and the step parses', async () => {
+        const engine = makeEngine();
+        engine.registerNodeExecutor({
+            type: 'sloppy_write',
+            async execute() {
+                return { success: true, warnings: ['amount was not written', 42, { text: 'x' }], droppedFields: SLOPPY_DROPS };
+            },
+        } as never);
+        engine.registerFlow('sloppy', chain('sloppy', [{ id: 'w', type: 'sloppy_write', label: 'W' }]));
+
+        await engine.execute('sloppy', CALLER);
+        const step = await stepOf(engine, 'sloppy', 'w');
+
+        expect(step.droppedFields).toEqual([{ object: 'expense_line', fields: ['amount'], reason: 'readonly' }]);
+        expect(step.warnings).toEqual(['amount was not written']);
+        expect(ExecutionStepLogSchema.safeParse(step).success).toBe(true);
+    });
+
+    it('a failure step applies the same filter', async () => {
+        const engine = makeEngine();
+        engine.registerNodeExecutor({
+            type: 'sloppy_fail',
+            async execute() {
+                return { success: false, error: 'boom', warnings: [7], droppedFields: SLOPPY_DROPS };
+            },
+        } as never);
+        engine.registerFlow('sloppy_fail', chain('sloppy_fail', [{ id: 'f', type: 'sloppy_fail', label: 'F' }]));
+
+        await engine.execute('sloppy_fail', CALLER);
+        const step = await stepOf(engine, 'sloppy_fail', 'f');
+
+        expect(step.status).toBe('failure');
+        expect(step.droppedFields).toEqual([{ object: 'expense_line', fields: ['amount'], reason: 'readonly' }]);
+        expect('warnings' in step).toBe(false);
+        expect(ExecutionStepLogSchema.safeParse(step).success).toBe(true);
+    });
+
+    it('a value that is not a list at all (array-like, a truthy `length`) leaves neither member on the step', async () => {
+        const engine = makeEngine();
+        engine.registerNodeExecutor({
+            type: 'sloppy_write',
+            async execute() {
+                return {
+                    success: true,
+                    warnings: 'status was not written',
+                    droppedFields: { length: 1, 0: { object: 'expense_report', fields: ['status'], reason: 'readonly' } },
+                };
+            },
+        } as never);
+        engine.registerFlow('sloppy', chain('sloppy', [{ id: 'w', type: 'sloppy_write', label: 'W' }]));
+
+        await engine.execute('sloppy', CALLER);
+        const step = await stepOf(engine, 'sloppy', 'w');
+
+        expect('droppedFields' in step).toBe(false);
+        expect('warnings' in step).toBe(false);
+        expect(ExecutionStepLogSchema.safeParse(step).success).toBe(true);
+    });
+
+    it('control: the in-tree write node\'s well-typed members reach its step unchanged', async () => {
+        const engine = makeEngine();
+        engine.registerFlow('submit_for_approval', submitFlow('submit_for_approval', { status: 'submitted', note: 'please' }));
+
+        await engine.execute('submit_for_approval', CALLER);
+        const [run] = await engine.listRuns('submit_for_approval');
+        const step = run.steps.find((s) => s.nodeType === 'update_record')!;
+
+        expect(step.droppedFields).toEqual([{ object: 'expense_report', fields: ['status'], reason: 'readonly' }]);
+        expect(step.warnings).toHaveLength(1);
+        expect(typeof step.warnings![0]).toBe('string');
+        expect(ExecutionStepLogSchema.safeParse(step).success).toBe(true);
+    });
+});
+
 describe('dropEventsOf — the fold over the step log', () => {
     const AT = '2026-11-13T00:00:00.000Z';
     const step = (nodeId: string, droppedFields?: Drop[]): StepLogEntry => ({

packages/spec/authorable-surface/automation.json

@@ -159,6 +159,7 @@
     "automation/ExecutionLog:variables",
     "automation/ExecutionStepLog:branch",
     "automation/ExecutionStepLog:completedAt",
+    "automation/ExecutionStepLog:droppedFields",
     "automation/ExecutionStepLog:durationMs",
     "automation/ExecutionStepLog:error",
     "automation/ExecutionStepLog:input",
@@ -174,6 +175,7 @@
     "automation/ExecutionStepLog:skippedBy",
     "automation/ExecutionStepLog:startedAt",
     "automation/ExecutionStepLog:status",
+    "automation/ExecutionStepLog:warnings",
     "automation/ExecutionStepMetrics:acted",
     "automation/ExecutionStepMetrics:failures",
     "automation/ExecutionStepMetrics:selected",

packages/spec/scripts/lib/skill-map-guards.ts

@@ -302,6 +302,32 @@ export const TRANSITIVE_ALLOWLIST: Record<string, readonly string[]> = {
     'shared/http.zod.ts',
     'shared/suggestions.zod.ts',
   ],
+  // The closure as it stood when `automation/execution.zod.ts` (core) gained
+  // one import edge -- `data/data-engine.zod.ts`, for the drop events a flow
+  // step log carries -- that alone dragged five pointers into this index (the
+  // data engine, the Unified Query DSL's filter and query, the execution
+  // context, the access explanation) that the objectstack-automation SKILL.md
+  // does not teach. These are the transitive pointers the index already
+  // published; a new one joins by a row here, not by an import edge.
+  'objectstack-automation': [
+    'automation/api-trigger-secret-ref.zod.ts',
+    'automation/api-trigger-signature.zod.ts',
+    'automation/control-flow.zod.ts',
+    'automation/schedule-organization.zod.ts',
+    'automation/schemaless-node-config.zod.ts',
+    'data/context-tokens.zod.ts',
+    'data/date-macros.zod.ts',
+    'data/hook-body.zod.ts',
+    'kernel/metadata-protection.zod.ts',
+    'kernel/override-declaration.zod.ts',
+    'shared/expression.zod.ts',
+    'shared/identifiers.zod.ts',
+    'shared/protection.zod.ts',
+    'shared/retry-policy.zod.ts',
+    'shared/suggestions.zod.ts',
+    'shared/value-domain.zod.ts',
+    'system/job.zod.ts',
+  ],
 };
 
 /** Whether a `packages/spec/src`-relative path exists on disk; injected so the guard stays pure. */