feat(service-automation): the automation engine holds a connector instance by its package and name, and a dispatch addresses the one its caller's name resolves to

.changeset/31699-connector-instance-by-package.md

@@ -0,0 +1,45 @@
+---
+'@objectstack/service-automation': patch
+---
+
+feat(service-automation): the automation engine holds a connector instance by its package and name, and a dispatch addresses the one instance its caller's name resolves to (ADR-0150 D4, #31699)
+
+Clause-②: yes (widening) — additive in-process surface only: `AutomationEngine` gains `resolveConnectorInstance`, `resolveConnectorInstanceFor` and `resolveConnectorActionInputSchema`; `registerConnector`, `registerDegradedConnector` and `unregisterConnector` take an optional instance (its package, and `soleOfName`); `resolveConnectorAction`, `resolveConnectorActionEffect`, `getConnectorOrigin`, `getConnectorDegradedReason` and `getConnectorDegradedHold` take an optional package; the barrel exports the types `ConnectorInstanceResolution` and `ConnectorInstanceOptions`. Dispatch answers change in one transitional state (an unstamped entry whose name gains a second holder after boot), as an intended tightening; the phase-1 one-holder refusal keeps a supported install out of that state, since it refuses a second holder at install and at package registration. Three smaller answers move too, each listed under "What answers differently" below: a dispatch that joins a refresh in flight for another connection row is refused 503 once, the store's two-holder refusal names a different remedy, and a re-registered instance moves to the end of the connector list. Every other dispatch, door and HTTP answer is as before: the declared-connector reconcile still keeps one instance per name (the one-holder backstop, its duplicate-name refusal, and `soleOfName` on every declarative registration), and a lookup by bare name answers that one instance.
+Metadata-protocol: none
+Skills-docs: none — nothing a skill or a docs page teaches changes while one instance per name is kept; the two-holder connector docs come with the PR that narrows the backstop.
+
+Part of #31699 (ADR-0150 child B, PR 3b). PR 3a-ii made the connection store, the connect door and the OAuth callback address an instance by package and name; this PR makes the engine hold and dispatch it so.
+
+**The engine's instances.**
+
+- The registered connectors are held by `(package, name)` (`ConnectorInstances`): a declarative instance by the package the connection door addresses it by (`connectorInstancePackage`: its real stamp; none for the environment's stored entry; the name's durable holder for an unstamped or compatibility-default entry), a plugin's connector by no package.
+- A plugin's connector and a declarative instance still never share a name, whatever package the instance is of (ADR-0097 §4). With `soleOfName`, a declarative registration drops any other declarative instance of the name: the reconcile registers every instance so, and keeps one per name, until the backstop narrows.
+- A read by bare name answers the one instance of the name, and none when there are two, never one of them. `getRegisteredConnectors` lists each name once; `getConnectorDescriptors` lists every instance.
+
+**Dispatch.**
+
+- `connector_action` resolves its `connectorId` for the calling flow's package (`resolveConnectorInstance`): a refused name is refused as before, naming every candidate; a resolved holder is dispatched to its own instance, and its action's `inputSchema` and `effect` are read from that instance. A caller with no package, a name no registry holds, or a resolved holder with no instance registered under it (a plugin's connector of a declared name) is answered by the one instance of the name, as the dispatch by name answered; two are not dispatched. `connectorNameRefusal` answers as before.
+- The OAuth2 grant handle asks its instance's package on every read of the connection store, so an unstamped entry's connections are read under the package its name resolves to now, not when the handle was made. A cached access token is served only for the connection row it was obtained for (its row id, beside its revision): a read that now chooses another row (another package's connection of the organization) refreshes, never serves the other row's token.
+
+**The reconcile.**
+
+- The instance package is part of a provider-bound entry's desired signature, so an unstamped entry whose name's holder changes (installed, uninstalled, a second holder) is registered again under the package it resolves to now, on the next reconcile. The engine's instance and the connection the grant handle reads stay one package.
+- An instance the reconcile drops (no longer declared, degraded and removed) is unregistered by name, every instance of it: the reconcile keeps one per name, whatever package it was registered under.
+- Limit: the boot's second reconcile (packages registered during the boot) is skipped when the declared list's fingerprint is unchanged, and the fingerprint carries no package, so a holder that changes in that window without the declared list changing is followed on the next reload or restart.
+
+**What answers differently.**
+
+- An unstamped (or compatibility-default) declared entry whose name gains a second durable holder: its dispatch was answered with the connection of the package that held the name at boot (200) until a restart. It is now read under the package the name resolves to at that moment, which is no one package, so it is refused 503 (no connection of its own): under two holders a row is its own package's only (PR 3a-ii's store rule), as the connection door already refuses that name (409). It is recovered by stamping the entry or by uninstalling the second holder. A holder that is uninstalled or replaced, leaving one holder, changes no answer: every row of the name still answers the one instance, and the next reconcile registers the instance again under the package it resolves to now (any provider-bound instance is re-materialized on such a change). A re-materialized OAuth2 instance starts with a new grant handle whose access-token cache is empty, so a connection with no refresh token reads `reconnect_required` at its next dispatch, as after a restart.
+- A dispatch that joins an access-token refresh already in flight for another connection row of the organization (the instance's package changed while the refresh was out) is refused 503, never served that row's token; the next dispatch refreshes its own row.
+- The store's refusal of a row of a two-holder name that names none of the holders now names the remedy that works for it: `os migrate stored-references bind` for a row naming no package, and uninstalling one of the holders for a row already naming another package, which the bind refuses (PR 3a-ii contract review N4).
+- On a change of holder, the instance is re-registered, so its name moves to the end of `getRegisteredConnectors()` and `GET /automation/connectors`.
+
+**Not in this PR (moved to the PR that narrows the backstop).**
+
+- The materializer's own bookkeeping (`desired`, the materialized, degraded and credential-refusal maps, `grantHandles`, retry and dematerialize) stays keyed by name: under the backstop the reconcile cannot hold two instances of one name, so re-keying it is observable only once the duplicate-name refusal is lifted, and lands with it and with a two-live-instance test. Its registrations and unregistrations already pass the instance package.
+- The connector pull keeps reading its connector by bare name (the one instance): whether a mapping's connector is resolved for the job's package, and refused where that name is refused, changes an answer and is decided with the narrowing.
+- The resolved-holder fallback above (a resolved holder with no registered instance answered by the one instance of the name) is right only while one instance per name is kept; with two instances alive the narrowing PR makes it `none`.
+- `ConnectorDescriptor.packageId`, the addressed connection door (`?package=`), connector into the namespace-scoped type sets, and the backstop itself.
+- `registerConnector(def, handlers, 'declarative', { packageId })` without `soleOfName` is public and keeps two instances of one name side by side (the engine tests use it). So does `registerConnector(def, handlers, 'declarative')` with no instance argument over a name the reconcile holds under a package: it is registered under no package beside that instance instead of replacing it, and a bare-name read answers neither until the next reconcile. Nothing in-tree registers either way outside the tests; the reconcile always passes the instance with `soleOfName`.
+
+**In-process APIs.** Listed under Clause-② above. All additive; every new parameter is optional and its absence answers as before, except the declarative registration with no instance argument named in the list above.

packages/services/service-automation/src/builtin/connector-nodes.ts

@@ -65,13 +65,17 @@ export function registerConnectorNodes(engine: AutomationEngine, ctx: PluginCont
             // [ADR-0150 D3, epic #31527 child E5] A packaged flow names its
             // connector from its package: refused (every candidate named) when
             // the name does not resolve for that package — never dispatched on
-            // a guess.
-            const nameRefusal = engine.connectorNameRefusal(cfg.connectorId, context);
-            if (nameRefusal) {
-                return refuseNode(`connector_action '${node.id}': ${nameRefusal.message}`);
+            // a guess. [ADR-0150 D4, #31699] Otherwise the step dispatches the
+            // one instance that answer addresses, by its package.
+            const instance = engine.resolveConnectorInstance(cfg.connectorId, context);
+            if (instance.status === 'refused') {
+                return refuseNode(`connector_action '${node.id}': ${instance.refusal.message}`);
             }
+            // `none`: an address no instance answers; every read below then
+            // finds nothing registered, as for a name never registered.
+            const packageId = instance.status === 'instance' ? instance.packageId : undefined;
 
-            const handler = engine.resolveConnectorAction(cfg.connectorId, cfg.actionId);
+            const handler = packageId === undefined ? undefined : engine.resolveConnectorAction(cfg.connectorId, cfg.actionId, packageId);
             if (!handler) {
                 // A degraded declarative instance (#3017) is registered but has no
                 // handlers — say WHY dispatch fails and what recovers it, instead
@@ -80,9 +84,9 @@ export function registerConnectorNodes(engine: AutomationEngine, ctx: PluginCont
                 // was refused. [#28016] Recovery is automatic only on the backoff:
                 // a held instance (a configuration fault met on a retry, a refused
                 // credential the platform will not re-send) names its fix instead.
-                const degraded = engine.getConnectorDegradedReason(cfg.connectorId);
+                const degraded = packageId === undefined ? undefined : engine.getConnectorDegradedReason(cfg.connectorId, packageId);
                 if (degraded) {
-                    const hold = engine.getConnectorDegradedHold(cfg.connectorId);
+                    const hold = engine.getConnectorDegradedHold(cfg.connectorId, packageId);
                     // A reason that is a full sentence keeps one full stop.
                     const reason = degraded.replace(/\.\s*$/, '');
                     return {
@@ -123,7 +127,7 @@ export function registerConnectorNodes(engine: AutomationEngine, ctx: PluginCont
             // #4395 lets the action declare `effect`, and this is where the
             // declaration is spent. Resolved BEFORE dispatch so both the
             // success and the failure path read one value.
-            const effect = engine.resolveConnectorActionEffect(cfg.connectorId, cfg.actionId);
+            const effect = engine.resolveConnectorActionEffect(cfg.connectorId, cfg.actionId, packageId);
 
             // #27327 — `input` is a value-slot map: a top-level CEL envelope is
             // evaluated in the run's scope, every other value handed on as a
@@ -143,7 +147,7 @@ export function registerConnectorNodes(engine: AutomationEngine, ctx: PluginCont
             // (#28040). A `runtime` failure, not a guard: the input can come
             // from run data, so a `fault` edge may handle it.
             const verdict = judgeConnectorInput(
-                findConnectorActionInputSchema(engine, cfg.connectorId, cfg.actionId),
+                findConnectorActionInputSchema(engine, cfg.connectorId, cfg.actionId, packageId),
                 input,
             );
             if (verdict) {

packages/services/service-automation/src/connector-connection-door.test.ts

@@ -247,6 +247,7 @@ async function boot(
 ) {
     const ql = await dataEngine();
     const provider = grantedProvider();
+    let captured: any;
     const kernel = new LiteKernel({ logger: { level: 'debug', format: 'json' } } as never);
     kernels.push(kernel);
     kernel.use(new AutomationServicePlugin({
@@ -260,6 +261,7 @@ async function boot(
         version: '1.0.0',
         dependencies: ['com.objectstack.service-automation'],
         async init(ctx: any) {
+            captured = ctx;
             const listItems = (t: string) => (t === 'connector' ? declared : []);
             const registry = opts.registry ? Object.assign(Object.create(opts.registry), { listItems }) : { listItems };
             ctx.registerService('objectql', { registry });
@@ -283,7 +285,9 @@ async function boot(
         const query = opts2.query ?? { code: opts2.code ?? GOOD_CODE, state };
         return { begun, state, done: () => engine.completeConnectorConnection({ query: { state, ...query }, context: (opts2.as ?? context) as never }) };
     };
-    return { ql, engine, provider, call, connect, kernel };
+    /** Fire the reload hook: the runtime reconcile, over `declared` as it stands now. */
+    const reload = async () => { await captured.trigger('metadata:reloaded'); await flush(); };
+    return { ql, engine, provider, call, connect, kernel, reload };
 }
 
 const rowsOf = async (ql: ObjectQL) => (await ql.find(CONNECTOR_CONNECTION_OBJECT, { context: SYSTEM } as never)) as any[];
@@ -1105,6 +1109,66 @@ describe('ADR-0150 D4 [#31699] — the connection door connects one instance, by
         expect(await rowsOf(ql)).toHaveLength(0);
     });
 
+    it('[3b] the materializer registers the instance by its package, and the dispatch handle reads the package as the name resolves now', async () => {
+        const registry = registryHolding('com.example.crm');
+        const { connect, call, engine } = await boot([crm({ _packageId: undefined })], { registry });
+        await (await connect()).done();
+        expect(engine.resolveConnectorInstanceFor('crm', null)).toEqual({ status: 'instance', packageId: 'com.example.crm' });
+        await expect(call('org_a')).resolves.toMatchObject({ status: 200 });
+        // A second durable holder: the unstamped entry is no one package's any
+        // more, so its connection (the package's row) is not its own.
+        hold(registry, OTHER);
+        await expect(call('org_a')).rejects.toMatchObject({ status: 503 });
+    });
+
+    it('[3b] the holder of an unstamped entry is replaced, leaving one holder: the dispatch keeps its connection', async () => {
+        const registry = registryHolding('com.example.crm');
+        const { connect, call, engine } = await boot([crm({ _packageId: undefined })], { registry });
+        await (await connect()).done();
+        registry.uninstallPackage('com.example.crm');
+        registry.unregisterItemsByPackage('com.example.crm');
+        hold(registry, OTHER);
+        expect(engine.resolveConnectorInstanceFor('crm', null)).toEqual({ status: 'instance', packageId: 'com.example.crm' });
+        await expect(call('org_a')).resolves.toMatchObject({ status: 200 });
+    });
+
+    it('[3b, review M1] the package an unstamped entry is registered under follows the name\'s holder, and a removed entry leaves no instance behind', async () => {
+        // Not connected: the instance is the degraded husk.
+        const registry = registryHolding('com.example.crm');
+        const declared: unknown[] = [crm({ _packageId: undefined })];
+        const { engine, reload } = await boot(declared, { registry });
+        expect(engine.resolveConnectorInstanceFor('crm', null)).toEqual({ status: 'instance', packageId: 'com.example.crm' });
+        expect(engine.getConnectorDegradedReason('crm')).toBeDefined();
+        // The holder is uninstalled; the declared list is unchanged. The
+        // reconcile registers the instance again, under no package.
+        registry.uninstallPackage('com.example.crm');
+        registry.unregisterItemsByPackage('com.example.crm');
+        await reload();
+        expect(engine.resolveConnectorInstanceFor('crm', null)).toEqual({ status: 'instance', packageId: null });
+        expect(engine.getConnectorDescriptors().filter((d) => d.name === 'crm')).toHaveLength(1);
+        // The entry is removed: no instance of the name is left, under any package.
+        declared.length = 0;
+        await reload();
+        expect(engine.getRegisteredConnectors()).not.toContain('crm');
+        expect(engine.getConnectorDegradedReason('crm')).toBeUndefined();
+    });
+
+    it('[3b, review M1] an entry removed while its package holds the name: both the degraded and the live instance are dropped', async () => {
+        const declared: unknown[] = [crm({ _packageId: undefined })];
+        const degraded = await boot(declared, { registry: registryHolding('com.example.crm') });
+        declared.length = 0;
+        await degraded.reload();
+        expect(degraded.engine.getRegisteredConnectors()).not.toContain('crm');
+
+        const declaredLive: unknown[] = [crm({ _packageId: undefined })];
+        const live = await boot(declaredLive, { registry: registryHolding('com.example.crm') });
+        await (await live.connect()).done();
+        expect(live.engine.getConnectorDegradedReason('crm')).toBeUndefined();
+        declaredLive.length = 0;
+        await live.reload();
+        expect(live.engine.getRegisteredConnectors()).not.toContain('crm');
+    });
+
     it('a resolved holder no declared entry is the instance of: the one declared entry is served, with its own package', async () => {
         const { connect, ql } = await boot([crm({ _packageId: OTHER })], { registry: registryHolding('com.example.crm') });
         await expect((await connect()).done()).resolves.toMatchObject({ status: 'connected' });

packages/services/service-automation/src/connector-connection-package.test.ts

@@ -175,7 +175,9 @@ describe('[#31699, contract review F1] a declaration that moves keeps its connec
         const store = storeOver(ql, twoHolders);
         const refusal = await store.read('org_a', 'crm', A).then(() => undefined, (e: unknown) => e);
         expect(refusal).toBeInstanceOf(ConnectorConnectionConflictError);
-        expect((refusal as Error).message).toMatch(/os migrate stored-references bind/);
+        // The bind never moves a row that already names a package: the remedy is to uninstall one holder.
+        expect((refusal as Error).message).toMatch(/Uninstall one of the packages/);
+        expect((refusal as Error).message).not.toMatch(/stored-references bind/);
         await expect(store.revoke('org_a', 'crm', undefined, A)).rejects.toBeInstanceOf(ConnectorConnectionConflictError);
         expect(await packages(ql)).toEqual(['com.acme.gone']);