fix(metadata-protocol)!: the metadata lists serve nothing a disabled package's customization pair or a dependent's added bundle member carries for it (ADR-0150 D3)

.changeset/32354-list-doors-disabled-holder.md

@@ -0,0 +1,24 @@
+---
+'@objectstack/metadata-protocol': patch
+---
+
+fix(metadata-protocol)!: the metadata lists serve nothing a disabled package's customization pair or a dependent's added bundle member carries for it (ADR-0150 D3, #32354)
+
+Clause-②: no (narrowing)
+Metadata-protocol: none
+
+<!-- adr-0087: not-required (no-migration-prescription) No key of any schema is added, removed, renamed or re-shaped, and no export changes shape. The moved answers are list reads under a reversible package state: no stored row changes, so there is nothing for a migration to rewrite and no tombstone. -->
+
+**BREAKING** for a reader of a metadata list while a package is disabled. A disabled package holds no name (ADR-0150 D3), and the by-name reads already withheld what it carries; three lists still served it, by the order things were registered and saved. Measured over a real `SchemaRegistry`, both registration orders, the pair's row stored before and after: package A ships `welcome` in `en-US`; its pair `A.custom` stores a `fr-FR` row of it, or package DEP (depending on A) ships a `fr-FR` addition to A's template; A is disabled.
+
+- `GET /meta/email_template` and the execution list (`getMetaItemsForExecution`, no package): the pair's `fr-FR` row and DEP's `fr-FR` addition were listed → not listed, with one holder or beside an enabled second holder of the name. A send naming no package reads this list, so it no longer sends either while A is disabled.
+- `GET /meta/email_template?package=A` and its execution list: the pair's `fr-FR` row was listed → `[]`, as `?package=A&member=fr-FR` answers 404.
+- `GET /meta/email_template?package=A.custom`: the pair's `fr-FR` row was listed → not listed, as `?package=A.custom&member=fr-FR` answers 404.
+
+The rule is the exact read's: a list scoped to a package serves no name whose exact read at that package is a disabled package's withheld item; the list naming no package leaves out an item whose own package's address is, and a bundle addition whose every base in its dependency closure is disabled. It applies to every package-scoped type, and only while some package is disabled.
+
+**Not moved.** With nothing disabled, every list answers as before. DEP's own list (`?package=DEP`) keeps its addition, as its exact read does. An enabled holder's lists, the pair's own item of a name A ships nothing of, and the per-package layer (the package export, which keeps a disabled package's items) answer as before. Re-enabling A restores every list.
+
+Until the email send door judges the served holders (#32670), a send made from DEP itself reads the list naming no package for this name, so it no longer finds DEP's `fr-FR` while A is disabled.
+
+The fix for a reader: re-enable the package to list its customization again; a dependent's addition stays readable at the dependent's own address.

packages/metadata-protocol/src/protocol-meta-list-reads.ts

@@ -18,7 +18,7 @@
  * not move.
  */
 
-import { ITEM_KEY_DISCRIMINATORS, isTenantAuthored, pairsOf } from '@objectstack/metadata-core';
+import { ITEM_KEY_DISCRIMINATORS, isTenantAuthored, packageDependencyClosure, pairsOf } from '@objectstack/metadata-core';
 import { PLURAL_TO_SINGULAR, SINGULAR_TO_PLURAL } from '@objectstack/spec/shared';
 import { isAggregatedViewContainer } from '@objectstack/spec/ui';
 import {
@@ -49,7 +49,13 @@ import { storedEntryRow } from './protocol-stored-overlay-types.js';
 import { viewIdentityPatch } from './protocol-view-normalization.js';
 import { isServedOverrideRowOf, type ServedOverrideRow } from './served-override.js';
 import { installedManifestsOfEngine } from './installed-manifests.js';
-import { declinesPackageLessRowAt, holdsNameForExactRead, packageLessRowWithheld, pairMemberWithheldAt } from './protocol-exact-qualifier.js';
+import {
+    declinesPackageLessRowAt,
+    disabledExactAddressHolder,
+    holdsNameForExactRead,
+    packageLessRowWithheld,
+    pairMemberWithheldAt,
+} from './protocol-exact-qualifier.js';
 
 /** The body of `ObjectStackProtocolImplementation#getMetaItems`; the method's docblock there is its contract. */
 export async function getMetaItems(host: Pick<ProtocolHost, 'readFlattenedMetaItems'>, request: { type: string; packageId?: string; names?: readonly string[]; previewDrafts?: boolean }) {
@@ -92,6 +98,68 @@ function decorateListedItem(type: string, item: unknown): unknown {
     return { ...served, name: (item as { name: string }).name, ...(_diagnostics !== undefined ? { _diagnostics } : {}) };
 }
 
+/**
+ * [#32354, ADR-0150 D3] Whether the effective or execution list withholds an
+ * item of `name` it serves at `packageId` because the item belongs to a
+ * disabled holder: a disabled package holds no name, and the list hides what
+ * the by-name read naming the same address hides, independent of the order
+ * things were registered or saved. `undefined` (nothing withheld) for a type
+ * the registry does not read by package, a registry that cannot answer, and
+ * while no package is disabled, so no list moves then.
+ *
+ *  - `'address'` (a list scoped to `packageId`): the exact read of the name at
+ *    that address is the disabled package's withheld item
+ *    ({@link disabledExactAddressHolder}: the package is disabled, or it is a
+ *    customization pair whose every base holding the name is). Before, the
+ *    list scoped to a disabled base, or to its pair, listed the pair's row at
+ *    a member the base does not ship while both exact reads answered 404.
+ *  - `'unscoped'` (the list naming no package, `packageId` the item's own
+ *    binding): the same, which leaves out a disabled holder's pair row; and,
+ *    for a bundle (`ITEM_KEY_DISCRIMINATORS`), an addition to a disabled
+ *    holder's bundle: `packageId` holds the name only as an addition (the
+ *    registry's served holders leave it out for every other reader) and each
+ *    holder in its declared dependency closure is disabled. Both were listed,
+ *    so a send naming no package, which reads this list, sent them. The
+ *    dependent's own address keeps serving its addition, as its exact read
+ *    does.
+ *
+ * Every holder is read with `includeDisabledPackages`, so a disable never
+ * promotes a dependent's addition to a holder. Answers are cached per call.
+ */
+function disabledHolderListing(
+    host: Pick<ProtocolHost, 'engine'>,
+    type: string,
+): ((name: string, packageId: string, naming: 'address' | 'unscoped') => boolean) | undefined {
+    const registry: any = host.engine?.registry;
+    if (typeof registry?.packageScopedHoldersOf !== 'function' || typeof registry.isPackageDisabled !== 'function'
+        || typeof registry.getAllPackages !== 'function') return undefined;
+    if (!(registry.getAllPackages() as ReadonlyArray<{ enabled?: unknown; status?: unknown } | undefined>)
+        .some((pkg) => pkg?.enabled === false || pkg?.status === 'disabled')) return undefined;
+    const disabled = (id: string): boolean => registry.isPackageDisabled(id) === true;
+    const isBundle = ITEM_KEY_DISCRIMINATORS[type] !== undefined;
+    const answers = new Map<string, boolean>();
+    const judge = (name: string, packageId: string, naming: 'address' | 'unscoped'): boolean => {
+        if (disabledExactAddressHolder(host, { type, name, packageId }) !== undefined) return true;
+        if (naming !== 'unscoped' || !isBundle) return false;
+        const every: readonly string[] = registry.packageScopedHoldersOf(type, name, null, { includeDisabledPackages: true }) ?? [];
+        if (every.includes(packageId)) return false;
+        const own: readonly string[] = registry.packageScopedHoldersOf(type, name, packageId, { includeDisabledPackages: true }) ?? [];
+        if (!own.includes(packageId)) return false;
+        const reach = new Set(packageDependencyClosure(packageId, registry)?.dependencies ?? []);
+        const bases = every.filter((holder) => holder !== packageId && reach.has(holder));
+        return bases.length > 0 && bases.every(disabled);
+    };
+    return (name, packageId, naming) => {
+        const key = `${naming}\u0000${packageId}\u0000${name}`;
+        let answer = answers.get(key);
+        if (answer === undefined) {
+            answer = judge(name, packageId, naming);
+            answers.set(key, answer);
+        }
+        return answer;
+    };
+}
+
 /** The body of `ObjectStackProtocolImplementation#readFlattenedMetaItems`; the method's docblock there is its contract. */
 export async function readFlattenedMetaItems(host: Pick<ProtocolHost, 'artifactLockLayerAt' | 'baseServesBundleMember' | 'convertStoredItem' | 'declinesStoredRow' | 'disabledActionsOf' | 'engine' | 'environmentId' | 'expandStoredViewContainers' | 'findEnvironmentRows' | 'flowRegistrationsOf' | 'foldObjectExtendersFromRegistry' | 'getServicesRegistry' | 'governServedObject' | 'hydrateOverlayIntoRegistry' | 'isStoredEntryOfDeclinedName' | 'isUnservedSealedOverlay' | 'isWritablePackage' | 'lookupArtifactItem' | 'namesWithOwnStoredRow' | 'readActiveOverlayRows' | 'rethrowUnlessMetadataStoreUnprovisioned' | 'shippedArtifactsOf' | 'shippingPackagesOf' | 'shipsBundleName' | 'storedOverlayEntries'>, request: { type: string; packageId?: string; names?: readonly string[]; previewDrafts?: boolean },
         audience: 'served' | 'execution',
@@ -870,6 +938,21 @@ export async function readFlattenedMetaItems(host: Pick<ProtocolHost, 'artifactL
     // package's items installed elsewhere as an empty package.
     if (request.type !== 'package' && !keepDisabledPackages) {
         items = (items as any[]).filter((it) => !isItemOfDisabledPackage(host.engine.registry, it));
+        // [#32354, ADR-0150 D3] …and an item another package carries for a
+        // disabled holder ({@link disabledHolderListing}): the list scoped to
+        // a package serves no name its exact read withholds as a disabled
+        // package's, and the list naming no package leaves out a disabled
+        // holder's pair row and a dependent's addition to its bundle.
+        const withheld = disabledHolderListing(host, request.type);
+        if (withheld) {
+            items = (items as any[]).filter((it) => {
+                const itemName = (it as { name?: unknown } | null)?.name;
+                if (typeof itemName !== 'string') return true;
+                if (packageId) return !withheld(itemName, packageId, 'address');
+                const bound = boundPackageOf((it as { _packageId?: unknown })._packageId);
+                return bound === undefined || !withheld(itemName, bound, 'unscoped');
+            });
+        }
     }
     // [#28877] …and, on the effective served list, an action that launches
     // into a disabled package ({@link isActionWithheldByDisabledPackage}),

packages/objectql/src/package-scoped-exact-qualifier.test.ts

@@ -2528,3 +2528,95 @@ describe('[#32183] an App\'s pair two modules share, one of them disabled', () =
     expect(JSON.stringify(neither ?? null)).not.toMatch(/(suitex|suitey) dashboard/);
   });
 });
+
+/**
+ * [#32354, ADR-0150 D3, epic note 6820147040] The list doors beside a disabled
+ * holder: a disabled package holds no name, so no list serves what it carries
+ * for one. FROM (measured on `2b2e5b41`, both registration orders and both save
+ * orders): with A disabled, the list naming no package and its execution face
+ * listed the row A's customization pair stores at a member A does not ship,
+ * and a dependent's added locale of A's template; the lists scoped to A and to
+ * A's pair listed the pair's row too, while the exact reads of both addresses
+ * answered 404. A send naming no package reads that execution list. TO: none
+ * of them. The dependent's own list keeps its addition, as its exact read
+ * does, and an enabled holder's lists are unchanged. Re-enabled, as before.
+ */
+describe('[#32354] the list doors serve nothing a disabled holder\'s pair or dependent carries for it', () => {
+  const DEP = 'com.acme.dep';
+  const tpl = (packageId: string, locale: string) => registry.registerItem('email_template', {
+    name: 'welcome', label: `${packageId.split('.').pop()} ${locale}`, locale, subject: locale, bodyHtml: '<p>x</p>',
+  }, 'name' as never, packageId);
+  const pairRow = (name: string, label: string) => rows.push({
+    id: `row_${name}`, type: 'email_template', name, state: 'active', package_id: PAIR_A, bundle_member: 'fr-FR',
+    metadata: JSON.stringify({ name, label, locale: 'fr-FR', subject: label, bodyHtml: '<p>p</p>' }),
+  });
+  const labelsOf = async (read: Promise<{ items?: unknown[] }>, name = 'welcome') => ((await read).items ?? [])
+    .filter((it) => (it as { name?: unknown }).name === name)
+    .map((it) => (it as { label?: string }).label)
+    .sort();
+  const answers = async () => ({
+    list: await labelsOf(protocol.getMetaItems({ type: 'email_template' })),
+    execution: await labelsOf(protocol.getMetaItemsForExecution({ type: 'email_template' })),
+    atA: await labelsOf(protocol.getMetaItems({ type: 'email_template', packageId: A })),
+    executionAtA: await labelsOf(protocol.getMetaItemsForExecution({ type: 'email_template', packageId: A })),
+    atPair: await labelsOf(protocol.getMetaItems({ type: 'email_template', packageId: PAIR_A })),
+    atDep: await labelsOf(protocol.getMetaItems({ type: 'email_template', packageId: DEP })),
+    atB: await labelsOf(protocol.getMetaItems({ type: 'email_template', packageId: B })),
+  });
+  type Answers = Awaited<ReturnType<typeof answers>>;
+  const lists = (unscoped: string[], atA: string[], rest: Pick<Answers, 'atPair' | 'atDep' | 'atB'>): Answers =>
+    ({ list: unscoped, execution: unscoped, atA, executionAtA: atA, ...rest });
+
+  /** `[state, holders, carried, enabled answers, A-disabled answers]`. */
+  const STATES: ReadonlyArray<[string, readonly string[], 'pair row' | 'addition', Answers, Answers]> = [
+    ['one holder, its pair\'s row', [A], 'pair row',
+      lists(['alpha en-US', 'pair fr-FR'], ['alpha en-US', 'pair fr-FR'], { atPair: ['pair fr-FR'], atDep: [], atB: [] }),
+      lists([], [], { atPair: [], atDep: [], atB: [] })],
+    ['two holders, the disabled one\'s pair\'s row', [A, B], 'pair row',
+      lists(['alpha en-US', 'beta en-US', 'pair fr-FR'], ['alpha en-US', 'pair fr-FR'], { atPair: ['pair fr-FR'], atDep: [], atB: ['beta en-US'] }),
+      lists(['beta en-US'], [], { atPair: [], atDep: [], atB: ['beta en-US'] })],
+    ['one holder, a dependent\'s addition', [A], 'addition',
+      lists(['alpha en-US', 'dep fr-FR'], ['alpha en-US'], { atPair: [], atDep: ['dep fr-FR'], atB: [] }),
+      lists([], [], { atPair: [], atDep: ['dep fr-FR'], atB: [] })],
+    ['two holders, a dependent of the disabled one\'s addition', [A, B], 'addition',
+      lists(['alpha en-US', 'beta en-US', 'dep fr-FR'], ['alpha en-US'], { atPair: [], atDep: ['dep fr-FR'], atB: ['beta en-US'] }),
+      lists(['beta en-US'], [], { atPair: [], atDep: ['dep fr-FR'], atB: ['beta en-US'] })],
+  ];
+  const CASES = STATES.flatMap(([state, holders, carried, enabled, disabled]) =>
+    (['as listed', 'reversed'] as const).flatMap((order) => (carried === 'pair row'
+      ? (['stored first', 'stored last'] as const)
+      : (['no row'] as const)).map((save) => [state, order, save, holders, carried, enabled, disabled] as const)));
+
+  beforeEach(() => {
+    registry.installPackage({ ...(manifest(DEP) as object), dependencies: { [A]: '^1.0.0' } } as never, 'code');
+    registry.installPackage(manifest(PAIR_A), 'code');
+  });
+
+  it.each(CASES)('%s, registered %s, row %s: A disabled, no list serves what is carried for A; re-enabled, as before', async (_state, order, save, holders, carried, enabled, disabled) => {
+    if (save === 'stored first') pairRow('welcome', 'pair fr-FR');
+    const shipped: Array<[string, string]> = [...holders.map((id): [string, string] => [id, 'en-US']), ...(carried === 'addition' ? [[DEP, 'fr-FR'] as [string, string]] : [])];
+    for (const [id, locale] of order === 'reversed' ? [...shipped].reverse() : shipped) tpl(id, locale);
+    if (save === 'stored last') pairRow('welcome', 'pair fr-FR');
+
+    // CONTROL — nothing disabled: every list as on FROM.
+    expect(await answers()).toEqual(enabled);
+    registry.disablePackage(A);
+    expect(await answers()).toEqual(disabled);
+    // Read again, after the first reads hydrated the stored row: the same answers.
+    expect(await answers()).toEqual(disabled);
+    registry.enablePackage(A);
+    expect(await answers()).toEqual(enabled);
+  });
+
+  it.each(['as listed', 'reversed'] as const)('CONTROL — registered %s: the pair\'s own item of a name A ships nothing of stays listed while A is disabled, as its by-name read serves it', async (order) => {
+    const arrange = [() => tpl(A, 'en-US'), () => pairRow('farewell', 'pair own farewell')];
+    for (const step of order === 'reversed' ? [...arrange].reverse() : arrange) step();
+    registry.disablePackage(A);
+    expect(labelOf(await protocol.getMetaItem({ type: 'email_template', name: 'farewell', member: 'fr-FR' } as never))).toBe('pair own farewell');
+    for (const read of [
+      protocol.getMetaItems({ type: 'email_template' }),
+      protocol.getMetaItemsForExecution({ type: 'email_template' }),
+      protocol.getMetaItems({ type: 'email_template', packageId: PAIR_A }),
+    ]) expect(await labelsOf(read, 'farewell')).toEqual(['pair own farewell']);
+  });
+});