fix: Finder live checks, select-all receipts, Finder focus, and Finder renames that never happen

CHANGELOG.md

@@ -10,13 +10,19 @@ Notable changes to Desktop Tools, newest first. Each release's GitHub release no
 
 ### Changed
 
+- `desktop_inspect` keeps containers an agent can scope into, such as a Finder window's file list after its long sidebar, when its result is cut to size, dropping other elements first. ([#126](https://github.com/githubnext/desktop-tools/pull/126))
 - Agent input aimed at any Desktop Tools process, including another build of it, or sent while one of its prompts or menus is open, is refused with `SELF_TARGET_REFUSED`, so only the user can answer its prompts or choose Pause All Input. Agents can still observe them. ([#120](https://github.com/githubnext/desktop-tools/pull/120))
 
 ### Fixed
 
 - `desktop_clipboard_read` with `save_path` saves images again; the host never enabled image export, so every such read was refused. The build now fails if the host would drop an operation the app allows. ([#125](https://github.com/githubnext/desktop-tools/pull/125))
 - A clipboard paste whose receiver never took it, such as an image pasted into a TextEdit `.rtf` document, no longer blocks every clipboard write and paste until that app quits: closing the receiving window releases it too, and the refusal says so. ([#125](https://github.com/githubnext/desktop-tools/pull/125))
 - `window_appears` no longer counts a window macOS keeps listed but ordered out on the current Space as open, so `expect` with `window_appears` for Spotlight works on macOS 27, where Siri AI (`com.apple.campo`) hosts the panel. ([#125](https://github.com/githubnext/desktop-tools/pull/125))
+- Command-A in a text field is `completed` again instead of `unknown` with a receipt mismatch: the client checked the host's receipt for a layout-resolved key without the keyboard layout, which only the main thread can read. ([#126](https://github.com/githubnext/desktop-tools/pull/126))
+- Finder windows with a long sidebar keep their focused file list in observations, so `desktop_insert` with `format: "files"` pastes into them instead of being refused for having no focused control. ([#126](https://github.com/githubnext/desktop-tools/pull/126))
+- Renaming a file through Finder's Name column with `desktop_set_cell`, or its name field with `desktop_type` or `desktop_set_value`, used to report `completed` while Finder only showed the new name and never renamed the file. It is now refused before input with `CELL_NOT_EDITABLE`. ([#126](https://github.com/githubnext/desktop-tools/pull/126))
+- A `desktop_key` sequence that stops partway reports `KEY_SEQUENCE_STOPPED` whatever stopped it, as when Command-N moves focus to a new window, instead of a bare `DESKTOP_ACTION_FAILED`, and says when the key that stopped it may have been sent. ([#126](https://github.com/githubnext/desktop-tools/pull/126))
+- `desktop_read` with `table: true` no longer reports a role description such as "cell", "column" or "image" as cell text or a column title, or a folder's disclosure triangle as `0` in its Name cell, and names a sorted column by its title rather than its sort direction. ([#126](https://github.com/githubnext/desktop-tools/pull/126))
 
 ## [0.13.1] - 2026-10-17
 

dist/build.js

@@ -74,6 +74,8 @@ const patches = [
     "peekaboo-caller-admission.patch",
     "peekaboo-clipboard-image-export-allowlist.patch",
     "peekaboo-paste-window-release.patch",
+    "peekaboo-layout-shortcuts.patch",
+    "peekaboo-focus-budget.patch",
 ].map((name) => join(native, "patches", name));
 function spawn(command, env) {
     return spawnSync(command[0], command.slice(1), { encoding: "utf8", env: env ?? process.env });

dist/client.d.ts

@@ -24,5 +24,7 @@ export declare function hostRefusal<T extends {
     message: string;
     hint?: unknown;
 }>(error: T): T;
+/** An inspection's JSON within `limit`, keeping the containers an agent can scope into over other elements. */
+export declare function boundedElements(data: Record<string, unknown>, limit?: number): string;
 /** Creates a client bound to one host socket and signed client; calls are independent processes. */
 export declare function createDesktop(config: DesktopConfig): DesktopClient;

dist/client.js

@@ -170,15 +170,33 @@ const caller = (config) => async (args, signal, options = {}) => {
         ...(value.target_receipt ? { target_receipt: value.target_receipt } : {}),
     };
 };
-function bounded(data, field, limit = MAX_TEXT) {
+/**
+ * Containers an agent can pass as `scope` to list what a long result left out. A Finder window's long sidebar can
+ * fill the result before its file list, so trimming keeps these and drops other elements first.
+ */
+const SCOPE_ROLES = new Set([
+    "AXWindow", "AXSheet", "AXDrawer", "AXSplitGroup", "AXScrollArea", "AXOutline", "AXTable", "AXList", "AXBrowser",
+    "AXTabGroup", "AXWebArea",
+]);
+function scopeContainer(element) {
+    const role = element?.attributes?.role;
+    return typeof role === "string" && SCOPE_ROLES.has(role);
+}
+/** An inspection's JSON within `limit`, keeping the containers an agent can scope into over other elements. */
+export function boundedElements(data, limit = MAX_TEXT) {
+    return bounded(data, "ui_elements", limit, scopeContainer);
+}
+/** Fits `field` within `limit` by dropping its last values, those `keep` accepts only after all others. */
+function bounded(data, field, limit = MAX_TEXT, keep) {
     const source = data[field];
     if (!Array.isArray(source))
         throw new Error(`Native inspection returned no ${field} array.`);
     const values = [...source];
     const value = { ...data, [field]: values };
     let text = JSON.stringify(value);
     while (Buffer.byteLength(text) > limit && values.length) {
-        values.pop();
+        const index = keep ? values.findLastIndex((v) => !keep(v)) : -1;
+        values.splice(index >= 0 ? index : values.length - 1, 1);
         text = JSON.stringify({
             ...value,
             truncated: true,
@@ -367,15 +385,15 @@ async function inspect(native, request, signal) {
             : undefined;
         const { image, width, height } = await screenshot(path, join(directory, "image.jpg"), signal);
         const { screenshot_raw: _raw, screenshot_annotated: _annotated, ...observation } = data;
-        const text = bounded({
+        const text = boundedElements({
             ...observation,
             ...(saved ? { saved } : {}),
             image: {
                 width,
                 height,
                 note: "Screenshot, scaled down when large but never up; Accessibility bounds remain in their original coordinate system. Pointer points use fractions of this image: x from the left edge and y from the top, each >= 0 and < 1.",
             },
-        }, "ui_elements", limit);
+        }, limit);
         return { text, image };
     }
     catch (error) {
@@ -1266,7 +1284,7 @@ async function act(native, request, signal) {
             throw new Error("The target application changed after the action.");
         }
         return {
-            text: bounded({ ...fresh, action: data }, "ui_elements"),
+            text: boundedElements({ ...fresh, action: data }),
             image: observation.image,
             outcome,
             isError: false,
@@ -1429,7 +1447,7 @@ async function observeManagement(native, request, action, signal) {
             throw new Error("The application changed process generation before the later inspection.");
         }
         return {
-            text: bounded({ ...data, ...fresh }, "ui_elements"),
+            text: boundedElements({ ...data, ...fresh }),
             image: observation.image,
             outcome: "completed",
             isError: false,

dist/table.js

@@ -10,8 +10,12 @@ const MAX_TEXT = 96_000;
 const HEADER_REACH = 40;
 export class TableError extends Error {
 }
+/** The element's label, unless it is only its role description, as an unnamed cell, column or icon reports. */
+function ownLabel(element) {
+    return element.label?.trim() && element.label.trim() === element.role_description?.trim() ? null : element.label;
+}
 function text(element) {
-    for (const piece of [element.value, element.title, element.label]) {
+    for (const piece of [element.value, element.title, ownLabel(element)]) {
         if (typeof piece === "string" && piece.trim())
             return piece.trim();
     }
@@ -63,6 +67,9 @@ function tableLayout(elements) {
                 next = end(next) - 1;
                 continue;
             }
+            // A folder's disclosure triangle reads 0 or 1; the row's expanded says the same.
+            if (element.role === "AXDisclosureTriangle")
+                continue;
             const piece = text(element);
             if (piece && !pieces.includes(piece))
                 pieces.push(piece);
@@ -90,7 +97,7 @@ function tableLayout(elements) {
     const columnTitles = inside
         .filter((index) => parents[index] === table && elements[index].role === "AXColumn")
         .sort((a, b) => elements[a].bounds.x - elements[b].bounds.x)
-        .map((index) => (elements[index].title || elements[index].label || elements[index].description || "").trim());
+        .map((index) => (elements[index].title || ownLabel(elements[index]) || elements[index].description || "").trim());
     if (columnTitles.some(Boolean))
         columns = columnTitles;
     const bounds = elements[table].bounds;
@@ -105,8 +112,10 @@ function tableLayout(elements) {
             && element.bounds.y + element.bounds.height / 2 < top
             && element.bounds.x < bounds.x + bounds.width && element.bounds.x + element.bounds.width > bounds.x)
             .sort((a, b) => a.element.bounds.x - b.element.bounds.x);
-        if (buttons.length)
-            columns = buttons.map(({ element }) => text(element) || (element.description ?? "").trim());
+        // A sorted column's button holds its sort direction as its value, so its name comes first.
+        if (buttons.length) {
+            columns = buttons.map(({ element }) => (element.title || ownLabel(element) || element.description || text(element)).trim());
+        }
     }
     if (!columns && rows.length) {
         // A web table's header cells say so in their role description.

dist/tools.js

@@ -714,7 +714,7 @@ export function desktopTools({ place, app }) {
         {
             name: "desktop_set_cell",
             op: "set-cell",
-            description: `Write one cell of a table read with desktop_read table true, then read the table again to confirm it. Pass that read's pid, window, scope snapshot as snapshot and scope element as table, the row's element ID, and column as an exact title or 0-based index. The cell's first control is written through desktop_type or desktop_set_value, with their rules. Completed only when the control reads back the value and the row's other cells are unchanged; otherwise unknown, with cell.readback differs, row_changed or unreadable. Cells with only text, as in Numbers and Excel, are refused: click the cell, desktop_insert, then desktop_key enter.`,
+            description: `Write one cell of a table read with desktop_read table true, then read the table again to confirm it. Pass that read's pid, window, scope snapshot as snapshot and scope element as table, the row's element ID, and column as an exact title or 0-based index. The cell's first control is written through desktop_type or desktop_set_value, with their rules. Completed only when the control reads back the value and the row's other cells are unchanged; otherwise unknown, with cell.readback differs, row_changed or unreadable. Cells with only text, as in Numbers and Excel, are refused: click the cell, desktop_insert, then desktop_key enter. Finder file names are refused with CELL_NOT_EDITABLE, since Finder would show the name without renaming the file; rename with a shell mv.`,
             parameters: {
                 type: "object",
                 required: ["pid", "window", "snapshot", "table", "row", "column", "value"],

docs/recipes.md

@@ -106,6 +106,21 @@ To copy files picked in Finder instead, `desktop_click` the first one and the re
 - A name that already exists is refused with `PASTE_NAME_EXISTS` before anything is pasted. A manual Command-V would open a Replace / Keep Both / Stop dialog instead.
 - A drag between Finder windows also copies, but its result is always `unknown`.
 
+## Rename a file in Finder
+
+A shell `mv` is simpler and can be checked. Finder shows a name written into a file's name field through Accessibility without renaming the file, so `desktop_set_cell`, `desktop_type` and `desktop_set_value` refuse it with `CELL_NOT_EDITABLE`. In Finder itself the rename is foreground input *(unverified)*:
+
+1. `desktop_focus` the Finder window target, `desktop_click` the file's row, then inspect.
+2. `desktop_key` `enter` to edit the name, then inspect.
+3. `desktop_insert` the new name and `desktop_key` `enter`.
+
+**Verify:** `ls` the folder. The row's text alone doesn't prove the rename.
+
+**Goes wrong:**
+
+- A name already in the folder makes Finder ask to use another name.
+- With the window in the background, the name editor doesn't hold the window's keyboard focus and the insert is refused.
+
 ## Open something with Spotlight
 
 `desktop_open` or `desktop_launch` is better when the target is known. Use Spotlight only for what needs its search.