Skip to content

Paste clipboard images into remote and Codespace loops (#819) - #859

Merged
scgopi merged 3 commits into
mainfrom
feat/819-remote-image-paste
Oct 18, 2026
Merged

scgopi merged 3 commits into
mainfrom
feat/819-remote-image-paste

Conversation

@scgopi

@scgopi scgopi commented Oct 17, 2026 •

Copy link
Copy Markdown
Owner

Closes #819.

What changes

You can now paste an image from the Mac's clipboard into a loop running over SSH or in a GitHub Codespace.

  • Live terminal: pressing Cmd-V or Ctrl-V with only an image on the clipboard, in a remote loop's agent pane, copies the image to the host. GraphCode then types Image available at /home/…/attachments/paste-….png - inspect it. into the composer. Nothing is submitted. Text pastes, local loops and plain shell panes behave exactly as before. A short notice in the pane shows progress or the error.
  • New Node form: remote drafts accept images again. They are staged locally and uploaded when Create is pressed. If the upload fails, the form stays open with the error and no node is created. Cancelling discards local files and, best effort, anything already uploaded.

How

  • RemoteAttachmentUpload (GraphcodeKit/Sessions) sends the raw bytes on one ssh process's stdin, through the existing RemoteProjectLocation.sshInvocation. That covers plain ssh, custom ports, IPv6, ControlMaster, and both Codespace dials. There is no PTY, no base64 and no new transport.
  • On the host, a python3 writer checks the byte count and SHA-256 and writes to a temp file in ~/.graphcode/memory/<slug>/<node>/attachments. It then fsyncs, sets mode 0600 (0700 on the directory) and does an atomic os.replace. Only the count, the digest and paths GraphCode generates go into argv, each single-quoted. File names never come from the clipboard.
  • Every remote launch now grants the node's memory directory (workspacePaths) and creates its attachments/ folder before the agent starts. Copilot and Codex can therefore open images pasted mid-session.
  • Deleting a remote node or graph runs a detached, best-effort rm -rf of that node's remote memory.

Evidence

RED: xcodebuild test -only-testing:graphcodeTests/RemoteAttachmentUploadTests with main's sources and this PR's tests -> exit 65, compile errors "cannot find 'RemoteAttachmentUpload' in scope" (52), 'RemoteImagePaste' (20), 'RemoteAttachmentClient' (4)
GREEN: xcodebuild test -only-testing RemoteAttachmentUploadTests, DraftAttachmentReducerTests, RemoteImagePasteKeyTests, PromptAttachmentTests, RemoteSessionLaunchTests -> exit 0, 63 tests passed
GREEN: loopback sshd with a fake HOME (TEST_RUNNER_GRAPHCODE_LOOPBACK_RIG) -> aRealSSHHopDeliversTheExactBytes passed, 300282 bytes byte-identical on the host, mode 0600
REGRESSION: xcodebuild test -scheme graphcode (private DerivedData) -> exit 0, 2321 tests in 262 suites passed
REGRESSION: xcodebuild build -scheme graphcoded and -scheme graphcode-cli -> exit 0 and exit 0; swift build --target GraphcodeKit -> exit 0; swift-portable swift test -> exit 0
REGRESSION: swiftlint lint -> 0 errors, and the new complexity warning was removed; swift format lint --strict -> exit 0
RED: resume-grant tests with BackendCommand.swift from 2d0bbe8 -> exit 65, aResumedLoopKeepsItsDirectoryGrants failed for copilotCLI/allowTools and codex/workspace
GREEN: xcodebuild test -only-testing RemoteAttachmentUploadTests, RemoteSessionResumeTests, CopilotBackendTests, ZmxSessionLauncherTests -> exit 0, 96 tests passed
REGRESSION: xcodebuild test -scheme graphcode after ebb4e1f -> exit 0, 2325 tests in 262 suites passed; swift-portable swift test -> exit 0

Tests cover binary payloads with NUL, invalid UTF-8, newlines and shell metacharacters. They also cover a wrong digest, truncated input and extra bytes: in each case nothing is published and no temp file is left behind. The rest check argv for every host kind (including a multiplexed Codespace), the 10 MB limit, extension and file-name refusal, parsing past login-shell noise, error mapping, the launch grant, mkdir happening before zmx run, upload-then-create, a failed upload creating nothing, a stale reply for a cancelled draft creating nothing, local drafts never dialing, and a pasteboard holding text plus an image staying text.

Not verified

  • A real ⌘V in the app. The terminal interception (GhosttyTerminalNSView.keyDown) is unit-tested only for key matching. No GUI automation is available on this machine.
  • A real Codespace. It uses the same sshInvocation that every other Codespace dial uses, but it was not tried against a live or stopped Codespace.

@scgopi

scgopi commented Oct 17, 2026

Copy link
Copy Markdown
Owner Author

Independent review of 2d25c98

Verdict: no blocking issues. The transport, quoting and stale-reply handling are sound. I found one medium issue, on delete-time cost for Codespace graphs, and three low ones. Every finding below was checked against the code at this head.

Findings, by severity

# Severity Where Finding
1 🟠 Medium GraphcodeKit/Sources/ProjectRegistry.swift:731, :1319 → GraphcodeKit/Sources/Sessions/RemoteAttachmentUpload.swift:175 Every remote node delete now dials the host, with no throttle and no check for a stopped Codespace
2 🟡 Low graphcode/Sources/Infrastructure/Ghostty/GhosttyTerminalNSView+RemoteImagePaste.swift:49, GhosttyTerminalNSView.swift:318 swallowedPasteKeyCode can go stale after ⌘V and later swallow the release of a plain v
3 🟡 Low GraphcodeKit/Sources/Domain/BackendCommand.swift:115, ZmxSessionLauncher.swift:1710 The new attachments grant does not survive a resume
4 ⚪ Nit GhosttyTerminalNSView+RemoteImagePaste.swift:72 When two pastes overlap, the first to finish hides the second one's "Copying…" notice

1. Deleting nodes starts stopped Codespaces and fans out dials. removeRemoteMemory runs one detached Task per node. Each task makes its own ssh call, or gh codespace ssh for a Codespace. The repo already records that this is expensive: CodespaceSSHUser.swift:8 says "gh starts a stopped codespace it is pointed at", and ProjectPersistence+Export.swift:100-104 limits export to 4 concurrent dials to avoid "thirty tunnels racing to start a stopped codespace."

  • Failure scenario: a 30-loop Codespace graph whose codespace has idle-stopped gets deleted. That fires 30 concurrent gh codespace ssh … rm -rf calls, and each one tries to start the codespace. The cost lands on the user's billing and on the Codespaces API rate limit, just to remove directories. For almost every node those directories never held an attachment.
  • Deleting a single node from an idle Codespace graph also wakes the codespace.
  • This contradicts the PR's own rule for drafts ("a draft that is cancelled never dials, nor starts a stopped Codespace").
  • Suggestion: at minimum, skip Codespaces whose state isn't known to be live, and bound the concurrency the way export does. Better, record on the node that it uploaded something and dial only for those nodes. Alternatively, let the next ensure on that host sweep orphaned memory dirs.

2. A stale swallowed key release. interceptRemoteImagePaste stores the ⌘V key code and expects the matching keyUp to clear it. AppKit generally doesn't deliver keyUp for Command-modified keys, so after a ⌘V paste the flag stays set. The next plain v release is then dropped, and the surface sees a press with no release. Under the kitty keyboard protocol that is visible to the program as a stuck key. ⌃V is fine because its keyUp is delivered.

  • I couldn't verify this in the GUI here. A cheap fix is to clear the flag on the next keyDown as well, or to only set it when .command is absent.

3. The grant is lost on resume. launchArguments returns before any --add-dir when prompt is nil (BackendCommand.swift:115), and the resume path at ZmxSessionLauncher.swift:1710 passes prompt: nil. Take a remote Copilot loop on .allowTools, or a Codex loop on .workspace, that resumed after a host reboot or app relaunch: an image pasted mid-session may be refused, despite the PR text ("Copilot and Codex can therefore open images pasted mid-session").

  • The resume behaviour itself predates this PR, and the defaults (.allowEverything / .yolo) emit no grants anyway, so this is low. I'd soften that claim in the PR body or add a resume test.

4. Overlapping paste notices. hidePasteNotice() on success bumps the generation and hides the label even when a newer paste set the sticky notice. The second paste still completes; only its progress text disappears early. This is cosmetic.

Checked and fine

Area Result Evidence
Shell quoting / argv injection ✅ Only the count, digest and a path built from a sanitized slug, a UUID and isSafeFileName reach argv. Each is shellQuoted and then wrapped once more by remoteLoginShellCommand. The Python program is a quoted -c literal and never interpolates values.
Path traversal ✅ The file name is generated (RemoteImagePaste.fileName / the draft staging name) and refused unless it matches [A-Za-z0-9._-] with an image extension and no leading dot. SessionBriefing.slug maps everything else to -. In rm -rf ~/'…/<slug>/<UUID>', everything after ~/ is quoted and always ends in a UUID.
Partial-file exposure ✅ mkstemp writes in the same directory, then fsync, then the count and SHA-256 checks, then chmod 0600, then os.replace. The temp file is unlinked in finally. A killed ssh leaves an EOF short read, so nothing is published.
Image bytes in logs ✅ FoundationProcessRunner writes standardInput to the pipe and never logs it. Errors carry only stderr's last line or the exit code.
SIGPIPE when ssh exits before reading stdin ✅ The app doesn't ignore SIGPIPE itself, but libghostty's global init does (ThirdParty/ghostty/src/global.zig:210). The early exit therefore becomes an EPIPE that ProcessPipe.write swallows.
Stale upload replies / cancellation ✅ .uploaded checks both draftID and isUploading. Cancel resets DraftAttachments and cancels CancelID.draftUpload, which kills the child. draftID is only re-minted in openNodeForm. Add, remove and a second Create are all gated while an upload runs.
Surface teardown mid-upload ✅ [weak self], then a re-check of remoteImagePaste == target, !processHasExited and a non-nil surface before ghostty_surface_text.
Remote launch argv ✅ The memory dir is added after remotePath and deduped against the wake dir. remoteQuotedCommand keeps ~/.graphcode/… expandable. With default settings Copilot (.allowEverything) and Codex (.yolo) emit no --add-dir, so the default typed command is byte-identical. The mkdir -p -m 700 runs in the outer ensure script before zmx run, not in the typed (MAX_CANON-bound) line, and its failure can't block the launch.
New Node form ✅ Local graphs never dial (pendingRemoteUploads is empty). A failed upload keeps the form open and creates nothing. On success the local staging is discarded only once the form has closed.

Minor observation, not a finding: the upload's stdin passes through exec zsh -l -i -c (RemoteProjectLocation.swift:239). An rc file that reads stdin would consume image bytes. The digest check turns that into a clean "arrived corrupted" error rather than a bad file, so it fails safe.

I didn't rebuild or rerun the suite. This review is from reading the code at 2d25c98a, alongside the PR's own gate evidence.

scgopi added a commit that referenced this pull request Oct 17, 2026
- Remote memory removal is batched by RemoteMemoryJanitor into one rm -rf
  per project, sent one project at a time. Deleting a 30-loop graph is now one
  dial instead of 30.
- Node and graph deletion never dials a Codespace, because gh would start a
  stopped one. A cancelled draft that has just uploaded still cleans up.
- Clear the swallowed Cmd-V release on the next keyDown, since AppKit often
  sends no keyUp for a Cmd chord.
- Overlapping pastes keep the "Copying image" notice up until the last one
  finishes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: scgopi <scgopireddy@gmail.com>
@scgopi

scgopi commented Oct 17, 2026

Copy link
Copy Markdown
Owner Author

Review follow-up, in ce09ce3:

# Finding Outcome
1 Remote cleanup dials on every delete and starts stopped Codespaces Fixed. RemoteMemoryJanitor batches removals into one rm -rf per project and sends them one project at a time, so a 30-loop graph is one dial. Node and graph deletion never dial a Codespace. A cancelled draft that just uploaded still cleans up, because the codespace is known to be running.
2 A stale swallowedPasteKeyCode eats the next v release Fixed. It is cleared on every keyDown.
3 The resume path drops --add-dir grants Deferred. This is an existing gap: launchArguments returns before any grant (workspace, wake digest, briefing) when there is no prompt. Fixing it changes every Copilot/Codex resume command, so it belongs in its own change. Default settings emit no grants.
4 Overlapping paste notices hide each other Fixed. An in-flight count keeps the "Copying image" notice up until the last upload finishes.

New tests: one dial for 30 deletions plus a skipped codespace; codespace removal is refused unless explicitly included; removal of several nodes in one command.

GREEN: xcodebuild test -only-testing RemoteAttachmentUploadTests, DraftAttachmentReducerTests, RemoteImagePasteKeyTests, RemoteSessionLaunchTests -> exit 0, 54 tests passed
REGRESSION: full graphcode scheme -> 2323 tests, 1 failure in CodexReadinessGateTests (a 5s timing test that this change doesn't touch); that suite alone rerun -> exit 0, passed. graphcoded and graphcode-cli builds -> exit 0. swiftlint 0 errors, swift format --strict exit 0.

scgopi added a commit that referenced this pull request Oct 17, 2026
launchArguments returned before any --add-dir when there was no prompt,
so a resumed session lost the workspace and attachments grants it was
launched with. Defaults (Copilot YOLO, Codex YOLO) still emit none.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: scgopi <scgopireddy@gmail.com>
@scgopi

scgopi commented Oct 17, 2026

Copy link
Copy Markdown
Owner Author

Finding 3 is fixed in ebb4e1f. All four review findings are now addressed in this PR.

# Finding Outcome
3 Resume passed prompt: nil, so launchArguments returned before any --add-dir ✅ Fixed: a prompt-less launch now still emits Copilot's readableDirectories and Codex's writableDirectories grants. Default settings (both set to YOLO) still emit none, so default argv is unchanged.

RED: resume-grant tests with BackendCommand.swift from 2d0bbe8 -> exit 65, both copilotCLI/allowTools and codex/workspace cases failed
GREEN: focused suites -> 96 passed
REGRESSION: full suite -> 2325 passed; swift-portable -> exit 0

scgopi and others added 3 commits October 17, 2026 16:10
An image on the Mac's pasteboard now reaches a loop running over SSH or in a
GitHub Codespace. RemoteAttachmentUpload sends the raw bytes on one ssh
process's stdin, through the existing sshInvocation (plain ssh, ports,
multiplexing, both Codespace dials). A python3 writer on the host checks the
byte count and SHA-256, writes 0600 into ~/.graphcode/memory/<slug>/<node>/
attachments, and publishes with an atomic rename.

- Live terminal: an image-only Cmd-V or Ctrl-V on a remote agent surface is
  uploaded, and "Image available at <path> - inspect it." is typed into the
  composer. Nothing is submitted. Text pastes are unchanged.
- New Node form: remote drafts take images again. They are staged locally and
  uploaded when Create is pressed. A failed upload keeps the form open and
  creates nothing.
- Every remote launch grants and creates the node's memory directory, so
  Copilot and Codex can open images pasted mid-session.
- Deleting a remote node or graph removes its remote memory, best effort.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: scgopi <scgopireddy@gmail.com>
- Remote memory removal is batched by RemoteMemoryJanitor into one rm -rf
  per project, sent one project at a time. Deleting a 30-loop graph is now one
  dial instead of 30.
- Node and graph deletion never dials a Codespace, because gh would start a
  stopped one. A cancelled draft that has just uploaded still cleans up.
- Clear the swallowed Cmd-V release on the next keyDown, since AppKit often
  sends no keyUp for a Cmd chord.
- Overlapping pastes keep the "Copying image" notice up until the last one
  finishes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: scgopi <scgopireddy@gmail.com>
launchArguments returned before any --add-dir when there was no prompt,
so a resumed session lost the workspace and attachments grants it was
launched with. Defaults (Copilot YOLO, Codex YOLO) still emit none.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: scgopi <scgopireddy@gmail.com>
@scgopi
scgopi force-pushed the feat/819-remote-image-paste branch from ebb4e1f to 76ce315 Compare October 17, 2026 23:10
@scgopi
scgopi merged commit ebd1c8c into main Oct 18, 2026
24 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support clipboard image paste in remote nodes, including GitHub Codespaces

1 participant