Skip to content

Commit e66c7d6

Browse files
authored
feat(cutline): send a clip, moment, marker or media reference to the chat (#99)
* feat(cutline): send a clip, moment, marker or media reference to the chat Right-click a clip, an empty spot on a track, the ruler, a marker or a file in the Project bin and choose "Send to chat". The Host shows its own confirmation and appends a plain-text reference to the chat draft: clip, track and media ids, timeline seconds and frame, the open sequence and composition, and the project revision, so the Agent can act on "here" without guessing. - chat-context.js builds the reference from pure functions; labels follow the interface language, ids, seconds and user text stay as stored, user text is reduced to one bounded line and the whole text stays under 8,000 characters. - Right-clicking a marker now opens "Edit marker..." / "Send marker to chat"; double-click still opens the editor. - The items stay visible and disabled when the client has no message.append. - Unit tests for the text and frame snapping at seven frame rates; a Chromium test drives every menu with the Host bridge stubbed. - Checked on MiniMax Code 3.1.9 for Windows. Version 0.2.0. * fix(cutline): list referenced clips top to bottom and label the media id as "id" "Clips here" and multi-clip references now follow the timeline's lane order (V3, V2, V1, A1, A2...) instead of the order the clips were added. The editor passes its live track order; without one, video lanes run from the highest number down and audio lanes from A1 up. A clip's source line reads "id m_..." instead of an untranslated "media m_...". * test(cutline): retry an MCP request once when its reused socket was reset fetch keeps an idle socket for 4 s and the server closes it after 5 s. When a synchronous test step blocks the process long enough, both timers fire together and the reused socket is reset before the request is read, which made "the tool answers prepare, status and cancel" fail under full-suite load. Only ECONNRESET / UND_ERR_SOCKET is retried, once.
1 parent 03d147e commit e66c7d6

15 files changed

Lines changed: 928 additions & 11 deletions

File tree

‎plugins/klarkxy/cutline/.minimax-plugin/plugin.json‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22
"schemaVersion": 1,
33
"name": "cutline",
44
"displayName": "Cutline",
5-
"version": "0.1.1",
5+
"version": "0.2.0",
66
"description": "浏览器里的视频剪辑台:时间线剪切、字幕、动效、调色与导出,聊天里一句话就能剪片。",
77
"author": "klarkxy",
88
"icon": "icon.png",

‎plugins/klarkxy/cutline/PATCHES.md‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,9 @@
11
# Patches to the vendored FableCut source
22

3+
## Send to chat
4+
5+
Right-click menus can hand the conversation a plain-text reference to what the user points at. The clip menu ends with **Send to chat** (the selection plus the right-clicked moment); an empty spot on a track and the ruler offer **Send this moment to chat**; the media menu offers **Send to chat** after Open in Source. A right-click on a marker now opens a two-item menu, **Edit marker…** and **Send marker to chat**; double-click still opens the editor directly. The new `miniapp/client/chat-context.js` (global `FableCutChat`, loaded before `app.js`) builds the text from pure functions: ids, track ids and timeline seconds as the MCP tools address them, timecode in the editor's own `MM:SS:FF`, frame numbers, the open sequence, the composition path and the project revision. User text is reduced to one bounded line, lists are capped and the whole reference stays under 8,000 characters. Labels go through `T()`; ids, seconds and user text do not. The page calls `window.mcode.miniapp.message.append`, which needs no manifest declaration and shows the Host's own confirmation; the items stay visible and disabled with a tooltip when the bridge is missing, and each rejection code gets its own toast. Thirty dictionary keys were added to both languages. `tests/chat-context.test.mjs` covers the text, frame snapping at seven frame rates, hostile names and the size budget; `tests/send-to-chat.test.mjs` drives every menu in Chromium with the bridge stubbed, a cancelled confirmation, Chinese labels and a client without the bridge.
6+
37
## Compact monitor and localized menus
48

59
The monitor header keeps one row of tabs and icon controls. Canvas aspect, frame rate, export framing and resolution move into a keyboard-accessible popover; the original controls and stored values remain unchanged. Snap targets and sequence actions follow the selected language while user sequence names stay untouched and escaped. Playback controls wrap when the monitor is narrow, avoiding overlap with timecodes. Six browser tests cover menu localization, raw values, escaped names, popover operation and layout at five viewport widths.

‎plugins/klarkxy/cutline/README.md‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -33,6 +33,10 @@ You can also ask in chat:
3333

3434
Manual and Agent-driven edits operate on the same project, so you can take over at any point.
3535

36+
### Point the Agent at a spot
37+
38+
Right-click a clip, an empty spot on a track, the ruler, a marker or a file in the Project bin and choose **Send to chat**. MiniMax Code asks you to confirm, then adds a short reference to the chat input: clip and media ids, tracks, timeline seconds and frame, the open sequence and composition, and the project revision. Type what you want changed after it and press Send. A clip reference also carries the moment you right-clicked, so "split here" or "the caption at this point" lands where you meant. The text stays in the draft until you send it, and you can edit it first.
39+
3640
## Choose an export method
3741

3842
![Cutline export dialog in English: range, presets, video and audio settings](docs/export-en.png)
@@ -54,6 +58,7 @@ Standalone export needs a local Chrome or Edge installation and uses the project
5458
## Tested environment
5559

5660
- Target: MiniMax Code 3.1.2 or later. Installation, media import, timeline editing and standalone export were exercised on Windows 11 with Google Chrome, including a 180-second render.
61+
- **Send to chat** was checked on MiniMax Code 3.1.9 for Windows: all five right-click entries appear, the Host shows its confirmation, and the confirmed reference lands in the chat draft with correct ids, times and frames at 24 fps. A Chromium browser test with the Host bridge stubbed covers the text for each menu, a cancelled confirmation, both interface languages and a client without the API.
5762
- macOS, every effect combination and sustained long-form or high-resolution workloads remain on the validation roadmap. Current records cover the tested scenarios.
5863

5964
Run `npm test --prefix plugins/klarkxy/cutline` from the repository root for plugin tests, and `npm run check` for repository validation.
@@ -65,6 +70,7 @@ Run `npm test --prefix plugins/klarkxy/cutline` from the repository root for plu
6570
- **Network:** the editor and MCP endpoint use the Host-assigned local address. HTTPS imports contact the supplied URL; optional font loading contacts `fonts.googleapis.com` and `fonts.gstatic.com`, and optional background removal contacts `cdn.jsdelivr.net`. Standalone export is isolated from the external network.
6671
- **Processes:** media operations invoke `ffmpeg` and `ffprobe`; the Agent starts and manages Node plus Chrome or Edge for standalone export.
6772
- **Agent turns and conversations:** Agent turns start from your chat. The plugin exposes MCP tools, leaves `agentAccess` undeclared and does not read conversation content or other sessions. All conversations use the same open project.
73+
- **Chat input:** **Send to chat** calls the page API `message.append`, which needs no declaration. Each use shows the Host's confirmation with the exact text; after you confirm, the text joins the unsent draft of the conversation the Mini App is open in. The reference holds ids, times, track names, media file names, clip names, the first line of a title, marker labels and the project name. The page sends nothing on its own and reads no chat history. On a client without the API the menu items stay visible and disabled.
6874
- **Host data and devices:** Host configuration, credentials, private logs, session databases and the microphone sit outside the plugin's access.
6975

7076
## Upstream and maintenance

‎plugins/klarkxy/cutline/README.zh-CN.md‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -33,6 +33,10 @@ Cutline 是基于 [FableCut](https://github.com/ronak-create/FableCut) 的社区
3333

3434
手动编辑和 Agent 编辑使用同一个工程,你可以随时接管。
3535

36+
### 给 Agent 指一个位置
37+
38+
在片段、轨道空白处、时间标尺、标记或项目素材上点右键,选 **发送到对话**。MiniMax Code 弹出确认后,会在对话输入框里加一段简短引用:片段与素材 id、轨道、时间线秒数和帧号、当前序列与合成、工程修订号。接着写上要改什么,再按发送即可。片段引用还带着你右键的那个时间点,所以“从这里切开”“这个时间点的字幕”都能落在你指的位置。引用文字先进入草稿,发送前可以随意修改。
39+
3640
## 选择导出方式
3741

3842
![Cutline 中文导出面板:范围、预设、视频和音频设置](docs/export-zh-CN.png)
@@ -54,6 +58,7 @@ Cutline 是基于 [FableCut](https://github.com/ronak-create/FableCut) 的社区
5458
## Tested environment
5559

5660
- 目标客户端:MiniMax Code 3.1.2 或更高版本。已在 Windows 11 与 Google Chrome 环境中验证安装、素材导入、时间线编辑和独立导出,包括一次 180 秒渲染。
61+
- **发送到对话** 已在 Windows 版 MiniMax Code 3.1.9 上实测:五个右键入口都会出现,Host 会弹出确认框,确认后引用写入对话草稿,24 fps 下的 id、时间和帧号均正确。另有 Chromium 浏览器测试(Host 桥接为测试替身),覆盖各菜单生成的文字、取消确认、中英两种界面语言,以及不支持该 API 的客户端。
5762
- macOS、全部效果组合及长片/高分辨率负载仍待完整验证。当前记录代表已实测场景。
5863

5964
在仓库根目录运行 `npm test --prefix plugins/klarkxy/cutline` 执行插件测试,运行 `npm run check` 执行仓库校验。
@@ -65,6 +70,7 @@ Cutline 是基于 [FableCut](https://github.com/ronak-create/FableCut) 的社区
6570
- **网络:** 编辑器和 MCP 使用 Host 分配的本机地址。HTTPS 导入访问你提供的 URL;可选字体功能访问 `fonts.googleapis.com`、`fonts.gstatic.com`,可选背景移除访问 `cdn.jsdelivr.net`。独立导出与外部网络隔离。
6671
- **进程:** 媒体处理调用 `ffmpeg`、`ffprobe`;独立导出由 Agent 启动并管理 Node、Chrome 或 Edge 后台任务。
6772
- **Agent 与会话:** Agent 回合由你在聊天中发起。插件提供 MCP 工具,未声明 `agentAccess`,不会读取聊天内容或其他会话;所有会话共用当前打开的工程。
73+
- **对话输入框:** **发送到对话** 调用页面 API `message.append`,该 API 无需声明。每次使用都会弹出 Host 自己的确认框并显示完整文字;确认后文字追加到 Mini App 所在对话的未发送草稿。引用内容包括 id、时间、轨道名、素材文件名、片段名、字幕首行、标记名和工程名。页面不会自行发送消息,也不读取聊天记录。客户端不支持该 API 时,菜单项保持可见并置灰。
6874
- **Host 数据与设备:** Host 配置、凭据、私有日志、会话数据库和麦克风均位于访问范围之外。
6975

7076
## 上游与维护

‎plugins/klarkxy/cutline/miniapp/client/app.js‎

Lines changed: 99 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -406,8 +406,76 @@ function showTrackCtxMenu(clientX, clientY, track) {
406406
menu.style.top = Math.max(pad, y) + "px";
407407
btn.focus();
408408
}
409+
/* ── Send to chat: hand the conversation a reference to a clip, a moment, a
410+
marker or a media file (chat-context.js builds the text). The Host shows its
411+
own confirmation and appends the text to the chat input; nothing is sent
412+
until the user presses Send. ── */
413+
const ChatRef = self.FableCutChat;
414+
function chatAppendApi() {
415+
const message = window.mcode?.miniapp?.message;
416+
return message && typeof message.append === "function" ? message : null;
417+
}
418+
/** Why the menu item is disabled, or null when the chat input can take text. */
419+
function chatUnavailable() {
420+
return chatAppendApi() ? null : T("This version of MiniMax Code cannot add text to the chat input");
421+
}
422+
/** Where the user is: canvas fps, project, revision, open sequence, composition path. */
423+
function chatContext(extra = {}) {
424+
const root = seqRoot();
425+
const seqId = SeqLib.activeId(root);
426+
const seq = SeqLib.find(root, seqId) || SeqLib.MAIN;
427+
const compPath = state.compCtx
428+
? state.compCtx.path.map((id) => ({ id, name: (CompLib.getComp(state.compCtx.root, id) || {}).name || id }))
429+
: [];
430+
return {
431+
fps: projectFps(), projectName: root.name || "", revision: project.revision ?? null,
432+
sequence: { id: seq.id || seqId, name: seq.name || seqId }, compPath,
433+
getMedia, getComp, clips: project.clips.filter((c) => trackOf(c)), trackOrder: TRACKS.map((t) => t.id), ...extra,
434+
};
435+
}
436+
let chatSending = false;
437+
async function sendToChat(text) {
438+
const message = chatAppendApi();
439+
if (!message) { toast(chatUnavailable()); return; }
440+
if (chatSending) return;
441+
chatSending = true;
442+
try {
443+
await message.append(text);
444+
toast(T("Added to the chat input — add your request and press Send"));
445+
} catch (err) {
446+
const code = err && err.message;
447+
if (code === "user_cancelled") toast(T("Not added to the chat"));
448+
else if (code === "target_unavailable") toast(T("No conversation to add to — open Cutline from a chat"));
449+
else if (code === "result_unknown") toast(T("It may already be in the chat input — check there before trying again"));
450+
else toast(T("Could not add to the chat ({code})", { code: code || "error" }));
451+
} finally {
452+
chatSending = false;
453+
}
454+
}
455+
/** A small right-click menu of [label, run, disabledReason] rows, placed at (x, y). */
456+
function showChatMenu(clientX, clientY, rows) {
457+
hideTrackCtxMenu();
458+
const menu = document.createElement("div");
459+
menu.className = "ctx-menu";
460+
menu.id = "trackCtxMenu";
461+
for (const [label, run, why] of rows) {
462+
const btn = document.createElement("button");
463+
btn.type = "button";
464+
btn.className = "ctx-item";
465+
btn.textContent = label;
466+
if (why) { btn.disabled = true; btn.title = why; }
467+
else btn.addEventListener("click", () => { hideTrackCtxMenu(); run(); });
468+
menu.appendChild(btn);
469+
}
470+
document.body.appendChild(menu);
471+
trackCtxMenu = menu;
472+
const pad = 6;
473+
menu.style.left = Math.max(pad, Math.min(clientX, window.innerWidth - menu.offsetWidth - pad)) + "px";
474+
menu.style.top = Math.max(pad, Math.min(clientY, window.innerHeight - menu.offsetHeight - pad)) + "px";
475+
menu.querySelector(".ctx-item:not(:disabled)")?.focus();
476+
}
409477
/* Clip right-click menu: enable / lock / link for the clip's selection. */
410-
function showClipCtxMenu(clientX, clientY, c) {
478+
function showClipCtxMenu(clientX, clientY, c, atTime = null) {
411479
hideTrackCtxMenu();
412480
if (!state.selIds.has(c.id)) selectClip(c.id);
413481
const group = withLinked(selectedClips());
@@ -447,6 +515,11 @@ function showClipCtxMenu(clientX, clientY, c) {
447515
const why = linkRefusal(selectedClips());
448516
item(T("Link audio / video"), "Ctrl+L", toggleLinkSelected, why);
449517
}
518+
const sep = document.createElement("div");
519+
sep.className = "ctx-sep";
520+
menu.appendChild(sep);
521+
const picked = selectedClips();
522+
item(T("Send to chat"), "", () => sendToChat(ChatRef.clipsReference(picked, chatContext({ at: atTime }), T)), chatUnavailable());
450523
document.body.appendChild(menu);
451524
trackCtxMenu = menu;
452525
const pad = 6;
@@ -2806,6 +2879,7 @@ function showMediaMenu(x, y, m) {
28062879
};
28072880
const sep = () => { const d = document.createElement("div"); d.className = "ctx-sep"; menu.appendChild(d); };
28082881
item(T("Open in Source"), () => loadSourceFromMedia(m));
2882+
item(T("Send to chat"), () => sendToChat(ChatRef.mediaReference(m, MediaLib.placements(usageDoc(), m.id), chatContext(), T)), chatUnavailable());
28092883
item(T("Find timeline instances ({uses})", { uses }), () => showMediaInstances(m, x, y), uses ? null : "Not used on the timeline");
28102884
item(T("Media info…"), () => showMediaInfo(m, x, y));
28112885
item(isMediaOffline(m.id) ? T("Relink…") : T("Relink to another copy…"), () => openRelinkDialog([m]), state.connected ? null : T("Needs the Cutline server"));
@@ -5960,14 +6034,23 @@ function startTrimToolGesture(e, c, mode) {
59606034
window.addEventListener("pointermove", onMove);
59616035
window.addEventListener("pointerup", onUp);
59626036
}
5963-
/* Right-click a clip: enable / lock / link menu. */
6037+
/* Right-click a clip: enable / lock / link menu. Right-click an empty spot on a
6038+
track: send that moment to the chat. */
59646039
els.tracksContent.addEventListener("contextmenu", (e) => {
6040+
const at = ChatRef.snapToFrame(timeAtEvent(e), projectFps());
59656041
const clipDiv = e.target.closest(".clip");
5966-
if (!clipDiv) return;
6042+
if (!clipDiv) {
6043+
const track = trackAtEvent(e);
6044+
if (!track) return;
6045+
e.preventDefault();
6046+
showChatMenu(e.clientX, e.clientY, [[T("Send this moment to chat"),
6047+
() => sendToChat(ChatRef.timeReference(at, chatContext({ track }), T)), chatUnavailable()]]);
6048+
return;
6049+
}
59676050
const c = getClip(clipDiv.dataset.id);
59686051
if (!c) return;
59696052
e.preventDefault();
5970-
showClipCtxMenu(e.clientX, e.clientY, c);
6053+
showClipCtxMenu(e.clientX, e.clientY, c, at);
59716054
});
59726055
els.tracksContent.addEventListener("pointerdown", (e) => {
59736056
if (e.button !== 0) return;
@@ -6315,10 +6398,19 @@ els.ruler.addEventListener("dblclick", (e) => {
63156398
if (mk) openMarkerEditor(mk, e.clientX - 20, e.clientY + 10);
63166399
});
63176400
els.ruler.addEventListener("contextmenu", (e) => {
6318-
const mk = markerAtRulerEvent(e);
6319-
if (!mk) return;
63206401
e.preventDefault();
6321-
openMarkerEditor(mk, e.clientX, e.clientY);
6402+
const mk = markerAtRulerEvent(e);
6403+
const x = e.clientX, y = e.clientY;
6404+
if (mk) {
6405+
showChatMenu(x, y, [
6406+
[T("Edit marker…"), () => openMarkerEditor(mk, x, y), null],
6407+
[T("Send marker to chat"), () => sendToChat(ChatRef.markerReference(mk, chatContext(), T)), chatUnavailable()],
6408+
]);
6409+
return;
6410+
}
6411+
const at = ChatRef.snapToFrame(timeAtEvent(e), projectFps());
6412+
showChatMenu(x, y, [[T("Send this moment to chat"),
6413+
() => sendToChat(ChatRef.timeReference(at, chatContext(), T)), chatUnavailable()]]);
63226414
});
63236415
els.ruler.addEventListener("pointermove", (e) => {
63246416
if (state.gesture) return;

0 commit comments

Comments
 (0)