Control protocol & MCP

Desktop control channel

kinema --control 9876 (or KINEMA_CONTROL_PORT) listens on 127.0.0.1:<port> (loopback only). The listener requires KINEMA_CONTROL_TOKEN: 64 random hex characters shared with authorized clients through their inherited environment. Generate a fresh token for each desktop session, then start the desktop and bridge from that environment:

export KINEMA_CONTROL_TOKEN="$(python3 -c 'import secrets; print(secrets.token_hex(32))')"
kinema --control 9876 &
kinema-cli mcp --bridge 127.0.0.1:9876

The CLI / MCP bridge reads this variable automatically. Keep it private: don't put the value in command-line arguments, logs, checked-in MCP configuration, or shared files. Without a valid value, the desktop leaves the listener closed. Every raw JSON request must include the same top-level token string; missing or incorrect credentials get an error and close the connection before any request is dispatched. Loopback alone does not restrict clients to the desktop owner's OS account.

Socket reads and writes time out after two minutes without progress, releasing the connection slot. Expiration closes the connection without an unsolicited reply; reconnect before the next request (the CLI / MCP bridge does this automatically). Waiting for a UI reply has its separate 60-second limit. A partial request that times out is not dispatched.

One JSON request per line → one JSON reply per line. The app opts out of macOS App Nap (apps/kinema/src/app_nap.rs): a hidden window would otherwise drop the whole process to background priority, and on a busy machine it would stop answering.

Only requests are read. Every line must be a JSON object with a string method (id and params are optional; blank lines are skipped). Anything else gets one error reply ({"ok": false, "error": "… closing the connection"}) and the server closes the connection, so nothing sent after it on that connection runs. That covers text that isn't JSON, a JSON array or number, an object without method, invalid UTF-8, and a line longer than 4 MiB. An HTTP request (for example a web page's cross-origin fetch to 127.0.0.1:<port>) therefore can't smuggle a command in its body: its request line is rejected first. At most 16 connections are served at once; further ones get an error line and are closed. Clients that get an error reply should reconnect. Transport: apps/kinema/src/control_server.rs.

{"id": 1, "token": "<KINEMA_CONTROL_TOKEN>", "method": "engine.execute", "params": {"command": "sequence.addEdit", "params": {"seconds": 3}}}
{"id": 1, "ok": true, "result": {"cuts": 2}}

Methods (handlers in crates/ui-egui/src/control.rs):

For ui.click and ui.drag, omitted or null counts/steps use the defaults. Out-of-range or non-integer values return {"ok": false, "error": "…"} before queuing any pointer input. These limits also apply to the MCP bridge's ui_click, ui_drag and ui_control tools.

Timeline thumbnails. ui.set {timeline: {thumbnails}} sets the Timeline wrench menu's Show Video Thumbnails: off, head, headAndTail or continuous.

Selection or explicit targets. A command that acts on the selection is "not available right now" when nothing is selected. When params name the targets under a key the command documents (clips / clip: timeline clips of the active sequence, items / item: project items), enablement is checked against those instead and the selection is left as it is; the command's other conditions still apply. engine.commands and the menus always show the selection's enablement.

Focus. Driving the app never steals the user's keyboard. Started with --control, the app opens without activating itself. UI requests and screenshots that need a rendered frame bring the window forward without making it key (macOS orderFrontRegardless), so the user's typing keeps going to the app they are in. Only an explicit ui.focus {} activates Kinema. Injected keys work on focused widgets even when the native window is in the background.

ui.focus {id} targets an enabled, visible widget registered with a real egui response. It rejects unknown IDs, non-widget regions and hidden tool aliases. The reply acknowledges the request; ui.inspect.keyboardFocus reports the actual widget ID, matching element IDs, label and active focus. This is separate from ui.focused, which identifies the active panel. Tools, dock tabs, tab menus and divider handles expose widget IDs through ui.elements. Tab/Shift-Tab move widget focus; Space/Enter activate focused buttons and tabs, with visible focus outlines.

For unattended validation, start with --control <port> --control-input-only. This opt-in mode accepts widget input through control requests and excludes native keyboard, pointer, file-drop and menu commands. Window close/quit and screenshots remain available. Ordinary launches and --control alone keep native input enabled. ui.inspect.input distinguishes mode, the observed nativeWindowFocused, and egui's effectiveWindowFocused; effective focus does not activate the OS window or prove physical-keyboard delivery.

Element ids are stable, e.g. timeline.clip.<id>, timeline.track.V1.lock, tools.Razor, project.item.<id>, effects.item.gaussian_blur, panel.tab.Timeline, program.transport.playback.toggle.

Audio mixing: mixer.* commands (strips "A1", "S1", "Mix" or ids; lanes volume, pan, mute, send.<i>.level, fx.<slot>.<param>): mixer.inspect, mixer.setStrip (volume, pan, mute, solo, record arm, solo safe, mode Off/Read/Latch/Touch/Write, output, input map, channels), mixer.setValue, mixer.touch / mixer.release (a fader gesture; recorded during an automation pass), mixer.recordStart / mixer.recordStop (playback runs them), mixer.addSubmix, mixer.deleteSubmix, mixer.addInsert / removeInsert / setInsert, mixer.addSend / setSend / removeSend, mixer.setKeyframe / deleteKeyframe / moveKeyframe / clearLane, mixer.writeAutomation; clipMixer.set; clip.audioGain {clips?, mode: set|adjust|normalizeMax|normalizeAll, db}, clip.audioPeak; effects.setDefaultTransition. UI ids: mixer.<A1|S1|Mix>.<fader|value|pan|panValue|mode|mute|solo| record|soloSafe|output|input|fx.<n>|send.<n>|meter|name> (popup entries below them, e.g. mixer.A1.mode.Touch, mixer.A1.fx.0.studio_reverb), mixer.showEffects, mixer.transport.<cmd>, clipMixer.A1.<fader|pan|mute|solo|keyframe|value>, timeline.track.A1.keyframes[.<lane>|.clip], timeline.track.A1.lane[.kf.<n>], audioGain.<set|adjust|normalizeMax|normalizeAll|ok|cancel|peak>.

Essential Sound: essentialSound.inspect, essentialSound.setType {type: dialogue|music|sfx|ambience}, essentialSound.clearType, essentialSound.set {key, value} or {values: {key: value}} (keys such as repair.noise.on, repair.noise.amount (0–10), repair.humHz, clarity.eqPreset, clarity.enhanceTone, creative.reverbPreset, ducking.against, ducking.reduceDb, ducking.fadeS, pan.value, volume.levelDb, mute, section switches <section>.enabled), essentialSound.applyPreset {preset, type?}, essentialSound.savePreset {name}, essentialSound.deletePreset {name}, essentialSound.autoMatch {target?}, essentialSound.generateDucking; all take clips (default: the selected audio clips). UI ids: essentialSound.tab.<Browse|Edit>, essentialSound.type.<Dialogue|Music|SFX|Ambience>, essentialSound.clearType, essentialSound.preset[.save|.delete|.name|.ok], essentialSound.section.<Name>[.toggle], a row per setting key (essentialSound.repair.noise.on, essentialSound.repair.noise.amount, essentialSound.repair.humHz.<i>), essentialSound.autoMatch, essentialSound.ducking.against.<Type>, essentialSound.generateDucking, essentialSound.volume.on, essentialSound.volume.levelDb, essentialSound.mute, essentialSound.browse.<Type>.<preset>.

Project files, auto-save, crash recovery and preferences commands (file.recover, prefs.set, …) and their automation ids are listed in project-files.md. That file also lists the media management commands and dialog ids: offline media and relinking (media.findMissing, media.relink, media.autoRelink, media.search, media.makeOffline, media.status, linkMedia.*), proxies (media.createProxies, media.attachProxies, media.toggleProxies, proxies.*), ingest (project.ingestSettings) and the Project Manager (file.projectManager, pm.*).

Moving clips: timeline.move {moves: [{clip, track, time}], insert?, linked?} → {moved: [clip ids]}. With Linked Selection on (the default; linked overrides the toggle for one call) the linked partners of a listed clip that are not listed themselves move by the same time offset and stay on their own tracks, so picture and sound stay in sync. Like a listed clip, a partner overwrites what is at its destination (or inserts, with insert). A partner on a locked track stays where it is, and so does one whose position would not change (the listed clip only changed track). moved lists every clip that moved, partners included. No clip is moved before the sequence start: if one would be, all clips of the call land later by the same amount (their spacing is kept). A time beyond the representable range is an "invalid parameters" error. The timeline panel's drag passes linked: false, because it already lists exactly the clips it moves. With copy: true (Option-drag in the timeline) the clips stay where they are and copies land at the destinations instead (linked partners are copied too, unless linked: false); the result lists the new clips as copied, they are selected, and copies of linked clips are linked to each other.

Trim mode (crates/engine/src/trim.rs): trim.selectEditPoint / trim.selectNearest enter trim mode (the Program monitor becomes the Trim Monitor, ids trimMonitor.*); trim.monitor returns what it shows. Dynamic trimming takes an explicit clock (seconds) so it is deterministic: trim.shuttle {direction, slow?, clock} (J/L), trim.tick {clock}, trim.shuttleStop {clock?} (K, one undo step), trim.cancelDynamic (Esc), trim.playAround {clock, loop?} (Space / Shift+K).

Replace With Clip: clip.replaceFromSource, clip.replaceFromSourceMatchFrame and clip.replaceFromBin {clips?, item?, keepSourceIn?} swap the media of the clips and return {clips: [ids], item, short: [{clip, shortByFrames}]}. short lists the replaced clips that now run past the end of their media (the renderer holds the last frame there) with the number of sequence frames that have no media, at most the clip's length; it is empty when the new media covers every clip. The edit is the same either way. From Bin starts each clip at the new item's In point; with keepSourceIn: true each clip keeps its own source In instead. A subclip that restricts trims has no media outside its range, so there a kept In is moved to the subclip's first or last frame, and the result's sourceInClamped: [ids] (present with keepSourceIn) names the clips it was moved for.

Colour (crates/engine/src/color.rs): sequence.colorSettings {workingSpace: rec709|rec2100-pq| rec2100-hlg, wideGamut, autoToneMap}, clip.interpretFootage {items?, colorSpace?: auto|<id>, pixelAspect?: [num, den]|file}, color.spaces, media.colorInfo {item}; LUTs: lut.import {path, name?}, lut.list, lut.remove {id}, lut.export {lut, path, format?}, lumetri.setInputLut / lumetri.setLook {clip?, lut: lib:<id>|builtin:<id>|"", path?}, lumetri.setSection {clip?, section, on?}, lumetri.applyMatch {clip?, referenceTime|referenceFrame|referenceTimecode, faceDetection?}. Without params the menu entries open dialogs (ids colorDialog.space.<id>, colorDialog.working.<id>, colorDialog.wideGamut, colorDialog.autoToneMap, colorDialog.ok|cancel). Lumetri panel ids: lumetri.input_lut, lumetri.look_lut, lumetri.switch.<section>, lumetri.section.<section>, lumetri.match.*; HDR scopes: scopes.hdrWaveform. file.exportMedia {sdr: true} exports an HDR sequence as tone-mapped SDR.

Keyboard shortcuts (crates/engine/src/shortcuts.rs): shortcuts.list {query?, panel?}, shortcuts.get, shortcuts.set {command, keys, panel?, add?, keepConflicts?}, shortcuts.clear, shortcuts.undo / shortcuts.redo, shortcuts.conflicts {platform?}, shortcuts.forKey {key}, shortcuts.resolve {keys, panel?}, shortcuts.presets, shortcuts.loadPreset / savePreset / deletePreset {name}, shortcuts.export / import {path}, shortcuts.audit. Keys use Cmd (⌘ / Ctrl), Ctrl (macOS ⌃), Alt, Shift. The dialog (Edit ▸ Keyboard Shortcuts…, ⌥⌘K) uses ids shortcuts.* (shortcuts.key.K, shortcuts.cell.<command>, shortcuts.ok, …).

MCP

kinema-cli mcp serves MCP on stdio: headless (in-process session; --demo / --project p.fcproj) or --bridge 127.0.0.1:9876 to drive the running app. Tools: command_list, command_run, command_batch, doc_inspect, render_preview, project_inspect, sequence_inspect, media_import, render_frame, ui_inspect, ui_elements, ui_click, ui_drag, ui_key, ui_type, ui_screenshot, ui_control; resources kinema://document and kinema://commands. .mcp.json registers both. Annotations, argument checking, errors and export progress / cancellation: agents.md § Conventions.

Time is in ticks (254 016 000 000 per second); commands also accept seconds, frame or timecode.

Panel docking commands

ui.menu.invoke and engine.execute route these frontend commands:

Invalid source/target identities and non-finite or out-of-range geometry return an error with the layout unchanged. ui.inspect.ui.floating reports floating groups and their logical-point rectangles. panel.drag.<Panel> locates each panel's header grip; panel.float.<index>.move and .resize locate floating geometry handles. Named workspace saves include these groups.

window.dockAction accepts {action} using the shared docking action's JSON form. It covers tab moves and ordering, split and floating geometry, activation, and closing. For example, {"action":{"ResizeSplit":{"path":[],"size":{"FixedFirst":360}}}} resizes the root split; path entries use false for the first child and true for the second. {"action":{"MoveFloating":{"panel":"Info","rect":[45,65,420,320]}}} repositions and resizes the floating group containing Info. Invalid actions leave the layout unchanged. Pointer gestures dispatch through this same catalog command.