Control a PS5 with a script or an AI agent

Orbit Play can let scripts, automation tools and AI agents on your computer press the PS5 controller’s buttons and move its sticks during a stream, while you keep using another app. An agent can find its way around the PS5’s menus, test a game for you, or repeat a sequence of presses, and take a picture of the stream to see what happened. This works on a Mac, Windows and Linux, with a jailbroken PS5 and Orbit Play 0.7.0 or later (Windows from 0.8.0).

Only your own user account on the computer can send these commands. Nothing reaches the PS5 from another computer, and what you send is never saved.

Turn it on

  1. In Orbit Play, open Playback, tick Let other apps on this computer control the player and choose Save playback settings. It’s off until you turn it on.
  2. Start a stream. The player reads the setting when a stream starts, so stop and start again if one was already open.

The command

Orbit Play itself is the command: run it with control first.

Computer Command
Mac "/Applications/Orbit Play.app/Contents/MacOS/OrbitPlay" control
Windows & "$env:LOCALAPPDATA\Programs\Orbit Play\Orbit Play\OrbitPlay.exe" control in PowerShell
Linux /path/to/OrbitPlay-<version>-linux-x64.AppImage control

On Windows the app is a windowed program, so add | Out-String after the command in PowerShell to see its reply. Every command also names the copy of Orbit Play that streams your PS5, so a command can never press buttons on the wrong console. For the usual copy, set this once in your terminal (Mac shown; the folder is %APPDATA%\Orbit Play on Windows and ~/.config/orbit-play on Linux):

export ORBIT_PLAY_STATE_DIR="$HOME/Library/Application Support/Orbit Play"
alias orbit-player-control='"/Applications/Orbit Play.app/Contents/MacOS/OrbitPlay" control'

Press buttons and move sticks

orbit-player-control status                 # is the stream ready, and what is held
orbit-player-control press cross            # one press (100 ms)
orbit-player-control press circle 250       # press for 250 ms, then release
orbit-player-control hold r2 500            # hold for 500 ms, then release
orbit-player-control stick left 0 -1 500    # left stick fully up for 500 ms
orbit-player-control release all            # let go of everything, centre the sticks

Buttons are cross, circle, square, triangle, up, down, left, right, l1, r1, l2, r2, l3, r3, options, create, ps and touchpad. Stick values go from -1 to 1 (for y, -1 is up), and times are whole milliseconds from 1 to 10000. Each command waits until it has finished, so commands sent one after another never run into each other. Use single presses to move through menus: a held D-pad direction keeps moving.

From a script

Each command prints one line of JSON and sets its exit status: 0 when it was done, 1 when it was refused, 2 when no stream is listening. A Python script can use that directly:

import json, os, subprocess

ORBIT_PLAY = "/Applications/Orbit Play.app/Contents/MacOS/OrbitPlay"
STATE = os.path.expanduser("~/Library/Application Support/Orbit Play")

def control(*words):
    """Run one control command; return its exit status and its reply."""
    done = subprocess.run([ORBIT_PLAY, "control", "--state-dir", STATE, *map(str, words)],
                          capture_output=True, text=True)
    return done.returncode, json.loads(done.stdout or "{}")

status, reply = control("status")
if status == 2:
    raise SystemExit("Start a stream with player control turned on first.")
for button in ("down", "down", "cross"):
    control("press", button)
control("release", "all")

Give it to an AI agent

An AI agent that can run terminal commands, such as a coding assistant, needs two things: a way to press buttons, which is the command above, and a way to see the screen.

status names the player window, for example {"ok": true, "held": [], "window": 206186, "title": "Orbit Play Player", …}. On a Mac, screencapture -x -o -l 206186 screen.png takes a picture of just that window without bringing it forward, as long as it’s on the current desktop and not minimized. On Linux, window is the X11 window ID, which tools such as ImageMagick’s import -window can capture.

Then tell the agent how to work, for example:

You can press the PS5 controller’s buttons with orbit-player-control press <button> and see the screen by running orbit-player-control status, then screencapture -x -o -l <window> screen.png and looking at screen.png. Press one button at a time and look at the screen after each press before deciding the next one. When you’re done, run orbit-player-control release all.

Looking after every press keeps the agent from getting lost in menus.

When a command is refused

Reply What to do
not_running Turn the setting on, then start or restart a stream
no_ps5_chosen Set ORBIT_PLAY_STATE_DIR or put --state-dir before the command
unknown_button Use a button name from the list above, such as cross, not x
bad_duration Use whole milliseconds from 1 to 10000
rate_limited Send fewer than 100 commands a second
stopped The stream ended while the command was running

The full reference is in the guide to control the player from other apps.

Sources

Updated 2026-11-12