Control a PS5 with a script or an AI agent
Orbit Play can let scripts, automation tools and AI agents on your computer press the PS5 controller’s buttons and move its sticks during a stream, while you keep using another app. An agent can find its way around the PS5’s menus, test a game for you, or repeat a sequence of presses, and take a picture of the stream to see what happened. This works on a Mac, Windows and Linux, with a jailbroken PS5 and Orbit Play 0.7.0 or later (Windows from 0.8.0).
Only your own user account on the computer can send these commands. Nothing reaches the PS5 from another computer, and what you send is never saved.
Turn it on
- In Orbit Play, open Playback, tick Let other apps on this computer control the player and choose Save playback settings. It’s off until you turn it on.
- Start a stream. The player reads the setting when a stream starts, so stop and start again if one was already open.
The command
Orbit Play itself is the command: run it with control first.
| Computer | Command |
|---|---|
| Mac | "/Applications/Orbit Play.app/Contents/MacOS/OrbitPlay" control |
| Windows | & "$env:LOCALAPPDATA\Programs\Orbit Play\Orbit Play\OrbitPlay.exe" control in PowerShell |
| Linux | /path/to/OrbitPlay-<version>-linux-x64.AppImage control |
On Windows the app is a windowed program, so add | Out-String after the
command in PowerShell to see its reply. Every command also names the copy of Orbit
Play that streams your PS5, so a command can never press buttons on the wrong
console. For the usual copy, set this once in your terminal (Mac shown; the
folder is %APPDATA%\Orbit Play on Windows and ~/.config/orbit-play on
Linux):
export ORBIT_PLAY_STATE_DIR="$HOME/Library/Application Support/Orbit Play"
alias orbit-player-control='"/Applications/Orbit Play.app/Contents/MacOS/OrbitPlay" control'
Press buttons and move sticks
orbit-player-control status # is the stream ready, and what is held
orbit-player-control press cross # one press (100 ms)
orbit-player-control press circle 250 # press for 250 ms, then release
orbit-player-control hold r2 500 # hold for 500 ms, then release
orbit-player-control stick left 0 -1 500 # left stick fully up for 500 ms
orbit-player-control release all # let go of everything, centre the sticks
Buttons are cross, circle, square, triangle, up, down, left,
right, l1, r1, l2, r2, l3, r3, options, create, ps and
touchpad. Stick values go from -1 to 1 (for y, -1 is up), and times are whole
milliseconds from 1 to 10000. Each command waits until it has finished, so
commands sent one after another never run into each other. Use single presses
to move through menus: a held D-pad direction keeps moving.
From a script
Each command prints one line of JSON and sets its exit status: 0 when it was done, 1 when it was refused, 2 when no stream is listening. A Python script can use that directly:
import json, os, subprocess
ORBIT_PLAY = "/Applications/Orbit Play.app/Contents/MacOS/OrbitPlay"
STATE = os.path.expanduser("~/Library/Application Support/Orbit Play")
def control(*words):
"""Run one control command; return its exit status and its reply."""
done = subprocess.run([ORBIT_PLAY, "control", "--state-dir", STATE, *map(str, words)],
capture_output=True, text=True)
return done.returncode, json.loads(done.stdout or "{}")
status, reply = control("status")
if status == 2:
raise SystemExit("Start a stream with player control turned on first.")
for button in ("down", "down", "cross"):
control("press", button)
control("release", "all")
Give it to an AI agent
An AI agent that can run terminal commands, such as a coding assistant, needs two things: a way to press buttons, which is the command above, and a way to see the screen.
status names the player window, for example
{"ok": true, "held": [], "window": 206186, "title": "Orbit Play Player", …}.
On a Mac, screencapture -x -o -l 206186 screen.png takes a picture of just
that window without bringing it forward, as long as it’s on the current desktop
and not minimized. On Linux, window is the X11 window ID, which tools such as
ImageMagick’s import -window
can capture.
Then tell the agent how to work, for example:
You can press the PS5 controller’s buttons with
orbit-player-control press <button>and see the screen by runningorbit-player-control status, thenscreencapture -x -o -l <window> screen.pngand looking atscreen.png. Press one button at a time and look at the screen after each press before deciding the next one. When you’re done, runorbit-player-control release all.
Looking after every press keeps the agent from getting lost in menus.
When a command is refused
| Reply | What to do |
|---|---|
not_running |
Turn the setting on, then start or restart a stream |
no_ps5_chosen |
Set ORBIT_PLAY_STATE_DIR or put --state-dir before the command |
unknown_button |
Use a button name from the list above, such as cross, not x |
bad_duration |
Use whole milliseconds from 1 to 10000 |
rate_limited |
Send fewer than 100 commands a second |
stopped |
The stream ended while the command was running |
The full reference is in the guide to control the player from other apps.
Sources
- ImageMagick: import.