# Control a PS5 with a script or an AI agent

Source: https://orbitplay.dev/guides/control-ps5-script-ai-agent/ · Updated 2026-11-12

Orbit Play can let scripts, automation tools and AI agents on your computer
press the PS5 controller's buttons and move its sticks during a stream, while
you keep using another app. An agent can find its way around the PS5's menus,
test a game for you, or repeat a sequence of presses, and take a picture of the
stream to see what happened. This works on a Mac, Windows and Linux, with a
jailbroken PS5 and Orbit Play 0.7.0 or later (Windows from 0.8.0).

Only your own user account on the computer can send these commands. Nothing
reaches the PS5 from another computer, and what you send is never saved.

## Turn it on

1. In Orbit Play, open **Playback**, tick **Let other apps on this computer
   control the player** and choose **Save playback settings**. It's off until
   you turn it on.
2. Start a stream. The player reads the setting when a stream starts, so stop
   and start again if one was already open.

## The command

Orbit Play itself is the command: run it with `control` first.

| Computer | Command |
| --- | --- |
| Mac | `"/Applications/Orbit Play.app/Contents/MacOS/OrbitPlay" control` |
| Windows | `& "$env:LOCALAPPDATA\Programs\Orbit Play\Orbit Play\OrbitPlay.exe" control` in PowerShell |
| Linux | `/path/to/OrbitPlay-<version>-linux-x64.AppImage control` |

On Windows the app is a windowed program, so add `| Out-String` after the
command in PowerShell to see its reply. Every command also names the copy of Orbit
Play that streams your PS5, so a command can never press buttons on the wrong
console. For the usual copy, set this once in your terminal (Mac shown; the
folder is `%APPDATA%\Orbit Play` on Windows and `~/.config/orbit-play` on
Linux):

```sh
export ORBIT_PLAY_STATE_DIR="$HOME/Library/Application Support/Orbit Play"
alias orbit-player-control='"/Applications/Orbit Play.app/Contents/MacOS/OrbitPlay" control'
```

## Press buttons and move sticks

```sh
orbit-player-control status                 # is the stream ready, and what is held
orbit-player-control press cross            # one press (100 ms)
orbit-player-control press circle 250       # press for 250 ms, then release
orbit-player-control hold r2 500            # hold for 500 ms, then release
orbit-player-control stick left 0 -1 500    # left stick fully up for 500 ms
orbit-player-control release all            # let go of everything, centre the sticks
```

Buttons are `cross`, `circle`, `square`, `triangle`, `up`, `down`, `left`,
`right`, `l1`, `r1`, `l2`, `r2`, `l3`, `r3`, `options`, `create`, `ps` and
`touchpad`. Stick values go from -1 to 1 (for y, -1 is up), and times are whole
milliseconds from 1 to 10000. Each command waits until it has finished, so
commands sent one after another never run into each other. Use single presses
to move through menus: a held D-pad direction keeps moving.

## From a script

Each command prints one line of JSON and sets its exit status: 0 when it was
done, 1 when it was refused, 2 when no stream is listening. A Python script can
use that directly:

```python
import json, os, subprocess

ORBIT_PLAY = "/Applications/Orbit Play.app/Contents/MacOS/OrbitPlay"
STATE = os.path.expanduser("~/Library/Application Support/Orbit Play")

def control(*words):
    """Run one control command; return its exit status and its reply."""
    done = subprocess.run([ORBIT_PLAY, "control", "--state-dir", STATE, *map(str, words)],
                          capture_output=True, text=True)
    return done.returncode, json.loads(done.stdout or "{}")

status, reply = control("status")
if status == 2:
    raise SystemExit("Start a stream with player control turned on first.")
for button in ("down", "down", "cross"):
    control("press", button)
control("release", "all")
```

## Give it to an AI agent

An AI agent that can run terminal commands, such as a coding assistant, needs
two things: a way to press buttons, which is the command above, and a way to
see the screen.

`status` names the player window, for example
`{"ok": true, "held": [], "window": 206186, "title": "Orbit Play Player", …}`.
On a Mac, `screencapture -x -o -l 206186 screen.png` takes a picture of just
that window without bringing it forward, as long as it's on the current desktop
and not minimized. On Linux, `window` is the X11 window ID, which tools such as
[ImageMagick's `import -window`](https://imagemagick.org/script/import.php)
can capture.

Then tell the agent how to work, for example:

> You can press the PS5 controller's buttons with `orbit-player-control press
> <button>` and see the screen by running `orbit-player-control status`, then
> `screencapture -x -o -l <window> screen.png` and looking at `screen.png`.
> Press one button at a time and look at the screen after each press before
> deciding the next one. When you're done, run `orbit-player-control release
> all`.

Looking after every press keeps the agent from getting lost in menus.

## When a command is refused

| Reply | What to do |
| --- | --- |
| `not_running` | Turn the setting on, then start or restart a stream |
| `no_ps5_chosen` | Set `ORBIT_PLAY_STATE_DIR` or put `--state-dir` before the command |
| `unknown_button` | Use a button name from the list above, such as `cross`, not `x` |
| `bad_duration` | Use whole milliseconds from 1 to 10000 |
| `rate_limited` | Send fewer than 100 commands a second |
| `stopped` | The stream ended while the command was running |

The full reference is in the guide to
[control the player from other apps](https://orbitplay.dev/guides/player-control/).

## Sources

- ImageMagick: [import](https://imagemagick.org/script/import.php).
