Browser agent

Goal: Run an LLM-driven browser agent inside a MIOSA sandbox. Start the sandbox browser, connect Playwright to it over CDP, and loop: screenshot, ask the model what to do, act on the same page.

What you’ll use: Sandboxes, the sandbox browser, Playwright over CDP

What you’ll build

A BrowserAgent that:

1. Creates a sandbox and starts its browser.
2. Connects Playwright to the same browser the live viewer shows.
3. Enters a screenshot -> LLM -> action loop.
4. Stops when the model signals completion.

The LLM is your choice. The code below uses Anthropic Claude’s vision API, but the pattern is model-agnostic.

Prerequisites

• A MIOSA workspace API key (msk_u_*) - see API Keys
• An Anthropic API key (or substitute your preferred vision-capable model)
• Node 22+ or Python 3.11+

Step 1 - Install and configure

Playwright runs on your machine as the driver. The browser it drives runs inside the sandbox.

Step 2 - Create a sandbox and start its browser

Create a sandbox, then start its browser. Nothing about the browser is chosen at create time: any template works, as long as its image includes Chromium and Node (the standard images do).

Step 3 - Connect Playwright over CDP

The sandbox browser speaks the Chrome DevTools Protocol. Connect Playwright to cdp_url (with the stream_auth token appended as ?token=) and reuse the context and page that are already open, so you act on the session a viewer would see.

Step 4 - Screenshot, decide, act

The loop is the same as any vision agent: capture a screenshot, ask the model what to do next, act, repeat. The model returns a JSON action; your code executes it on the page.

Step 5 - Execute actions on the page

Playwright works in pixels; the model works in a 0-1000 space. Scale on the way in.

Step 6 - Run the full loop

Tips for reliable browser agents

• Wait after every action. Pages animate and reflow. page.wait_for_timeout(500) after each step is cheap insurance against acting on a stale screenshot.
• Save every screenshot. The debug loop is: screenshot N shows the state the model reasoned about to produce action N. When the agent gets stuck, inspect the saved frames.
• Cap MAX_TURNS tightly (10-25). A runaway loop burns credits. Surface an incomplete-task status instead of retrying forever.
• Prefer goto over clicking through navigation when the goal is simply to reach a URL.
• Watch it live. The live viewer shows the same browser your agent is driving, so you can take over when the model stalls instead of restarting the sandbox.
• Snapshot after login. If the task needs authenticating, snapshot immediately after login succeeds, then restore for later tasks to skip the login step. See Snapshots and persistence.

When to use a Computer instead

The sandbox browser is the default for web automation: it is one browser, it bills as ordinary sandbox usage, and an agent and a person can share it.

Reach for a Computer when the task is not a web page:

• Full desktop applications. A native app, an installer, a file manager, or anything that is not a browser tab.
• Non-browser GUIs. Windows apps, IDE windows, or a desktop that needs a real window manager.
• Whole-screen computer use. When the model must see and control the entire desktop, not one page. See Computer use.
• Human takeover of the machine. When a person needs to drive a real desktop, not just a page.

Computer use drives screenshots, clicks, and key presses against a full desktop VM. The action loop is the same shape as above, but the primitives are computer.screenshot(), computer.left_click(), computer.type(), and computer.key(). See Computer use API reference. See also Computer use vs browser use.

What you learned

• The browser is part of the sandbox; there is no separate machine to boot.
• Playwright connects over CDP to the same browser a viewer can watch, so automation and a human share one session.
• The agent loop is yours: the browser executes actions, your code decides what to do next.
• Coordinates are 0-1000 in the model’s view; scale to pixels with the page’s inner dimensions.
• Detaching the driver (browser.close()) does not stop the sandbox browser; use the browser API to free its memory.

Next