Skip to content

Browser Server

browser_server (doc_processing_std_tools.browser, MCP name browser-tools) lets an agent drive a real Chromium browser: open pages, read them as text, click, type, fill forms, manage tabs and download files. It is modelled on Anthropic's browser use toolset (browser_toolset_20260801): each member of that toolset is one tool here, named browser_<member>, and the tools work as ordinary function tools with every model provider. New in 0.6.0.

Off by default, and acting on live websites

The composed launcher adds this server only when DPTTOOLS_ENABLE_BROWSER=true. A browser agent can buy, send, delete and sign in on real sites, and it reads untrusted page content. Read Security before you enable it: with the default settings, approving consequential actions is your application's job.

Install and enable

pip install "doc-processing-std-tools[browser] @ git+https://github.com/rzoeb/doc-processing-std-tools.git@v0.6.1"
playwright install chromium            # in a container: playwright install --with-deps chromium

The browser extra installs Playwright and Pillow; Chromium itself is a separate download. Only Chromium is supported. The module imports without the extra — the tools report what is missing when they run.

Route How
Composed launcher Set DPTTOOLS_ENABLE_BROWSER=true. The browser's settings are validated at start-up only then, so a bad browser setting cannot stop the other servers
In-process Pass browser_server to Agent(tools=[...]), or build one with make_browser_server(tools="full", enable=["read_console"])

How an agent uses a page

The model reads a page as text and acts by reference:

  1. browser_read_page (or browser_find) returns the page's accessibility tree, with a [ref_N] tag on each element.
  2. The action tools take ref="ref_N". A target can also be x and y in screenshot pixels, from browser_screenshot.
  3. A ref stays valid until its tab navigates or the element is removed. A stale ref returns an error telling the model to read the page again; live refs are never renumbered.

Every result is text that ends with a Tab Context footer listing the open tabs, plus a line for each tab that opened and each download that started or finished since the last call. Pointer and form tools name the element they acted on ("Clicked button 'Place order'"), which also serves audit logs. Every tool takes an optional intent (a note on why, logged only), and every tool except the four tab tools takes an optional tab_id (default: the active tab).

Reading text is much cheaper than looking at screenshots, so the model should prefer browser_read_page and browser_get_page_text, and use browser_screenshot to check layout or visual state. DPTTOOLS_BROWSER_AUTO_SCREENSHOT=true attaches a screenshot to every action that changes the page, for models that work better that way.

Tools

The core profile (default): 18 tools

Tool Key parameters What it does
browser_navigate url Open a URL (a scheme-less URL gets https://), or "back", "forward", "reload". Only http and https
browser_screenshot — An image of the viewport; the other tools' x/y are in its pixels
browser_zoom x0, y0, x1, y1 One region of the page, enlarged up to 4×, to read small text
browser_left_click ref or x+y, modifiers Click; the result names the element clicked
browser_hover ref or x+y, modifiers Move the mouse over an element or point, for menus and tooltips
browser_scroll scroll_direction, scroll_amount (1–10, default 3), optional target Scroll the page, or the area under a target
browser_scroll_to ref Scroll an element into view
browser_type text Type into the focused element
browser_key text, repeat=1 Press a key, a chord or a sequence (Enter, ctrl+a, ctrl+a Delete)
browser_wait duration Wait up to 30 seconds
browser_read_page filter="interactive" (or "all"), depth=15, ref The accessibility tree with refs; ref reads a subtree
browser_find query Elements matching a short description, with refs. It matches words, not meaning
browser_get_page_text — The page's readable text, main content first
browser_form_input ref, value Set a text field, select, checkbox or radio directly
browser_new_tab url Open a tab, optionally at a URL, and make it active
browser_list_tabs — The open tabs with their tab id, title and URL
browser_switch_tab tab_id Make another tab active
browser_close_tab tab_id Close a tab

Also registered: fetch_webpage_rendered

Both profiles also register fetch_webpage_rendered(url, max_length, offset, format, include_assets). It returns the same JSON object as fetch_webpage, plus "rendered": true, from a page that Chromium loaded with its JavaScript run — for single-page apps and other pages fetch_webpage sees as an empty shell (it says so with likely_js_rendered).

  • Each call uses a fresh, credential-free browser context behind the same network policy, closed afterwards. In persistent and cdp modes it uses a separate headless browser, never the user's.
  • It is slower than fetch_webpage — seconds rather than milliseconds — so try that first.

So the browser server registers 19 tools with the core profile and 28 with full (18 or 27 browser_* tools, plus fetch_webpage_rendered), and the composed launcher has 75 or 84 tools with it (92 or 101 with the computer server enabled as well).

The full profile: 9 more

DPTTOOLS_BROWSER_TOOLS=full adds the rest of Anthropic's 27 default members:

Tool Key parameters What it does
browser_right_click ref or x+y, modifiers Right-click (context menu)
browser_middle_click ref or x+y, modifiers Middle-click (often opens a link in a new tab)
browser_double_click ref or x+y, modifiers Double-click (for example, select a word)
browser_triple_click ref or x+y, modifiers Triple-click (for example, select a paragraph)
browser_left_click_drag from_x, from_y, then ref or x+y Drag from a point to an element or point
browser_left_mouse_down — Press and hold the left button where the mouse is
browser_left_mouse_up — Release the left button
browser_mouse_move ref or x+y Move the mouse without clicking
browser_hold_key text, duration Hold a key or chord for up to 30 seconds

Every tool's schema costs tokens on every turn, which is why the smaller profile is the default.

Opt-in tools

Off by default because of what they can do; enable each by name in DPTTOOLS_BROWSER_ENABLE (comma-separated), or with make_browser_server(enable=[...]).

Tool Key parameters What it does
browser_javascript_exec expression Run a JavaScript expression in the page and return its value (a promise is awaited)
browser_read_console — Console messages since the last read, per tab
browser_read_network — Network requests since the last read, per tab, with secret-looking URL parameters redacted
browser_file_upload ref, paths Attach files to a file input — only files inside DPTTOOLS_BROWSER_UPLOAD_DIR, which must differ from the download directory
browser_run_script code Run a Python script with Playwright against the tab; returns printed output and the final expression's value

browser_run_script runs out of process. Each session's scripts run in one long-lived child process, so variables persist between calls, like a REPL. The script gets page, context, await screenshot() and await read_page(), using Playwright's async API.

  • It is not a sandbox. The script is model-written Python running as the server's user. It can open network connections directly, bypassing the egress proxy, and read the process environment through /proc, although the child starts without the variables whose names look like secrets (API keys, tokens, passwords). Enable it only where execute_command would also be acceptable, and rely on the container's network policy.
  • In launch and persistent modes it attaches to the session's browser over a DevTools port on 127.0.0.1, which std-tools opens only while this tool is enabled. In cdp mode it attaches to DPTTOOLS_BROWSER_CDP_URL, the endpoint you provide.
  • A script can reach the whole browser, not only its own tab: it is arbitrary Python running as the server's user. So while run_script is enabled, only one browser session is open at a time (a second client is refused until the first has been idle for DPTTOOLS_BROWSER_IDLE_TIMEOUT_S), in launch mode each new session gets a freshly started browser, and cdp mode requires DPTTOOLS_BROWSER_CDP_SHARE_TABS=true. To serve several clients with scripts enabled, run one std-tools server, in its own container, per client.
  • In launch and persistent modes the browser's own traffic still goes through the egress proxy, which the child cannot reconfigure. In cdp mode there is no proxy: std-tools checks the requests of the tabs it manages, and a script can open pages outside them (in a new context, say). Connections the script opens itself are never checked.
  • A script that runs longer than DPTTOOLS_BROWSER_SCRIPT_TIMEOUT_S (60) is stopped with its whole process tree, and its variables are lost.
  • Any local process can reach std-tools' DevTools port while it is open. That is acceptable inside a sandbox container; on a shared machine, leave the tool off.

Downloads

A download is saved to the session's folder under DPTTOOLS_BROWSER_DOWNLOAD_DIR (default: a new temporary folder per session) and reported once, on the next result:

Download started: statement.pdf from https://example.com/...
Download completed: statement.pdf saved to /tmp/.../statement.pdf (48213 bytes)

or Download failed: …. The path opens directly with read_pdf, read_document_pages_as_images, OCR and the other Documents tools. Downloaded files are kept when the session closes; manage the folder if you set it.

Downloads have no size or count limit of their own: Chromium writes the whole file before std-tools sees it. The container's disk quota is the control, so give the download folder one.

Browser modes

DPTTOOLS_BROWSER_MODE chooses whose browser the agent uses:

Mode What std-tools does Typical use
launch (default) Starts its own headless Chromium, with a fresh, credential-free context for each MCP session A sandbox container
persistent Starts its own Chromium on a saved profile (DPTTOOLS_BROWSER_PROFILE_DIR), usually headed (DPTTOOLS_BROWSER_HEADLESS=false). One session at a time A person signs in once and watches
cdp Attaches to a browser someone else started, at DPTTOOLS_BROWSER_CDP_URL Driving the user's own browser

For cdp:

  • Chrome 136 and later refuse remote debugging on the default profile. Start Chrome with --remote-debugging-port=9222 --user-data-dir=<a dedicated folder> and sign in there.
  • Bind the port to 127.0.0.1 only: whoever reaches it controls the browser.
  • The agent sees and uses only the tabs it opened, unless DPTTOOLS_BROWSER_CDP_SHARE_TABS=true.
  • The egress proxy cannot apply to a browser std-tools did not launch; requests are checked one by one instead, without IP pinning. See Security.

Isolated vs Attached compares the modes in more depth.

Sessions

  • Each MCP session gets its own browser context, so cookies and storage are not shared between sessions. In-process calls share one default session.
  • A session unused for DPTTOOLS_BROWSER_IDLE_TIMEOUT_S (600) seconds is closed with its tabs; at most DPTTOOLS_BROWSER_MAX_SESSIONS (4) are open at once, and only one while run_script is enabled.
  • Playwright runs on its own thread with a long-lived event loop, so the browser survives callers that create a new event loop per call, such as the toolkit's run_sync.
  • The viewport is DPTTOOLS_BROWSER_VIEWPORT (1440x900). Screenshots pass through the image limits; at the default viewport nothing is scaled.

Environment variables

Variable Default Effect
DPTTOOLS_ENABLE_BROWSER false Adds the browser server to the composed launcher
DPTTOOLS_BROWSER_TOOLS core core (18 browser_* tools) or full (27), plus fetch_webpage_rendered
DPTTOOLS_BROWSER_ENABLE empty Opt-in tools: javascript_exec, file_upload, read_console, read_network, run_script
DPTTOOLS_BROWSER_MODE launch launch, persistent or cdp
DPTTOOLS_BROWSER_HEADLESS true false shows a window (needs a display)
DPTTOOLS_BROWSER_PROFILE_DIR — The saved profile; required for persistent
DPTTOOLS_BROWSER_CDP_URL — The DevTools endpoint; required for cdp
DPTTOOLS_BROWSER_CDP_SHARE_TABS false In cdp mode, let the agent use tabs it did not open
DPTTOOLS_BROWSER_MAX_SESSIONS 4 Browser sessions open at once (always 1 while run_script is enabled)
DPTTOOLS_BROWSER_IDLE_TIMEOUT_S 600 Idle seconds before a session is closed
DPTTOOLS_BROWSER_VIEWPORT 1440x900 Viewport size
DPTTOOLS_BROWSER_AUTO_SCREENSHOT false Attach a screenshot after every page-changing action
DPTTOOLS_BROWSER_DOWNLOAD_DIR a temporary folder per session Where downloads are saved
DPTTOOLS_BROWSER_UPLOAD_DIR — The only folder browser_file_upload may read; required to enable it
DPTTOOLS_BROWSER_ALLOWED_DOMAINS empty (any public host) Only these domains and their subdomains
DPTTOOLS_BROWSER_BLOCKED_DOMAINS empty Never these domains; wins over the allowed list
DPTTOOLS_BROWSER_ALLOW_HOSTS empty Exact hostnames allowed past the private-address block
DPTTOOLS_BROWSER_CONFIRM off The confirmation gate: off, deny or elicit (Security)
DPTTOOLS_BROWSER_SCRIPT_TIMEOUT_S 60 Timeout for one browser_run_script call

The Configuration page gives accepted values and when each is read.

Examples

A typical sequence, written as Python calls (the model sends the same arguments as JSON):

browser_navigate(url="https://example.com/statements")
browser_read_page(filter="interactive")     # the tree, with a [ref_N] tag on each element
browser_form_input(ref="ref_3", value="2026-09")                  # the "Month" field
browser_left_click(ref="ref_7", intent="download the statement")  # the "Download" button
browser_get_page_text()

Adding the safety instructions to an agent's system prompt:

from doc_processing_toolkit import Agent
from doc_processing_std_tools.browser import BROWSER_SAFETY_PROMPT, browser_server

agent = Agent(
    model="claude-sonnet-5",
    instructions="You collect monthly statements from supplier portals.\n\n" + BROWSER_SAFETY_PROMPT,
    tools=[browser_server],
)

BROWSER_SAFETY_PROMPT tells the model that page content is untrusted data, to ask before consequential actions, and never to bypass access controls. std-tools never injects it; your application decides.

Pitfalls

  • browser_find is lexical. It matches words in element names, labels and nearby text, not meaning. When it misses, use browser_read_page(filter="interactive").
  • Refs go stale after navigation or a page update. Read the page again rather than reusing old refs.
  • Only http and https. file:, javascript:, data: and chrome: URLs are refused.
  • Nothing asks for confirmation by default. With DPTTOOLS_BROWSER_CONFIRM=off, approve consequential actions with toolkit hooks, or set the gate to deny. elicit needs an MCP client that supports elicitation — the toolkit has none today — and otherwise refuses every flagged action.
  • persistent mode allows one session. A profile folder cannot be shared between browsers.
  • Headed mode needs a display. DPTTOOLS_BROWSER_HEADLESS=false in a container without one fails to start the browser.
  • The full profile costs tokens on every turn. Use it when the model needs the extra pointer tools.
  • A missing Chromium is reported when a tool runs, not at import: run playwright install chromium (with --with-deps in a container).
  • Placeholder arguments are tolerated. Strict function calling (OpenAI) fills unused optional arguments with placeholders instead of leaving them out, so the tools read them as "not given": tab_id 0 means the active tab, and browser_navigate opens a tab when none is open whatever tab_id says; ref="" means no ref; x=0, y=0 next to a ref is ignored; and browser_scroll at (0, 0) scrolls at the centre of the page.

Recipes