Skip to content

Recipe: Portal Download to Text

Goal

Have an agent sign in to a supplier portal, pick a monthly statement, download it with the browser server, and read its text with the Documents tools.

The agent acts on a live website

A browser agent can submit, buy and delete on real sites, and it reads untrusted page content. Read Security first, narrow the browser to the portal's domain, and decide how actions are approved (step 6).

Prerequisites

  • The browser and docs extras, and Chromium:

    pip install "doc-processing-std-tools[browser,docs] @ git+https://github.com/rzoeb/doc-processing-std-tools.git@v0.6.1"
playwright install chromium            # in a container: playwright install --with-deps chromium

  • The browser and document tools on the same machine or container, so a downloaded file's path opens with read_pdf.

Steps

Tool calls are written as Python calls; the model sends the same arguments as JSON. The results are from a run against a test portal, with the Tab Context footer left out.

1. Configure the browser for one portal

DPTTOOLS_ENABLE_BROWSER=true                       # composed launcher only
DPTTOOLS_BROWSER_ALLOWED_DOMAINS=portal.example.com
DPTTOOLS_BROWSER_DOWNLOAD_DIR=/workspace/downloads
  • DPTTOOLS_BROWSER_ALLOWED_DOMAINS keeps the browser on the portal and its subdomains, so a link or an instruction planted in a page cannot take it elsewhere.
  • DPTTOOLS_BROWSER_DOWNLOAD_DIR keeps downloads in a folder you choose. Each session saves into its own subfolder, and files are kept after the session closes. Without it, downloads go to a new temporary folder per session.

2. Decide how the agent signs in

The model sends every argument of every tool call to the model provider, so a password the agent types has passed through the model, even though the tool's result shows it as (hidden). Pick one of these:

Approach How When
A person signs in once (recommended) DPTTOOLS_BROWSER_MODE=persistent with DPTTOOLS_BROWSER_PROFILE_DIR and DPTTOOLS_BROWSER_HEADLESS=false. A person signs in in the visible window, and the login stays in the profile folder Two-factor sign-in, or a password the model should never see. See Isolated vs Attached
The agent types the credentials The default launch mode; your application puts the username and password in the prompt A low-value test account only

With DPTTOOLS_BROWSER_CONFIRM=deny, password entry is refused whichever tool tries it:

Error: Refused by the confirmation gate (DPTTOOLS_BROWSER_CONFIRM=deny): enter a password is a consequential action. Ask the user to do it, or to approve it in the app.

So an unattended pipeline that uses deny needs the saved-profile approach.

3. Sign in

browser_navigate(url="https://portal.example.com/login")
browser_read_page()

Page: "Supplier portal - sign in" (https://portal.example.com/login)
- textbox "Username" [ref_1] value=""
- textbox "Password" [ref_2] value=""
- button "Sign in" [ref_3]

The model acts on the refs:

browser_form_input(ref="ref_1", value="acme-accounts")
# Set textbox 'Username' [ref_1] to 'acme-accounts'.
browser_form_input(ref="ref_2", value="...")
# Set textbox 'Password' [ref_2] to '(hidden)'.
browser_left_click(ref="ref_3", intent="sign in")
# Clicked button 'Sign in' [ref_3].

With a saved profile, the agent is already signed in and starts at step 4.

4. Pick the statement and download it

browser_read_page()

Page: "Statements" (https://portal.example.com/statements)
- combobox "Month" [ref_4] selected="2026-08"
- button "Download statement" [ref_5]

browser_form_input(ref="ref_4", value="2026-09")
# Set combobox 'Month' [ref_4] to '2026-09'.
browser_left_click(ref="ref_5", intent="download the September statement")

Clicked button 'Download statement' [ref_5].
Download started: statement-2026-09.pdf from https://portal.example.com/statement-2026-09.pdf

The download is reported once, on the first result after it finishes. A short wait is enough:

browser_wait(duration=1)

Waited 1 seconds.
Download completed: statement-2026-09.pdf saved to /workspace/downloads/session-27b1678d/statement-2026-09.pdf (1,049 bytes)

URL parameters that look like credentials (a token= in a signed download link, for example) are redacted in these lines.

5. Read the statement

The path in the Download completed line opens directly with the document tools:

read_pdf(path="/workspace/downloads/session-27b1678d/statement-2026-09.pdf")

--- Page 1 ---
Supplier statement 2026-09
Opening balance 1,200.00 Closing balance 950.00

If read_pdf reports a scanned PDF, continue with ocr_pdf (OCR a Scanned PDF). For an Excel or CSV download, use read_excel or read_file.

6. Put it together in an agent

from doc_processing_toolkit import Agent
from doc_processing_toolkit.agent import HookDirective
from doc_processing_std_tools import browser_server, document_processing_server
from doc_processing_std_tools.browser import BROWSER_SAFETY_PROMPT

ACTS = {"browser_left_click", "browser_form_input", "browser_type", "browser_key"}

async def approve(event):
    if event.event_type == "before_tool_call" and event.metadata["tool_name"] in ACTS:
        if not await ask_the_user(event.metadata["tool_name"], event.metadata["arguments"]):
            return HookDirective(action="skip_tool", message="The user declined this action.")
    return None

agent = Agent(
    model="claude-sonnet-5",
    instructions=(
        "You download monthly statements from the supplier portal and report their balances.\n\n"
        + BROWSER_SAFETY_PROMPT
    ),
    tools=[
        browser_server,
        {"type": "fastmcp", "server": document_processing_server,
         "allowed_tools": ["get_file_overview", "read_pdf", "ocr_pdf"]},
    ],
    hooks=approve,
)
result = await agent.run("Download the September 2026 statement and give me its closing balance.")

ask_the_user stands for your application's approval step. A hook sees the tool and its arguments (browser_left_click(ref="ref_5")), not the button's name, so approve per tool or per domain, and log the results, which name each element acted on.

Navigation (browser_navigate, browser_new_tab) is left out of ACTS on purpose: approving every page load would bury the actions that matter, and DPTTOOLS_BROWSER_ALLOWED_DOMAINS (step 1) already keeps the browser on the portal. Without that setting, add both tools to ACTS, and approve by the domain in args["url"].

For a reviewed, repeatable job, you can instead leave the hook out and rely on DPTTOOLS_BROWSER_ALLOWED_DOMAINS and DPTTOOLS_BROWSER_CONFIRM=deny. Using with the Toolkit covers the options.

What to validate

  • browser_read_page lists the sign-in fields and buttons with refs.
  • browser_form_input on a password field reports (hidden), and with DPTTOOLS_BROWSER_CONFIRM=deny it is refused.
  • Each click result names the element clicked.
  • Download completed appears once, with a path and a byte count, and credential-like URL parameters are redacted.
  • read_pdf opens the downloaded path.
  • With DPTTOOLS_BROWSER_ALLOWED_DOMAINS set, navigating to another domain is refused.

Test sources

  • tests/integration/test_browser_opt_in.py: test_download_is_reported_with_a_path_the_document_tools_open (download, redaction, read_pdf).
  • tests/integration/test_browser_server.py: test_form_input_sets_fields, test_type_and_key_submit_a_form, and the ref tests.
  • tests/integration/test_browser_policy.py: test_allowed_domains_limit_navigation_and_subresources, test_deny_refuses_flagged_actions_only.

These run on real Chromium and are skipped when it is missing.