# Document Processing Standard Tools — full documentation > MCP tool servers that give AI agents file, document, OCR, web, browser and desktop capabilities, designed to run in a container. Documentation for doc-processing-std-tools. Every page of the site follows, in navigation order, as its Markdown source. API reference pages appear as their mkdocstrings `:::` stubs; the site renders them from the package docstrings. # Document Processing Standard Tools `doc-processing-std-tools` is a library of [MCP](https://modelcontextprotocol.io/) tool servers that give an AI agent practical capabilities: reading and editing files, searching the web, reading documents and images, running code, keeping memory, and using Agent Skills. It also has two optional servers, off by default: a browser and a desktop the agent can operate. It is the standard tool library for the `Agent` class in [doc-processing-toolkit](https://doc-processing-toolkit.vercel.app/), but every server is an ordinary `FastMCP` server, so any MCP-compatible host can use it. Designed and built by Rounak Zoeb as the tool layer for the doc-processing-toolkit agent framework. !!! note "Source code" The source code is in a private repository. These pages document every tool, setting and API. To review the code, message [Rounak Zoeb](https://www.linkedin.com/in/rounak-zoeb-b19aaa14b) on LinkedIn with your GitHub username. !!! info "Which version this site describes" These pages describe release **0.6.1** (tag `v0.6.1`). What changed since 0.5.0 is marked *new in 0.6.0* or *new in 0.6.1*, with an "Upgrading from …" note where you need to act. The [Changelog](changelog.md) lists every change. ## The servers at a glance | Server | Import | Tools | What it gives the agent | Extra to install | |---|---|---|---|---| | [File System](servers/file_system.md) | `file_system_server` | 12 | Read, write, edit, patch, move and search files | — | | [Web](servers/web.md) | `web_search_server` | 4 | Web search (Tavily, Serper, Brave), page and image fetching with SSRF protection, citations | `[web]`, optional `[extract]` | | [Memory](servers/memory.md) | `memory_server` | 11 | Persistent key-value and file-based memory | — | | [Documents](servers/documents.md) | `document_processing_server` | 13 | PDF, Word, Excel, PowerPoint and image reading; OCR; page rendering for vision; conversion with LibreOffice | `[docs]`, `[ocr]` | | [Code Execution](servers/code_execution.md) | `code_execution_server` | 2 | A shell session that keeps its working directory | — | | [Screenshot](servers/screenshot.md) | `screenshot_server` | 2 | Screen capture of the machine the server runs on | `[screenshot]` | | [Image Tools](servers/image_tools.md) | `image_tools_server` | 7 | Crop, resize, zoom, rotate, annotate and convert images | Pillow (`[docs]` or `[screenshot]`) | | [Skills](servers/skills.md) | `skills_server` | 5 | Discover, load and run [Agent Skills](https://agentskills.io/specification) | — | | [Browser](servers/browser.md) | `browser_server` | 19 or 28 (18 or 27 `browser_*` tools, plus `fetch_webpage_rendered`) | *Optional, off by default:* a Chromium browser driven through `browser_*` tools, and rendered page fetching | `[browser]` + Chromium | | [Computer (Desktop)](servers/computer.md) | `computer_server` | 17 | *Optional, off by default:* a desktop operated by screenshot, mouse and keyboard through `computer_*` tools — its own virtual display, or the user's real screen | `[computer]` + Xvfb (virtual) | The eight always-on servers expose **56 tools**. The optional servers add to the composed launcher only when enabled: | Composed launcher | Tools | |---|---| | The eight always-on servers | 56 | | With the browser server (`DPTTOOLS_ENABLE_BROWSER=true`) | 75 (`core` profile) or 84 (`full`) | | With the computer server (`DPTTOOLS_ENABLE_COMPUTER=true`) | 73 | | With both | 92 (`core`) or 101 (`full`) | Every tool name is unique across the servers, so they can be combined freely. ## Two ways to run them - **In-process.** Import a server and hand it to the toolkit: `Agent(tools=[file_system_server])`. The tools run inside your application's process. - **Out-of-process.** `python -m doc_processing_std_tools` composes all eight servers, plus the optional ones you enable, into one MCP endpoint over stdio (default) or HTTP. Any MCP client can connect to it. [Architecture](architecture.md) explains both and when to choose which. ## Start here - [Quickstart](quickstart.md) — install, attach to an agent, run the launcher. - [Architecture](architecture.md) — how the servers are built, composed and secured. - [Compatibility](guides/compatibility.md) — Python, `mcp` and toolkit versions. - The server pages, listed above, for each server's tools, settings and pitfalls. ## Recipes Worked tasks, each checked against the code: steps, what to validate, and the tests behind it. - [OCR a Scanned PDF](recipes/ocr_a_scanned_pdf.md) — tell a scan from a text PDF and OCR the pages you need. - [Office Files to Page Images](recipes/office_to_page_images.md) — show Word, Excel and PowerPoint pages to the model. - [Search, Then Summarise](recipes/search_then_summarise.md) — a summary that cites the pages it read. - [JavaScript-Rendered Pages](recipes/js_rendered_page.md) — read pages built by JavaScript. - [Portal Download to Text](recipes/portal_download_to_text.md) — sign in, download a statement, read it. - [Desktop Actions with Hooks](recipes/desktop_with_hooks.md) — approve each desktop action from your application. - [Skills Setup](recipes/skills_setup.md) — write a skill and attach the skills server. - [The Launcher with Other MCP Clients](recipes/launcher_with_other_mcp_clients.md) — desktop assistants, IDEs and MCP SDK clients. ## With doc-processing-toolkit The toolkit's documentation covers the agent side of the same integration: - [Tools](https://doc-processing-toolkit.vercel.app/concepts/tools/) — the tool formats an `Agent` accepts, including FastMCP servers and `allowed_tools`. - [MCP](https://doc-processing-toolkit.vercel.app/concepts/mcp/) — how the toolkit consumes MCP servers, in-process and over stdio. - [Hooks and steering](https://doc-processing-toolkit.vercel.app/concepts/hooks/) — approving or vetoing tool calls from your application. - [Skills auto-injection](https://doc-processing-toolkit.vercel.app/concepts/skills/) — how the toolkit reads this package's `list_skills` result. - [Client integration](https://doc-processing-toolkit.vercel.app/concepts/client_integration/) — pinning, upgrading and running the toolkit inside an application. - [Speech and images from an agent in your app](https://doc-processing-toolkit.vercel.app/recipes/agent_media_in_apps/) — letting the model look at an image through `read_image`. ## For coding agents Two plain-text files at the site root are generated from this site's navigation: - [llms.txt](llms.txt) — a short map of every page, with links, in the [llms.txt](https://llmstxt.org/) format. - [llms-full.txt](llms-full.txt) — every page's Markdown source in one file. ## Source code The source code is in a private repository. These pages document every tool, setting and API. To review the code, message [Rounak Zoeb](https://www.linkedin.com/in/rounak-zoeb-b19aaa14b) on LinkedIn with your GitHub username. # Quickstart ## Install `doc-processing-std-tools` needs **Python 3.10 or newer**. It is not published to PyPI: install it from Git at a release tag (this needs access to the repository). !!! note "Source code" The source code is in a private repository. These pages document every tool, setting and API. To review the code, message [Rounak Zoeb](https://www.linkedin.com/in/rounak-zoeb-b19aaa14b) on LinkedIn with your GitHub username. ```bash pip install "doc-processing-std-tools @ git+https://github.com/rzoeb/doc-processing-std-tools.git@v0.6.1" ``` The base install covers the File System, Memory, Code Execution and Skills servers. The other servers need optional dependencies, installed as extras: | Extra | Installs | Needed for | |---|---|---| | `docs` | pdfplumber, python-docx, openpyxl, python-pptx, Pillow, pymupdf | the [Documents](servers/documents.md) readers and renderers, and [Image Tools](servers/image_tools.md) | | `ocr` | llmwhisperer-client 2.x | `ocr_pdf` and `ocr_image` (also needs `LLMWHISPERER_API_KEY`) | | `web` | httpx, Pillow | the [Web](servers/web.md) server | | `extract` | trafilatura | optional: main-content extraction and better Markdown in `fetch_webpage` | | `screenshot` | mss, Pillow | the [Screenshot](servers/screenshot.md) server | | `browser` | Playwright, Pillow | the optional [Browser](servers/browser.md) server; then run `playwright install chromium` | | `computer` | mss, Pillow, pynput (not on Linux) | the optional [Computer (Desktop)](servers/computer.md) server; its `virtual` backend also needs Xvfb, `xdotool` and a window manager from the system's package manager | | `all` | all seven above | everything (Chromium and the desktop's system packages are still installed separately) | ```bash pip install "doc-processing-std-tools[docs,web] @ git+https://github.com/rzoeb/doc-processing-std-tools.git@v0.6.1" ``` A missing extra never breaks an import. The tool that needs it returns an `Error: ...` message naming the extra to install. Two things are not Python packages and come from the operating system or a provider: - **LibreOffice**, on `PATH` as `libreoffice` or `soffice` (or named in `DPTTOOLS_SOFFICE_BIN`), for `convert_document` and to render Word, Excel and PowerPoint pages as images. PDFs render without it. - **An LLMWhisperer API key** in `LLMWHISPERER_API_KEY`, for OCR. !!! tip "Installing alongside doc-processing-toolkit" Resolve your application's requirements, the toolkit and std-tools in **one** `pip install` command, so pip sees every constraint at once. See [Compatibility](guides/compatibility.md) for the versions that work together. ## Attach servers to a toolkit agent Each server is a `FastMCP` object. Pass the ones you need straight to the toolkit's `Agent`: ```python from doc_processing_toolkit import Agent from doc_processing_std_tools import document_processing_server, file_system_server agent = Agent( model="claude-sonnet-5", instructions="You read documents in the workspace and answer questions about them.", tools=[file_system_server, document_processing_server], ) result = agent.run_sync("List the PDFs in ./reports and summarise the newest one.") print(result.output) ``` The tools run inside your process and read the `DPTTOOLS_*` variables from its environment. (Skills are the exception with toolkit 0.5.2 and earlier: see [Compatibility](guides/compatibility.md#known-toolkit-issue-skills-auto-injection-toolkit-052-and-earlier).) To expose only some of a server's tools (toolkit 0.5.1 or later), use the toolkit's dict form: ```python tools = [ { "type": "fastmcp", "server": document_processing_server, "allowed_tools": ["read_pdf", "read_image"], }, ] ``` !!! warning "Image-returning tools in-process need toolkit 0.5.1+" `read_image`, the page renderers and `take_screenshot` return images. Attached in-process to toolkit 0.5.0 or older, they reached the model as the text `<...Image object at 0x...>`. Use toolkit 0.5.1 or later, or run the servers out of process. ## Run the composed launcher To serve all eight servers as one MCP endpoint, run the package: ```bash python -m doc_processing_std_tools # stdio (default) python -m doc_processing_std_tools --transport http --port 8000 # streamable HTTP on 127.0.0.1 ``` That serves 56 tools. The two optional servers join only when enabled: set `DPTTOOLS_ENABLE_BROWSER=true` for the [browser](servers/browser.md) (75 tools with its `core` profile), `DPTTOOLS_ENABLE_COMPUTER=true` for the [computer](servers/computer.md) server (73), or both (92). The launcher checks the `DPTTOOLS_*` settings before serving and exits with a message naming any invalid one. Each server can also run on its own over stdio, for example `python -m doc_processing_std_tools.file_system`. A toolkit agent can start the launcher itself: ```python import sys agent = Agent( model="claude-sonnet-5", tools=[ { "type": "mcp_command", "command": sys.executable, "args": ["-m", "doc_processing_std_tools"], "name": "std-tools", } ], ) ``` !!! warning "A launched server does not see your environment variables" The MCP Python SDK starts a stdio server with only a short list of safe variables (such as `PATH` and `HOME`) unless the client passes an explicit `env`, and the toolkit's `mcp_command` and `mcp_script` configs pass none (toolkit 0.5.2 and earlier). So `DPTTOOLS_*` and `LLMWHISPERER_API_KEY` set in your application **do not reach** a launcher the toolkit starts, and skills auto-injection sees an empty catalog. Open the stdio session yourself with an explicit `env` and pass it to the agent as a pre-built session — the code is in [Using with the Toolkit](guides/with_the_toolkit.md#out-of-process-a-session-you-open-yourself). ## First calls With the servers attached, a model calls tools by name. A few calls that work on a fresh install with the `docs` extra: | Call | Returns | |---|---| | `list_directory(path=".")` | JSON list of entries with name, type and size | | `get_file_overview(path="report.pdf")` | JSON summary: page count, dimensions, text preview per page | | `read_pdf(path="report.pdf", pages="1-3")` | Text with `--- Page N ---` headers | | `read_document_pages_as_images(path="report.pdf", pages="1-2")` | One image per page, for the model to look at | | `execute_command(command="python --version")` | JSON with `stdout`, `stderr`, `exit_code`, `timed_out` | Every tool returns a string (or an image) and reports failures as a string that starts with `Error:`, rather than raising. ## Next steps - [Architecture](architecture.md) — in-process versus out-of-process, the security posture. - The server pages — each server's tools, settings and pitfalls. - [Compatibility](guides/compatibility.md) — version requirements. - [The Launcher with Other MCP Clients](recipes/launcher_with_other_mcp_clients.md) — registering the launcher with desktop assistants and IDEs. - The toolkit's [Local MCP Tools](https://doc-processing-toolkit.vercel.app/recipes/local_mcp_tools/) recipe — every way an agent can mount an MCP server. # Architecture ![Architecture diagram. An MCP host (doc-processing-toolkit agents, IDE coding agents, desktop assistants, custom apps) talks MCP to the launcher, which runs inside the container, the security boundary. The launcher composes eight always-on servers with 56 tools (Files 12, Documents 13, Web 4, Memory 11, Image tools 7, Agent Skills 5, Code execution 2, Screenshot 2) and two optional servers that are off by default (Browser, 19 to 28 tools; Desktop, 17). Calls to web pages, search APIs and the OCR API leave the container, guarded by the safety controls: an SSRF check with the IP pinned on every redirect hop, the browser egress proxy, a confirmation gate for consequential actions, and size caps, timeouts and process-tree kill.](assets/images/architecture.png) ## One FastMCP server per module Each server is a `FastMCP` instance defined in its own module of `doc_processing_std_tools`. A tool is a plain Python function registered on that server — synchronous for the eight always-on servers, `async` for the browser and computer servers. | Module | Server object | MCP server name | Tools | |---|---|---|---| | `file_system` | `file_system_server` | `file-system-tools` | 12 | | `web_search` | `web_search_server` | `web-search-tools` | 4 | | `memory` | `memory_server` | `memory-tools` | 11 | | `document_processing` | `document_processing_server` | `document-processing-tools` | 13 | | `code_execution` | `code_execution_server` | `code-execution-tools` | 2 | | `screenshot` | `screenshot_server` | `screenshot-tools` | 2 | | `image_tools` | `image_tools_server` | `image-tools` | 7 | | `skills` | `skills_server` (and `make_skills_server()`) | `skills-tools` | 5 | | `browser` (optional) | `browser_server` (and `make_browser_server()`) | `browser-tools` | 19 (`core`) or 28 (`full`): 18 or 27 `browser_*` tools plus `fetch_webpage_rendered`, and opt-ins | | `computer` (optional) | `computer_server` (and `make_computer_server()`) | `computer-tools` | 17 | Three conventions hold across every server: - **The docstring is the tool description.** FastMCP sends a tool's docstring to the model as its description, and builds the parameter schema from the type hints. Changing a docstring changes what the model reads when it chooses a tool. The [Tool Reference](reference/tools.md) shows that text, formatted for reading, with the exact text in a collapsed block under each tool. - **Errors are returned, not raised.** Tools report failures as a string beginning with `Error:` — a missing file, a bad argument, a missing optional library — so the model can read it and recover. Messages never include API keys, and network errors never echo the URL or IP. - **Optional dependencies are imported lazily.** A tool imports its library (pdfplumber, httpx, mss, Pillow, …) when it runs. If the library is missing, the tool returns an error naming the extra to install; the server still imports and its other tools still work. The package itself imports lazily too (PEP 562): `from doc_processing_std_tools import document_processing_server` imports only that module, so an install with only the `docs` extra never fails because of another server's missing dependency. ## Two ways to run the servers ### In-process Import the server objects and pass them to the toolkit's `Agent`: ```python from doc_processing_toolkit import Agent from doc_processing_std_tools import file_system_server, memory_server agent = Agent(model="claude-sonnet-5", tools=[file_system_server, memory_server]) ``` The toolkit calls the tool functions inside your application's process. There is no subprocess and no transport. See the toolkit's [Tools](https://doc-processing-toolkit.vercel.app/concepts/tools/) and [MCP](https://doc-processing-toolkit.vercel.app/concepts/mcp/) pages for the formats an `Agent` accepts. ### Out-of-process: the composed launcher `python -m doc_processing_std_tools` builds one parent server, `FastMCP("doc-processing-std-tools")`, and runs it over stdio (the default) or streamable HTTP: ```bash python -m doc_processing_std_tools python -m doc_processing_std_tools --transport http --host 127.0.0.1 --port 8000 ``` How it composes (`compose_servers()` in `__main__.py`): - Every tool of every sub-server is **re-registered on the parent** with the same function, name, description and annotations, so a tool behaves identically through either route. The `mcp` SDK has no public composition API, so this reads each server's registered tools through a private FastMCP attribute — the reason `mcp` is pinned below 2 ([Compatibility](guides/compatibility.md)). - **Tool names are not prefixed.** If two sub-servers ever registered the same name, the launcher would stop at start-up with a `RuntimeError` naming both, rather than let one shadow the other. - Tools that return images keep their unstructured output setting, so image results pass through unchanged. - **Blocking tools run on worker threads** (*new in 0.6.0*). Every tool is a synchronous function, and FastMCP would call it on its event loop, so one slow call would freeze every other request and the transport itself. The launcher therefore registers each synchronous tool as an `async` wrapper that runs it on a worker thread (`anyio.to_thread.run_sync`). Names, descriptions and schemas are unchanged; a unit test checks this. - **The browser server is optional** (*new in 0.6.0*). It is composed only when `DPTTOOLS_ENABLE_BROWSER=true`, and its `DPTTOOLS_BROWSER_*` settings are validated only then, so a bad browser setting cannot stop the other servers. Its tools are `async` and run on the browser's own runtime thread, so they are registered as they are. - **The computer server is optional too** (*new in 0.6.0*). It is composed only when `DPTTOOLS_ENABLE_COMPUTER=true`, and its `DPTTOOLS_COMPUTER_*` settings are validated only then. Its tools are `async` (they move blocking work to a thread themselves), so they are also registered as they are. With both optional servers the launcher has 92 tools (`core` browser profile) or 101 (`full`). - Before serving, the launcher validates the configuration and exits with a message naming any invalid variable. !!! warning "The HTTP transport has no authentication" `--transport http` binds to `127.0.0.1` by default. The server does not authenticate callers, so bind another interface (`--host 0.0.0.0`) only behind an authenticating proxy or a network policy that limits who can reach it. Each sub-server can also run alone over stdio, for example `python -m doc_processing_std_tools.document_processing`. A sub-server run this way skips the launcher's start-up validation; a bad setting surfaces as an `Error:` from the first tool that reads it. ### Which to choose | | In-process | Composed launcher (stdio or HTTP) | |---|---|---| | Where tools run | Your application's process | A separate process | | Configuration | Your process's environment | The launched process's environment — see [Configuration](#configuration) | | Image results (`read_image`, renderers, screenshots) | Need toolkit 0.5.1 or later | Work with any toolkit version | | Clients | One `Agent` | Any MCP client; several can share one HTTP server | | Shared state | Per process (for example, the shell session's working directory) | Shared by every client of that server process | ### Concurrency Because the composed server runs tools on worker threads, calls from one or several clients can overlap. What that means: - **A slow tool no longer blocks the others.** An OCR job (up to 10 minutes), a LibreOffice conversion (up to 2 minutes), a long `execute_command` or a slow page fetch runs while other calls proceed. - **File changes are locked.** Every tool in the File System and Memory servers that changes a file (writes, creates, edits, patches, deletes, moves, copies, renames) holds its server's lock from its checks to the end of the change. Overlapping calls therefore behave as if they ran one after the other: no lost edits, no interleaved overwrites, and a deleted file is not written back. The lock is per server process, not across processes. - **The shell session is serialised.** `execute_command` and `restart_session` share one session per process and take turns. - **A started call runs to the end.** A worker thread cannot be interrupted, so if a client cancels a call, the tool keeps running until it finishes or reaches its own timeout. Each sub-server run on its own (`python -m doc_processing_std_tools.`) does not use worker threads: FastMCP runs its tools one at a time on the event loop. In-process, the agent decides how tools are called; see [Using with the Toolkit](guides/with_the_toolkit.md#which-to-choose). ## Configuration Everything is configured through environment variables; the [Configuration](guides/configuration.md) page lists every one, generated from the table in the code. In short: - std-tools reads the **process environment only**. It never loads a `.env` file itself; load one in whatever starts std-tools (`docker run --env-file .env`, `env_file:` in Docker Compose, or your application's own `load_dotenv()`). - Values are read **once per process**, on first use, and cached — except `LLMWHISPERER_API_KEY`, which is read on every OCR call. Restart the process after changing one. - A numeric variable that is not a positive integer is an **error naming the variable**. The composed launcher refuses to start; other entry points fail on the first tool call that reads it. - `DPTTOOLS_MEMORY_DIR` and `DPTTOOLS_SKILLS_DIR` are resolved to absolute paths at that first read, relative to the process's working directory at the time. !!! warning "Stdio servers started by a client get a minimal environment" When an MCP client launches a stdio server, the MCP Python SDK passes the child only a short list of safe variables (`PATH`, `HOME` and a few others) unless the client supplies an explicit `env`. Pass every `DPTTOOLS_*` variable and API key you use in `env`. The toolkit's `mcp_command` and `mcp_script` configs pass none (toolkit 0.5.2 and earlier), so open the session yourself; see [Using with the Toolkit](guides/with_the_toolkit.md#passing-configuration-to-a-stdio-server). ## Security posture std-tools is designed to run **inside an isolated container** that the client application controls. The container is the security boundary: its mounts, network settings and resource limits decide what the agent can reach. What that means in practice: - **File paths are not confined.** The file, document and image tools accept relative paths with `..` and absolute paths, because moving around the sandbox is legitimate. `sanitize_path` only rejects an embedded NUL byte. Anything the process can read, the agent can read. - **Commands are not filtered.** `execute_command` runs whatever the model sends, as the server's user. It uses a subprocess — never `eval` or `exec` — with a timeout that kills the whole process tree. Defence in depth that is always on: | Measure | Where | |---|---| | File-size limit before reading (`DPTTOOLS_MAX_FILE_SIZE_MB`) | `read_file`, `search_in_files`, document, image and skill-resource reads | | Output truncation (`DPTTOOLS_OUTPUT_TRUNCATION_LIMIT`) | File and document text, command and script output, skills | | Timeouts | `execute_command` (kills the whole process tree), `execute_skill_script`, LibreOffice conversion, OCR | | SSRF protection: non-public addresses refused unless allow-listed, every redirect re-checked, the connection pinned to the validated IP | `fetch_webpage`, `fetch_image` | | Real path containment | The [Memory](servers/memory.md) tools (inside `DPTTOOLS_MEMORY_DIR`) and the [Skills](servers/skills.md) tools (inside `DPTTOOLS_SKILLS_DIR`) | | A pixel cap against decompression bombs (about 256 million pixels) | [Image Tools](servers/image_tools.md) | | Secrets kept out of results: errors have request URLs and API keys removed; the search key is excluded from `repr` | `web_search`, OCR, configuration | A typical production container (adapt the mounts to your application): ```bash docker run --rm \ -v /host/project:/workspace:rw \ -v /host/memory:/workspace/.memory:rw \ -v /host/skills:/workspace/.agent_skills:ro \ --memory=2g --cpus=2 \ -e DPTTOOLS_MEMORY_DIR=/workspace/.memory \ -e DPTTOOLS_SKILLS_DIR=/workspace/.agent_skills \ -e DPTTOOLS_WEB_SEARCH_PROVIDER=tavily \ -e DPTTOOLS_WEB_SEARCH_API_KEY=tvly-xxx \ my-agent-image python run_agent.py ``` - Run as a non-root user. - Mount read-only whatever the agent should not change (here, the skills directory). - Add `--network=none` when the agent needs no network. The Web server and OCR need outbound access, so leave it off if you use them. The browser and computer servers bring their own safety layers — for the browser, an egress proxy with IP pinning and domain lists; for the desktop, a separate virtual display per session, and a kill switch and a pause on human activity in host mode; for both, a confirmation gate that is off by default — and a split of responsibilities between std-tools and the client application. The [Security](guides/security.md#the-browser-and-desktop-servers) guide covers them. ## Optional extras | Extra | Packages | Servers or tools | |---|---|---| | (base) | `mcp>=1.25,<2`, `pydantic`, `pyyaml` | File System, Memory, Code Execution, Skills | | `docs` | pdfplumber, python-pptx, openpyxl, python-docx, Pillow, pymupdf | Documents (readers and renderers), Image Tools | | `ocr` | `llmwhisperer-client>=2,<3` | `ocr_pdf`, `ocr_image` | | `web` | `httpx>=0.27`, Pillow | Web (Pillow for `fetch_image`) | | `extract` | `trafilatura>=2.0` | Optional: main-content extraction in `fetch_webpage` | | `screenshot` | `mss`, Pillow | Screenshot | | `browser` | `playwright>=1.49,<2`, Pillow | Browser (plus `playwright install chromium`) | | `computer` | `mss`, Pillow, `pynput` (not on Linux) | Computer (the `virtual` backend also needs Xvfb, `xdotool` and a window manager) | | `all` | `docs`, `ocr`, `web`, `extract`, `screenshot`, `browser`, `computer` | Everything | std-tools deliberately does **not** depend on `doc-processing-toolkit`: OCR talks to LLMWhisperer directly, and the servers work with any MCP host. # File System Server `file_system_server` (`doc_processing_std_tools.file_system`, MCP name `file-system-tools`) gives an agent 12 tools to explore, read, write, edit and search files. It needs no extra. ```python from doc_processing_std_tools import file_system_server ``` ## Tools | Tool | Key parameters | Returns | |---|---|---| | `get_directory_structure` | `path`, `max_depth=3`, `include_hidden=False` | A tree drawn with box characters, directories first | | `list_directory` | `path`, `include_hidden=False` | JSON list of `{name, type, size}` (size for files only), directories first | | `read_file` | `path`, `offset` (1-based line), `limit` (lines), `encoding="utf-8"` | The text, truncated at the output limit | | `write_file` | `path`, `content`, `create_dirs=False`, `encoding="utf-8"` | Confirmation with the bytes written; overwrites an existing file | | `create_file` | `path`, `content`, `create_dirs=False` | Confirmation; fails if the file already exists | | `edit_file` | `path`, `old_string`, `new_string` | Confirmation; `old_string` must occur exactly once | | `apply_patch` | `path`, `patch` (unified diff) | Confirmation with the hunk count | | `delete_file` | `path` | Confirmation; files only, not directories | | `move_file` | `source`, `destination` | Confirmation; moves files or directories | | `copy_file` | `source`, `destination` | Confirmation; copies a file with its metadata | | `get_file_metadata` | `path` | JSON: `path`, `size_bytes`, `size_human`, `modified`, `created`, `is_file`, `is_dir`, `extension` | | `search_in_files` | `path`, `query`, `file_pattern="*"`, `is_regex=False`, `max_results=50` | JSON list of `{file, line, content}` matches | Timestamps in `get_file_metadata` are ISO 8601 in UTC. Hidden entries are names that start with a dot. ## Environment variables | Variable | Default | Effect here | |---|---|---| | `DPTTOOLS_MAX_FILE_SIZE_MB` | `50` | `read_file` refuses larger files; `search_in_files` skips them | | `DPTTOOLS_OUTPUT_TRUNCATION_LIMIT` | `50000` | `read_file` output is cut at this many characters, with a notice | ## Examples Tool calls are written here as Python calls; the model sends the same arguments as JSON. Read lines 100–149 of a log: ```python read_file(path="logs/run.log", offset=100, limit=50) ``` Search Python files for a regular expression: ```python search_in_files(path="src", query=r"def \w+_server", file_pattern="*.py", is_regex=True) ``` Change one line with `apply_patch(path="config.ini", patch=...)`, where `patch` is: ```diff --- a/config.ini +++ b/config.ini @@ -1,3 +1,3 @@ [server] -port = 8000 +port = 8080 host = 127.0.0.1 ``` ## Pitfalls - **Paths are not confined to a directory.** Relative `..` paths and absolute paths are accepted; the container is the boundary. See [Architecture](../architecture.md#security-posture). - **`apply_patch` is strict.** Each hunk's context and removed lines must match the file exactly at the stated line numbers, and the hunk body must contain exactly the number of lines its `@@` header declares. A mismatch is rejected — as "malformed patch" when the diff itself is inconsistent, or as "does not match current file content" when the file has changed — and the file is left untouched. For a single replacement, `edit_file` is simpler. - **`edit_file` needs a unique match.** If `old_string` occurs zero times or more than once, the call fails and says how many times it was found. Include more surrounding text to make it unique. - **Overlapping calls.** Through the composed launcher, tools run on worker threads and calls can overlap. Every tool that changes a file (`write_file`, `create_file`, `edit_file`, `apply_patch`, `delete_file`, `move_file`, `copy_file`) holds one lock from its checks to the end of the change, so overlapping calls behave as if they ran one after the other: no lost edits, no interleaved overwrites, and a deleted or moved file is not written back. The lock is per server process. - **`search_in_files` searches only text it can read.** It skips files over the size limit and files it cannot open. A PDF is searched only through a companion text file named `.pdf.txt` next to it; a PDF without one is skipped. The glob in `file_pattern` is applied recursively, and the search stops at `max_results` matching lines. - **`write_file` and `create_file` do not create folders** unless `create_dirs=True`. - **Encodings.** `read_file` replaces undecodable bytes instead of failing. `edit_file`, `apply_patch` and `create_file` always use UTF-8. - **Large listings are not truncated.** `get_directory_structure` and `list_directory` return the whole result; keep `max_depth` small on big trees. # Web Server `web_search_server` (`doc_processing_std_tools.web_search`, MCP name `web-search-tools`) gives an agent web search through a configured provider, a page fetcher and an image fetcher with SSRF protection, and a formatter that adds citation markers. ```python from doc_processing_std_tools import web_search_server ``` | Extra | Installs | Needed for | |---|---|---| | `web` | httpx, Pillow | every tool here (Pillow for `fetch_image`) | | `extract` | trafilatura | optional: main-content extraction and better Markdown in `fetch_webpage` | `[all]` includes both. ## Tools | Tool | Key parameters | Returns | |---|---|---| | `web_search` | `query`, `num_results=5` (capped at 20), `search_type="general"` (or `"news"`), `search_depth="basic"` | JSON `{"provider", "query", "results": [{"title", "url", "snippet"}], "usage"}` | | `fetch_webpage` | `url`, `format="text"` (or `"markdown"`, `"html"`), `max_length=10000`, `offset=0`, `include_assets=False`, `structured=True`, `extract_text=True` | A JSON object with the page's text and metadata (below); the plain text with `structured=False` | | `fetch_image` | `url` | A caption (source, format, size) followed by the image for the model to look at | | `format_search_results` | `results` — the `web_search` output as it is, or a bare JSON array of `{title, url, snippet or content}` | Numbered entries (`[1] Title`) followed by a `References:` list | ## Environment variables | Variable | Default | Effect here | |---|---|---| | `DPTTOOLS_WEB_SEARCH_PROVIDER` | none | `tavily`, `serper` or `brave`. Required by `web_search` | | `DPTTOOLS_WEB_SEARCH_API_KEY` | none | The provider's API key. Required by `web_search` | | `DPTTOOLS_WEB_FETCH_TIMEOUT` | `30` | HTTP timeout for `fetch_webpage` and `fetch_image`, in seconds, applied to each network operation; connecting is capped at 10 s | | `DPTTOOLS_WEB_FETCH_MAX_BYTES` | `2000000` | Most bytes `fetch_webpage` downloads from one response; a longer page is cut and the result says `byte_cap_reached` | | `DPTTOOLS_WEB_FETCH_MAX_REDIRECTS` | `5` | Most redirects the fetch tools follow | | `DPTTOOLS_WEB_FETCH_ALLOW_HOSTS` | empty | Comma-separated exact hostnames the fetch tools may reach although they resolve to a private address | | `DPTTOOLS_FETCH_IMAGE_MAX_BYTES` | `10000000` | Largest image `fetch_image` downloads | | `DPTTOOLS_IMAGE_MAX_EDGE_PX`, `DPTTOOLS_IMAGE_MAX_MB` | `1568`, `3` | The size `fetch_image` scales an image to before the model sees it ([Images](../guides/images.md)) | All except the first two are *new in 0.6.0*. `fetch_webpage`, `fetch_image` and `format_search_results` work without a search provider. The [Configuration](../guides/configuration.md) page lists every variable. !!! warning "Upgrading from 0.5.0" - **`fetch_webpage` returns a JSON object by default.** Code that used its result as the page text should read the `text` field, or pass `structured=False` for the plain string 0.5.0 returned. - **Non-text responses are refused.** Images, PDFs and other binary content now return an error (with their size) instead of garbled text; plain text and JSON are no longer passed through the HTML-to-text step. - **`fetch_webpage`'s timeout is `DPTTOOLS_WEB_FETCH_TIMEOUT`**, no longer `DPTTOOLS_SANDBOX_TIMEOUT`. If you raised the sandbox timeout to allow slow pages, set the new variable instead. - **`web_search` output has a `usage` key**, and `search_type="news"` no longer forces Tavily's 2-credit `advanced` depth. Strict-schema parsers must allow the new key. ## `fetch_webpage` ### The result ```json { "url": "https://example.com/report", "final_url": "https://www.example.com/report", "status": 200, "content_type": "text/html", "title": "Annual report", "format": "text", "extractor": "trafilatura", "text": "...", "truncated": true, "total_chars": 48210, "offset": 0, "next_offset": 10000, "has_more": true, "byte_cap_reached": false, "likely_js_rendered": false } ``` | Field | Meaning | |---|---| | `final_url`, `status`, `content_type` | Where the request ended after redirects, and what the server sent | | `title` | The HTML ``; `null` for non-HTML content | | `format` | How HTML was returned: `text`, `markdown` or `html`. `text` for non-HTML content | | `extractor` | `trafilatura` (the `extract` extra found main content), `stdlib` (the built-in extractor) or `none` (returned as-is) | | `text` | At most `max_length` characters, starting at `offset` | | `total_chars`, `has_more`, `next_offset` | The full length; when `has_more` is true, call again with `offset=next_offset` | | `truncated` | More text exists past this window, or the download was cut | | `byte_cap_reached` | The response was larger than `DPTTOOLS_WEB_FETCH_MAX_BYTES` and was cut, so the text ends early | | `likely_js_rendered` | The page looks built by JavaScript in the browser: little text with several scripts, or an empty app root such as `<div id="root"></div>` | With `include_assets=True` the object also has: - `images` — up to 30 `{url, alt, width, height, source}`, from `<img>`, `og:image` / `twitter:image` and icons; - `links` — up to 100 `{url, text}`; - `assets_truncated` — true when a list hit its cap. All URLs are absolute. **Nothing is downloaded**: pass an image URL to `fetch_image` to look at it. ### What it reads, and how - **Text content only:** HTML, plain text, Markdown, JSON, XML, CSV, other `text/*` types, and `+json` / `+xml` types. Anything else is refused from its headers, **before the body is downloaded**. A response with no content type is downloaded and refused if it looks binary. - **Only HTML is converted.** An HTML page — including one served as `text/plain` — becomes text, Markdown or stays HTML, per `format`. JSON, plain text, CSV and other text are returned as they are. - **Extraction.** With the `extract` extra, `text` and `markdown` hold the page's **main content**, without navigation, sidebars and footers. Without it, or when trafilatura finds no main content, the built-in extractor converts the whole page; its Markdown keeps headings, lists, links and tables. `structured=False` always uses the built-in extractor. - **Encoding** comes from a byte-order mark, then the `Content-Type` charset, then an HTML `<meta charset>`, then UTF-8. Undecodable bytes are replaced rather than failing. - `extract_text=False` is the older spelling of `format="html"`. ### SSRF policy `fetch_webpage` and `fetch_image` share one policy for reaching the network: 1. Only `http` and `https` URLs are accepted. 2. Each hop's hostname is resolved **once**, and **every** resolved address must be public. Refused: every block the IANA special-purpose address registries list as not globally reachable (private, loopback, link-local `169.254.0.0/16`, which covers cloud metadata endpoints, carrier-grade NAT `100.64.0.0/10`, `0.0.0.0/8`, the IETF protocol block `192.0.0.0/24`, unique-local `fc00::/7`, and the documentation and benchmarking ranges), multicast, reserved addresses, and any IPv6 address outside the public unicast block `2000::/3`. The blocks are listed in the code, so the result is the same on every Python release. An IPv6 address that carries an IPv4 one (IPv4-mapped `::ffff:…`, 6to4, Teredo, NAT64 `64:ff9b::/96`) is judged by that IPv4 address, so `::ffff:127.0.0.1` is refused like `127.0.0.1`. 3. The connection goes to the **first validated address** itself, with the original hostname in the `Host` header and for TLS, so a second DNS answer cannot redirect it (DNS rebinding). 4. Redirects are followed by hand, at most `DPTTOOLS_WEB_FETCH_MAX_REDIRECTS`, and every hop is checked again. 5. No credentials are sent — std-tools sets no auth headers — so a redirect cannot carry any to another host. 6. Bodies stop at the byte limit, whatever the server sends. 7. Network errors are reported without the URL or the resolved address. An operator can let named internal hosts through with `DPTTOOLS_WEB_FETCH_ALLOW_HOSTS` — exact hostnames, no wildcards, empty by default. Those hosts are still resolved once and pinned; only the private-address check is skipped for them. !!! warning "Upgrading from 0.6.0" The address check is wider in 0.6.1. Carrier-grade NAT addresses (`100.64.0.0/10`, which Tailscale uses), `0.0.0.0`, the IETF protocol block `192.0.0.0/24` and IPv4 addresses inside IPv6 forms were reachable before and are now refused. If the fetch tools reach an internal host at such an address, add its hostname to `DPTTOOLS_WEB_FETCH_ALLOW_HOSTS`, and to `DPTTOOLS_BROWSER_ALLOW_HOSTS` for the browser. ## `fetch_image` `fetch_image(url)` downloads one image and shows it to the model: - **PNG, JPEG, GIF and WebP** only. The content type is checked before download, and the real format is then checked with Pillow, so a mislabelled file is refused. - An image whose declared size is over `DPTTOOLS_FETCH_IMAGE_MAX_BYTES` is refused **before download**; one that turns out larger while downloading is refused too. Images over 64 million pixels are refused. - The image is scaled to the model limits (1568 px, 3 MB by default) before it is returned; the caption says when it was scaled, for example `Image from https://example.com/chart.png: PNG, 3200x1800 px, scaled to 1568x882 for the model.` - The same SSRF policy as `fetch_webpage` applies, including `DPTTOOLS_WEB_FETCH_ALLOW_HOSTS`. ## Search providers and usage | Provider | Authentication | `search_type="news"` | `search_depth` | `usage` in the result | |---|---|---|---|---| | Tavily | `Authorization: Bearer <key>` header | Searches the news topic | `basic` (1 credit), `advanced` (2 credits), `fast` (1), `ultra-fast` (1) | What Tavily reports, plus `search_depth`, `request_id` and `response_time` | | Serper | `X-API-KEY` header | Searches news | Ignored | `{"provider_reported": false}` | | Brave | `X-Subscription-Token` header | Passes a news search type | Ignored | `{"provider_reported": false}` | - For Tavily, std-tools asks for usage and passes on what Tavily returns, marked `"provider_reported": true`. If Tavily reports nothing, the result carries an estimate instead, marked `"estimated": true` with `estimated_credits`. - Serper and Brave report no cost, and std-tools does not invent one. - `search_type="news"` does not change the depth: pass `search_depth="advanced"` explicitly to pay for it. - Provider calls use a fixed 30-second timeout. Provider errors are returned with request URLs replaced by `<url>` and the API key by `<redacted>`. ## Examples Tool calls are written as Python calls; the model sends the same arguments as JSON. ```python web_search(query="LibreOffice headless convert-to options", num_results=5) web_search(query="central bank rate decision", search_type="news", search_depth="advanced") fetch_webpage(url="https://example.com/article", format="markdown") fetch_webpage(url="https://example.com/article", offset=10000) # the next part fetch_webpage(url="https://example.com/gallery", include_assets=True) fetch_image(url="https://example.com/images/chart.png") format_search_results(results=search_output) # the whole web_search output ``` ## Pitfalls - **Read long pages in parts.** `text` holds `max_length` characters; follow `next_offset` while `has_more` is true rather than raising `max_length` a long way. An `offset` past the end is an error. - **`likely_js_rendered` means the text is probably incomplete.** `fetch_webpage` reads the HTML the server sends and runs no JavaScript. - **Binary files are refused, not downloaded.** For a PDF or an Office file, download it with `execute_command` and use the [Documents](documents.md) tools; for an image, use `fetch_image`. - **Internal hosts need the allow-list.** Every non-public address is refused unless its hostname is in `DPTTOOLS_WEB_FETCH_ALLOW_HOSTS`. - **An unknown `format`, `search_type` or `search_depth` is an error**, not a silent default. - **Search results are not truncated** by `DPTTOOLS_OUTPUT_TRUNCATION_LIMIT`; `num_results` bounds them. - **A cancelled fetch keeps running.** Through the composed launcher, tools run on worker threads, which cannot be interrupted; a fetch ends when it completes or times out. ### JavaScript-rendered pages When `likely_js_rendered` is true, the text is probably incomplete. If the [Browser](browser.md) server is enabled, retry with `fetch_webpage_rendered`: it returns the same JSON object (plus `"rendered": true`) from a page Chromium loaded with its JavaScript run, using a fresh, credential-free context behind the browser's network policy. It is slower — seconds rather than milliseconds — so `fetch_webpage` stays the first choice. `fetch_webpage`'s description tells the model to do this when the tool is available. ## Recipes - [Search, Then Summarise](../recipes/search_then_summarise.md): search, read the best sources and write a summary that cites them. - [JavaScript-Rendered Pages](../recipes/js_rendered_page.md): fall back to `fetch_webpage_rendered` when a page is an empty shell. <!-- source: docs/servers/memory.md --> # Memory Server `memory_server` (`doc_processing_std_tools.memory`, MCP name `memory-tools`) gives an agent memory that survives across sessions, stored as files in one directory. It offers two interfaces over the same directory: a **key-value** interface and a **file-based** interface modelled on Anthropic's memory tool. It needs no extra. ```python from doc_processing_std_tools import memory_server ``` ## Tools ### Key-value Each key is stored as `<key>.json` in the memory directory, holding `{"key": ..., "value": ...}`. | Tool | Key parameters | Returns | |---|---|---| | `write_memory` | `key`, `value` | Confirmation; overwrites an existing value | | `read_memory` | `key` | The stored value, or an error if the key does not exist | | `search_memory` | `query` | JSON list of `{key, value}` whose key or value contains `query` (case-insensitive) | | `list_memory_keys` | — | JSON list of keys | | `delete_memory` | `key` | Confirmation | ### File-based Paths are relative to the memory directory. Use `/` or `""` for the directory itself. | Tool | Key parameters | Returns | |---|---|---| | `memory_view` | `path`, `view_range=[start, end]` (1-based, optional) | A directory listing, or the file with line numbers | | `memory_create` | `path`, `file_text` | Confirmation; creates parent folders; fails if the file exists | | `memory_str_replace` | `path`, `old_str`, `new_str` | Confirmation; `old_str` must occur exactly once | | `memory_insert` | `path`, `insert_line` (0-based), `insert_text` | Confirmation; line 0 inserts at the top, the line count appends | | `memory_delete` | `path` | Confirmation; deletes a file, or a folder **with everything in it** | | `memory_rename` | `old_path`, `new_path` | Confirmation; fails if the destination exists | ## Environment variables | Variable | Default | Effect here | |---|---|---| | `DPTTOOLS_MEMORY_DIR` | `./agent_memory` | Where memory is stored. Resolved to an absolute path, against the process's working directory, the first time configuration is read | `write_memory` creates the directory if it does not exist. Mount it as a persistent volume in a container, or memory is lost when the container stops. ## Examples Tool calls are written as Python calls; the model sends the same arguments as JSON. ```python write_memory(key="client:acme:preferred_format", value="PDF, one file per invoice") search_memory(query="acme") memory_create(path="projects/acme/notes.md", file_text="# Acme\n\n- Invoices arrive monthly.\n") memory_view(path="projects/acme/notes.md", view_range=[1, 20]) ``` ## Pitfalls - **Paths cannot leave the memory directory.** The file-based tools resolve every path inside `DPTTOOLS_MEMORY_DIR` and refuse one that resolves outside it, including through a symbolic link. This is real containment, unlike the File System server. - **The store root cannot be deleted.** `memory_delete` refuses an empty path, `.`, `/` or any path that resolves to the memory directory itself. Any other folder is deleted with all its contents and without confirmation. - **Keys become flat file names.** `/` and `\` in a key are replaced with `_`, so `a/b` and `a_b` are the same key. The original key is kept inside the file and returned by `list_memory_keys`. - **The two interfaces share one folder.** The key-value tools treat every top-level `.json` file as an entry, so a file created with `memory_create(path="notes.json", ...)` appears in `list_memory_keys` under its file name. Keep file-based memory in subfolders, or use another extension, to keep the two apart. - **Values are strings.** Store structured data as a JSON string and parse it after reading. - **Locking is per process.** Within one server process, every tool that changes the store (`write_memory`, `delete_memory`, `memory_create`, `memory_str_replace`, `memory_insert`, `memory_delete`, `memory_rename`) holds one lock from its checks to the end of the change, so overlapping calls through the composed launcher behave as if they ran one after the other: no lost edits, no half-written entries, and a deleted file is not written back. Two separate server processes writing the same key or file can still overwrite each other's changes, and `write_memory` simply replaces a value. - **Long results are not truncated.** `memory_view` and `search_memory` return everything; use `view_range` on large files. <!-- source: docs/servers/documents.md --> # Documents Server `document_processing_server` (`doc_processing_std_tools.document_processing`, MCP name `document-processing-tools`) gives an agent 13 tools to read PDFs, Word, Excel and PowerPoint files and images, to OCR scans, to render pages as images the model can look at, and to convert documents between formats. ```python from doc_processing_std_tools import document_processing_server ``` Writing documents from scratch is not part of this server: an agent creates them with code through the [Code Execution](code_execution.md) server, and can then convert them with `convert_document`. ## Install | Need | Install | |---|---| | The readers, renderers and image tools | the `docs` extra (pdfplumber, python-docx, openpyxl, python-pptx, Pillow, pymupdf) | | `ocr_pdf`, `ocr_image` | the `ocr` extra (llmwhisperer-client 2.x) **and** `LLMWHISPERER_API_KEY` | | `convert_document`, and rendering Word, Excel, PowerPoint or OpenDocument files | LibreOffice: on Debian or Ubuntu, `apt-get install libreoffice-core libreoffice-writer libreoffice-calc libreoffice-impress`. std-tools runs `libreoffice`, then `soffice`, from `PATH`, or the executable in `DPTTOOLS_SOFFICE_BIN` | PDFs render with pymupdf alone; LibreOffice is only needed to convert Office files. ## Tools ### Reading text | Tool | Key parameters | Returns | |---|---|---| | `read_pdf` | `path`, `pages="all"`, `extract_tables=False` | Text with `--- Page N ---` headers; tables as `|`-separated rows. Says so, and suggests `ocr_pdf`, when no page has text | | `read_word` | `path` (`.docx`) | Text: headings as Markdown `#` lines, then header and footer lines, tables, text boxes, and an embedded-image count when there are images | | `read_excel` | `path` (`.xlsx`), `sheet` (default: the active sheet), `range` (for example `"A1:D10"`) | JSON: `sheet`, `headers` (first row), `rows`, `total_rows`, `merged_ranges`, `has_images`, `has_charts`, `has_media` | | `read_powerpoint` | `path` (`.pptx`) | JSON: `slides` (each with `slide`, `text`, `speaker_notes`, `image_count`, `chart_count`, `has_media`), `slide_count`, `has_media` | | `get_file_overview` | `path` | JSON summary for the file type: PDF page count, sizes and text previews; Excel sheets, sizes and sample rows; Word headings and counts; PowerPoint slide titles; image size; or size, line count and first and last lines for any other file | ### OCR (LLMWhisperer) | Tool | Key parameters | Returns | |---|---|---| | `ocr_pdf` | `path` (`.pdf`), `pages="all"` | OCR text with `--- Page N ---` headers | | `ocr_image` | `path` (`.png`, `.jpg`, `.jpeg`, `.webp`, `.tif`, `.tiff`, `.bmp`) | OCR text | ### Seeing pages and images These return MCP image content, so the model sees the pixels. Every image is first brought within model-safe limits — see [Image size limits](#image-size-limits) below. | Tool | Key parameters | Returns | |---|---|---| | `read_image` | `path` (PNG, JPEG, WEBP, GIF, BMP, TIFF) | The image, with its format taken from the bytes rather than the extension; the file itself when it is within the limits | | `get_image_metadata` | `path` | JSON: `format`, `width`, `height`, `mode`, `file_size`, `file_size_bytes` — no pixels | | `render_document_page` | `path`, `page=1`, `scale=2.0` | One page as a PNG | | `render_document_thumbnail` | `path`, `max_pages=5` | The first pages as PNGs at scale 1.0 (72 dpi) | | `read_document_pages_as_images` | `path`, `pages="all"`, `scale=2.0`, `max_pages=20` | One PNG per requested page | The renderers accept PDF, DOCX, XLSX, PPTX, DOC, XLS, PPT, ODT, ODS and ODP, and reject image files (use `read_image`). `scale` multiplies 72 dpi, so the default `2.0` renders an A4 page at about 1190 × 1684 pixels. ### Image size limits *New in 0.6.0.* `read_image`, `render_document_page`, `render_document_thumbnail` and `read_document_pages_as_images` scale an image down when its long edge is over `DPTTOOLS_IMAGE_MAX_EDGE_PX` (default 1568 px) or its encoded size is over `DPTTOOLS_IMAGE_MAX_MB` (default 3 MB). Those defaults are accepted by every provider the toolkit supports. An image within both limits is sent unchanged, byte for byte; a scaled one may arrive as JPEG. An A4 page at the default `scale=2.0` is 1684 px tall, so it is scaled to 1568 px. [Images](../guides/images.md) explains the rules. ### Converting documents *New in 0.6.0.* | Tool | Key parameters | Returns | |---|---|---| | `convert_document` | `path`, `to`, `outdir=None`, `page="all"`, `overwrite=False` | JSON `{"output_files": [...], "pages": ..., "engine": ...}` — the files it wrote, not their content | - **Targets (`to`):** `pdf`, `docx`, `xlsx`, `pptx`, `odt`, `ods`, `odp`, `csv`, `html`, `txt`, `png`, `jpg` (`jpeg` is accepted as `jpg`). - **Sources:** Office and OpenDocument files (DOCX, XLSX, PPTX, DOC, XLS, PPT, ODT, ODS, ODP, RTF and similar). A **PDF can only become `png` or `jpg`**; to get its text, use `read_pdf`. - **`png` and `jpg`** write one image per page, named `<name>-p<N>.png` (or `.jpg`), rendered at 144 dpi. The document is converted to PDF first, then each page is rendered with pymupdf; `page` picks the pages (`"all"`, `"1-3"`, `"2,5"`). For other targets `page` must be `"all"`. - **`csv`** exports only the first sheet of a spreadsheet. `txt` and `csv` are written as UTF-8. - **Other targets** write one file, `<name>.<to>`. - **Where:** next to the source file, or into `outdir`, which must already exist. An existing output file is **never replaced** unless `overwrite=True`; otherwise the call fails and names the file. - **`pages`** in the result is the number of images written for `png`/`jpg`, the page count for `pdf` (when pymupdf is installed), and `null` otherwise. **`engine`** is `libreoffice`, `libreoffice+pymupdf` or `pymupdf`. `convert_document` writes files and returns paths; it does not show anything to the model. To look at pages, use `read_document_pages_as_images`, or `read_image` on a converted page image. ### LibreOffice runs `convert_document` and the Office-to-PDF step of the three render tools run LibreOffice headless. Each run gets **its own LibreOffice profile and output folder**, which are removed afterwards, so concurrent conversions in one container do not collide (*new in 0.6.0*). A run is stopped after `DPTTOOLS_CONVERT_TIMEOUT_S` seconds (default 120). ## Choosing a tool | The file is… | Start with | Then, if needed | |---|---|---| | Unknown | `get_file_overview` | the matching reader | | A PDF with selectable text | `read_pdf` | `read_document_pages_as_images` for charts and diagrams | | A scanned PDF | `ocr_pdf` (needs a key; costs per page) | or `read_document_pages_as_images`, which needs no key — the model reads the page images | | A Word, Excel or PowerPoint file | `read_word`, `read_excel`, `read_powerpoint` | `read_document_pages_as_images` when the result is near-empty or reports media (`has_media`, the image count) | | A photo or screenshot of text | `ocr_image` | or `read_image` | | An image to inspect visually | `read_image` | `get_image_metadata` to check its size first | | A file to deliver in another format | `convert_document` | `read_document_pages_as_images` to check the result | ## Environment variables | Variable | Default | Effect here | |---|---|---| | `LLMWHISPERER_API_KEY` | none | Required by `ocr_pdf` and `ocr_image`. Read on every call | | `DPTTOOLS_OCR_TIMEOUT` | `180` | Deadline for one OCR call (upload plus waiting for the result), in seconds, clamped to 30–600. Read once per process; a value that is not a positive integer is an error naming the variable | | `DPTTOOLS_MAX_FILE_SIZE_MB` | `50` | Every tool here refuses files larger than this | | `DPTTOOLS_IMAGE_MAX_EDGE_PX` | `1568` | Long-edge limit for images returned to the model. *New in 0.6.0* | | `DPTTOOLS_IMAGE_MAX_MB` | `3` | Encoded-size limit for images returned to the model. *New in 0.6.0* | | `DPTTOOLS_OUTPUT_TRUNCATION_LIMIT` | `50000` | Text results are cut at this many characters. OCR output is cut on a page boundary | | `DPTTOOLS_SOFFICE_BIN` | none | The LibreOffice executable (a path, or a command on `PATH`). When unset, `libreoffice` then `soffice` on `PATH`. A value that does not resolve is an error naming the variable. *New in 0.6.0* | | `DPTTOOLS_CONVERT_TIMEOUT_S` | `120` | Timeout for one LibreOffice conversion, in seconds. *New in 0.6.0* | ## Examples Tool calls are written as Python calls; the model sends the same arguments as JSON. ```python get_file_overview(path="statements/2026-09.pdf") read_pdf(path="statements/2026-09.pdf", pages="1-3", extract_tables=True) ocr_pdf(path="scans/contract.pdf", pages="1-3,7") ocr_pdf(path="scans/contract.pdf", pages="21-") # page 21 to the end read_document_pages_as_images(path="deck.pptx", pages="1-5", max_pages=5) convert_document(path="reports/q3.docx", to="pdf") convert_document(path="deck.pptx", to="png", page="1-3", outdir="exports") ``` A `convert_document` result: ```json { "output_files": ["/workspace/exports/deck-p1.png", "/workspace/exports/deck-p2.png", "/workspace/exports/deck-p3.png"], "pages": 3, "engine": "libreoffice+pymupdf" } ``` ## Pitfalls - **The `pages` syntax differs between tools.** `read_pdf` and `read_document_pages_as_images` take `"all"`, a page (`"3"`), a range (`"1-5"`) or a comma list mixing both (`"1-3,7"`); a range that runs past the end is clipped, but a start page past the end is an error. `ocr_pdf` takes the same forms plus an open-ended range, `"21-"`. - **`pages="all"` can flood the context.** `read_document_pages_as_images` returns one image per page, up to `max_pages` (20); pages past the cap are dropped without an error. Ask for a range. - **A higher `scale` does not mean more detail for the model.** Pages are scaled back to the long-edge limit, so beyond about `scale=2.0` a larger value mostly costs time. To read small print, crop or zoom with [Image Tools](image_tools.md) first, then `read_image` the result. - **An image that cannot fit is an error.** If an image would have to shrink below 256 px on its long edge to fit the byte limit, the tool returns an `Error:` suggesting a larger `DPTTOOLS_IMAGE_MAX_MB` or a crop. - **Long OCR output is cut on a page boundary.** The result then names the pages left out and the `pages` value to request them with. `ocr_image` has no pagination. - **OCR prechecks the PDF.** An empty file and a file without a PDF header are refused before anything is uploaded; so are a password-protected PDF and a PDF with no pages, when pymupdf is installed. - **Excel values are strings, and formulas are not returned.** `read_excel` reads the values saved with the workbook. A workbook generated by code and never opened in Excel may have no saved results for its formulas, which then read as empty strings. Only `.xlsx` workbooks are read; render an old `.xls` file with `read_document_pages_as_images` instead. - **Old binary formats.** `read_word` and `read_powerpoint` read `.docx` and `.pptx` only. `.doc` and `.ppt` files can still be rendered as images, through LibreOffice. - **Office conversion needs LibreOffice.** Without it, `convert_document` and rendering a Word, Excel or PowerPoint file return an error with the install command. - **Fonts come from the machine.** LibreOffice lays out pages with the fonts installed where it runs; install CJK fonts (for example `fonts-noto-cjk`) for Chinese, Japanese or Korean documents. - **`convert_document` will not overwrite.** Pass `overwrite=True`, or another `outdir`, when the output already exists. Converting a file to its own format is an error. - **Large documents.** A conversion that takes longer than `DPTTOOLS_CONVERT_TIMEOUT_S` is stopped; raise it for very long documents. - **In-process image results need toolkit 0.5.1 or later.** See [Compatibility](../guides/compatibility.md). !!! warning "Upgrading from 0.5.0" `DPTTOOLS_OCR_TIMEOUT` used to be read on every call, and a non-integer value silently fell back to 180. It is now read once per process, and an invalid value is an error naming the variable; the composed launcher refuses to start. Values outside 30–600 are still clamped. !!! tip "Skills that run LibreOffice by hand" Skill content that tells the model to write its own `soffice` commands through `execute_command` can be retired: `convert_document` handles the per-run profile, the timeout, cleanup and the "exit 0 but no output" case. Keep the policy — when to convert, naming and folders — in the skill, and have it call `convert_document`. ## Recipes - [OCR a Scanned PDF](../recipes/ocr_a_scanned_pdf.md): tell a scan from a text PDF, OCR the pages you need, and read a long document in parts. - [Office Files to Page Images](../recipes/office_to_page_images.md): show Word, Excel and PowerPoint pages to the model, or save them as image files. <!-- source: docs/servers/code_execution.md --> # Code Execution Server `code_execution_server` (`doc_processing_std_tools.code_execution`, MCP name `code-execution-tools`) gives an agent a shell session that remembers its working directory between calls. It is how an agent runs scripts and creates documents with code. It needs no extra. ```python from doc_processing_std_tools import code_execution_server ``` ## Tools | Tool | Key parameters | Returns | |---|---|---| | `execute_command` | `command`, `timeout` (seconds, optional) | JSON: `stdout`, `stderr`, `exit_code` (int), `timed_out` (bool) | | `restart_session` | — | Confirmation naming the working directory it reset to | `command` can use pipes, redirects and chaining (`&&`, `||`, `;`). ## How the session works - Each `execute_command` runs in a **fresh subprocess**, started in the session's current working directory. After the command ends, the session records the directory the command finished in, so a `cd` carries over to the next call. - **Only the working directory carries over.** Variables set with `export FOO=bar` or `source .env` are gone on the next call. Set them in the same command (`source .env && python job.py`), or write them to a file and read it each time. Files written to disk do persist. - `restart_session` returns to the directory the server started in. - On Linux and macOS the command runs in `/bin/bash -c`. On Windows it runs in `cmd.exe` through a temporary batch file, so use `cmd` syntax there. - There is **one session per server process**, shared by every caller of that process. Calls are serialised, so two commands never run at once in the same session. ## Environment variables | Variable | Default | Effect here | |---|---|---| | `DPTTOOLS_SANDBOX_TIMEOUT` | `30` | The default timeout **and the ceiling**: a larger `timeout` argument is lowered to this value | | `DPTTOOLS_OUTPUT_TRUNCATION_LIMIT` | `50000` | `stdout` and `stderr` are each cut at this many characters | ## Examples Tool calls are written as Python calls; the model sends the same arguments as JSON. ```python execute_command(command="cd /workspace/reports && ls -la") execute_command(command="python make_summary.py --month 2026-09", timeout=120) execute_command(command='python -c "import docx; print(docx.__version__)"') ``` A result: ```json {"stdout": "1.1.2\n", "stderr": "", "exit_code": 0, "timed_out": false} ``` ## Pitfalls - **The timeout cannot exceed `DPTTOOLS_SANDBOX_TIMEOUT`.** To allow long jobs, raise the variable. On timeout the whole process tree is killed and the result has `"timed_out": true` and `"exit_code": -1`, with whatever output was captured. - **A zero or negative `timeout`** returns `exit_code: -1` with an explanation in `stderr`. - **Commands are not filtered.** The command runs with the server process's permissions. The container is the security boundary — see [Architecture](../architecture.md#security-posture). - **Interactive commands are not supported.** There is no terminal, and a command that waits for input can block until the timeout. Pass flags such as `-y` or `--yes`. - **Background processes.** On Linux and macOS, a process started with `&` that keeps the command's output open makes the call wait until the timeout, and is then killed with the rest of the process group. Redirect its output (`job > job.log 2>&1 &`) if it must outlive the call. - **The working directory is per server process.** Restarting the server, or using a second server process, starts again from the process's own working directory. - **Long commands no longer block other tools** in the composed launcher (*new in 0.6.0*): it runs each call on a worker thread, so file, search and document calls proceed while a command runs. Commands still take turns in the one session. A command cannot be cancelled from the client; it runs until it finishes or reaches its timeout. <!-- source: docs/servers/screenshot.md --> # Screenshot Server `screenshot_server` (`doc_processing_std_tools.screenshot`, MCP name `screenshot-tools`) captures the screen of **the machine the server process runs on** and returns it to the model as an image, or saves it as a PNG. It needs the `screenshot` extra (mss and Pillow). Its tools capture the real screen of the machine the server runs on: in a container, usually nothing useful; on a user's machine, whatever is on their screen. For an agent's own desktop, use the [computer server](computer.md)'s `computer_screenshot`. ```python from doc_processing_std_tools import screenshot_server ``` !!! warning "This is the real screen" `take_screenshot` captures the actual monitors of the computer running std-tools. On a person's machine that includes everything open in other windows, and the image is sent to the model provider. Inside a container with no display, it returns an error. Attach this server only where capturing that screen is intended. It has no input capability: it cannot click or type. To act on a screen, use the [Computer (Desktop)](computer.md) server, which works on its own virtual display by default and on the user's real screen only when explicitly configured. ## Tools | Tool | Key parameters | Returns | |---|---|---| | `take_screenshot` | `output_path` (optional), `monitor` (optional), `region` (optional) | An image for the model when `output_path` is omitted; otherwise saves a PNG and returns its path | | `take_window_screenshot` | `window_title`, `output_path` | Always an error explaining that window capture is not supported, with a `take_screenshot(region=...)` example | `take_screenshot` options: - `monitor`: `0` is all monitors combined (the default), `1` the first monitor, `2` the second, and so on. An index out of range returns an error listing the valid ones. - `region`: `{"left": int, "top": int, "width": int, "height": int}` in screen pixels. It overrides `monitor`. - `output_path`: the folder must already exist; the tool does not create folders. ## Image size *New in 0.6.0.* The image returned to the model is scaled down when its long edge is over `DPTTOOLS_IMAGE_MAX_EDGE_PX` (default 1568 px) or its encoded size is over `DPTTOOLS_IMAGE_MAX_MB` (default 3 MB), and may then arrive as JPEG. A capture within both limits is sent unchanged. A file written to `output_path` is always the full-resolution PNG. See [Images](../guides/images.md). ## Environment variables | Variable | Default | Effect here | |---|---|---| | `DPTTOOLS_IMAGE_MAX_EDGE_PX` | `1568` | Long-edge limit for the image returned to the model | | `DPTTOOLS_IMAGE_MAX_MB` | `3` | Encoded-size limit for the image returned to the model | ## Examples Tool calls are written as Python calls; the model sends the same arguments as JSON. ```python take_screenshot(monitor=1) take_screenshot(region={"left": 0, "top": 0, "width": 1280, "height": 720}) take_screenshot(output_path="/workspace/captures/step-3.png") ``` ## Pitfalls - **The default captures every monitor at once.** Two full-HD screens give one 3840 × 1080 image, which is scaled to 1568 × 441 for the model, so text becomes hard to read. Prefer `monitor=1` or a `region`. - **Small text may need a region.** A 2560 × 1440 monitor is scaled to 1568 × 882. Capture a `region` around what matters, or save to `output_path` and zoom with [Image Tools](image_tools.md). - **In-process image results need toolkit 0.5.1 or later.** See [Compatibility](../guides/compatibility.md). - **No display, no capture.** In a headless container the capture fails with an error. A virtual desktop started by the computer server is not this server's screen: capture it with `computer_screenshot`. <!-- source: docs/servers/image_tools.md --> # Image Tools Server `image_tools_server` (`doc_processing_std_tools.image_tools`, MCP name `image-tools`) gives an agent seven tools to crop, resize, zoom, rotate, annotate, convert and inspect images with Pillow. It needs Pillow, which the `docs` and `screenshot` extras both install. ```python from doc_processing_std_tools import image_tools_server ``` The editing tools **write a new file and return its path**; they do not show the image to the model. To let the model see the result, pass the path to [`read_image`](documents.md). ## Tools | Tool | Key parameters | Returns | |---|---|---| | `image_crop` | `path`, `left`, `top`, `right`, `bottom`, `output_path`, `frame=0` | Path of the cropped image (default `<name>_cropped.<ext>`) | | `image_resize` | `path`, `width`, `height`, `output_path`, `frame=0` | Path of the resized image (default `<name>_resized.<ext>`). Give one dimension to keep the aspect ratio, or both for an exact size | | `image_zoom` | `path`, `region={"left", "top", "width", "height"}`, `zoom_factor=2.0`, `output_path`, `frame=0` | Path of the enlarged region (default `<name>_zoomed.<ext>`) | | `image_rotate` | `path`, `angle` (degrees, counter-clockwise), `output_path`, `frame=0` | Path of the rotated image (default `<name>_rotated.<ext>`); the canvas grows to fit | | `image_annotate` | `path`, `annotations` (a JSON list), `output_path`, `frame=0` | Path of the annotated image (default `<name>_annotated.<ext>`) | | `image_convert` | `path`, `format` (`png`, `jpeg`, `webp`, `bmp`, `tiff`), `output_path`, `frame=0` | Path of the converted image (default: same name with the new extension — `.jpg` for JPEG, `.tif` for TIFF) | | `image_get_info` | `path` | JSON: `width`, `height`, `format`, `mode`, `n_frames`, `is_animated`, `file_size_bytes`, `file_size_human` | Annotation objects for `image_annotate`: ```json [ {"type": "rectangle", "x": 40, "y": 60, "width": 200, "height": 80, "color": "red"}, {"type": "text", "x": 40, "y": 30, "text": "Total", "font_size": 24, "color": "red"}, {"type": "circle", "cx": 300, "cy": 200, "radius": 25, "color": "blue"}, {"type": "arrow", "x1": 10, "y1": 10, "x2": 120, "y2": 90, "color": "green"} ] ``` ## How images are handled - **Orientation.** Images are opened with their EXIF orientation applied, so coordinates match what a viewer shows. `image_get_info` reports the corrected dimensions. - **Quality and metadata.** Saving keeps the ICC colour profile, EXIF data and DPI where the format allows. JPEG is written at quality 95 without chroma subsampling, WebP at quality 95. - **Format.** The output format follows the output file's extension. Only `image_convert` changes the format. Converting to JPEG flattens transparency onto a **white** background. - **Multi-frame images.** For an animated GIF or a multi-page TIFF, `frame` picks the frame (0-based); an out-of-range frame is an error. - **Size cap.** Images above about 256 million pixels are refused as likely decompression bombs. ## Environment variables | Variable | Default | Effect here | |---|---|---| | `DPTTOOLS_MAX_FILE_SIZE_MB` | `50` | Every tool refuses input files larger than this | ## Examples Tool calls are written as Python calls; the model sends the same arguments as JSON. ```python image_get_info(path="scan.jpg") image_zoom(path="scan.jpg", region={"left": 800, "top": 1200, "width": 400, "height": 200}, zoom_factor=3) image_resize(path="photo.png", width=1200, output_path="photo_small.png") ``` ## Pitfalls - **The default output path sits next to the input** and is overwritten if it exists. Pass `output_path` to choose the location. `image_convert` to the format the file already has, without an `output_path`, writes over the input itself. - **Coordinates are in pixels of the corrected image.** Call `image_get_info` first to learn the dimensions; `image_crop` takes the box's right and bottom edges, while `image_zoom` takes a width and height. - **Edits do not reach the model by themselves.** Follow an edit with `read_image` on the returned path when the model needs to see it. Zooming into a region first is a cheap way to let the model read small print. - **These tools do not scale for the model.** They write files at the size you ask for. When the model looks at a result through `read_image`, that tool scales it to the model-safe limits ([Images](../guides/images.md)). <!-- source: docs/servers/skills.md --> # Skills Server `skills_server` (`doc_processing_std_tools.skills`, MCP name `skills-tools`) lets an agent discover, load and run **Agent Skills**: folders of instructions, reference files and scripts that follow the [Agent Skills specification](https://agentskills.io/specification). Skills work the same with every model provider. The server needs no extra (PyYAML is a base dependency). ```python from doc_processing_std_tools import skills_server ``` ## How skills are laid out Each skill is a folder under `DPTTOOLS_SKILLS_DIR` containing a `SKILL.md` with YAML frontmatter: ```text .agent_skills/ invoice-extraction/ SKILL.md scripts/ normalise_totals.py references/ field-glossary.md ``` ```markdown --- name: invoice-extraction description: Extracts header fields and line items from supplier invoices into JSON. license: MIT compatibility: Needs the docs extra. metadata: owner: finance-team --- # Invoice extraction 1. Run `get_file_overview` on the invoice ... ``` The agent reads skills in three levels, loading only what it needs: 1. `list_skills` — the name, description and path of every skill. 2. `load_skill` — one skill's full `SKILL.md`. 3. `load_skill_resource` and `execute_skill_script` — the files and scripts the skill refers to. ## Tools | Tool | Key parameters | Returns | |---|---|---| | `list_skills` | — | JSON: `skills`, `skills_dir`, `count`, and `diagnostics` when there are warnings (see below) | | `load_skill` | `skill_name` | The `SKILL.md` wrapped in `<skill_content name="..." directory="...">`, with a `<skill_resources>` list of the skill's files and the skill directory | | `load_skill_resource` | `skill_name`, `resource_path` (relative to the skill folder) | The file's text | | `execute_skill_script` | `skill_name`, `script_path` (a `.py` file in the skill), `args=""` | JSON: `stdout`, `stderr`, `exit_code` | | `create_skill` | `skill_name`, `description` | Confirmation with the new skill's folder, which holds a template `SKILL.md` | ### The `list_skills` payload ```json { "skills": [ { "name": "invoice-extraction", "description": "Extracts header fields and line items from supplier invoices into JSON.", "path": "/workspace/.agent_skills/invoice-extraction/SKILL.md", "directory": "/workspace/.agent_skills/invoice-extraction", "license": "MIT", "compatibility": "Needs the docs extra.", "metadata": {"owner": "finance-team"} } ], "skills_dir": "/workspace/.agent_skills", "count": 1, "diagnostics": [ {"skill": "draft-skill", "level": "error", "code": "missing_description", "message": "SKILL.md frontmatter is missing a non-empty 'description' field."} ] } ``` - `path` and `directory` are absolute. `license`, `compatibility` and `metadata` appear only when the frontmatter has them; `metadata` values are turned into strings. - `diagnostics` is present only when something was wrong. Each entry has `skill`, `level` (`warning` or `error`), `code` and `message`. Errors skip the skill; warnings do not. - When the skills folder does not exist, the result has an empty `skills` list, `count: 0` and a `note` saying so. ### Frontmatter rules | Situation | Result | |---|---| | No frontmatter, or YAML that cannot be parsed | Skill skipped, `error` diagnostic | | Missing or empty `description` | Skill skipped, `error` diagnostic | | An unquoted colon in `description` (common in skills written for other clients) | Quoted automatically and loaded, `warning` diagnostic | | Missing `name` | The folder name is used, `warning` diagnostic | | `name` differs from the folder, breaks the naming pattern, or is too long | Loaded, `warning` diagnostic | | `description` over 1024 characters, `compatibility` over 500, `metadata` not a mapping | Loaded (a non-mapping `metadata` is dropped), `warning` diagnostic | The `allowed-tools` field is not surfaced: this stack does not restrict tools per skill. ## Environment variables | Variable | Default | Effect here | |---|---|---| | `DPTTOOLS_SKILLS_DIR` | `./.agent_skills` | The skills folder, resolved to an absolute path when configuration is first read | | `DPTTOOLS_SANDBOX_TIMEOUT` | `30` | Timeout for `execute_skill_script`, in seconds | | `DPTTOOLS_MAX_FILE_SIZE_MB` | `50` | `load_skill_resource` refuses larger files | | `DPTTOOLS_OUTPUT_TRUNCATION_LIMIT` | `50000` | Cuts `load_skill`, resource and script output | ## A server only when skills exist `skills_server` always registers all five tools, so a skill added during a session is found by the next `list_skills` call. For a static deployment that should advertise no skills tools at all when the folder is empty, use the factory, which returns `None` in that case: ```python from doc_processing_std_tools import file_system_server, make_skills_server skills = make_skills_server() # None if DPTTOOLS_SKILLS_DIR is missing or has no skill agent = Agent(model="claude-sonnet-5", tools=[s for s in [skills, file_system_server] if s]) ``` The factory checks the folder once, when it is called. ## With doc-processing-toolkit The toolkit reads this server automatically ([Skills auto-injection](https://doc-processing-toolkit.vercel.app/concepts/skills/)): - It calls `list_skills` and puts each skill's `name`, `description` and `path` into the system prompt, and logs each `diagnostics` entry as a warning. Diagnostics never reach the model. - When there are no skills, it hides `load_skill`, `load_skill_resource` and `execute_skill_script` from the model to save tokens, and keeps `list_skills` and `create_skill` visible. - It finds the skills server by name: the name must contain `skill`. `skills_server` is named `skills-tools`, which matches; the composed server, `doc-processing-std-tools`, does not. Those three tool names, the naming rule and the `path` and `diagnostics` fields are a contract between the two packages; std-tools has supplied the fields since 0.2.0. !!! warning "Toolkit 0.5.2 and earlier: connect the skills server through a pre-built session" In those toolkit versions, auto-injection works only when your application starts the server itself, passes `DPTTOOLS_SKILLS_DIR` in the session's `env`, and hands the toolkit `{"session": session, "name": "...skill..."}`; `DPTTOOLS_SKILLS_DIR` must also be set in the application's own environment. In-process `Agent(tools=[skills_server])` and `mcp_command` configs return an empty catalog, and the agent never hears of your skills. This is a toolkit issue; [Compatibility](../guides/compatibility.md#known-toolkit-issue-skills-auto-injection-toolkit-052-and-earlier) has the working code. ## Pitfalls - **Skill names are strict in tool calls.** `load_skill`, `load_skill_resource`, `execute_skill_script` and `create_skill` accept only 1–64 lowercase letters, digits and single hyphens, not starting or ending with a hyphen, and the name must be the **folder** name. - **Paths cannot leave the skills folder.** Resource and script paths are resolved inside `DPTTOOLS_SKILLS_DIR`; one that escapes is refused. - **Scripts must be Python.** `execute_skill_script` runs only `.py` files, with the same Python interpreter as the server, in the script's own folder. `args` is split with POSIX shell rules, so quote arguments that contain spaces. - **A script that times out** returns `exit_code: -1` and a timeout message in `stderr`. - **Large `SKILL.md` files** are cut to fit the output limit, but the resource list and the skill directory are always kept, so the agent can read the rest with `load_skill_resource`. - **Resource listings are bounded.** `load_skill` lists the files in the skill folder and its immediate subfolders (`scripts/normalise_totals.py`, but not `scripts/lib/util.py`), at most 50 with a count of the rest, skipping hidden files and folders such as `__pycache__` and `node_modules`. Deeper files can still be read with `load_skill_resource`. - **`create_skill` writes a template.** The new `SKILL.md` has the frontmatter and a `TODO` body; the agent fills in the instructions with a file tool. ## Recipes - [Skills Setup](../recipes/skills_setup.md): write a skill, check it with `list_skills`, and attach the server to a toolkit agent. <!-- source: docs/servers/browser.md --> # Browser Server `browser_server` (`doc_processing_std_tools.browser`, MCP name `browser-tools`) lets an agent drive a real Chromium browser: open pages, read them as text, click, type, fill forms, manage tabs and download files. It is modelled on Anthropic's browser use toolset (`browser_toolset_20260801`): each member of that toolset is one tool here, named `browser_<member>`, and the tools work as ordinary function tools with every model provider. *New in 0.6.0.* !!! warning "Off by default, and acting on live websites" The composed launcher adds this server only when `DPTTOOLS_ENABLE_BROWSER=true`. A browser agent can buy, send, delete and sign in on real sites, and it reads untrusted page content. Read [Security](../guides/security.md) before you enable it: with the default settings, **approving consequential actions is your application's job**. ## Install and enable ```bash pip install "doc-processing-std-tools[browser] @ git+https://github.com/rzoeb/doc-processing-std-tools.git@v0.6.1" playwright install chromium # in a container: playwright install --with-deps chromium ``` The `browser` extra installs Playwright and Pillow; Chromium itself is a separate download. Only Chromium is supported. The module imports without the extra — the tools report what is missing when they run. | Route | How | |---|---| | Composed launcher | Set `DPTTOOLS_ENABLE_BROWSER=true`. The browser's settings are validated at start-up only then, so a bad browser setting cannot stop the other servers | | In-process | Pass `browser_server` to `Agent(tools=[...])`, or build one with `make_browser_server(tools="full", enable=["read_console"])` | ## How an agent uses a page The model reads a page as text and acts **by reference**: 1. `browser_read_page` (or `browser_find`) returns the page's accessibility tree, with a `[ref_N]` tag on each element. 2. The action tools take `ref="ref_N"`. A target can also be `x` and `y` in **screenshot pixels**, from `browser_screenshot`. 3. A ref stays valid until its tab navigates or the element is removed. A stale ref returns an error telling the model to read the page again; live refs are never renumbered. Every result is text that ends with a **Tab Context** footer listing the open tabs, plus a line for each tab that opened and each download that started or finished since the last call. Pointer and form tools name the element they acted on ("Clicked button 'Place order'"), which also serves audit logs. Every tool takes an optional `intent` (a note on why, logged only), and every tool except the four tab tools takes an optional `tab_id` (default: the active tab). Reading text is much cheaper than looking at screenshots, so the model should prefer `browser_read_page` and `browser_get_page_text`, and use `browser_screenshot` to check layout or visual state. `DPTTOOLS_BROWSER_AUTO_SCREENSHOT=true` attaches a screenshot to every action that changes the page, for models that work better that way. ## Tools ### The `core` profile (default): 18 tools | Tool | Key parameters | What it does | |---|---|---| | `browser_navigate` | `url` | Open a URL (a scheme-less URL gets `https://`), or `"back"`, `"forward"`, `"reload"`. Only `http` and `https` | | `browser_screenshot` | — | An image of the viewport; the other tools' `x`/`y` are in its pixels | | `browser_zoom` | `x0`, `y0`, `x1`, `y1` | One region of the page, enlarged up to 4×, to read small text | | `browser_left_click` | `ref` or `x`+`y`, `modifiers` | Click; the result names the element clicked | | `browser_hover` | `ref` or `x`+`y`, `modifiers` | Move the mouse over an element or point, for menus and tooltips | | `browser_scroll` | `scroll_direction`, `scroll_amount` (1–10, default 3), optional target | Scroll the page, or the area under a target | | `browser_scroll_to` | `ref` | Scroll an element into view | | `browser_type` | `text` | Type into the focused element | | `browser_key` | `text`, `repeat=1` | Press a key, a chord or a sequence (`Enter`, `ctrl+a`, `ctrl+a Delete`) | | `browser_wait` | `duration` | Wait up to 30 seconds | | `browser_read_page` | `filter="interactive"` (or `"all"`), `depth=15`, `ref` | The accessibility tree with refs; `ref` reads a subtree | | `browser_find` | `query` | Elements matching a short description, with refs. It matches words, not meaning | | `browser_get_page_text` | — | The page's readable text, main content first | | `browser_form_input` | `ref`, `value` | Set a text field, select, checkbox or radio directly | | `browser_new_tab` | `url` | Open a tab, optionally at a URL, and make it active | | `browser_list_tabs` | — | The open tabs with their tab id, title and URL | | `browser_switch_tab` | `tab_id` | Make another tab active | | `browser_close_tab` | `tab_id` | Close a tab | ### Also registered: `fetch_webpage_rendered` Both profiles also register `fetch_webpage_rendered(url, max_length, offset, format, include_assets)`. It returns the same JSON object as [`fetch_webpage`](web.md#fetch_webpage), plus `"rendered": true`, from a page that Chromium loaded with its JavaScript run — for single-page apps and other pages `fetch_webpage` sees as an empty shell (it says so with `likely_js_rendered`). - Each call uses a fresh, credential-free browser context behind the same network policy, closed afterwards. In `persistent` and `cdp` modes it uses a separate headless browser, never the user's. - It is slower than `fetch_webpage` — seconds rather than milliseconds — so try that first. So the browser server registers **19 tools** with the `core` profile and **28** with `full` (18 or 27 `browser_*` tools, plus `fetch_webpage_rendered`), and the composed launcher has 75 or 84 tools with it (92 or 101 with the [computer server](computer.md) enabled as well). ### The `full` profile: 9 more `DPTTOOLS_BROWSER_TOOLS=full` adds the rest of Anthropic's 27 default members: | Tool | Key parameters | What it does | |---|---|---| | `browser_right_click` | `ref` or `x`+`y`, `modifiers` | Right-click (context menu) | | `browser_middle_click` | `ref` or `x`+`y`, `modifiers` | Middle-click (often opens a link in a new tab) | | `browser_double_click` | `ref` or `x`+`y`, `modifiers` | Double-click (for example, select a word) | | `browser_triple_click` | `ref` or `x`+`y`, `modifiers` | Triple-click (for example, select a paragraph) | | `browser_left_click_drag` | `from_x`, `from_y`, then `ref` or `x`+`y` | Drag from a point to an element or point | | `browser_left_mouse_down` | — | Press and hold the left button where the mouse is | | `browser_left_mouse_up` | — | Release the left button | | `browser_mouse_move` | `ref` or `x`+`y` | Move the mouse without clicking | | `browser_hold_key` | `text`, `duration` | Hold a key or chord for up to 30 seconds | Every tool's schema costs tokens on every turn, which is why the smaller profile is the default. ### Opt-in tools Off by default because of what they can do; enable each by name in `DPTTOOLS_BROWSER_ENABLE` (comma-separated), or with `make_browser_server(enable=[...])`. | Tool | Key parameters | What it does | |---|---|---| | `browser_javascript_exec` | `expression` | Run a JavaScript expression in the page and return its value (a promise is awaited) | | `browser_read_console` | — | Console messages since the last read, per tab | | `browser_read_network` | — | Network requests since the last read, per tab, with secret-looking URL parameters redacted | | `browser_file_upload` | `ref`, `paths` | Attach files to a file input — only files inside `DPTTOOLS_BROWSER_UPLOAD_DIR`, which must differ from the download directory | | `browser_run_script` | `code` | Run a Python script with Playwright against the tab; returns printed output and the final expression's value | **`browser_run_script` runs out of process.** Each session's scripts run in one long-lived child process, so variables persist between calls, like a REPL. The script gets `page`, `context`, `await screenshot()` and `await read_page()`, using Playwright's async API. - **It is not a sandbox.** The script is model-written Python running as the server's user. It can open network connections directly, bypassing the egress proxy, and read the process environment through `/proc`, although the child starts without the variables whose names look like secrets (API keys, tokens, passwords). Enable it only where `execute_command` would also be acceptable, and rely on the container's network policy. - In `launch` and `persistent` modes it attaches to the session's browser over a DevTools port on `127.0.0.1`, which std-tools opens only while this tool is enabled. In `cdp` mode it attaches to `DPTTOOLS_BROWSER_CDP_URL`, the endpoint you provide. - **A script can reach the whole browser**, not only its own tab: it is arbitrary Python running as the server's user. So while `run_script` is enabled, only one browser session is open at a time (a second client is refused until the first has been idle for `DPTTOOLS_BROWSER_IDLE_TIMEOUT_S`), in `launch` mode each new session gets a freshly started browser, and `cdp` mode requires `DPTTOOLS_BROWSER_CDP_SHARE_TABS=true`. To serve several clients with scripts enabled, run one std-tools server, in its own container, per client. - In `launch` and `persistent` modes the browser's own traffic still goes through the egress proxy, which the child cannot reconfigure. In `cdp` mode there is no proxy: std-tools checks the requests of the tabs it manages, and a script can open pages outside them (in a new context, say). Connections the script opens itself are never checked. - A script that runs longer than `DPTTOOLS_BROWSER_SCRIPT_TIMEOUT_S` (60) is stopped with its whole process tree, and its variables are lost. - Any local process can reach std-tools' DevTools port while it is open. That is acceptable inside a sandbox container; on a shared machine, leave the tool off. ## Downloads A download is saved to the session's folder under `DPTTOOLS_BROWSER_DOWNLOAD_DIR` (default: a new temporary folder per session) and reported once, on the next result: ```text Download started: statement.pdf from https://example.com/... Download completed: statement.pdf saved to /tmp/.../statement.pdf (48213 bytes) ``` or `Download failed: …`. The path opens directly with `read_pdf`, `read_document_pages_as_images`, OCR and the other [Documents](documents.md) tools. Downloaded files are kept when the session closes; manage the folder if you set it. Downloads have no size or count limit of their own: Chromium writes the whole file before std-tools sees it. The container's disk quota is the control, so give the download folder one. ## Browser modes `DPTTOOLS_BROWSER_MODE` chooses whose browser the agent uses: | Mode | What std-tools does | Typical use | |---|---|---| | `launch` (default) | Starts its own headless Chromium, with a fresh, credential-free context for each MCP session | A sandbox container | | `persistent` | Starts its own Chromium on a saved profile (`DPTTOOLS_BROWSER_PROFILE_DIR`), usually headed (`DPTTOOLS_BROWSER_HEADLESS=false`). One session at a time | A person signs in once and watches | | `cdp` | Attaches to a browser someone else started, at `DPTTOOLS_BROWSER_CDP_URL` | Driving the user's own browser | For `cdp`: - Chrome 136 and later refuse remote debugging on the default profile. Start Chrome with `--remote-debugging-port=9222 --user-data-dir=<a dedicated folder>` and sign in there. - Bind the port to `127.0.0.1` only: whoever reaches it controls the browser. - The agent sees and uses only the tabs it opened, unless `DPTTOOLS_BROWSER_CDP_SHARE_TABS=true`. - The egress proxy cannot apply to a browser std-tools did not launch; requests are checked one by one instead, without IP pinning. See [Security](../guides/security.md#browser-network-policy). [Isolated vs Attached](../guides/isolated_vs_attached.md) compares the modes in more depth. ## Sessions - Each MCP session gets its own browser context, so cookies and storage are not shared between sessions. In-process calls share one default session. - A session unused for `DPTTOOLS_BROWSER_IDLE_TIMEOUT_S` (600) seconds is closed with its tabs; at most `DPTTOOLS_BROWSER_MAX_SESSIONS` (4) are open at once, and only one while `run_script` is enabled. - Playwright runs on its own thread with a long-lived event loop, so the browser survives callers that create a new event loop per call, such as the toolkit's `run_sync`. - The viewport is `DPTTOOLS_BROWSER_VIEWPORT` (`1440x900`). Screenshots pass through the [image limits](../guides/images.md); at the default viewport nothing is scaled. ## Environment variables | Variable | Default | Effect | |---|---|---| | `DPTTOOLS_ENABLE_BROWSER` | `false` | Adds the browser server to the composed launcher | | `DPTTOOLS_BROWSER_TOOLS` | `core` | `core` (18 `browser_*` tools) or `full` (27), plus `fetch_webpage_rendered` | | `DPTTOOLS_BROWSER_ENABLE` | empty | Opt-in tools: `javascript_exec`, `file_upload`, `read_console`, `read_network`, `run_script` | | `DPTTOOLS_BROWSER_MODE` | `launch` | `launch`, `persistent` or `cdp` | | `DPTTOOLS_BROWSER_HEADLESS` | `true` | `false` shows a window (needs a display) | | `DPTTOOLS_BROWSER_PROFILE_DIR` | — | The saved profile; required for `persistent` | | `DPTTOOLS_BROWSER_CDP_URL` | — | The DevTools endpoint; required for `cdp` | | `DPTTOOLS_BROWSER_CDP_SHARE_TABS` | `false` | In `cdp` mode, let the agent use tabs it did not open | | `DPTTOOLS_BROWSER_MAX_SESSIONS` | `4` | Browser sessions open at once (always 1 while `run_script` is enabled) | | `DPTTOOLS_BROWSER_IDLE_TIMEOUT_S` | `600` | Idle seconds before a session is closed | | `DPTTOOLS_BROWSER_VIEWPORT` | `1440x900` | Viewport size | | `DPTTOOLS_BROWSER_AUTO_SCREENSHOT` | `false` | Attach a screenshot after every page-changing action | | `DPTTOOLS_BROWSER_DOWNLOAD_DIR` | a temporary folder per session | Where downloads are saved | | `DPTTOOLS_BROWSER_UPLOAD_DIR` | — | The only folder `browser_file_upload` may read; required to enable it | | `DPTTOOLS_BROWSER_ALLOWED_DOMAINS` | empty (any public host) | Only these domains and their subdomains | | `DPTTOOLS_BROWSER_BLOCKED_DOMAINS` | empty | Never these domains; wins over the allowed list | | `DPTTOOLS_BROWSER_ALLOW_HOSTS` | empty | Exact hostnames allowed past the private-address block | | `DPTTOOLS_BROWSER_CONFIRM` | `off` | The confirmation gate: `off`, `deny` or `elicit` ([Security](../guides/security.md#the-confirmation-gate-at-a-glance)) | | `DPTTOOLS_BROWSER_SCRIPT_TIMEOUT_S` | `60` | Timeout for one `browser_run_script` call | The [Configuration](../guides/configuration.md#browser) page gives accepted values and when each is read. ## Examples A typical sequence, written as Python calls (the model sends the same arguments as JSON): ```python browser_navigate(url="https://example.com/statements") browser_read_page(filter="interactive") # the tree, with a [ref_N] tag on each element browser_form_input(ref="ref_3", value="2026-09") # the "Month" field browser_left_click(ref="ref_7", intent="download the statement") # the "Download" button browser_get_page_text() ``` Adding the safety instructions to an agent's system prompt: ```python from doc_processing_toolkit import Agent from doc_processing_std_tools.browser import BROWSER_SAFETY_PROMPT, browser_server agent = Agent( model="claude-sonnet-5", instructions="You collect monthly statements from supplier portals.\n\n" + BROWSER_SAFETY_PROMPT, tools=[browser_server], ) ``` `BROWSER_SAFETY_PROMPT` tells the model that page content is untrusted data, to ask before consequential actions, and never to bypass access controls. std-tools never injects it; your application decides. ## Pitfalls - **`browser_find` is lexical.** It matches words in element names, labels and nearby text, not meaning. When it misses, use `browser_read_page(filter="interactive")`. - **Refs go stale** after navigation or a page update. Read the page again rather than reusing old refs. - **Only `http` and `https`.** `file:`, `javascript:`, `data:` and `chrome:` URLs are refused. - **Nothing asks for confirmation by default.** With `DPTTOOLS_BROWSER_CONFIRM=off`, approve consequential actions with toolkit hooks, or set the gate to `deny`. `elicit` needs an MCP client that supports elicitation — the toolkit has none today — and otherwise refuses every flagged action. - **`persistent` mode allows one session.** A profile folder cannot be shared between browsers. - **Headed mode needs a display.** `DPTTOOLS_BROWSER_HEADLESS=false` in a container without one fails to start the browser. - **The `full` profile costs tokens** on every turn. Use it when the model needs the extra pointer tools. - **A missing Chromium is reported when a tool runs**, not at import: run `playwright install chromium` (with `--with-deps` in a container). - **Placeholder arguments are tolerated.** Strict function calling (OpenAI) fills unused optional arguments with placeholders instead of leaving them out, so the tools read them as "not given": `tab_id` 0 means the active tab, and `browser_navigate` opens a tab when none is open whatever `tab_id` says; `ref=""` means no ref; `x=0, y=0` next to a ref is ignored; and `browser_scroll` at (0, 0) scrolls at the centre of the page. ## Recipes - [Portal Download to Text](../recipes/portal_download_to_text.md): sign in to a portal, download a statement and read its text. - [JavaScript-Rendered Pages](../recipes/js_rendered_page.md): use `fetch_webpage_rendered` without the rest of the browser tools. <!-- source: docs/servers/computer.md --> # Computer (Desktop) Server `computer_server` (`doc_processing_std_tools.computer`, MCP name `computer-tools`) lets an agent operate a whole desktop by screenshot, mouse and keyboard. It is modelled on Anthropic's computer use toolset (`computer_toolset_20260801`): each of its 17 members is one tool here, named `computer_<member>`, and the tools work as ordinary function tools with every model provider. *New in 0.6.0.* Use it for native applications, operating-system dialogs and flows that cross applications. For anything inside web pages, the [Browser](browser.md) server is the better tool, because it reads the page itself instead of looking at pixels. !!! warning "Off by default — and `host` mode is the user's real computer" The composed launcher adds this server only when `DPTTOOLS_ENABLE_COMPUTER=true`. The default `virtual` backend gives the agent its own desktop inside a Linux container. The `host` backend operates **the real screen, mouse and keyboard** of the machine std-tools runs on: run it only with the user's knowledge, because every screenshot, with whatever is on the screen, goes to the model provider. Read [Security](../guides/security.md#the-desktop-server) before enabling it. ## Install and enable ```bash pip install "doc-processing-std-tools[computer] @ git+https://github.com/rzoeb/doc-processing-std-tools.git@v0.6.1" # virtual backend (Linux): apt-get install xvfb xdotool openbox (x11vnc optional) ``` The `computer` extra installs `mss`, Pillow and, except on Linux, `pynput`. On Linux the extra leaves `pynput` out, because its `evdev` dependency compiles from source and would break `[all]` on slim images. Linux **host** mode installs it separately: ```bash pip install python-xlib && pip install --no-deps pynput ``` | Route | How | |---|---| | Composed launcher | Set `DPTTOOLS_ENABLE_COMPUTER=true` | | In-process | Pass `computer_server` to `Agent(tools=[...])` | Optional packages are imported only when a backend starts, so the module imports without the extra. ## How an agent uses the desktop - **Look, then act.** `computer_screenshot` returns an image of the desktop. The other tools take `x` and `y` in **the pixels of the latest screenshot**. - **Large screens are scaled.** A screen larger than `DPTTOOLS_IMAGE_MAX_EDGE_PX` (1568 px) is scaled down for the model, and the coordinates the model sends are mapped back to real pixels ([Images](../guides/images.md)). The default virtual display, 1440 × 900, is not scaled. - **Every result names the display or monitor** it acted on. Every tool takes an optional `intent` (logged only). - **Starting applications** on a virtual desktop: use [`execute_command`](code_execution.md) with the display the results name, for example `DISPLAY=:99 libreoffice --calc report.xlsx &`. ## Tools | Tool | Key parameters | What it does | |---|---|---| | `computer_screenshot` | — | A screenshot of the desktop (an image) | | `computer_zoom` | `x0`, `y0`, `x1`, `y1` | A region of the screen, enlarged up to 4×, to read small text | | `computer_left_click` | `x`, `y`, `modifiers` | Click, optionally holding keys (`ctrl`, `shift+ctrl`) | | `computer_right_click` | `x`, `y`, `modifiers` | Right-click | | `computer_middle_click` | `x`, `y`, `modifiers` | Middle-click | | `computer_double_click` | `x`, `y`, `modifiers` | Double-click | | `computer_triple_click` | `x`, `y`, `modifiers` | Triple-click | | `computer_left_click_drag` | `from_x`, `from_y`, `x`, `y` | Drag from one point to another | | `computer_mouse_move` | `x`, `y` | Move the mouse without clicking | | `computer_left_mouse_down` | — | Press and hold the left button where the mouse is | | `computer_left_mouse_up` | — | Release the left button | | `computer_cursor_position` | — | Where the cursor is, in screenshot pixels | | `computer_scroll` | `scroll_direction`, `scroll_amount` (1–10, default 3), optional `x`, `y` | Scroll at a point, or at the screen centre | | `computer_type` | `text` | Type text into whatever has focus. Tabs and newlines are typed; other control characters are refused (press keys with `computer_key`), *new in 0.6.1* | | `computer_key` | `text`, `repeat=1` | Press a key, a chord or a sequence, with xdotool key names: `Return`, `ctrl+a`, `alt+Tab`, `ctrl+a BackSpace` | | `computer_hold_key` | `text`, `duration` | Hold a key or chord for up to 30 seconds | | `computer_wait` | `duration` | Wait up to 30 seconds | With the computer server enabled, the composed launcher has **73 tools** (56 + 17), or **92** with the browser server's `core` profile as well. ## Backends `DPTTOOLS_COMPUTER_BACKEND` chooses whose desktop the agent uses. ### `virtual` (default): the agent's own desktop - Each MCP session gets its **own Xvfb display** (`DPTTOOLS_COMPUTER_DISPLAY_SIZE`, default `1440x900`) with a light window manager. Input goes through `xdotool` on that display only, so sessions never see each other's desktops. - **Linux only**, so in practice a container — see the [Deployment](../guides/deployment.md#with-the-browser-and-a-virtual-desktop) pattern. - At most `DPTTOOLS_COMPUTER_MAX_SESSIONS` (2) desktops are open at once; one unused for 10 minutes is closed. - `DPTTOOLS_COMPUTER_VNC_PORT` serves each display over VNC on `127.0.0.1` (the first session on that port, the next on port + 1; needs `x11vnc`), so a person can watch or take over. - **The desktop sees the server's environment.** Xvfb, the window manager, `x11vnc` and `xdotool` start with the server's environment variables, API keys included, and so does any program started from the desktop (the window manager's menu, for example). Give the computer server only the keys it needs: run it in a launcher process of its own, with no API keys in its environment. ### `host`: the real screen - Operates the real screen, mouse and keyboard of the machine std-tools runs on, through `pynput` and `mss`. - **Platforms:** Windows and Linux X11. macOS is untested (it needs Accessibility and Screen Recording permission). Wayland is not supported. - **One session at a time** — there is only one real cursor. A second client is refused until the first has been idle for a minute. - `DPTTOOLS_COMPUTER_MONITOR` picks the monitor (default: the primary one); coordinates are relative to it, never to the combined virtual screen. - On Windows the server declares per-monitor DPI awareness, so screenshots and clicks share physical pixels. ## Host-mode safeguards These are **always on in host mode**, whatever the confirmation gate says: - **Kill switch.** Moving the mouse into the **top-left corner** of the screen, or pressing `DPTTOOLS_COMPUTER_KILL_HOTKEY` (default `ctrl+alt+shift+k`), stops the agent. Every later call returns an error until the server restarts. - **Pause on human activity.** If the user moved the mouse or typed in the last second, the next action is refused ("the user is using the computer right now"). On Windows, input that arrives *during* an action also stops that action ("the user took over"). - **On Linux X11** the system does not mark the agent's own input reliably, so std-tools ignores input during an action and for 0.4 s after it. There the pause, and the top-left corner, work **between actions only**. The kill hotkey works at any time, on both platforms. - **Every screenshot goes to the model provider**, with whatever is on the screen — other windows, notifications, messages. Close what the agent should not see. ## The confirmation gate `DPTTOOLS_COMPUTER_CONFIRM` takes the same three values as the browser's gate: | Value | Effect | |---|---| | `off` (default) | std-tools asks nothing; the client application approves actions through its own hooks | | `deny` | Key chords that close, quit or delete are refused: `alt+f4`, `ctrl+w`, `ctrl+q`, `super+w`, `super+q`, `delete`, `shift+delete`. A chord that contains one of them counts too (`ctrl+shift+w`, `ctrl+delete`, `Alt_L+F4`, `alt+0xffc1`), and so do a click's `modifiers` (`modifiers="alt+F4"`), *new in 0.6.1* | | `elicit` | std-tools asks a human through MCP elicitation before the first action of each host session ("Allow the agent to control this computer for this session?") and before each of those chords; it refuses if nobody can answer. The toolkit has no elicitation support today | The gate knows only these chords: an application's own shortcuts, keys such as `XF86Close`, and typed text, such as a command in a terminal, pass it. Nor can it tell what a click will do — pixels carry no element names — so **approval in your application matters more here than for the browser**. See [Using with the Toolkit](../guides/with_the_toolkit.md#desktop-actions). ## Environment variables | Variable | Default | Effect | |---|---|---| | `DPTTOOLS_ENABLE_COMPUTER` | `false` | Adds the computer server to the composed launcher | | `DPTTOOLS_COMPUTER_BACKEND` | `virtual` | `virtual` or `host` | | `DPTTOOLS_COMPUTER_DISPLAY_SIZE` | `1440x900` | Size of each virtual display (`virtual` only) | | `DPTTOOLS_COMPUTER_MAX_SESSIONS` | `2` | Virtual desktops open at once (`host` always allows one) | | `DPTTOOLS_COMPUTER_VNC_PORT` | `off` | VNC on `127.0.0.1` for each virtual display (needs `x11vnc`) | | `DPTTOOLS_COMPUTER_MONITOR` | `primary` | The monitor `host` uses: `primary` or a number | | `DPTTOOLS_COMPUTER_KILL_HOTKEY` | `ctrl+alt+shift+k` | The `host` kill switch chord | | `DPTTOOLS_COMPUTER_CONFIRM` | `off` | The confirmation gate: `off`, `deny` or `elicit` | The [Configuration](../guides/configuration.md#computer) page gives accepted values and when each is read. ## Examples A typical sequence on a virtual desktop, written as Python calls (the model sends the same arguments as JSON): ```python execute_command(command="DISPLAY=:99 libreoffice --calc /workspace/report.xlsx &") computer_wait(duration=3) computer_screenshot() # look before acting computer_left_click(x=412, y=230, intent="select cell B4") computer_type(text="2026-09") computer_key(text="Return") computer_screenshot() # check the result ``` ## Pitfalls - **Coordinates come from the latest screenshot.** After the screen changes, take a new screenshot before clicking again. - **Screenshots are the expensive part.** Each one costs image tokens; zoom into a region with `computer_zoom` to read small text instead of raising the image limits. - **`virtual` needs Linux** with `xvfb`, `xdotool` and a window manager installed. On Windows or macOS, only `host` is available. - **Linux host mode needs `pynput` installed by hand** (see above); without it the backend cannot start. - **Key names are xdotool's**, such as `Return`, `BackSpace`, `Page_Down` and `super`. Each key in `computer_key`, `computer_hold_key` and the click tools' `modifiers` must be a single character or a key name (letters, digits and `_`, such as `XF86AudioMute`); anything else is refused with an `Error:` naming it (*new in 0.6.1*). A name that starts with a digit must be a hex keysym (`0x1008ff13`) or a `3270_*` name, never a key code such as `119`. Names other than the common ones (`ctrl`, `Return`, `Tab`, `F1`–`F12` …) are passed on as written, and X key names are case-sensitive: `ISO_Left_Tab`, not `iso_left_tab`. - **In host mode the agent can see everything on the screen.** Close windows and notifications the agent should not read, and tell the user that their screen goes to the model provider. - **A killed host session stays killed** until the server restarts. - **`computer_scroll` at (0, 0) scrolls at the centre of the screen**, because strict function calling (OpenAI) sends 0 as a placeholder for unused optional arguments. To scroll at the very top-left, move a pixel in: (1, 1). ## Recipes - [Desktop Actions with Hooks](../recipes/desktop_with_hooks.md): drive a desktop application while a person approves each action from your application. <!-- source: docs/guides/configuration.md --> # Configuration Everything in std-tools is configured through environment variables, each with a default. This page lists every variable the code reads. The tables are generated from one table in the code (`ENV_VARS` in `_config.py`) by `scripts/gen_reference.py`, which also writes the README's configuration section and `.env.example`, so the three always agree. ## How configuration is loaded - **std-tools reads the process environment only.** It never loads a `.env` file itself. To use one, load it in whatever starts std-tools: `docker run --env-file .env`, `env_file:` in Docker Compose, or `load_dotenv()` in your application before std-tools is first used. The repository's `.env.example` lists every variable, commented out, ready to copy. - **A stdio server sees only the environment you pass it.** The MCP SDK starts a stdio server with a short allow-list of variables (`PATH`, `HOME` or `USERPROFILE`, `TEMP` and a few others), so pass the variables below in the client's `env`. See [Using with the Toolkit](with_the_toolkit.md#passing-configuration-to-a-stdio-server). - **Values are read once per process**, then cached, unless the table says otherwise. Restart the server after changing one. - **Invalid numbers are errors, not silent defaults.** A numeric variable that is not a positive integer raises an error naming the variable. The composed launcher (`python -m doc_processing_std_tools`) checks every variable at start-up and refuses to start; other entry points fail on the first tool call that reads it. - **Relative directories** (`DPTTOOLS_MEMORY_DIR`, `DPTTOOLS_SKILLS_DIR`) are resolved against the process's working directory when configuration is first read. The variables, by area: <!-- BEGIN GENERATED: configuration (scripts/gen_reference.py; do not edit by hand) --> ## General | Variable | Default | Accepted values | When it is read | Description | |---|---|---|---|---| | `DPTTOOLS_MAX_FILE_SIZE_MB` | `50` | Positive integer (MB) | First use; cached | Largest file the read tools will open. | | `DPTTOOLS_OUTPUT_TRUNCATION_LIMIT` | `50000` | Positive integer (characters) | First use; cached | Longest text a tool returns to the model; longer output is truncated with a notice. | | `DPTTOOLS_MEMORY_DIR` | `./agent_memory` | Directory path | First use; cached | Where the memory tools keep their files. A relative path is resolved against the working directory. | | `DPTTOOLS_SKILLS_DIR` | `./.agent_skills` | Directory path | First use; cached | Where the skills tools look for skills (one subdirectory with a `SKILL.md` per skill). | ## Code execution | Variable | Default | Accepted values | When it is read | Description | |---|---|---|---|---| | `DPTTOOLS_SANDBOX_TIMEOUT` | `30` | Positive integer (seconds) | First use; cached | Default and maximum timeout for `execute_command`, and the timeout for `execute_skill_script`. | ## Documents and OCR | Variable | Default | Accepted values | When it is read | Description | |---|---|---|---|---| | `DPTTOOLS_OCR_TIMEOUT` | `180` | Integer seconds; clamped to 30–600 | First use; cached | Wall-clock deadline for one OCR job, upload and polling included. A value outside 30–600 is clamped; one that is not a positive integer is an error. | | `LLMWHISPERER_API_KEY` | *(none)* | LLMWhisperer API key | Every OCR call | Required by `ocr_pdf` and `ocr_image` (install the `[ocr]` extra). | | `DPTTOOLS_CONVERT_TIMEOUT_S` | `120` | Positive integer (seconds) | First use; cached | Timeout for one LibreOffice conversion: `convert_document`, and the Office-to-PDF step of `render_document_page`, `render_document_thumbnail` and `read_document_pages_as_images`. | | `DPTTOOLS_SOFFICE_BIN` | *(none)* | Path to `soffice`, or a command name on `PATH` | First use; cached | LibreOffice executable to run. When unset, std-tools looks for `libreoffice`, then `soffice`, on `PATH`. A value that does not resolve to an executable is an error naming the variable. | ## Web | Variable | Default | Accepted values | When it is read | Description | |---|---|---|---|---| | `DPTTOOLS_WEB_SEARCH_PROVIDER` | *(none)* | `tavily`, `serper` or `brave` | First use; cached | Search provider for `web_search`. Without it, `web_search` returns a configuration error. | | `DPTTOOLS_WEB_SEARCH_API_KEY` | *(none)* | The provider's API key | First use; cached | API key for the search provider. Tavily receives it as a Bearer token. | | `DPTTOOLS_WEB_FETCH_TIMEOUT` | `30` | Positive integer (seconds) | First use; cached | Timeout for `fetch_webpage`, independent of `DPTTOOLS_SANDBOX_TIMEOUT`. The connect phase is capped at 10 seconds. | | `DPTTOOLS_WEB_FETCH_MAX_BYTES` | `2000000` | Positive integer (bytes) | First use; cached | Most bytes `fetch_webpage` downloads from one response. A longer page is cut at this point and the result says `byte_cap_reached`. | | `DPTTOOLS_WEB_FETCH_MAX_REDIRECTS` | `5` | Positive integer | First use; cached | Most redirects `fetch_webpage` follows. Every hop is checked against the private-address block. | | `DPTTOOLS_WEB_FETCH_ALLOW_HOSTS` | *(none)* | Comma-separated exact hostnames | First use; cached | Hosts the fetch tools may reach even though they resolve to a private or loopback address (for example an internal wiki). Exact names only, no wildcards. Empty by default: every private address is refused. | | `DPTTOOLS_FETCH_IMAGE_MAX_BYTES` | `10000000` | Positive integer (bytes) | First use; cached | Largest image `fetch_image` downloads. A larger image is refused (its declared size is checked before download); one within the limit is then scaled to the image limits below. | ## Images | Variable | Default | Accepted values | When it is read | Description | |---|---|---|---|---| | `DPTTOOLS_IMAGE_MAX_EDGE_PX` | `1568` | Positive integer (pixels) | First use; cached | Long-edge limit for every image returned to the model; larger images are scaled down. Files written to an `output_path` keep full resolution. | | `DPTTOOLS_IMAGE_MAX_MB` | `3` | Positive integer (MB) | First use; cached | Encoded-size limit for every image returned to the model. | ## Browser | Variable | Default | Accepted values | When it is read | Description | |---|---|---|---|---| | `DPTTOOLS_ENABLE_BROWSER` | `false` | `true` or `false` | Launcher start-up | Adds the browser server to the composed launcher (`python -m doc_processing_std_tools`). In-process users pass `browser_server` to `Agent(tools=[...])` instead. | | `DPTTOOLS_BROWSER_MODE` | `launch` | `launch`, `persistent` or `cdp` | First use; cached | `launch`: std-tools starts its own Chromium with a fresh profile per session. `persistent`: its own Chromium on a saved profile (one session at a time). `cdp`: attach to a browser someone else started (the user's own browser). | | `DPTTOOLS_BROWSER_HEADLESS` | `true` | `true` or `false` | First use; cached | Run Chromium without a window. Set `false` to watch the agent (needs a display). | | `DPTTOOLS_BROWSER_PROFILE_DIR` | *(none)* | Directory path | First use; cached | Saved browser profile for `persistent` mode (required there), so a person can sign in once. | | `DPTTOOLS_BROWSER_CDP_URL` | *(none)* | A DevTools endpoint URL | First use; cached | Browser to attach to in `cdp` mode (required there), for example `http://127.0.0.1:9222`. Chrome 136 and later need a dedicated `--user-data-dir` for remote debugging. | | `DPTTOOLS_BROWSER_CDP_SHARE_TABS` | `false` | `true` or `false` | First use; cached | In `cdp` mode, let the agent see and use tabs it did not open. By default it works only in its own tabs. Required to enable `run_script` in `cdp` mode, because a script can reach every tab. | | `DPTTOOLS_BROWSER_TOOLS` | `core` | `core` or `full` | Server creation | `core`: 18 `browser_*` tools. `full`: all 27 default members of Anthropic's browser toolset (adds right/middle/double/triple click, drag, mouse down/up/move and hold_key). Both also register `fetch_webpage_rendered`. | | `DPTTOOLS_BROWSER_ENABLE` | *(none)* | Comma-separated: `javascript_exec`, `file_upload`, `read_console`, `read_network`, `run_script` | Server creation | Opt-in tools, off by default because of what they can do. A `run_script` script can reach the whole browser, so while it is enabled only one browser session is open at a time, and `cdp` mode also needs `DPTTOOLS_BROWSER_CDP_SHARE_TABS=true`. | | `DPTTOOLS_BROWSER_MAX_SESSIONS` | `4` | Positive integer | First use; cached | Most browser sessions (one per MCP session) open at once. Always 1 while `run_script` is enabled. | | `DPTTOOLS_BROWSER_IDLE_TIMEOUT_S` | `600` | Positive integer (seconds) | First use; cached | A session unused for this long is closed, with its tabs. | | `DPTTOOLS_BROWSER_VIEWPORT` | `1440x900` | `WIDTHxHEIGHT` | First use; cached | Page viewport size in pixels. Screenshots are this size unless the image limits scale them. | | `DPTTOOLS_BROWSER_AUTO_SCREENSHOT` | `false` | `true` or `false` | First use; cached | Attach a screenshot to the result of every action that changes the page. Off by default: reading the page as text is cheaper. | | `DPTTOOLS_BROWSER_DOWNLOAD_DIR` | *(none)* | Directory path | First use; cached | Where downloads are saved, one subdirectory per session. Default: a new temporary directory per session. | | `DPTTOOLS_BROWSER_UPLOAD_DIR` | *(none)* | Directory path | First use; cached | The only directory `browser_file_upload` may read from (required to enable it). Must differ from the download directory. | | `DPTTOOLS_BROWSER_ALLOWED_DOMAINS` | *(none)* | Comma-separated domains | First use; cached | If set, the browser may only load these domains and their subdomains. Empty: any public host. Private and reserved addresses stay blocked either way. | | `DPTTOOLS_BROWSER_BLOCKED_DOMAINS` | *(none)* | Comma-separated domains | First use; cached | Domains (and their subdomains) the browser may never load. Wins over the allowed list. | | `DPTTOOLS_BROWSER_ALLOW_HOSTS` | *(none)* | Comma-separated exact hostnames | First use; cached | Hosts the browser may reach even though they resolve to a private or loopback address. Exact names only. | | `DPTTOOLS_BROWSER_CONFIRM` | `off` | `off`, `deny` or `elicit` | First use; cached | Confirmation gate for consequential actions (purchases, deletions, sending, passwords, uploads). `off`: the client app approves actions through its own hooks. `deny`: refuse them. `elicit`: ask a human through MCP elicitation; refuse if nobody can answer. | | `DPTTOOLS_BROWSER_SCRIPT_TIMEOUT_S` | `60` | Positive integer (seconds) | First use; cached | Timeout for one `browser_run_script` call. | ## Computer | Variable | Default | Accepted values | When it is read | Description | |---|---|---|---|---| | `DPTTOOLS_ENABLE_COMPUTER` | `false` | `true` or `false` | Launcher start-up | Adds the desktop computer server to the composed launcher. In-process users pass `computer_server` to `Agent(tools=[...])` instead. | | `DPTTOOLS_COMPUTER_BACKEND` | `virtual` | `virtual` or `host` | First use; cached | `virtual`: the agent's own Xvfb display per session (Linux containers). `host`: the real screen, mouse and keyboard of the machine std-tools runs on, with a kill switch; only with the user's knowledge. | | `DPTTOOLS_COMPUTER_DISPLAY_SIZE` | `1440x900` | `WIDTHxHEIGHT` | First use; cached | Size of each virtual display (`virtual` only). | | `DPTTOOLS_COMPUTER_MAX_SESSIONS` | `2` | Positive integer | First use; cached | Most virtual desktops open at once (`virtual` only; `host` always allows one session). | | `DPTTOOLS_COMPUTER_VNC_PORT` | `off` | Port number, or `off` | First use; cached | Serve each virtual display over VNC on `127.0.0.1` (first session on this port, the next on port + 1) so a person can watch or take over. Needs `x11vnc` (`virtual` only). | | `DPTTOOLS_COMPUTER_MONITOR` | `primary` | `primary` or a monitor number (1, 2, ...) | First use; cached | Which monitor the `host` backend uses. `primary` is the one the system marks primary (on Linux, the XRandR primary output). Coordinates are relative to it. | | `DPTTOOLS_COMPUTER_KILL_HOTKEY` | `ctrl+alt+shift+k` | A key chord | First use; cached | In `host` mode, pressing this (or moving the mouse into the top-left corner) stops the agent until the server restarts. | | `DPTTOOLS_COMPUTER_CONFIRM` | `off` | `off`, `deny` or `elicit` | First use; cached | Confirmation gate. `elicit`: ask before the first action of each host session and before key chords that close, quit or delete (`alt+f4`, `ctrl+w`, `ctrl+q`, `super+w`, `super+q`, `delete`, `shift+delete`), also with extra keys such as `ctrl+shift+w` and as click modifiers. `deny`: refuse those chords. `off`: the client app approves actions through its own hooks. | <!-- END GENERATED: configuration --> <!-- source: docs/guides/security.md --> # Security std-tools gives a model real capabilities: it reads and writes files, runs commands, fetches web pages, can drive a browser, can operate a desktop and can capture a screen. This page sets out what std-tools enforces, and what the client application that deploys it must do. The mechanics are described in [Architecture](../architecture.md#security-posture). ## The model: the container is the boundary std-tools is built to run **inside an isolated container that the client application controls**. The container's mounts, network settings, user and resource limits decide what the agent can reach. std-tools adds defence in depth inside that boundary, but does not try to be the boundary itself: - File paths are not confined to a folder. The file, document and image tools accept `..` and absolute paths, and can read anything the server process can. - `execute_command` runs whatever the model sends, with the server process's permissions. ## What std-tools enforces | Protection | Where | |---|---| | A size limit before a file is read (`DPTTOOLS_MAX_FILE_SIZE_MB`) | File, document, image and skill-resource reads | | Output truncation (`DPTTOOLS_OUTPUT_TRUNCATION_LIMIT`) | File and document text, command and script output, and skill content. Directory listings, memory views and memory search results are not cut | | Timeouts | `execute_command` (kills the whole process tree), `execute_skill_script`, LibreOffice conversion, OCR, web fetches | | SSRF protection: every address that is not public refused (private, loopback, link-local, carrier-grade NAT, and IPv4 inside IPv6 forms), every redirect re-checked, the connection pinned to the validated address | `fetch_webpage`, `fetch_image` | | Text-only page reads (binary responses refused before download) and size caps on page and image downloads | `fetch_webpage`, `fetch_image` | | Path containment | The memory tools (inside `DPTTOOLS_MEMORY_DIR`) and the skills tools (inside `DPTTOOLS_SKILLS_DIR`) | | Refusal to delete the whole memory store | `memory_delete` | | A pixel cap against decompression bombs | Image Tools | | Errors without request URLs or API keys | `web_search`, OCR | | No `eval` or `exec` of model input in the server process (`browser_run_script` runs model code in a separate child process); commands run as subprocesses | Code execution, skill scripts, `browser_run_script` | | The HTTP transport binds to `127.0.0.1` by default | Composed launcher | ## What the client application must do | Responsibility | Why | |---|---| | **Run std-tools in an isolated container**, as a non-root user, with only the mounts the agent needs (read-only where it should not write) | The container is the only real limit on files and commands | | **Keep secrets out of reach.** Anything in the server process's environment — including `DPTTOOLS_WEB_SEARCH_API_KEY` and `LLMWHISPERER_API_KEY` — is visible to commands the agent runs, because they inherit that environment. Do not mount credentials the agent should not read | `execute_command` and skill scripts can print the environment and read any readable file | | **Restrict the network** (`--network=none`, or an egress policy) when the agent needs no internet. The Web server and OCR need outbound access | The fetch tools block private addresses, but commands the agent runs do not | | **Keep `DPTTOOLS_WEB_FETCH_ALLOW_HOSTS` short.** Each host on it is reachable by the fetch tools, and the model chooses the URLs | It is the one way past the private-address block | | **Put an authenticating proxy in front of the HTTP transport**, or keep it on `127.0.0.1` | The composed server does not authenticate callers | | **Approve consequential tool calls** — deleting files, running commands, writing outside a work folder — with toolkit hooks or your own policy | The general-purpose tools never ask for confirmation; see [Using with the Toolkit](with_the_toolkit.md#approving-tool-calls-with-hooks) | | **Treat tool output as untrusted.** Web pages, documents, images and OCR text can contain instructions aimed at the model | std-tools returns content as it finds it | | **Attach `take_screenshot` only where capturing that screen is intended** | It captures the real screen of the machine the server runs on, and sends it to the model provider | | **Run the computer server's `host` backend only with the user's knowledge** | It operates their real screen, mouse and keyboard, and every screenshot goes to the model provider; see [the desktop server](#the-desktop-server) | | **Pin a release and restart after upgrades** | Configuration and code are loaded once per process | ## The browser and desktop servers The [browser server](../servers/browser.md) acts on live websites and the [computer server](../servers/computer.md) on a desktop, so both have their own safeguards on top of the container boundary. Both are off unless you enable them: `DPTTOOLS_ENABLE_BROWSER=true` and `DPTTOOLS_ENABLE_COMPUTER=true` for the launcher, or by passing `browser_server` or `computer_server` in-process. ### Browser network policy In `launch` and `persistent` modes, **every connection Chromium makes** — pages, sub-resources, fetch and XHR, WebSockets — goes through an **egress proxy** inside std-tools on `127.0.0.1`. For each connection the proxy: 1. checks the domain lists, `DPTTOOLS_BROWSER_ALLOWED_DOMAINS` (empty: any public host) and `DPTTOOLS_BROWSER_BLOCKED_DOMAINS` (which wins); 2. resolves the host once, and refuses every address that is not public (private, loopback, link-local, carrier-grade NAT, multicast, reserved, and IPv4 addresses inside IPv6 forms), unless the host is listed in `DPTTOOLS_BROWSER_ALLOW_HOSTS`; 3. connects to the address it validated. Chromium never resolves names itself, so DNS rebinding cannot redirect a connection. HTTPS is tunnelled unchanged: there is no TLS interception. On top of the proxy, navigation refuses non-`http(s)` schemes (`file:`, `javascript:`, `data:`, `chrome:`) and re-checks the final URL after redirects, and service workers are blocked. In **`cdp` mode** std-tools did not launch the browser, so the proxy does not apply. The agent's own tabs are checked request by request — the domain lists and the private-address block — but connections cannot be pinned to a validated address. `fetch_webpage_rendered` uses a fresh, credential-free context per call behind the same policy. `browser_run_script` runs model-written Python as the server's user. It can open network connections directly, bypassing the egress proxy, and read the process environment through `/proc`. In `cdp` mode there is no proxy even for the browser's own traffic: std-tools checks the requests of the tabs it manages, and a script can open pages outside them (in a new context, say). Enable it only where `execute_command` would also be acceptable, and rely on the container's network policy. A script can also reach every page of the browser it attaches to, so while `run_script` is enabled std-tools allows one browser session at a time, in `launch` mode starts a fresh browser for each session, and in `cdp` mode requires `DPTTOOLS_BROWSER_CDP_SHARE_TABS=true`. Isolating clients from each other's scripts takes one std-tools server per client. ### Who stops a dangerous action — three layers | Layer | Where it runs | What it knows about the action | Example | Limits | |---|---|---|---|---| | **1. Server hard policy** | Inside std-tools; the operator sets it through env vars | The URL, the host, the file path, which tool is being called | Refuse `file://`; refuse a domain not on the allow-list; refuse an upload outside the upload directory; `javascript_exec` disabled; host mode off | Blunt yes/no rules; no human involved | | **2. Server confirmation gate** | Inside std-tools; it asks a human through **MCP elicitation** | The *meaning* of the action, because the server resolves `ref_12` to a real element: "button 'Place order'", "password field", "file input" | Clicking an element named like purchase / pay / delete / send / submit order; typing into a password field; `file_upload`; `javascript_exec`; the first action of a desktop host session | Needs an MCP client that supports elicitation | | **3. App approval** | In the application running the agent — the toolkit's hooks (`skip_tool`, `pause`) | Only the tool name and its arguments | "Pause before every `browser_left_click` and ask the user" | std-tools cannot count on it: any MCP client can connect, and many have no hooks. The app sees `ref_12`, not "Place order" | Layer 1 is always on. Layer 2 is `DPTTOOLS_BROWSER_CONFIRM` for the browser: `off` (default), `deny` (refuse flagged actions outright; suits unattended pipelines) or `elicit` (ask a human through MCP elicitation; refuse if nobody can answer, including in-process use and clients without elicitation — the toolkit has none today). The gate flags clicks on elements whose name reads like a purchase, payment, deletion, sending or publishing, account closure or accepting terms (a word-list heuristic), typing or entering a password, file uploads, running JavaScript and running scripts. For the desktop, `DPTTOOLS_COMPUTER_CONFIRM` takes the same three values: `elicit` asks before the first action of each host session ("Allow the agent to control this computer for this session?") and before every key chord that closes, quits or deletes (`alt+f4`, `ctrl+w`, `ctrl+q`, `super+w`, `super+q`, `delete`, `shift+delete`); `deny` refuses those chords. The desktop gate cannot tell what a click will do (pixels carry no element names), so app approval matters more there. Layer 3 — the app's hooks — is where human approval normally lives. ### The confirmation gate at a glance | Value | Browser: `DPTTOOLS_BROWSER_CONFIRM` | Desktop: `DPTTOOLS_COMPUTER_CONFIRM` | |---|---|---| | `off` (default) | std-tools does not stop flagged actions; the client application approves them | The same | | `deny` | Flagged actions are refused outright. Suits unattended pipelines | Key chords that close, quit or delete are refused | | `elicit` | std-tools asks a human through MCP elicitation before a flagged action | std-tools asks before the first action of each host session, and before each of those chords | With `elicit`, std-tools refuses when nobody can answer — including in-process use and clients without elicitation. The toolkit has no elicitation support today. !!! warning "On the desktop, approval in your application matters more" The browser gate reads element names, so it knows when a click lands on "button 'Place order'". The desktop gate sees only pixels and coordinates: it cannot tell what a click will do, and it flags only the close, quit and delete key chords. Approve desktop actions in your application — for example, a toolkit hook on the `computer_*` tool names — or run the agent on a `virtual` desktop that holds nothing it should not touch. ### The desktop server The `virtual` backend gives each MCP session its own Xvfb display inside the container, so the agent sees only the applications it started there. A virtual desktop needs no network; any VNC view (`DPTTOOLS_COMPUTER_VNC_PORT`) listens on `127.0.0.1` only. Key input cannot run commands. Each key in `computer_key`, `computer_hold_key` and the click tools' `modifiers` must be a single character or a key name, and each chord reaches `xdotool` as the only key of its own call, so no model text is read as an `xdotool` command (*new in 0.6.1*; before it, a word such as `exec` in `computer_key` started a program). The desktop's programs (Xvfb, the window manager, `x11vnc`, `xdotool`), and anything started from the desktop, inherit the server's environment, API keys included. Run the computer server in a launcher process of its own that holds no API keys, or accept that programs on the desktop can read them. **Host mode is the user's real computer.** Run it only with the user's knowledge: every screenshot, with whatever is on the screen, goes to the model provider. Safeguards that are always on in host mode, whatever the confirmation gate says: - **Kill switch.** Moving the mouse into the top-left corner of the screen, or pressing `DPTTOOLS_COMPUTER_KILL_HOTKEY` (default `ctrl+alt+shift+k`), stops the agent; every later call returns an error until the server restarts. - **Pause on human activity.** If the user moved the mouse or typed in the last second, the next action is refused ("the user is using the computer right now"). On Windows, input that arrives *during* an action also stops that action ("the user took over"). On Linux X11 the system does not mark the agent's own input reliably, so std-tools ignores input during an action and for 0.4 s after it: there the pause, and the top-left corner, work between actions only. The kill hotkey works at any time on both. The [computer server](../servers/computer.md#host-mode-safeguards) page has the details, and [Isolated vs Attached](isolated_vs_attached.md) compares the two setups. ### What `off` means: the security burden moves to the client app With the gate off, std-tools does not stop a consequential action on its own; the app has to. Layer 1's fixed rules still hold whatever the app does: | std-tools still enforces, always | The client app is responsible for | |---|---| | The browser and computer servers stay off unless enabled; `cdp` and `host` modes stay off unless configured | Approving consequential actions before they run — purchases and payments, sending messages or posts, deleting data, account or permission changes, accepting terms — with toolkit hooks (`pause`, `skip_tool`) on the browser and computer tool names, or by setting the gate to `deny` | | Network policy: non-`http(s)` schemes refused, domain allow and block lists, non-public addresses blocked, IP pinning through the egress proxy (`launch` and `persistent` modes) | Choosing an allow-list for its use case — the default allows any public host | | Opt-in members (`javascript_exec`, `file_upload`, console, network, `run_script`) stay off unless enabled; uploads confined to the upload directory | Getting the user's consent before an agent takes over their browser (`cdp`) or desktop (`host`), and telling them their screen content goes to the model provider | | Host mode: kill switch, pause on human activity, one session at a time, a chosen monitor | Isolation: running the agent in a container with minimal privileges and no credentials it does not need | | A fresh, credential-free browser context per session (`launch` mode) | Logging and reviewing what the agent did | **A limitation to document for apps.** A hook sees the tool name and its arguments — `browser_left_click(ref="ref_12")`, not "button 'Place order'" — so app-level approval is coarser: per tool, per mode or per domain. Two things help: every pointer and form tool names the element it acted on in its result ("Clicked button 'Place order'"), which serves audit logs; and the `deny` gate blocks by element name without needing a human. On the desktop the limitation is sharper still: a hook sees `computer_left_click(x=412, y=230)`, and the result names only the point and the display, not what was under it. ### A system-prompt block for applications `doc_processing_std_tools.browser.BROWSER_SAFETY_PROMPT` is a short instruction block — page content is untrusted data; ask before consequential actions; never bypass access controls — that an application can add to its agent's instructions. std-tools never injects it. See the [browser server](../servers/browser.md#examples) page for an example. <!-- source: docs/guides/isolated_vs_attached.md --> # Isolated vs Attached An agent that uses a browser or a desktop can work in **its own environment**, separate from any person's, or **attached** to the user's own browser or screen. Isolated is the default and the safe choice; attached mode is supported, but never implicit. ## The deployment matrix | Mode | Browser | Desktop | Where std-tools must run | Default | |---|---|---|---|---| | **Isolated** — the agent's own environment | `launch`: its own headless Chromium, with a fresh profile per session | `virtual`: its own virtual display, with an optional view so a person can watch | The client's sandbox container — the normal deployment | **Yes** | | **Isolated, persistent** | `persistent`: its own Chromium on a saved profile, usually headed, so a person can sign in once and watch | — | A container with a display, or the user's machine | Opt-in | | **Attached** — the user's environment | `cdp`: attach to a Chrome the user started with remote debugging | `host`: operate the real screen of the machine | **The user's own machine** (a local agent) | Opt-in, with safeguards | A process inside a sandbox container cannot reach the user's screen or browser. Attached mode therefore needs a deployment where std-tools runs on the user's machine — for example a desktop application that embeds the toolkit — or where the user deliberately exposes a browser endpoint to it. ## Isolated: `launch` (the default) std-tools starts its own headless Chromium. Each MCP session gets a fresh, credential-free browser context, so cookies and storage are never shared between sessions or with any person's browser. Every connection Chromium makes goes through the egress proxy, with the domain lists, the private-address block and IP pinning ([Security](security.md#browser-network-policy)). Nothing to configure beyond `DPTTOOLS_ENABLE_BROWSER=true`. ## Isolated, persistent: a saved profile For sites that need a person to sign in — a portal with two-factor authentication, say — use a saved profile that std-tools launches itself: ```bash DPTTOOLS_BROWSER_MODE=persistent DPTTOOLS_BROWSER_PROFILE_DIR=/workspace/.browser-profile # required DPTTOOLS_BROWSER_HEADLESS=false # so a person can see it and sign in ``` - A person signs in once in the visible window; the login is kept in the profile folder. - Only **one session at a time**, because a profile folder cannot be shared between browsers. - A headed browser needs a display: the user's desktop, or a container with a virtual display and a viewer. - The egress proxy still applies, because std-tools launched the browser. - The profile folder holds the site's session cookies: protect it like a credential. ## Attached: the user's own browser (`cdp`) In `cdp` mode the agent drives a Chrome that the user started: 1. **Start Chrome with a dedicated profile and remote debugging** (the executable's name and path depend on the platform): ```bash chrome --remote-debugging-port=9222 --user-data-dir="$HOME/agent-chrome-profile" ``` Chrome 136 and later refuse remote debugging on the default profile — Google made the change because attackers used the debugging port to steal cookies — so an agent cannot attach to the user's everyday profile. The user signs in to what the agent needs in this dedicated profile. 2. **Point std-tools at it:** ```bash DPTTOOLS_ENABLE_BROWSER=true DPTTOOLS_BROWSER_MODE=cdp DPTTOOLS_BROWSER_CDP_URL=http://127.0.0.1:9222 ``` What changes in `cdp` mode: - **The agent works only in tabs it opened**, unless `DPTTOOLS_BROWSER_CDP_SHARE_TABS=true` lets it see and use the user's other tabs. - **No egress proxy, so no IP pinning.** std-tools did not launch the browser, so it cannot route its traffic. The agent's own tabs are checked request by request — the domain lists and the private-address block — but connections cannot be pinned to a validated address. - **Whoever reaches the debugging port controls the browser** completely. Keep it on `127.0.0.1` and never expose it beyond the machine. - `fetch_webpage_rendered` uses a separate headless browser, never the user's. ## Consent and approval for the browser The confirmation gate is `off` by default in every mode, so in attached mode especially: - Get the user's **consent** before an agent takes over their browser, and tell them that what is on screen goes to the model provider. - **Approve consequential actions** in your application — toolkit hooks on the `browser_*` tools — or set `DPTTOOLS_BROWSER_CONFIRM=deny`. See [Using with the Toolkit](with_the_toolkit.md#browser-actions) and [Security](security.md). ## Isolated: a virtual desktop (`virtual`, the default) The [computer server](../servers/computer.md)'s default backend gives each MCP session its own Xvfb display inside the container, with a light window manager. The agent sees only the applications it starts there, and sessions never see each other's desktops. ```bash DPTTOOLS_ENABLE_COMPUTER=true DPTTOOLS_COMPUTER_BACKEND=virtual # the default DPTTOOLS_COMPUTER_DISPLAY_SIZE=1440x900 # the default DPTTOOLS_COMPUTER_VNC_PORT=5900 # optional: a person can watch or take over ``` - **Linux only**: the image needs `xvfb`, `xdotool` and a window manager such as `openbox` ([Deployment](deployment.md#with-the-browser-and-a-virtual-desktop)). - The agent starts applications with `execute_command`, on the display its results name: `DISPLAY=:99 libreoffice --calc report.xlsx &`. - At most `DPTTOOLS_COMPUTER_MAX_SESSIONS` (2) desktops are open at once; one unused for 10 minutes is closed. - The VNC view needs `x11vnc`. It listens on `127.0.0.1` inside the container (the first session on the port you set, the next on port + 1), without a password, so a plain port mapping does not reach it: connect through something you control that shares the container's network, such as an SSH tunnel, and never expose it more widely. - A virtual desktop needs no network. ## Attached: the user's real screen (`host`) In `host` mode the agent operates the real screen, mouse and keyboard of the machine std-tools runs on — so std-tools must run on the user's machine, not in a container. ```bash DPTTOOLS_ENABLE_COMPUTER=true DPTTOOLS_COMPUTER_BACKEND=host DPTTOOLS_COMPUTER_MONITOR=primary # or a monitor number DPTTOOLS_COMPUTER_KILL_HOTKEY=ctrl+alt+shift+k ``` - **Platforms:** Windows and Linux X11. macOS is untested (it needs Accessibility and Screen Recording permission); Wayland is not supported. On Linux, install `pynput` separately: `pip install python-xlib && pip install --no-deps pynput`. - **One session at a time**, because there is one real cursor. A second client is refused until the first has been idle for a minute. - **The person stays in control.** Moving the mouse into the top-left corner of the screen, or pressing the kill hotkey, stops the agent until the server restarts. If the person moved the mouse or typed in the last second, the next action is refused. On Linux X11 the pause and the corner work between actions only; the hotkey works at any time. - **Everything on the screen goes to the model provider**, in every screenshot. Close what the agent should not see. ## Consent and approval on the desktop - Get the user's **consent** before an agent takes over their screen in `host` mode, and tell them what is on screen goes to the model provider. `DPTTOOLS_COMPUTER_CONFIRM=elicit` asks for that consent at the start of each host session, in MCP clients that support elicitation. - **Approve consequential actions** in your application — toolkit hooks on the `computer_*` tools — because the desktop gate cannot tell what a click will do. See [Using with the Toolkit](with_the_toolkit.md#desktop-actions) and [Security](security.md#the-desktop-server). <!-- source: docs/guides/deployment.md --> # Deployment std-tools is designed to run inside a container that your application controls (see [Security](security.md)). This page gives the container patterns. ## A base image ```dockerfile # syntax=docker/dockerfile:1 FROM python:3.12-slim # git, for pip to install std-tools from its repository. LibreOffice, for # convert_document and to render Word, Excel and PowerPoint pages as images; leave it # out if the agent only reads and renders PDFs. RUN apt-get update && apt-get install -y --no-install-recommends \ git \ libreoffice-core libreoffice-writer libreoffice-calc libreoffice-impress \ && rm -rf /var/lib/apt/lists/* # std-tools with every extra, pinned to a release tag, in the same pip command as your # application's own requirements so pip resolves them together. The repository is # private: the GitHub token arrives as a build secret and git adds it to the URL for # this one command, so it is stored in no layer and not in pip's install record. COPY requirements.txt . RUN --mount=type=secret,id=github_token \ GIT_CONFIG_COUNT=1 \ GIT_CONFIG_KEY_0="url.https://x-access-token:$(cat /run/secrets/github_token)@github.com/.insteadOf" \ GIT_CONFIG_VALUE_0="https://github.com/" \ pip install --no-cache-dir -r requirements.txt \ "doc-processing-std-tools[all] @ git+https://github.com/rzoeb/doc-processing-std-tools.git@v0.6.1" # Run as a non-root user. RUN useradd -m agent USER agent WORKDIR /workspace ``` Notes: - **Installing from the repository needs read access to it** at build time. Build with a GitHub token passed as a build secret (Docker 23 or later; a fine-grained token with read-only **Contents** access is enough): ```bash GITHUB_TOKEN=<token> docker build --secret id=github_token,env=GITHUB_TOKEN -t my-agent-image . ``` Never put the token in a `RUN` command, an `ARG` or an `ENV`: those are kept in the image. - **Choose extras** instead of `[all]` to keep the image small: `docs` for document and image tools, `ocr`, `web`, `extract` (trafilatura, for cleaner page text), `screenshot`, `browser`, `computer`. See [Quickstart](../quickstart.md#install). - **Fonts.** LibreOffice renders with the fonts installed in the image. For documents in Chinese, Japanese or Korean, add a CJK font package such as `fonts-noto-cjk`, or pages render with missing characters. - If LibreOffice is installed somewhere other than `PATH`, point `DPTTOOLS_SOFFICE_BIN` at its `soffice` executable. - The repository also contains `docker/integration.Dockerfile`, a Linux image for running the integration tests (LibreOffice, Xvfb, `xdotool`, CJK fonts). It is a test image, not a deployment pattern. - `take_screenshot` needs a display; in a headless container it returns an error, so the `screenshot` extra is rarely useful there. ## Running it | Pattern | How | |---|---| | In your agent's process | Install std-tools in the agent's image and attach the servers in-process ([Using with the Toolkit](with_the_toolkit.md)) | | A launcher your agent starts | Start `python -m doc_processing_std_tools` over stdio from your application, passing configuration in `env` | | A separate HTTP service | `python -m doc_processing_std_tools --transport http --host 0.0.0.0 --port 8000` in its own container, reachable only by your agent, behind an authenticating proxy or network policy | A `docker run` for the first pattern: ```bash docker run --rm \ --user agent \ -v /host/project:/workspace:rw \ -v /host/memory:/workspace/.memory:rw \ -v /host/skills:/workspace/.agent_skills:ro \ --memory=2g --cpus=2 \ --env-file agent.env \ my-agent-image python run_agent.py ``` where `agent.env` holds, for example: ```bash DPTTOOLS_MEMORY_DIR=/workspace/.memory DPTTOOLS_SKILLS_DIR=/workspace/.agent_skills DPTTOOLS_WEB_SEARCH_PROVIDER=tavily DPTTOOLS_WEB_SEARCH_API_KEY=tvly-xxx ``` - **Memory** should live on a persistent volume, or it is lost with the container. - **Skills** can be mounted read-only unless the agent should create skills. - **Network.** Add `--network=none` when the agent needs no internet; the Web server and OCR need outbound access. - **Configuration** is read once per process, so restart after changing it. The [Configuration](configuration.md) page lists every variable. To check what is installed in a running container: ```bash python -c "import doc_processing_std_tools as d; print(d.__version__)" ``` `__version__` (*new in 0.6.0*) comes from the installed package metadata. ## With the browser and a virtual desktop The same pattern with Chromium for the [browser server](../servers/browser.md) and an Xvfb desktop for the [computer server](../servers/computer.md) added. Playwright keeps its browsers per user, so Chromium itself is installed as the agent user (or set `PLAYWRIGHT_BROWSERS_PATH` to a shared directory): ```dockerfile # syntax=docker/dockerfile:1 FROM python:3.12-slim # git for the install, a virtual desktop for the computer server (add x11vnc if a # person should watch), fonts, and the desktop applications the agent should use RUN apt-get update && apt-get install -y --no-install-recommends \ git \ xvfb xdotool openbox fonts-dejavu-core fonts-liberation \ libreoffice-core libreoffice-writer libreoffice-calc libreoffice-impress \ && rm -rf /var/lib/apt/lists/* # Installed as in the base image: the token is a build secret COPY requirements.txt . RUN --mount=type=secret,id=github_token \ GIT_CONFIG_COUNT=1 \ GIT_CONFIG_KEY_0="url.https://x-access-token:$(cat /run/secrets/github_token)@github.com/.insteadOf" \ GIT_CONFIG_VALUE_0="https://github.com/" \ pip install --no-cache-dir -r requirements.txt \ "doc-processing-std-tools[all] @ git+https://github.com/rzoeb/doc-processing-std-tools.git@v0.6.1" # Chromium's system libraries (as root) RUN python -m playwright install-deps chromium RUN useradd -m agent USER agent WORKDIR /workspace # The browser itself (as the agent user) RUN python -m playwright install chromium ENV DPTTOOLS_ENABLE_BROWSER=true \ DPTTOOLS_ENABLE_COMPUTER=true ``` The browser server needs network access to the sites it visits, so it does not combine with `--network=none`; narrow it with `DPTTOOLS_BROWSER_ALLOWED_DOMAINS`. The virtual desktop needs no network. Leave out either half if you need only one server: the `xvfb xdotool openbox` packages and `DPTTOOLS_ENABLE_COMPUTER` for the desktop, the two `playwright install` lines and `DPTTOOLS_ENABLE_BROWSER` for the browser. In-process, pass `browser_server` or `computer_server` instead of setting the `ENABLE` variables. For the browser: - The default `launch` mode is headless and needs no display. - Give the container outbound network access; the browser's own egress proxy then applies the domain lists and the private-address block. Set `DPTTOOLS_BROWSER_ALLOWED_DOMAINS` for the sites your use case needs. - Mount a volume for `DPTTOOLS_BROWSER_DOWNLOAD_DIR` if downloads must outlive the container. - Chromium needs memory and shared memory: size the container for the number of sessions you allow (`DPTTOOLS_BROWSER_MAX_SESSIONS`, default 4). For the desktop: - Each MCP session gets its own Xvfb display, up to `DPTTOOLS_COMPUTER_MAX_SESSIONS` (default 2); size the container for the applications the agent runs on them. - Install the applications the agent should use, and fonts for the documents it opens. - To let a person watch, add `x11vnc` and set `DPTTOOLS_COMPUTER_VNC_PORT`. The VNC server has no password and listens only on `127.0.0.1` inside the container, so reach it through a tunnel you control rather than by exposing it. - The `host` backend does not belong in a container: it operates the real screen of the machine std-tools runs on ([Isolated vs Attached](isolated_vs_attached.md)). <!-- source: docs/guides/with_the_toolkit.md --> # Using with the Toolkit std-tools is the standard tool library for the `Agent` class in [doc-processing-toolkit](https://doc-processing-toolkit.vercel.app/). This page covers the ways to attach the servers to an agent, what each way implies, and how your application can approve tool calls. Version requirements are on the [Compatibility](compatibility.md) page. The toolkit's own documentation covers the agent side: [Tools](https://doc-processing-toolkit.vercel.app/concepts/tools/), [MCP](https://doc-processing-toolkit.vercel.app/concepts/mcp/), [Hooks and steering](https://doc-processing-toolkit.vercel.app/concepts/hooks/), [Skills auto-injection](https://doc-processing-toolkit.vercel.app/concepts/skills/), [Client integration](https://doc-processing-toolkit.vercel.app/concepts/client_integration/) and [Speech and images from an agent in your app](https://doc-processing-toolkit.vercel.app/recipes/agent_media_in_apps/). ## Ways to attach ### In-process: pass the server objects ```python from doc_processing_toolkit import Agent from doc_processing_std_tools import document_processing_server, file_system_server agent = Agent( model="claude-sonnet-5", tools=[ file_system_server, {"type": "fastmcp", "server": document_processing_server, "allowed_tools": ["read_pdf", "read_image"]}, ], ) ``` The dict form with `allowed_tools` (toolkit 0.5.1 or later) shows the model only the tools you list. The tools run in your process and read their configuration from your process's environment. ### Out-of-process: a session you open yourself Start the composed launcher (or one server) over stdio, with the environment it needs, and hand the toolkit the open session: ```python import os import sys from mcp import ClientSession, StdioServerParameters from mcp.client.stdio import get_default_environment, stdio_client from doc_processing_toolkit import Agent env = get_default_environment() env.update({k: v for k, v in os.environ.items() if k.startswith("DPTTOOLS_")}) env["LLMWHISPERER_API_KEY"] = os.environ.get("LLMWHISPERER_API_KEY", "") # if you use OCR params = StdioServerParameters( command=sys.executable, args=["-m", "doc_processing_std_tools"], env=env ) async with stdio_client(params) as (read, write), ClientSession(read, write) as session: await session.initialize() agent = Agent(model="claude-sonnet-5", tools=[{"session": session, "name": "std-tools-skills"}]) result = await agent.run("Summarise ./reports/latest.pdf") ``` A pre-built session belongs to the event loop that opened it: run the agent with `await agent.run()` inside the `async with` block, not with `run_sync`. The name contains `skill` so that the toolkit also recognises the session as a skills server (see [Skills](#skills) below). ### Out-of-process: let the toolkit start the server The toolkit's `mcp_command` and `mcp_script` configs start a stdio server for you: ```python agent = Agent( model="claude-sonnet-5", tools=[{"type": "mcp_command", "command": sys.executable, "args": ["-m", "doc_processing_std_tools"], "name": "std-tools"}], ) ``` In toolkit 0.5.2 and earlier these configs pass the server **no environment** beyond the MCP SDK's allow-list, so the server runs on its defaults: no search key, no OCR key, default folders and limits. Use them only when the defaults are what you want; otherwise open the session yourself, as above. ### Which to choose | | In-process | Session you open | Toolkit-started (`mcp_command`) | |---|---|---|---| | Configuration | Your process's environment | The `env` you pass | Defaults only (toolkit ≤ 0.5.2) | | Image results | Need toolkit 0.5.1+ | Any toolkit version | Any toolkit version | | Skills auto-injection | Empty catalog (toolkit ≤ 0.5.2) | Works | Empty catalog (toolkit ≤ 0.5.2) | | A slow tool (OCR, LibreOffice, a long command) | Runs on the agent's event loop and holds up streaming and hooks until it returns (toolkit ≤ 0.5.2) | Runs on a worker thread in the composed server; the agent keeps going | Same as a session you open | | Arguments | Passed to the function as the model sent them | Validated against the tool's schema by the server | Validated by the server | | `run_sync` | Yes | No — `await agent.run()` | Yes | ## Passing configuration to a stdio server The MCP Python SDK starts a stdio server with only a short allow-list of variables: on Windows `APPDATA`, `HOMEDRIVE`, `HOMEPATH`, `LOCALAPPDATA`, `PATH`, `PATHEXT`, `PROCESSOR_ARCHITECTURE`, `SYSTEMDRIVE`, `SYSTEMROOT`, `TEMP`, `USERNAME` and `USERPROFILE`; elsewhere `HOME`, `LOGNAME`, `PATH`, `SHELL`, `TERM` and `USER`. Everything else — every `DPTTOOLS_*` variable and every API key — must be passed in the session's `env`, as in the example above. Build `env` from `get_default_environment()` so the allow-list is kept, then add what std-tools needs. The [Configuration](configuration.md) page lists every variable. Other MCP clients, such as desktop assistants and IDEs, take an `env` block in their server configuration for the same reason. ## Images Six tools return images: `read_image`, `render_document_page`, `render_document_thumbnail`, `read_document_pages_as_images`, `take_screenshot` and `fetch_image` — plus, when they are enabled, the browser server's `browser_screenshot` and `browser_zoom` and the computer server's `computer_screenshot` and `computer_zoom`. Each scales its image to model-safe limits first ([Images](images.md)). - **Out of process**, image results reach the model with any toolkit version. - **In-process**, they need **toolkit 0.5.1 or later**. Older versions turned them into the text `<...Image object at 0x...>`, and a model could then describe a picture it never saw. - On a model that cannot take images in tool results, the toolkit replaces the image with a short notice. To let a model look at a generated image, attach only `read_image`: `{"type": "fastmcp", "server": document_processing_server, "allowed_tools": ["read_image"]}`. ## Skills The toolkit reads the skills server automatically: it calls `list_skills`, puts each skill's name, description and path into the system prompt, and hides `load_skill`, `load_skill_resource` and `execute_skill_script` when there are no skills. The [Skills server](../servers/skills.md) page describes the payload. !!! warning "Toolkit 0.5.2 and earlier: use a pre-built session for skills" Auto-injection works only through a session you open yourself, as in the example above: - the session's `name` contains `skill`; - its server was started with `DPTTOOLS_SKILLS_DIR` in `env`; - `DPTTOOLS_SKILLS_DIR` is also set in your application's own environment and names a folder that exists there. In-process (`Agent(tools=[skills_server])`) and `mcp_command` configs return an empty catalog, and the agent is never told the skills exist. This is a toolkit issue; see [Compatibility](compatibility.md#known-toolkit-issue-skills-auto-injection-toolkit-052-and-earlier). ## Approving tool calls with hooks std-tools does not ask for confirmation: a tool call runs when the model makes it. To put a person or a policy in the loop, use a toolkit hook. A hook sees every `before_tool_call` event and can decline the call with `skip_tool`, so the tool never runs and the model is told why: ```python from doc_processing_toolkit.agent import Agent, HookDirective NEEDS_APPROVAL = {"execute_command", "delete_file", "move_file", "memory_delete", "write_file"} async def approve(event): if event.event_type == "before_tool_call" and event.metadata["tool_name"] in NEEDS_APPROVAL: allowed = await ask_the_user(event.metadata["tool_name"], event.metadata["arguments"]) if not allowed: return HookDirective(action="skip_tool", message="The user declined this action.") return None agent = Agent(model="claude-sonnet-5", tools=[...], hooks=approve) ``` `ask_the_user` stands for your application's own approval step. Hooks fire for tools the toolkit calls itself — in-process servers and MCP sessions alike. See the toolkit's [Hooks and steering](https://doc-processing-toolkit.vercel.app/concepts/hooks/) page for the full event list and directives. ### Browser actions The [browser server](../servers/browser.md) acts on live websites, where a click can submit an order or send a message. Its server-side confirmation gate, `DPTTOOLS_BROWSER_CONFIRM`, is **off by default**, so approving consequential actions is your application's job: - Hook `before_tool_call` for the browser tools that act — `browser_left_click`, `browser_form_input`, `browser_type`, `browser_key`, `browser_navigate` to a new domain — and the opt-in `browser_javascript_exec`, `browser_file_upload` and `browser_run_script`. - A hook sees `browser_left_click(ref="ref_12")`, not "button 'Place order'", so approval is per tool, per mode or per domain. The tool's result names the element it acted on, for your audit log. - For unattended pipelines, set `DPTTOOLS_BROWSER_CONFIRM=deny`: std-tools then refuses clicks on elements named like purchases, payments, deletions, sending or publishing, account closure or accepting terms, as well as password entry, uploads and scripts. - `DPTTOOLS_BROWSER_CONFIRM=elicit` asks a human through MCP elicitation. The toolkit does not support elicitation today, so with the toolkit `elicit` refuses every flagged action. - Add `BROWSER_SAFETY_PROMPT` to the agent's instructions ([Browser](../servers/browser.md#examples)). The [Security](security.md#the-browser-and-desktop-servers) guide sets out exactly what std-tools enforces and what your application must do. ### Desktop actions The [computer server](../servers/computer.md) operates a desktop: its own virtual display by default, or the user's real screen in `host` mode. Its confirmation gate, `DPTTOOLS_COMPUTER_CONFIRM`, is also **off by default**, and it can do less than the browser's: it sees pixels, not element names, so it cannot tell what a click will do. Approval in your application matters more here. - Hook `before_tool_call` for the computer tools that act — the clicks (`computer_left_click`, `computer_right_click`, `computer_middle_click`, `computer_double_click`, `computer_triple_click`), `computer_left_click_drag`, `computer_left_mouse_down`, `computer_type`, `computer_key` and `computer_hold_key`. `computer_screenshot`, `computer_zoom`, `computer_cursor_position` and `computer_wait` only look or wait. - A hook sees `computer_left_click(x=412, y=230)`. To show the user what is about to happen, keep the latest `computer_screenshot` and display it with the coordinates, or ask the model to state its purpose in each call's optional `intent` argument (std-tools only logs it). - `DPTTOOLS_COMPUTER_CONFIRM=deny` refuses the key chords that close, quit or delete (`alt+f4`, `ctrl+w`, `ctrl+q`, `super+w`, `super+q`, `delete`, `shift+delete`), also with extra keys and as a click's `modifiers`. It does not cover clicks on a "Delete" button. - `DPTTOOLS_COMPUTER_CONFIRM=elicit` would ask before the first action of each host session and before those chords. The toolkit does not support elicitation, so with the toolkit `elicit` refuses them. - In `host` mode, get the user's consent before the agent starts, and tell them their screen goes to the model provider. The kill switch and the pause on human activity are always on, whatever your hooks do ([Security](security.md#the-desktop-server)). <!-- source: docs/guides/images.md --> # Images Several std-tools tools return an image for the model to look at. Since 0.6.0, every one of them first brings the image within limits that every supported model provider accepts, so a large photo or a high-resolution page no longer fails the model request — and with it the agent's turn. ## Which tools scale | Tool | Server | |---|---| | `read_image` | [Documents](../servers/documents.md) | | `render_document_page` | [Documents](../servers/documents.md) | | `render_document_thumbnail` | [Documents](../servers/documents.md) | | `read_document_pages_as_images` | [Documents](../servers/documents.md) | | `take_screenshot` | [Screenshot](../servers/screenshot.md) | | `fetch_image` | [Web](../servers/web.md) | | `browser_screenshot`, `browser_zoom`, and images from `browser_run_script` | [Browser](../servers/browser.md) | | `computer_screenshot`, `computer_zoom` | [Computer (Desktop)](../servers/computer.md) | Only the image returned to the model is scaled. A file a tool writes to an `output_path` keeps its full resolution, and tools that write files and return paths — the [Image Tools](../servers/image_tools.md) server and `convert_document` — do not scale at all. ## The limits | Variable | Default | Limit | |---|---|---| | `DPTTOOLS_IMAGE_MAX_EDGE_PX` | `1568` | The longest side, in pixels | | `DPTTOOLS_IMAGE_MAX_MB` | `3` | The encoded file size, in megabytes | Both are read once per process and must be positive integers. The defaults come from the strictest values across the providers `doc-processing-toolkit` supports, as its model registry records them: | Provider | Longest side | File size | |---|---|---| | Anthropic | 1568 px | 5 MB | | OpenAI | 2000 px | 20 MB | | Google | — | 20 MB | | Fireworks | — | 10 MB | The size limit is 70 % of the smallest provider limit, rounded down, because sending an image as base64 adds about a third to its size. An image within both defaults is safe on every provider. Raise them only if every model you use accepts more; larger images also cost more image tokens. ## How an image is scaled 1. **An image within both limits is returned unchanged**, byte for byte, in its own format. 2. Otherwise it is resized with a Lanczos filter so its longest side fits, keeping the aspect ratio. Images are never enlarged. 3. It is encoded as **PNG** if the source was PNG or has transparency, and as **JPEG** (quality 85) otherwise. 4. While it is still over the size limit: - a JPEG steps its quality down, 85 → 75 → 65; - a PNG without transparency switches to JPEG; - then the scale shrinks by 15 % and the steps repeat. A PNG with transparency only shrinks. 5. If the image would have to shrink below **256 px** on its longest side, the tool returns an `Error:` suggesting a larger `DPTTOOLS_IMAGE_MAX_MB` or a crop. A scaled animated GIF or multi-page TIFF keeps only its first frame. Some sizes at the defaults: | Source | Sent to the model | |---|---| | A 1440 × 900 screenshot | Unchanged | | A 1920 × 1080 screenshot | 1568 × 882 | | Two full-HD monitors, 3840 × 1080 | 1568 × 441 | | An A4 page rendered at `scale=2.0`, 1190 × 1684 | 1108 × 1568 | | A 4000 × 3000 JPEG photo | 1568 × 1176 JPEG | ## Tips - **Crop rather than shrink for detail.** When small print matters, cut out the region first with `image_crop` or `image_zoom`, then `read_image` the result: the region is then sent at full detail. - **Check before sending.** `get_image_metadata` and `image_get_info` report an image's size without sending the pixels. - **In-process use needs toolkit 0.5.1 or later** for any image result to reach the model as an image. See [Compatibility](compatibility.md). ## Coordinates in the browser and computer servers `browser_screenshot` and `browser_zoom` pass through the same scaling, and the scaling step reports the factor it applied. The browser server keeps that factor per tab, so when the model gives `x` and `y` in screenshot pixels, the server maps them back to real page pixels before it clicks. At the default viewport of 1440 × 900 nothing is scaled, so the factor is 1. The [computer server](../servers/computer.md) works the same way (*new in 0.6.0*): `x` and `y` are in the pixels of the latest `computer_screenshot`. The default virtual display, 1440 × 900, is sent unchanged. A real screen in `host` mode is often larger: a 1920 × 1080 monitor is sent as 1568 × 882, and every coordinate the model sends is mapped back to real pixels. The screenshot's result text gives both sizes, for example "Screenshot of monitor 1 (1920x1080, scaled to 1568x882)." `computer_zoom` enlarges a region up to 4× to read small text. <!-- source: docs/guides/compatibility.md --> # Compatibility ## Tested versions std-tools 0.6.0 is tested against `doc-processing-toolkit` **0.5.1 and 0.5.2**, on Python **3.10.11 and 3.12**, with `mcp` **1.26 and 1.30**. The standing check is `tests/integration/test_toolkit_compat.py`: it runs whenever the toolkit is installed next to std-tools, makes no model call, and covers the points on this page. 0.6.1 changes nothing on this page: its changes are to key input, the address check, error wording and the docs. ## Python `doc-processing-std-tools` requires **Python 3.10 or newer** (`requires-python = ">=3.10"`). ## The `mcp` SDK std-tools requires **`mcp>=1.25,<2`**, the same range as the toolkit. (0.5.0 declared `mcp>=1.0.0,<2`; the old floor was never meaningful next to the toolkit.) The ceiling below 2 is deliberate: the composed launcher reads each server's registered tools through a private FastMCP attribute, because the SDK has no public composition API, and the toolkit also relies on FastMCP internals. A move to `mcp` 2.x has to be made in both packages together. ## doc-processing-toolkit std-tools does **not** depend on the toolkit and imports nothing from it; the servers work with any MCP host. The two packages meet at a few points, and these are the contract: | Contract | Detail | Since | |---|---|---| | Server objects as tools | The toolkit's `Agent(tools=[...])` accepts a `FastMCP` server object directly, which is how std-tools servers attach in-process | — | | Image results in-process | `read_image`, `render_document_page`, `render_document_thumbnail`, `read_document_pages_as_images`, `take_screenshot`, `fetch_image`, the browser's `browser_screenshot` and `browser_zoom`, and the computer server's `computer_screenshot` and `computer_zoom` return images. Attached in-process, they reach the model as images only from **toolkit 0.5.1**; earlier versions turned them into the text `<...Image object at 0x...>`. Out-of-process use works with any toolkit version | toolkit 0.5.1 | | The `list_skills` payload | Each entry in `skills` has a `path` (absolute path to its `SKILL.md`), and non-fatal problems come in a top-level `diagnostics` array. The toolkit's skills auto-injection reads both | std-tools 0.2.0 | | Skills tool names | The toolkit hides `load_skill`, `load_skill_resource` and `execute_skill_script` when the catalog is empty. Renaming any of them would silently break that | std-tools 0.2.0 | | Skills server name | The toolkit finds the skills server by its name, which must contain `skill`. `skills_server` is named `skills-tools`, which matches; the composed server is named `doc-processing-std-tools`, which does not | — | | `allowed_tools` | Exposing only some of a server's tools with `{"type": "fastmcp", "server": ..., "allowed_tools": [...]}` | toolkit 0.5.1 | **A stdio server only sees the environment you pass it.** The MCP SDK starts a stdio server with a short allow-list of variables (`PATH`, `TEMP`, `USERPROFILE` and a few others). Pass every `DPTTOOLS_*` variable and API key you use in the session's `env`, or the server falls back to its defaults — for example `./.agent_skills` for skills — and has no keys. ## Known toolkit issue: skills auto-injection (toolkit 0.5.2 and earlier) In toolkit 0.5.2 and earlier, skills auto-injection works **only through a pre-built session**. The other two routes return an empty catalog, and the toolkit then also hides the activation tools, so the agent never learns about your skills. The only trace is a WARNING in the toolkit's log. | Route | What happens | |---|---| | In-process, `Agent(tools=[skills_server])` | The toolkit calls `list_skills` with a `skills_dir` argument the tool has never taken, then misreads the result. Empty catalog | | `mcp_command` or `mcp_script` config | The toolkit passes the server no environment, so it never sees `DPTTOOLS_SKILLS_DIR`. Empty catalog | | **Pre-built session** whose server was started with `env` | **Works** | This is a toolkit issue, not a std-tools one; the fixes are planned on the toolkit side. Until a toolkit release fixes it, start the server yourself and hand the toolkit the session: ```python import os import sys from mcp import ClientSession, StdioServerParameters from mcp.client.stdio import get_default_environment, stdio_client from doc_processing_toolkit import Agent env = get_default_environment() env.update({k: v for k, v in os.environ.items() if k.startswith("DPTTOOLS_")}) env["LLMWHISPERER_API_KEY"] = os.environ.get("LLMWHISPERER_API_KEY", "") # if you use OCR params = StdioServerParameters( command=sys.executable, args=["-m", "doc_processing_std_tools"], env=env ) async with stdio_client(params) as (read, write), ClientSession(read, write) as session: await session.initialize() agent = Agent( model="claude-sonnet-5", tools=[{"session": session, "name": "std-tools-skills"}], ) result = await agent.run("Which skills do you have?") ``` Three details make this work: - The session's `name` contains `skill`, so the toolkit recognises it as a skills server. - `env` carries `DPTTOOLS_SKILLS_DIR`, so the server reads the right folder. - `DPTTOOLS_SKILLS_DIR` is also set in **your application's own environment**, pointing at a folder that exists there: the toolkit skips auto-injection otherwise. A pre-built session belongs to the event loop that opened it, so use `await agent.run(...)` inside the `async with` block, not `run_sync`. Other in-process servers (`file_system_server`, `memory_server` and the rest) are not affected. ## Versions and pinning std-tools is installed from Git, and a tag is the version: ```bash pip install "doc-processing-std-tools @ git+https://github.com/rzoeb/doc-processing-std-tools.git@v0.6.1" ``` Tags are `v<version>` and match `version` in `pyproject.toml`. A branch name is not a pin. A running deployment can report its version with `doc_processing_std_tools.__version__` (*new in 0.6.0*), which is read from the installed package metadata. After upgrading, reinstall rather than relying on an editable install's metadata, and restart long-running processes, because configuration is read once per process. See the [Changelog](../changelog.md) for what each release changed. <!-- source: docs/recipes/ocr_a_scanned_pdf.md --> # Recipe: OCR a Scanned PDF ## Goal Get the text out of a PDF whose pages are pictures, such as a scan, a fax or a photographed contract: tell that it is scanned, OCR only the pages you need with `ocr_pdf`, and read a long document in parts. ## Prerequisites - The `docs` and `ocr` extras (or `all`): ```bash pip install "doc-processing-std-tools[docs,ocr] @ git+https://github.com/rzoeb/doc-processing-std-tools.git@v0.6.1" ``` - `LLMWHISPERER_API_KEY` in the environment of the process that runs the tools. LLMWhisperer bills per page, so ask for the pages you need rather than `"all"`. ## Steps Tool calls are written as Python calls; the model sends the same arguments as JSON. ### 1. Check whether the PDF has text `get_file_overview` is cheap and needs no key. A scanned PDF has an empty text preview on every page and reports images: ```python get_file_overview(path="scans/contract.pdf") ``` ```json { "type": "pdf", "page_count": 3, "text_preview_per_page": ["", "", ""], "has_tables": false, "has_images": true } ``` `read_pdf` on the same file says so too, and names the tool to use: ```text --- Page 1 --- (no extractable text on this page) --- Page 2 --- (no extractable text on this page) --- Page 3 --- (no extractable text on this page) [No extractable text found — this looks like a scanned/image PDF. Use ocr_pdf for OCR extraction.] ``` A PDF with a text layer on some pages and scans on others shows text for some pages only. OCR just the empty ones. ### 2. OCR the pages you need ```python ocr_pdf(path="scans/contract.pdf", pages="1-2") ``` ```text --- Page 1 --- SUPPLY AGREEMENT - PAGE 1 Clause 1.1: The supplier delivers item 100 monthly. ... --- Page 2 --- SUPPLY AGREEMENT - PAGE 2 ... ``` `pages` takes `"all"`, a page (`"3"`), a range (`"1-5"`), a list (`"1-3,7"`) or an open-ended range (`"21-"`, page 21 to the end). Pages come back in document order, whatever order you list them in, and the `--- Page N ---` headers are their page numbers in the document. !!! note "Page labels in 0.6.0" In 0.6.0 the labels could be wrong: an open-ended range was numbered from 1 (`pages="21-"` came back as `--- Page 1 ---` onwards), and a list out of order (`"3,1"`) was labelled in the order given. The note in step 3 could also name pages you never asked for. This is fixed in 0.6.1 (see the [Changelog](../changelog.md)). On 0.6.0, ask for one closed range at a time, such as `"21-40"`. ### 3. Read a long document in parts OCR output longer than `DPTTOOLS_OUTPUT_TRUNCATION_LIMIT` (50,000 characters by default) is cut **on a page boundary**, and the result ends with exactly the pages left out, as a `pages` value: ```python ocr_pdf(path="scans/contract.pdf", pages="2-") ``` ```text --- Page 2 --- SUPPLY AGREEMENT - PAGE 2 ... [Output truncated on a page boundary. Page 3 was not included. Re-call ocr_pdf with pages='3' to read it.] ``` With several pages left out, the note lists them, for example `pages='5-9,12'`. The model follows the note with another `ocr_pdf` call. A dense 200-page document is better read in ranges of about 20 pages from the start: each call is quicker, and a timeout loses less. ### 4. Give an agent the tools Attach only the tools this job needs, and say when to use OCR: ```python from doc_processing_toolkit import Agent from doc_processing_std_tools import document_processing_server agent = Agent( model="claude-sonnet-5", instructions=( "Read PDFs with read_pdf. When it reports no extractable text, use ocr_pdf, " "asking for at most 20 pages per call." ), tools=[{"type": "fastmcp", "server": document_processing_server, "allowed_tools": ["get_file_overview", "read_pdf", "ocr_pdf"]}], ) result = agent.run_sync("What does clause 2.4 of scans/contract.pdf say?") ``` In-process, the tools read `LLMWHISPERER_API_KEY` from your process's environment. An OCR call takes seconds to minutes, and with toolkit 0.5.2 or earlier an in-process tool holds up the agent's event loop until it returns. To avoid that, run the tools out of process, as in [Using with the Toolkit](../guides/with_the_toolkit.md#out-of-process-a-session-you-open-yourself), and pass `LLMWHISPERER_API_KEY` in the session's `env`. ### Without an OCR key `read_document_pages_as_images(path="scans/contract.pdf", pages="1-2")` returns the pages as images, and a model that reads images can read the text itself. It costs image tokens instead of OCR pages, and it suits a few pages or a page with a layout that matters, such as a form or a table. ## What to validate - `read_pdf` on the scan ends with the note naming `ocr_pdf`. - `ocr_pdf` returns a `--- Page N ---` block for each requested page, in document order and numbered as in the document, including for an open-ended range such as `"2-"` and a list out of order such as `"3,1"`. - A result that was cut ends with the pages left out, and asking for exactly those returns the rest. - Without `LLMWHISPERER_API_KEY`, `ocr_pdf` returns an `Error:` naming the variable and uploads nothing. - An empty file, or a file that is not a PDF, is refused before anything is uploaded. ## Test sources - `tests/unit/test_document_processing.py`: `TestReadPdf::test_scanned_pdf_routes_to_ocr`, `TestOcrExtraction` (page headers, page ranges, cuts on a page boundary), `TestOcrLabels`, `TestPagesSpec`, `TestOcrInputValidation` and `TestOcrErrorClassification`. The OCR client is mocked. - `tests/integration/test_ocr_live.py`: real LLMWhisperer calls. They are paid, so they are left out of routine runs. ## Related pages - [Documents server](../servers/documents.md): every reading tool, and the OCR settings (`DPTTOOLS_OCR_TIMEOUT`). - [Configuration](../guides/configuration.md): `LLMWHISPERER_API_KEY` and the output limit. <!-- source: docs/recipes/office_to_page_images.md --> # Recipe: Office Files to Page Images ## Goal Let a model see Word, Excel and PowerPoint files as they look on the page: charts, pictures, diagrams and layout that the text readers can only count. Show the pages to the model, or save them as image files for a report or another step. ## Prerequisites - The `docs` extra (pymupdf, Pillow and the Office readers). - **LibreOffice** where the tools run. It turns each file into a PDF, which pymupdf then renders. On Debian or Ubuntu: ```bash apt-get install libreoffice-core libreoffice-writer libreoffice-calc libreoffice-impress ``` The [Deployment](../guides/deployment.md) page has a Dockerfile with it. Install fonts for the documents you expect, such as `fonts-noto-cjk` for Chinese, Japanese or Korean. PDFs render without LibreOffice, with pymupdf alone. ## Steps Tool calls are written as Python calls; the model sends the same arguments as JSON. The results are from a run in a Linux container with LibreOffice. ### 1. Read the text first, and look for media The text readers are fast and cost no image tokens. They also say when a file holds something they cannot read: ```python read_powerpoint(path="decks/q3_review.pptx") ``` ```json { "slides": [ {"slide": 1, "text": ["Q3 review", "Revenue and margin"], "image_count": 0, "chart_count": 0, "has_media": false}, {"slide": 2, "text": ["Revenue by region"], "image_count": 0, "chart_count": 1, "has_media": true}, {"slide": 3, "text": ["Next steps", "Hire two analysts"], "image_count": 0, "chart_count": 0, "has_media": false} ], "slide_count": 3, "has_media": true } ``` Slide 2 is a chart with a title: its numbers are not in the text. `read_excel` reports `has_charts`, `has_images` and `has_media` the same way. `read_word` ends with a line such as: ```text [media_summary] embedded_images=1 — this document contains images; if text looks incomplete, render it with read_document_pages_as_images. ``` ### 2. Show the model the pages that matter ```python read_document_pages_as_images(path="decks/q3_review.pptx", pages="2") ``` This returns one image per page, for the model to look at. In the run above, a standard 4:3 slide came back as a 1440 × 1080 PNG, and the LibreOffice step took about 1.5 seconds per file. | Tool | Use it for | |---|---| | `read_document_pages_as_images(path, pages, scale=2.0, max_pages=20)` | The pages you name: `"2"`, `"1-3"`, `"1-3,7"` | | `render_document_thumbnail(path, max_pages=5)` | A quick look at the first pages, at 72 dpi (a Letter page is 612 × 792) | | `render_document_page(path, page=1, scale=2.0)` | One page | Every image is brought within `DPTTOOLS_IMAGE_MAX_EDGE_PX` (1568 px) and `DPTTOOLS_IMAGE_MAX_MB` (3 MB) before the model sees it. A tall spreadsheet page rendered at 1109 × 1568 for that reason. A higher `scale` doesn't add detail past that limit. To read small print, crop the page with [Image Tools](../servers/image_tools.md) and `read_image` the crop. Ask for page ranges rather than `"all"`: each page costs image tokens, and pages past `max_pages` are dropped without an error. ### 3. Save page images as files When the images are an output rather than something for the model to look at, write them to disk with `convert_document`: ```python convert_document(path="decks/q3_review.pptx", to="png", page="1-2", outdir="exports") ``` ```json { "output_files": ["/workspace/exports/q3_review-p1.png", "/workspace/exports/q3_review-p2.png"], "pages": 2, "engine": "libreoffice+pymupdf" } ``` - Pages are rendered at 144 dpi and named `<name>-p<N>.png`; `to="jpg"` writes JPEGs. - `outdir` must already exist. Without it, the files go next to the source. - An existing file is never replaced unless you pass `overwrite=True`: ```text Error: '/workspace/exports/q3_review-p1.png' (and 1 more) already exists. Pass overwrite=True to replace it, or choose another outdir. ``` - `convert_document(path="sales.xlsx", to="pdf")` makes a PDF instead, with `pages` set to its page count. To check a saved page, `read_image(path="exports/q3_review-p2.png")` shows it to the model. ### 4. Give an agent the tools ```python from doc_processing_toolkit import Agent from doc_processing_std_tools import document_processing_server agent = Agent( model="claude-sonnet-5", instructions=( "Read Office files with read_word, read_excel or read_powerpoint first. When a result reports " "charts, images or media, look at those pages with read_document_pages_as_images, a few " "pages at a time." ), tools=[{"type": "fastmcp", "server": document_processing_server, "allowed_tools": ["get_file_overview", "read_word", "read_excel", "read_powerpoint", "read_document_pages_as_images", "convert_document", "read_image"]}], ) result = agent.run_sync("What does the revenue chart in decks/q3_review.pptx show?") ``` Image results reach the model in-process only with toolkit 0.5.1 or later ([Compatibility](../guides/compatibility.md)); out of process they work with any version. A LibreOffice run takes a second or more, so a busy agent is better served out of process, where the composed launcher runs it on a worker thread. ## What to validate - The text readers report media: `chart_count` and `has_media` (PowerPoint), `has_charts` (Excel), the `[media_summary]` line (Word). - `read_document_pages_as_images` returns one image per requested page, each within the image limits. - `convert_document(..., to="png", page="1-2")` writes exactly those pages and lists them in `output_files`. - Running the same conversion again fails, naming the existing file, until `overwrite=True`. - Without LibreOffice, rendering an Office file returns an `Error:` with the install command, and PDFs still render. ## Test sources - `tests/integration/test_libreoffice_live.py`: real conversions (DOCX to PNG and PDF, XLSX to CSV, two at once, `render_document_page` on a real DOCX). Skipped without LibreOffice; run it in the Linux image with `tests/integration/run_in_docker.py`. - `tests/unit/test_soffice.py`: finding LibreOffice, per-run profiles, timeouts, and the no-overwrite rule. - `tests/unit/test_document_processing.py`: `TestReadPowerPoint`, `TestReadExcel`, `TestReadWord` (media reporting), and the three render tools. ## Related pages - [Documents server](../servers/documents.md#converting-documents): every target `convert_document` supports. - [Images](../guides/images.md): how images are fitted to the model limits. <!-- source: docs/recipes/search_then_summarise.md --> # Recipe: Search, Then Summarise ## Goal Search the web, read the best sources in full, and write a summary that cites them, with the [Web server](../servers/web.md)'s `web_search`, `fetch_webpage` and `format_search_results`. ## Prerequisites - The `web` extra, and `extract` for main-content extraction (both are in `all`): ```bash pip install "doc-processing-std-tools[web,extract] @ git+https://github.com/rzoeb/doc-processing-std-tools.git@v0.6.1" ``` - A search provider account (Tavily, Serper or Brave) and its API key. ## Steps Tool calls are written as Python calls; the model sends the same arguments as JSON. The results are from a live run, shortened. ### 1. Configure the provider ```bash DPTTOOLS_WEB_SEARCH_PROVIDER=tavily # or serper, brave DPTTOOLS_WEB_SEARCH_API_KEY=tvly-xxx ``` Both go in the environment of the process that runs the tools: your own process in-process, or the server's `env` out of process. `fetch_webpage` and `format_search_results` need no provider. ### 2. Search ```python web_search(query="LibreOffice headless convert-to pdf command line", num_results=5) ``` ```json { "provider": "tavily", "query": "LibreOffice headless convert-to pdf command line", "results": [ {"title": "Tip of the day: batch PDF conversion with LibreOffice - anarcat", "url": "https://anarc.at/blog/2019-10-08-libreoffice-pdf-batch", "snippet": "> Fear not, shell scripting and headless mode flies in to the rescue! ..."}, {"title": "Convert-to command line parameter", "url": "https://ask.libreoffice.org/t/convert-to-command-line-parameter/840", "snippet": "More importantly, headless conversion silently does nothing if any LO is already open; ..."} ], "usage": {"provider_reported": true, "search_depth": "basic", "credits": 1, "response_time": 1.1, "request_id": "..."} } ``` - `num_results` defaults to 5 and is capped at 20. - `search_type="news"` searches news, for recent events. - With Tavily, `search_depth="basic"` costs 1 credit and `"advanced"` 2. `usage` reports what Tavily charged. Serper and Brave report no cost, and std-tools doesn't invent one. ### 3. Read the best sources A snippet is a fragment, not a source. Pick the two or three results that answer the question, and read each one: ```python fetch_webpage(url="https://anarc.at/blog/2019-10-08-libreoffice-pdf-batch", format="markdown") ``` ```json { "final_url": "https://anarc.at/blog/2019-10-08-libreoffice-pdf-batch/", "status": 200, "title": "Tip of the day: batch PDF conversion with LibreOffice - anarcat", "extractor": "trafilatura", "text": "# Tip of the day: batch PDF conversion with LibreOffice\n\nSomeone asked me today ...", "total_chars": 1560, "has_more": false, "likely_js_rendered": false } ``` - **Long pages come in parts.** When `has_more` is true, call again with `offset=next_offset` rather than raising `max_length`. - **`likely_js_rendered: true`** means the text is probably incomplete. Retry with `fetch_webpage_rendered` if the browser server is attached ([JavaScript-Rendered Pages](js_rendered_page.md)). - **A PDF or Office file is refused**, with its size. Download it with `execute_command` and read it with the [Documents](../servers/documents.md) tools. - **Private and internal addresses are refused**, and so is a redirect to one. ### 4. Number the sources and cite them `format_search_results` turns results into numbered entries and a reference list. Pass it the pages the model actually read, as a JSON array of `{title, url, snippet}`, so that the numbers match the sources used. Here the snippets are the points the summary takes from each page: ```python format_search_results(results=json.dumps([ {"title": "Tip of the day: batch PDF conversion with LibreOffice - anarcat", "url": "https://anarc.at/blog/2019-10-08-libreoffice-pdf-batch", "snippet": "libreoffice --headless --convert-to pdf *.docx converts every DOCX file in the folder."}, {"title": "Convert-to command line parameter", "url": "https://ask.libreoffice.org/t/convert-to-command-line-parameter/840", "snippet": "Headless conversion silently does nothing if LibreOffice is already open."}, ])) ``` ```text [1] Tip of the day: batch PDF conversion with LibreOffice - anarcat libreoffice --headless --convert-to pdf *.docx converts every DOCX file in the folder. [2] Convert-to command line parameter Headless conversion silently does nothing if LibreOffice is already open. --- References: [1] Tip of the day: batch PDF conversion with LibreOffice - anarcat — https://anarc.at/blog/2019-10-08-libreoffice-pdf-batch [2] Convert-to command line parameter — https://ask.libreoffice.org/t/convert-to-command-line-parameter/840 ``` Entries are numbered in the order given, with the titles given. The model writes its summary with `[1]`, `[2]` markers and ends with the reference list. The whole `web_search` output, passed as it is, works too: the numbers then follow the search results, with their snippets. ### 5. Give an agent the tools ```python from doc_processing_toolkit import Agent from doc_processing_std_tools import web_search_server agent = Agent( model="claude-sonnet-5", instructions=( "Answer with a short summary that cites its sources. Search with web_search, read the two " "or three best results with fetch_webpage, and use only what you read. Number the sources " "with format_search_results, cite them as [n] in the text, and end with the reference list." ), tools=[{"type": "fastmcp", "server": web_search_server, "allowed_tools": ["web_search", "fetch_webpage", "format_search_results"]}], ) result = agent.run_sync("How do I convert a folder of DOCX files to PDF with LibreOffice?") ``` `fetch_image` is left out here because a summary doesn't need it. Add it when the model should look at a chart or photo from a page: `fetch_webpage(..., include_assets=True)` lists a page's image URLs. ## What to validate - `web_search` returns `provider`, `query`, `results` (each with `title`, `url` and `snippet`) and `usage`, and Tavily's `usage` shows the credits charged. - Without the provider settings, `web_search` returns an `Error:` naming them, and the other three tools still work. - `fetch_webpage` returns the page's main text, and paging with `offset` returns the rest. - `format_search_results` numbers the entries from 1, and its reference list matches them. - An unknown `search_type` or `search_depth` is an error, not a silent default. - Error messages never contain the API key or the provider's request URL. ## Test sources - `tests/unit/test_web_search.py`: each provider (`TestWebSearch`), usage reporting (`TestSearchUsage`), citation formatting (`TestFormatSearchResults`), argument checks (`TestWebSearchInputValidation`) and the network policy (`TestSSRFProtection`). HTTP is mocked, so no key is needed. - `tests/unit/test_fetch_webpage_structured.py`: the `fetch_webpage` result, paging and the content-type check. - `tests/integration/test_web_search_mcp.py`: the tools through the server. ## Related pages - [Web server](../servers/web.md): every argument, the providers and the network policy. - [Configuration](../guides/configuration.md): the fetch limits (`DPTTOOLS_WEB_FETCH_*`). <!-- source: docs/recipes/js_rendered_page.md --> # Recipe: JavaScript-Rendered Pages ## Goal Read a page whose content appears only after JavaScript runs in the browser, such as a single-page app or a dashboard. Use the fast `fetch_webpage` first, and fall back to `fetch_webpage_rendered` only when the page turns out to be an empty shell. ## Prerequisites - The `web` extra for `fetch_webpage` (add `extract` for main-content extraction). - The `browser` extra and Chromium for `fetch_webpage_rendered`: ```bash pip install "doc-processing-std-tools[web,extract,browser] @ git+https://github.com/rzoeb/doc-processing-std-tools.git@v0.6.1" playwright install chromium # in a container: playwright install --with-deps chromium ``` - `fetch_webpage_rendered` belongs to the [browser server](../servers/browser.md), which is off unless enabled. You don't need the rest of the browser tools to use it (step 4). ## Steps Tool calls are written as Python calls; the model sends the same arguments as JSON. ### 1. Fetch the page normally ```python fetch_webpage(url="https://example.com/dashboard") ``` On a page built by JavaScript, the HTML the server sends holds little more than a title and an empty `<div id="root">`, and the result says so with `likely_js_rendered`: ```json { "title": "Results dashboard", "extractor": "stdlib", "text": "Results dashboard", "total_chars": 17, "likely_js_rendered": true } ``` `likely_js_rendered` is true when the page has little text and several scripts, or an empty app root such as `<div id="root"></div>`. `fetch_webpage` takes milliseconds and runs no JavaScript, so it stays the first choice: most pages don't need a browser. ### 2. Fetch it again, rendered ```python fetch_webpage_rendered(url="https://example.com/dashboard") ``` Chromium loads the page, runs its JavaScript and returns the same JSON object as `fetch_webpage`, plus `"rendered": true`: ```json { "url": "https://example.com/dashboard", "final_url": "https://example.com/dashboard", "status": 200, "content_type": "text/html", "title": "Results dashboard", "format": "text", "extractor": "trafilatura", "text": "Quarterly results\nRevenue grew 12% this quarter.\nDetails", "truncated": false, "total_chars": 56, "offset": 0, "next_offset": null, "has_more": false, "byte_cap_reached": false, "rendered": true } ``` It takes seconds rather than milliseconds. Each call uses a fresh browser context with no cookies or stored sign-ins, which is closed afterwards, so it reads pages as a signed-out visitor. For a page behind a sign-in, drive the browser tools instead ([Portal Download to Text](portal_download_to_text.md)). ### 3. Ask for Markdown, links or the next part The arguments match `fetch_webpage`'s: ```python fetch_webpage_rendered(url="https://example.com/dashboard", format="markdown", include_assets=True) fetch_webpage_rendered(url="https://example.com/dashboard", offset=10000) # the next part ``` ```text # Quarterly results Revenue grew 12% this quarter. [Details](https://example.com/details) ``` With `include_assets=True`, the result also lists the page's `links` and `images`, with absolute URLs. ### 4. Give an agent both tools `fetch_webpage`'s description already tells the model to retry with `fetch_webpage_rendered` when `likely_js_rendered` is true and that tool is available. So attach the two tools, and nothing else from the browser server: ```python from doc_processing_toolkit import Agent from doc_processing_std_tools import browser_server, web_search_server agent = Agent( model="claude-sonnet-5", tools=[ {"type": "fastmcp", "server": web_search_server, "allowed_tools": ["fetch_webpage"]}, {"type": "fastmcp", "server": browser_server, "allowed_tools": ["fetch_webpage_rendered"]}, ], ) ``` Through the composed launcher, set `DPTTOOLS_ENABLE_BROWSER=true` in the server's `env`. That adds all 19 browser tools (75 tools in all); use `allowed_tools` on the session to show the model only the ones it needs. ### Internal sites `fetch_webpage_rendered` uses the browser's network policy, which refuses private and internal addresses: ```text Error: Navigation refused: localhost resolves to a private or internal address; the browser may not reach it (add it to DPTTOOLS_BROWSER_ALLOW_HOSTS if it is a trusted internal host). ``` To read a trusted internal host, add its exact hostname to **both** allow-lists: `DPTTOOLS_WEB_FETCH_ALLOW_HOSTS` for `fetch_webpage` and `DPTTOOLS_BROWSER_ALLOW_HOSTS` for `fetch_webpage_rendered`. `DPTTOOLS_BROWSER_ALLOWED_DOMAINS` and `DPTTOOLS_BROWSER_BLOCKED_DOMAINS` apply to the rendered fetch as well. ## What to validate - `fetch_webpage` on the shell returns `likely_js_rendered: true`, without the content. - `fetch_webpage_rendered` returns `rendered: true`, the same `title`, and the content in `text`. - `format="markdown"` keeps headings and links, and `include_assets=True` lists the links with absolute URLs. - Paging works the same as `fetch_webpage`'s: `offset`, `next_offset`, `has_more`. - No browser session or tab is left open after a call. - A private address is refused unless its hostname is in `DPTTOOLS_BROWSER_ALLOW_HOSTS`, and `file:` URLs are refused. ## Test sources - `tests/integration/test_fetch_rendered.py`: the shell and rendered fetches of the same page, on real Chromium (skipped when Chromium is missing). - `tests/unit/test_fetch_webpage_structured.py` (`TestJsHint`): when `likely_js_rendered` is set. ## Related pages - [Web server](../servers/web.md#fetch_webpage): the result's fields, and the fetch's network policy. - [Browser server](../servers/browser.md): the rest of the browser tools, and its settings. <!-- source: docs/recipes/portal_download_to_text.md --> # Recipe: Portal Download to Text ## Goal Have an agent sign in to a supplier portal, pick a monthly statement, download it with the [browser server](../servers/browser.md), and read its text with the [Documents](../servers/documents.md) tools. !!! warning "The agent acts on a live website" A browser agent can submit, buy and delete on real sites, and it reads untrusted page content. Read [Security](../guides/security.md#the-browser-and-desktop-servers) first, narrow the browser to the portal's domain, and decide how actions are approved (step 6). ## Prerequisites - The `browser` and `docs` extras, and Chromium: ```bash pip install "doc-processing-std-tools[browser,docs] @ git+https://github.com/rzoeb/doc-processing-std-tools.git@v0.6.1" playwright install chromium # in a container: playwright install --with-deps chromium ``` - The browser and document tools on the **same machine or container**, so a downloaded file's path opens with `read_pdf`. ## Steps Tool calls are written as Python calls; the model sends the same arguments as JSON. The results are from a run against a test portal, with the Tab Context footer left out. ### 1. Configure the browser for one portal ```bash DPTTOOLS_ENABLE_BROWSER=true # composed launcher only DPTTOOLS_BROWSER_ALLOWED_DOMAINS=portal.example.com DPTTOOLS_BROWSER_DOWNLOAD_DIR=/workspace/downloads ``` - `DPTTOOLS_BROWSER_ALLOWED_DOMAINS` keeps the browser on the portal and its subdomains, so a link or an instruction planted in a page cannot take it elsewhere. - `DPTTOOLS_BROWSER_DOWNLOAD_DIR` keeps downloads in a folder you choose. Each session saves into its own subfolder, and files are kept after the session closes. Without it, downloads go to a new temporary folder per session. ### 2. Decide how the agent signs in The model sends every argument of every tool call to the model provider, so a password the agent types has passed through the model, even though the tool's result shows it as `(hidden)`. Pick one of these: | Approach | How | When | |---|---|---| | **A person signs in once** (recommended) | `DPTTOOLS_BROWSER_MODE=persistent` with `DPTTOOLS_BROWSER_PROFILE_DIR` and `DPTTOOLS_BROWSER_HEADLESS=false`. A person signs in in the visible window, and the login stays in the profile folder | Two-factor sign-in, or a password the model should never see. See [Isolated vs Attached](../guides/isolated_vs_attached.md#isolated-persistent-a-saved-profile) | | The agent types the credentials | The default `launch` mode; your application puts the username and password in the prompt | A low-value test account only | With `DPTTOOLS_BROWSER_CONFIRM=deny`, password entry is refused whichever tool tries it: ```text Error: Refused by the confirmation gate (DPTTOOLS_BROWSER_CONFIRM=deny): enter a password is a consequential action. Ask the user to do it, or to approve it in the app. ``` So an unattended pipeline that uses `deny` needs the saved-profile approach. ### 3. Sign in ```python browser_navigate(url="https://portal.example.com/login") browser_read_page() ``` ```text Page: "Supplier portal - sign in" (https://portal.example.com/login) - textbox "Username" [ref_1] value="" - textbox "Password" [ref_2] value="" - button "Sign in" [ref_3] ``` The model acts on the refs: ```python browser_form_input(ref="ref_1", value="acme-accounts") # Set textbox 'Username' [ref_1] to 'acme-accounts'. browser_form_input(ref="ref_2", value="...") # Set textbox 'Password' [ref_2] to '(hidden)'. browser_left_click(ref="ref_3", intent="sign in") # Clicked button 'Sign in' [ref_3]. ``` With a saved profile, the agent is already signed in and starts at step 4. ### 4. Pick the statement and download it ```python browser_read_page() ``` ```text Page: "Statements" (https://portal.example.com/statements) - combobox "Month" [ref_4] selected="2026-08" - button "Download statement" [ref_5] ``` ```python browser_form_input(ref="ref_4", value="2026-09") # Set combobox 'Month' [ref_4] to '2026-09'. browser_left_click(ref="ref_5", intent="download the September statement") ``` ```text Clicked button 'Download statement' [ref_5]. Download started: statement-2026-09.pdf from https://portal.example.com/statement-2026-09.pdf ``` The download is reported once, on the first result after it finishes. A short wait is enough: ```python browser_wait(duration=1) ``` ```text Waited 1 seconds. Download completed: statement-2026-09.pdf saved to /workspace/downloads/session-27b1678d/statement-2026-09.pdf (1,049 bytes) ``` URL parameters that look like credentials (a `token=` in a signed download link, for example) are redacted in these lines. ### 5. Read the statement The path in the `Download completed` line opens directly with the document tools: ```python read_pdf(path="/workspace/downloads/session-27b1678d/statement-2026-09.pdf") ``` ```text --- Page 1 --- Supplier statement 2026-09 Opening balance 1,200.00 Closing balance 950.00 ``` If `read_pdf` reports a scanned PDF, continue with `ocr_pdf` ([OCR a Scanned PDF](ocr_a_scanned_pdf.md)). For an Excel or CSV download, use `read_excel` or `read_file`. ### 6. Put it together in an agent ```python from doc_processing_toolkit import Agent from doc_processing_toolkit.agent import HookDirective from doc_processing_std_tools import browser_server, document_processing_server from doc_processing_std_tools.browser import BROWSER_SAFETY_PROMPT ACTS = {"browser_left_click", "browser_form_input", "browser_type", "browser_key"} async def approve(event): if event.event_type == "before_tool_call" and event.metadata["tool_name"] in ACTS: if not await ask_the_user(event.metadata["tool_name"], event.metadata["arguments"]): return HookDirective(action="skip_tool", message="The user declined this action.") return None agent = Agent( model="claude-sonnet-5", instructions=( "You download monthly statements from the supplier portal and report their balances.\n\n" + BROWSER_SAFETY_PROMPT ), tools=[ browser_server, {"type": "fastmcp", "server": document_processing_server, "allowed_tools": ["get_file_overview", "read_pdf", "ocr_pdf"]}, ], hooks=approve, ) result = await agent.run("Download the September 2026 statement and give me its closing balance.") ``` `ask_the_user` stands for your application's approval step. A hook sees the tool and its arguments (`browser_left_click(ref="ref_5")`), not the button's name, so approve per tool or per domain, and log the results, which name each element acted on. Navigation (`browser_navigate`, `browser_new_tab`) is left out of `ACTS` on purpose: approving every page load would bury the actions that matter, and `DPTTOOLS_BROWSER_ALLOWED_DOMAINS` (step 1) already keeps the browser on the portal. Without that setting, add both tools to `ACTS`, and approve by the domain in `args["url"]`. For a reviewed, repeatable job, you can instead leave the hook out and rely on `DPTTOOLS_BROWSER_ALLOWED_DOMAINS` and `DPTTOOLS_BROWSER_CONFIRM=deny`. [Using with the Toolkit](../guides/with_the_toolkit.md#browser-actions) covers the options. ## What to validate - `browser_read_page` lists the sign-in fields and buttons with refs. - `browser_form_input` on a password field reports `(hidden)`, and with `DPTTOOLS_BROWSER_CONFIRM=deny` it is refused. - Each click result names the element clicked. - `Download completed` appears once, with a path and a byte count, and credential-like URL parameters are redacted. - `read_pdf` opens the downloaded path. - With `DPTTOOLS_BROWSER_ALLOWED_DOMAINS` set, navigating to another domain is refused. ## Test sources - `tests/integration/test_browser_opt_in.py`: `test_download_is_reported_with_a_path_the_document_tools_open` (download, redaction, `read_pdf`). - `tests/integration/test_browser_server.py`: `test_form_input_sets_fields`, `test_type_and_key_submit_a_form`, and the ref tests. - `tests/integration/test_browser_policy.py`: `test_allowed_domains_limit_navigation_and_subresources`, `test_deny_refuses_flagged_actions_only`. These run on real Chromium and are skipped when it is missing. ## Related pages - [Browser server](../servers/browser.md): every tool, the modes and the settings. - [Isolated vs Attached](../guides/isolated_vs_attached.md): `launch`, `persistent` and `cdp`. - [Security](../guides/security.md#the-confirmation-gate-at-a-glance): what the confirmation gate covers. <!-- source: docs/recipes/desktop_with_hooks.md --> # Recipe: Desktop Actions with Hooks ## Goal Let an agent drive a desktop application through the [computer server](../servers/computer.md), while your application shows a person each consequential action, with a screenshot, and lets them approve or decline it through a toolkit hook. ## Prerequisites - A Linux container with a virtual desktop and the applications the agent should use: `xvfb`, `xdotool`, a window manager such as `openbox`, and std-tools with the `computer` extra. The [Deployment](../guides/deployment.md#with-the-browser-and-a-virtual-desktop) page has a Dockerfile. - Your application, with doc-processing-toolkit, running outside the container or inside it. - A model that reads images: the agent works from screenshots. The default `virtual` backend gives each MCP session its own display inside the container, so the agent never touches your real screen. `host` mode, which drives the real screen, is covered at the end. ## Steps ### 1. Start the server where the desktop is Open a stdio session to the composed launcher inside the container, with the computer server enabled: ```python from mcp import ClientSession, StdioServerParameters from mcp.client.stdio import get_default_environment, stdio_client params = StdioServerParameters( command="docker", args=["run", "-i", "--rm", "-e", "DPTTOOLS_ENABLE_COMPUTER=true", "my-agent-image", "python", "-m", "doc_processing_std_tools"], env=get_default_environment(), ) ``` `docker run -i` connects the container's standard input and output to the session. Pass std-tools settings with more `-e` flags; the container has none of your application's environment. If your application runs in the same container, start `python -m doc_processing_std_tools` directly instead. ### 2. Open the application on the desktop The first computer tool call starts the session's display, and every result names it: ```python computer_screenshot() # Screenshot of display :99 (1440x900). (followed by the image) ``` Your application then starts the program on that display with `execute_command`, through the same session, **before the agent starts**. Run it in the background with its output redirected, so that the command returns at once: ```python execute_command(command="DISPLAY=:99 nohup libreoffice --calc /workspace/report.xlsx > /dev/null 2>&1 &") ``` Step 4 shows these two calls in code. Don't give `execute_command` to the agent: it runs any shell command in the container, and an approval step built on screenshots can't judge a command. If the agent must start programs itself, add `execute_command` to the hook's tools in step 3, and show the person `args["command"]` instead of a screenshot. ### 3. Write the approval hook A hook sees every `before_tool_call` event and can decline the call, so the tool never runs. For the desktop, the tools that act are the clicks, drags, mouse-down, typing and keys. `computer_screenshot`, `computer_zoom`, `computer_cursor_position` and `computer_wait` only look or wait. The hook sees coordinates, not a button's name: `computer_left_click(x=595, y=492)`. To show the person what that means, it takes a screenshot of its own **through the same session**, so of the same display, and shows it with the coordinates and the model's stated intent: ```python from doc_processing_toolkit.agent import HookDirective ACTS = { "computer_left_click", "computer_right_click", "computer_middle_click", "computer_double_click", "computer_triple_click", "computer_left_click_drag", "computer_left_mouse_down", "computer_type", "computer_key", "computer_hold_key", } def make_approval_hook(session): async def approve(event): if event.event_type != "before_tool_call" or event.metadata["tool_name"] not in ACTS: return None name, args = event.metadata["tool_name"], event.metadata["arguments"] shot = await session.call_tool("computer_screenshot", {}) image = next(c for c in shot.content if c.type == "image") # base64 PNG in image.data if await ask_the_user(name, args, image): return None return HookDirective(action="skip_tool", message="The user declined this action.") return approve ``` `ask_the_user` stands for your application's approval step: draw a marker at `args["x"]`, `args["y"]` on the image, show `args.get("intent")`, and wait for an answer. The hook's screenshot goes to your application only, never to the model. ### 4. Run the agent ```python import re from doc_processing_toolkit import Agent async with stdio_client(params) as (read, write), ClientSession(read, write) as session: await session.initialize() # Your application opens the program (step 2); the agent never gets execute_command. shot = await session.call_tool("computer_screenshot", {}) display = re.search(r"display (:\d+)", shot.content[0].text).group(1) await session.call_tool("execute_command", {"command": f"DISPLAY={display} nohup libreoffice --calc /workspace/report.xlsx > /dev/null 2>&1 &"}) agent = Agent( model="claude-sonnet-5", instructions=( "You operate a Linux desktop with the computer tools. Take a screenshot before acting, " "and give every action an intent that says what it is for. If the user declines an " "action, take a new screenshot and decide again." ), tools=[{"session": session, "name": "std-tools", "allowed_tools": ["computer_screenshot", "computer_zoom", "computer_left_click", "computer_type", "computer_key", "computer_wait"]}], hooks=make_approval_hook(session), ) result = await agent.run("report.xlsx is open on the screen. Put 2026-09 in cell B4.") ``` The session belongs to the event loop that opened it, so run the agent with `await agent.run()` inside the `async with` block, not with `run_sync`. `allowed_tools` keeps the model's tool list to what the job needs, and every tool on it that acts is in the hook's list: the launcher with the computer server has 73 tools. In a test run against a dialog that asked "Delete all 1,204 customer records?" with **Keep** and **Delete** buttons, the hook was called once, with the model's click and intent: ```text computer_left_click {'x': 595, 'y': 492, 'intent': 'Click Keep'} ``` - **Declined:** the click did not run. The model reported that it had not clicked, said the dialog was still open, and asked how to continue. - **Approved:** the click ran, and the dialog recorded **Keep**. ### 5. Add a backstop in the server Your hook is the approval step. These settings limit what can happen if a call gets past it: | Setting | Effect | |---|---| | `DPTTOOLS_COMPUTER_CONFIRM=deny` | Refuses the key chords that close, quit or delete: `alt+f4`, `ctrl+w`, `ctrl+q`, `super+w`, `super+q`, `delete`, `shift+delete`, also with extra keys and as a click's `modifiers`. It cannot judge what a click does | | `DPTTOOLS_COMPUTER_VNC_PORT` | Serves each display over VNC on `127.0.0.1` in the container (needs `x11vnc`), so a person can watch or take over through a tunnel you control | | `DPTTOOLS_COMPUTER_MAX_SESSIONS` | How many desktops can be open at once (default 2) | `DPTTOOLS_COMPUTER_CONFIRM=elicit` would ask through MCP elicitation, which the toolkit does not support, so with the toolkit it refuses those chords. ### On the user's real screen (`host` mode) `DPTTOOLS_COMPUTER_BACKEND=host` drives the real screen, mouse and keyboard of the machine std-tools runs on, so it does not belong in a container. The same hook works, and matters more there: - Get the person's consent before the agent starts, and tell them that every screenshot, with whatever is on the screen, goes to the model provider. - The safeguards are always on: moving the mouse to the top-left corner, or pressing `ctrl+alt+shift+k`, stops the agent; and an action is refused while the person is using the mouse or keyboard. - Only one session at a time. See [Isolated vs Attached](../guides/isolated_vs_attached.md#attached-the-users-real-screen-host) and [Security](../guides/security.md#the-desktop-server). ## What to validate - The first computer call names the display, and an application started on it appears in the next screenshot. - The hook runs for each acting tool, with its coordinates and `intent`, and not for screenshots or waits. - A declined call does not run: the screen is unchanged, and the model is told it was declined. - An approved click lands where the model aimed. - With `DPTTOOLS_COMPUTER_CONFIRM=deny`, `computer_key(text="ctrl+w")` is refused. - Two sessions get two displays, and neither sees the other's windows. ## Test sources - `tests/integration/test_computer_virtual.py`: clicks, typing, keys, drags, zoom and one display per session, on a real Xvfb display (run it in the Linux image with `tests/integration/run_in_docker.py`). - `tests/unit/test_computer_unit.py`: the confirmation gate, coordinate scaling, the host-mode safeguards and the tool count, with the backends replaced by a recording fake. - The hook mechanics (`before_tool_call`, `skip_tool`) are tested in doc-processing-toolkit; see its [Hooks and steering](https://doc-processing-toolkit.vercel.app/concepts/hooks/) page. ## Related pages - [Computer server](../servers/computer.md): every tool, the backends and the settings. - [Using with the Toolkit](../guides/with_the_toolkit.md#desktop-actions): approving desktop and browser actions. <!-- source: docs/recipes/skills_setup.md --> # Recipe: Skills Setup ## Goal Create a skills folder, write a skill with a reference file and a script, check that the [skills server](../servers/skills.md) reads it cleanly, and attach the server so the agent knows the skill exists. ## Prerequisites - std-tools with no extra: PyYAML is a base dependency. The skill below also uses Documents tools, which need the `docs` extra. - For the agent step, doc-processing-toolkit. Toolkit 0.5.2 and earlier need the pre-built session shown in step 5. ## Steps ### 1. Choose the skills folder The server reads `DPTTOOLS_SKILLS_DIR`, by default `./.agent_skills` relative to the server's working directory. Set it to an absolute path so it doesn't depend on where the server starts: ```bash DPTTOOLS_SKILLS_DIR=/workspace/.agent_skills ``` In a container, mount the folder read-only unless the agent should create skills ([Deployment](../guides/deployment.md#running-it)). ### 2. Write the skill Each skill is a folder whose name is the skill's name: 1–64 lowercase letters, digits and single hyphens. It holds a `SKILL.md`, plus any files the instructions refer to: ```text .agent_skills/ invoice-extraction/ SKILL.md references/ fields.md scripts/ check_totals.py ``` `SKILL.md` starts with YAML frontmatter. The `description` is what the agent sees before it loads the skill, so say what the skill does **and when to use it**: ```markdown --- name: invoice-extraction description: Extracts the header fields and line items of a supplier invoice into JSON. Use when asked to read, check or summarise an invoice. metadata: owner: finance-team --- # Invoice extraction 1. Run `get_file_overview` on the invoice. If it has no text, use `ocr_pdf`. 2. Read the field names in `references/fields.md`. 3. Put the line items in a JSON file and run `scripts/check_totals.py <file>`. 4. Report the fields as JSON, and say whether the totals add up. ``` `scripts/check_totals.py` is ordinary Python. It runs with the server's interpreter, in the script's own folder: ```python import json, sys data = json.load(open(sys.argv[1], encoding="utf-8")) lines = sum(l["amount"] for l in data["lines"]) print(json.dumps({"lines_total": lines, "stated_total": data["total"], "matches": abs(lines - data["total"]) < 0.005})) ``` An agent can also start a skill itself: `create_skill(skill_name="invoice-extraction", description="...")` writes a template `SKILL.md` with the frontmatter and a `TODO` body, which the agent then fills in with a file tool. ### 3. Check what the server sees Call `list_skills` once, by hand or from a test, before giving the folder to an agent: ```json { "skills": [ {"name": "draft-notes", "description": "Notes: things to remember", "path": "/workspace/.agent_skills/draft-notes/SKILL.md", "directory": "/workspace/.agent_skills/draft-notes"}, {"name": "invoice-extraction", "description": "Extracts the header fields and line items of a supplier invoice into JSON. Use when asked to read, check or summarise an invoice.", "path": "/workspace/.agent_skills/invoice-extraction/SKILL.md", "directory": "/workspace/.agent_skills/invoice-extraction", "metadata": {"owner": "finance-team"}} ], "skills_dir": "/workspace/.agent_skills", "count": 2, "diagnostics": [ {"skill": "draft-notes", "level": "warning", "code": "yaml_lenient_retry", "message": "description value contained unquoted colons; lenient quoting applied"} ] } ``` `diagnostics` lists what was wrong. A `warning` still loads the skill, as here, where a colon in an unquoted `description` was quoted for you. An `error`, such as a missing `description`, skips the skill. The toolkit logs diagnostics and never shows them to the model, so fix them at this step. The [skills server](../servers/skills.md#frontmatter-rules) page lists every rule. ### 4. Walk through the skill as the agent will ```python load_skill(skill_name="invoice-extraction") ``` ```text <skill_content name="invoice-extraction" directory="/workspace/.agent_skills/invoice-extraction"> --- name: invoice-extraction ... 4. Report the fields as JSON, and say whether the totals add up. <skill_resources> <file>references/fields.md</file> <file>scripts/check_totals.py</file> </skill_resources> Skill directory: /workspace/.agent_skills/invoice-extraction Relative paths in this skill resolve against the skill directory. </skill_content> ``` ```python load_skill_resource(skill_name="invoice-extraction", resource_path="references/fields.md") execute_skill_script(skill_name="invoice-extraction", script_path="scripts/check_totals.py", args="/workspace/invoices/inv-0042.json") ``` ```json {"stdout": "{\"lines_total\": 150.5, \"stated_total\": 150.5, \"matches\": true}\n", "stderr": "", "exit_code": 0} ``` `args` is split with POSIX shell rules, so quote a path that contains spaces. A script that runs longer than `DPTTOOLS_SANDBOX_TIMEOUT` (30 seconds) returns `exit_code: -1`. ### 5. Attach the server to an agent The toolkit reads the skills server automatically: it calls `list_skills` and puts each skill's name, description and path into the system prompt ([Skills auto-injection](https://doc-processing-toolkit.vercel.app/concepts/skills/)). With toolkit 0.5.2 and earlier this works **only through a session you open yourself**: ```python import os import sys from mcp import ClientSession, StdioServerParameters from mcp.client.stdio import get_default_environment, stdio_client from doc_processing_toolkit import Agent os.environ["DPTTOOLS_SKILLS_DIR"] = "/workspace/.agent_skills" # the toolkit checks it too env = get_default_environment() env["DPTTOOLS_SKILLS_DIR"] = os.environ["DPTTOOLS_SKILLS_DIR"] env["LLMWHISPERER_API_KEY"] = os.environ.get("LLMWHISPERER_API_KEY", "") # the skill may OCR params = StdioServerParameters( command=sys.executable, args=["-m", "doc_processing_std_tools"], env=env ) async with stdio_client(params) as (read, write), ClientSession(read, write) as session: await session.initialize() agent = Agent( model="claude-sonnet-5", tools=[{"session": session, "name": "std-tools-skills"}], ) result = await agent.run("Check the totals on invoices/inv-0042.pdf.") ``` - The session's `name` contains `skill`, so the toolkit treats it as a skills server. - The server gets `DPTTOOLS_SKILLS_DIR` in `env`, and your application has it set too, pointing at a folder that exists. - The session runs the composed launcher, so the agent has the tools the skill calls for (`get_file_overview`, `ocr_pdf`, a file tool to write the JSON) as well as the skills tools. To give an agent only the skills tools, run `-m doc_processing_std_tools.skills` instead. `Agent(tools=[skills_server])` and `mcp_command` configs return an empty catalog with toolkit 0.5.2 and earlier, and the agent never hears of the skills ([Compatibility](../guides/compatibility.md#known-toolkit-issue-skills-auto-injection-toolkit-052-and-earlier)). ## What to validate - `list_skills` returns every skill with an absolute `path`, and `diagnostics` only for what was wrong. - `load_skill` returns the `SKILL.md` with a `<skill_resources>` list of the skill's files. - `load_skill_resource` and `execute_skill_script` work with paths relative to the skill folder, and a path that leaves the skills folder is refused. - A skill name that is not lowercase letters, digits and hyphens is refused. - Through the pre-built session, the toolkit finds the skills (the agent's system prompt lists them). ## Test sources - `tests/unit/test_skills.py`: frontmatter parsing and diagnostics (`TestParseFrontmatter`, `TestBuildSkillRecord`), names (`TestValidateSkillName`), and each tool. - `tests/integration/test_skills_mcp.py`: create, list and load through the server. - `tests/integration/test_toolkit_compat.py`: `test_skills_catalog_through_a_prebuilt_stdio_session` passes with the toolkit installed. `test_skills_catalog_in_process` and `test_skills_catalog_through_the_toolkit_command_config` are expected to fail on toolkit 0.5.2. ## Related pages - [Skills server](../servers/skills.md): the tools, the payload and the frontmatter rules. - [Agent Skills specification](https://agentskills.io/specification). <!-- source: docs/recipes/launcher_with_other_mcp_clients.md --> # Recipe: The Launcher with Other MCP Clients ## Goal Register the composed launcher, `python -m doc_processing_std_tools`, with MCP clients other than the toolkit: a desktop assistant, an IDE, or your own code using an MCP SDK. Pass it the configuration it needs, and check the connection. ## Prerequisites - std-tools installed with the extras you need, in its **own virtual environment**, so the client runs a known interpreter: ```bash python -m venv /opt/std-tools /opt/std-tools/bin/pip install "doc-processing-std-tools[all] @ git+https://github.com/rzoeb/doc-processing-std-tools.git@v0.6.1" ``` On Windows the interpreter is `C:\opt\std-tools\Scripts\python.exe`. - Any software the tools use, installed where the server runs: LibreOffice for Office conversion, and Chromium (`playwright install chromium`) if you enable the browser. ## Steps ### 1. Decide what the server needs A client starts a stdio server with **only the environment you give it**. Most clients pass a short allow-list such as `PATH` and `HOME`, and nothing from your shell. So write every setting into the client's configuration: | Setting | Why | |---|---| | `DPTTOOLS_MEMORY_DIR`, `DPTTOOLS_SKILLS_DIR` | Their defaults are relative (`./agent_memory`, `./.agent_skills`), and a client chooses the working directory. Use absolute paths | | `DPTTOOLS_WEB_SEARCH_PROVIDER`, `DPTTOOLS_WEB_SEARCH_API_KEY` | `web_search` returns an error without them | | `LLMWHISPERER_API_KEY` | For `ocr_pdf` and `ocr_image` | | `DPTTOOLS_ENABLE_BROWSER`, `DPTTOOLS_ENABLE_COMPUTER` | Only if you want the optional servers | The file tools also resolve a relative path against the server's working directory. Give absolute paths in prompts, or use a client that lets you set the working directory (`cwd`). [Configuration](../guides/configuration.md) lists every variable. ### 2. Register it with the client **JSON configuration with `mcpServers`** (Claude Desktop's `claude_desktop_config.json`, and many other clients): ```json { "mcpServers": { "std-tools": { "command": "/opt/std-tools/bin/python", "args": ["-m", "doc_processing_std_tools"], "env": { "DPTTOOLS_MEMORY_DIR": "/home/me/agent/memory", "DPTTOOLS_SKILLS_DIR": "/home/me/agent/skills", "DPTTOOLS_WEB_SEARCH_PROVIDER": "tavily", "DPTTOOLS_WEB_SEARCH_API_KEY": "tvly-xxx" } } } } ``` In JSON on Windows, write paths with forward slashes (`C:/opt/std-tools/Scripts/python.exe`) or doubled backslashes. **Claude Code**, from the command line: ```bash claude mcp add std-tools \ -e DPTTOOLS_MEMORY_DIR=/home/me/agent/memory \ -e DPTTOOLS_WEB_SEARCH_PROVIDER=tavily -e DPTTOOLS_WEB_SEARCH_API_KEY=tvly-xxx \ -- /opt/std-tools/bin/python -m doc_processing_std_tools ``` **VS Code** (`.vscode/mcp.json`) can prompt for a key instead of storing it in the file, and can set the working directory: ```json { "inputs": [ {"type": "promptString", "id": "tavily-key", "description": "Tavily API key", "password": true} ], "servers": { "std-tools": { "type": "stdio", "command": "/opt/std-tools/bin/python", "args": ["-m", "doc_processing_std_tools"], "cwd": "${workspaceFolder}", "env": { "DPTTOOLS_WEB_SEARCH_PROVIDER": "tavily", "DPTTOOLS_WEB_SEARCH_API_KEY": "${input:tavily-key}" } } } } ``` Each client's own documentation has the details: where its file lives, and how it handles secrets. ### 3. Or serve it over HTTP For a client that connects to a URL, or several clients sharing one server, run the launcher with streamable HTTP. It reads its configuration from its **own** environment, since no client passes one: ```bash export DPTTOOLS_MEMORY_DIR=/home/me/agent/memory python -m doc_processing_std_tools --transport http --port 8000 ``` The endpoint is `http://127.0.0.1:8000/mcp`. For Claude Code: `claude mcp add --transport http std-tools http://127.0.0.1:8000/mcp`. !!! warning "The HTTP transport has no authentication" It binds to `127.0.0.1` by default. Anyone who can reach the port can run every tool, including `execute_command`. Bind another interface (`--host 0.0.0.0`) only behind an authenticating proxy or a network policy ([Architecture](../architecture.md#out-of-process-the-composed-launcher)). ### 4. Check the connection The launcher checks its settings before serving. A bad value stops it with a message naming the variable, which the client shows as a server that failed to start: ```text doc-processing-std-tools failed to start: ConfigError: DPTTOOLS_SANDBOX_TIMEOUT must be an integer (got 'soon'). ``` Once it is connected, the client should list **56 tools**, or 75 with the browser, 73 with the computer server and 92 with both. A short script with the MCP Python SDK checks the same thing without a client: ```python import asyncio from mcp import ClientSession, StdioServerParameters from mcp.client.stdio import get_default_environment, stdio_client async def main(): env = get_default_environment() env["DPTTOOLS_MEMORY_DIR"] = "/home/me/agent/memory" params = StdioServerParameters( command="/opt/std-tools/bin/python", args=["-m", "doc_processing_std_tools"], env=env ) async with stdio_client(params) as (read, write), ClientSession(read, write) as session: await session.initialize() print(len((await session.list_tools()).tools)) # 56 result = await session.call_tool("write_memory", {"key": "report-folder", "value": "Reports go to /finance."}) print(result.content[0].text) # Memory key 'report-folder' saved. asyncio.run(main()) ``` For HTTP, use `streamablehttp_client("http://127.0.0.1:8000/mcp")` from `mcp.client.streamable_http` in the same way. The MCP Inspector (`npx @modelcontextprotocol/inspector`) does the same check interactively. ### 5. Keep the tool list small 56 tool descriptions cost tokens on every turn, and some clients limit how many tools they show. To serve fewer tools, run one server on its own: ```bash python -m doc_processing_std_tools.document_processing # the 13 Documents tools python -m doc_processing_std_tools.file_system ``` A server run this way skips the launcher's start-up check, so a bad setting shows up as an `Error:` from the first tool that reads it. Many clients can also hide tools you don't use. ## What to validate - The client lists 56 tools (more with the optional servers), with names unchanged and not prefixed. - A memory written through one connection is in `DPTTOOLS_MEMORY_DIR`, at the absolute path you set. - `web_search` works; without the provider settings in the client's `env`, it returns an `Error:` naming them. - A bad setting stops the launcher with a message naming the variable. - Over HTTP, the server answers at `/mcp` on `127.0.0.1` only. ## Test sources - `tests/integration/test_multi_server_launcher.py`: composition, the 56-tool count, no duplicate names, a call through each server, settings passed through. - `tests/unit/test_launcher_offload.py`: blocking tools run on worker threads in the launcher. - `tests/integration/test_toolkit_compat.py`: a stdio session started with `env`, as in step 4. ## Related pages - [Quickstart](../quickstart.md#run-the-composed-launcher): the launcher's options. - [Architecture](../architecture.md#out-of-process-the-composed-launcher): how the launcher composes the servers. - [Using with the Toolkit](../guides/with_the_toolkit.md#passing-configuration-to-a-stdio-server): the same `env` rule, for toolkit agents. <!-- source: docs/reference/tools.md --> # Tool Reference <!-- Generated by scripts/gen_docs_tools.py from the FastMCP tool registry. Do not edit by hand. --> Every tool the servers register, with the description and parameter schema an MCP client receives — the text the model reads when it chooses a tool. Each description is formatted for reading; the collapsed block under it holds the exact text the model receives, with its common indentation removed. For how to use each server, see its page under **Servers**. | Server | MCP name | Tools | |---|---|---| | [file_system](#file_system-server) | `file-system-tools` | 12 | | [web_search](#web_search-server) | `web-search-tools` | 4 | | [memory](#memory-server) | `memory-tools` | 11 | | [document_processing](#document_processing-server) | `document-processing-tools` | 13 | | [code_execution](#code_execution-server) | `code-execution-tools` | 2 | | [screenshot](#screenshot-server) | `screenshot-tools` | 2 | | [image_tools](#image_tools-server) | `image-tools` | 7 | | [skills](#skills-server) | `skills-tools` | 5 | | **Composed launcher, default** | `doc-processing-std-tools` | **56** | | [browser](#browser-server) (optional) | `browser-tools` | 19 (`core`), 28 (`full`), plus up to 5 opt-in | | **Composed launcher with the browser** | `doc-processing-std-tools` | **75** (`core`) or **84** (`full`), plus opt-ins | | [computer](#computer-server) (optional) | `computer-tools` | 17 | | **Composed launcher with the computer server** | `doc-processing-std-tools` | **73** | | **Composed launcher with both** | `doc-processing-std-tools` | **92** (`core`) or **101** (`full`), plus opt-ins | ## file_system server `file-system-tools` — see the [server page](../servers/file_system.md). ### `get_directory_structure` Return a markdown-formatted directory tree. Walks the directory recursively up to `max_depth` levels and formats the result as an indented tree using box-drawing characters. **Args** - `path`: Root directory path to list. - `max_depth`: Maximum depth of recursion (default 3). - `include_hidden`: Whether to include hidden files/directories (names starting with '.'). **Returns** A markdown-formatted directory tree string. ??? note "Exact text the model receives" ```text Return a markdown-formatted directory tree. Walks the directory recursively up to ``max_depth`` levels and formats the result as an indented tree using box-drawing characters. Args: path: Root directory path to list. max_depth: Maximum depth of recursion (default 3). include_hidden: Whether to include hidden files/directories (names starting with '.'). Returns: A markdown-formatted directory tree string. ``` | Parameter | Type | Required | Default | |---|---|---|---| | `path` | string | yes | — | | `max_depth` | integer | no | `3` | | `include_hidden` | boolean | no | `false` | ??? note "JSON schema" ```json { "properties": { "path": { "title": "Path", "type": "string" }, "max_depth": { "default": 3, "title": "Max Depth", "type": "integer" }, "include_hidden": { "default": false, "title": "Include Hidden", "type": "boolean" } }, "required": [ "path" ], "title": "get_directory_structureArguments", "type": "object" } ``` ### `read_file` Read file contents, optionally within a line range. **Args** - `path`: Path to the file to read. - `offset`: 1-based starting line number (optional). Lines before this are skipped. - `limit`: Maximum number of lines to return from the offset (optional). - `encoding`: File encoding (default utf-8). **Returns** The file contents as a string (possibly truncated). ??? note "Exact text the model receives" ```text Read file contents, optionally within a line range. Args: path: Path to the file to read. offset: 1-based starting line number (optional). Lines before this are skipped. limit: Maximum number of lines to return from the offset (optional). encoding: File encoding (default utf-8). Returns: The file contents as a string (possibly truncated). ``` | Parameter | Type | Required | Default | |---|---|---|---| | `path` | string | yes | — | | `offset` | integer or null | no | `null` | | `limit` | integer or null | no | `null` | | `encoding` | string | no | `"utf-8"` | ??? note "JSON schema" ```json { "properties": { "path": { "title": "Path", "type": "string" }, "offset": { "anyOf": [ { "type": "integer" }, { "type": "null" } ], "default": null, "title": "Offset" }, "limit": { "anyOf": [ { "type": "integer" }, { "type": "null" } ], "default": null, "title": "Limit" }, "encoding": { "default": "utf-8", "title": "Encoding", "type": "string" } }, "required": [ "path" ], "title": "read_fileArguments", "type": "object" } ``` ### `write_file` Write content to a file. Creates the file if it doesn't exist, overwrites if it does. **Args** - `path`: Path to the file to write. - `content`: Content to write. - `create_dirs`: Whether to create parent directories if they don't exist. - `encoding`: File encoding (default utf-8). **Returns** Confirmation message with the file path and bytes written. ??? note "Exact text the model receives" ```text Write content to a file. Creates the file if it doesn't exist, overwrites if it does. Args: path: Path to the file to write. content: Content to write. create_dirs: Whether to create parent directories if they don't exist. encoding: File encoding (default utf-8). Returns: Confirmation message with the file path and bytes written. ``` | Parameter | Type | Required | Default | |---|---|---|---| | `path` | string | yes | — | | `content` | string | yes | — | | `create_dirs` | boolean | no | `false` | | `encoding` | string | no | `"utf-8"` | ??? note "JSON schema" ```json { "properties": { "path": { "title": "Path", "type": "string" }, "content": { "title": "Content", "type": "string" }, "create_dirs": { "default": false, "title": "Create Dirs", "type": "boolean" }, "encoding": { "default": "utf-8", "title": "Encoding", "type": "string" } }, "required": [ "path", "content" ], "title": "write_fileArguments", "type": "object" } ``` ### `create_file` Create a new file. Fails if the file already exists. **Args** - `path`: Path to the file to create. - `content`: Content to write. - `create_dirs`: Whether to create parent directories if they don't exist. **Returns** Confirmation message with the file path. ??? note "Exact text the model receives" ```text Create a new file. Fails if the file already exists. Args: path: Path to the file to create. content: Content to write. create_dirs: Whether to create parent directories if they don't exist. Returns: Confirmation message with the file path. ``` | Parameter | Type | Required | Default | |---|---|---|---| | `path` | string | yes | — | | `content` | string | yes | — | | `create_dirs` | boolean | no | `false` | ??? note "JSON schema" ```json { "properties": { "path": { "title": "Path", "type": "string" }, "content": { "title": "Content", "type": "string" }, "create_dirs": { "default": false, "title": "Create Dirs", "type": "boolean" } }, "required": [ "path", "content" ], "title": "create_fileArguments", "type": "object" } ``` ### `edit_file` Replace exact text in a file (str_replace pattern). Finds the exact `old_string` in the file and replaces it with `new_string`. The old_string must appear exactly once in the file. **Args** - `path`: Path to the file to edit. - `old_string`: The exact text to find (must appear exactly once). - `new_string`: The replacement text. **Returns** Confirmation message. ??? note "Exact text the model receives" ```text Replace exact text in a file (str_replace pattern). Finds the exact ``old_string`` in the file and replaces it with ``new_string``. The old_string must appear exactly once in the file. Args: path: Path to the file to edit. old_string: The exact text to find (must appear exactly once). new_string: The replacement text. Returns: Confirmation message. ``` | Parameter | Type | Required | Default | |---|---|---|---| | `path` | string | yes | — | | `old_string` | string | yes | — | | `new_string` | string | yes | — | ??? note "JSON schema" ```json { "properties": { "path": { "title": "Path", "type": "string" }, "old_string": { "title": "Old String", "type": "string" }, "new_string": { "title": "New String", "type": "string" } }, "required": [ "path", "old_string", "new_string" ], "title": "edit_fileArguments", "type": "object" } ``` ### `apply_patch` Apply a unified diff (`git diff` format) to a file. Parses headers (`---`, `+++`, `@@`) and applies the hunks to the target file. **Rejects patches if context lines don't match exactly** (e.g., the file has drifted since the diff was generated). For safe single-string substitutions use `edit_file`; this tool is preferred for multi-line / structural edits where context lines are stable. **Args** - `path`: Path to the file to patch. - `patch`: Unified diff content to apply. **Returns** Confirmation message. ??? note "Exact text the model receives" ```text Apply a unified diff (``git diff`` format) to a file. Parses headers (``---``, ``+++``, ``@@``) and applies the hunks to the target file. **Rejects patches if context lines don't match exactly** (e.g., the file has drifted since the diff was generated). For safe single-string substitutions use ``edit_file``; this tool is preferred for multi-line / structural edits where context lines are stable. Args: path: Path to the file to patch. patch: Unified diff content to apply. Returns: Confirmation message. ``` | Parameter | Type | Required | Default | |---|---|---|---| | `path` | string | yes | — | | `patch` | string | yes | — | ??? note "JSON schema" ```json { "properties": { "path": { "title": "Path", "type": "string" }, "patch": { "title": "Patch", "type": "string" } }, "required": [ "path", "patch" ], "title": "apply_patchArguments", "type": "object" } ``` ### `delete_file` Delete a file. **Args** - `path`: Path to the file to delete. **Returns** Confirmation message. ??? note "Exact text the model receives" ```text Delete a file. Args: path: Path to the file to delete. Returns: Confirmation message. ``` | Parameter | Type | Required | Default | |---|---|---|---| | `path` | string | yes | — | ??? note "JSON schema" ```json { "properties": { "path": { "title": "Path", "type": "string" } }, "required": [ "path" ], "title": "delete_fileArguments", "type": "object" } ``` ### `move_file` Move or rename a file. **Args** - `source`: Current path of the file. - `destination`: New path for the file. **Returns** Confirmation message. ??? note "Exact text the model receives" ```text Move or rename a file. Args: source: Current path of the file. destination: New path for the file. Returns: Confirmation message. ``` | Parameter | Type | Required | Default | |---|---|---|---| | `source` | string | yes | — | | `destination` | string | yes | — | ??? note "JSON schema" ```json { "properties": { "source": { "title": "Source", "type": "string" }, "destination": { "title": "Destination", "type": "string" } }, "required": [ "source", "destination" ], "title": "move_fileArguments", "type": "object" } ``` ### `copy_file` Copy a file, preserving metadata. **Args** - `source`: Path of the file to copy. - `destination`: Path for the copy. **Returns** Confirmation message. ??? note "Exact text the model receives" ```text Copy a file, preserving metadata. Args: source: Path of the file to copy. destination: Path for the copy. Returns: Confirmation message. ``` | Parameter | Type | Required | Default | |---|---|---|---| | `source` | string | yes | — | | `destination` | string | yes | — | ??? note "JSON schema" ```json { "properties": { "source": { "title": "Source", "type": "string" }, "destination": { "title": "Destination", "type": "string" } }, "required": [ "source", "destination" ], "title": "copy_fileArguments", "type": "object" } ``` ### `get_file_metadata` Return file metadata as a JSON string. **Args** - `path`: Path to the file or directory. **Returns** JSON string with keys: path, size_bytes, size_human, modified, created, is_file, is_dir, extension. ??? note "Exact text the model receives" ```text Return file metadata as a JSON string. Args: path: Path to the file or directory. Returns: JSON string with keys: path, size_bytes, size_human, modified, created, is_file, is_dir, extension. ``` | Parameter | Type | Required | Default | |---|---|---|---| | `path` | string | yes | — | ??? note "JSON schema" ```json { "properties": { "path": { "title": "Path", "type": "string" } }, "required": [ "path" ], "title": "get_file_metadataArguments", "type": "object" } ``` ### `search_in_files` Search for text or regex across files in a directory. Walks the directory and searches within each matching file for lines containing the query. For PDF files, checks for co-located `.txt` files (e.g., `document.pdf.txt`) that contain extracted text. **Args** - `path`: Root directory to search in. - `query`: Search string or regex pattern. - `file_pattern`: Glob pattern to filter files (default '\*'). - `is_regex`: Whether the query is a regular expression. - `max_results`: Maximum number of matching lines to return (default 50). **Returns** JSON string with a list of matches, each containing file, line, and content. ??? note "Exact text the model receives" ```text Search for text or regex across files in a directory. Walks the directory and searches within each matching file for lines containing the query. For PDF files, checks for co-located ``.txt`` files (e.g., ``document.pdf.txt``) that contain extracted text. Args: path: Root directory to search in. query: Search string or regex pattern. file_pattern: Glob pattern to filter files (default '*'). is_regex: Whether the query is a regular expression. max_results: Maximum number of matching lines to return (default 50). Returns: JSON string with a list of matches, each containing file, line, and content. ``` | Parameter | Type | Required | Default | |---|---|---|---| | `path` | string | yes | — | | `query` | string | yes | — | | `file_pattern` | string | no | `"*"` | | `is_regex` | boolean | no | `false` | | `max_results` | integer | no | `50` | ??? note "JSON schema" ```json { "properties": { "path": { "title": "Path", "type": "string" }, "query": { "title": "Query", "type": "string" }, "file_pattern": { "default": "*", "title": "File Pattern", "type": "string" }, "is_regex": { "default": false, "title": "Is Regex", "type": "boolean" }, "max_results": { "default": 50, "title": "Max Results", "type": "integer" } }, "required": [ "path", "query" ], "title": "search_in_filesArguments", "type": "object" } ``` ### `list_directory` List files and directories with basic metadata. Returns entries sorted with directories first, then files, alphabetically within each group. **Args** - `path`: Path to the directory to list. - `include_hidden`: Whether to include hidden files/directories (names starting with '.'). **Returns** JSON list of entries, each with name, type ('file' or 'directory'), and size (in bytes, for files). ??? note "Exact text the model receives" ```text List files and directories with basic metadata. Returns entries sorted with directories first, then files, alphabetically within each group. Args: path: Path to the directory to list. include_hidden: Whether to include hidden files/directories (names starting with '.'). Returns: JSON list of entries, each with name, type ('file' or 'directory'), and size (in bytes, for files). ``` | Parameter | Type | Required | Default | |---|---|---|---| | `path` | string | yes | — | | `include_hidden` | boolean | no | `false` | ??? note "JSON schema" ```json { "properties": { "path": { "title": "Path", "type": "string" }, "include_hidden": { "default": false, "title": "Include Hidden", "type": "boolean" } }, "required": [ "path" ], "title": "list_directoryArguments", "type": "object" } ``` ## web_search server `web-search-tools` — see the [server page](../servers/web.md). ### `web_search` Search the web using the configured search provider. The search provider is configured via the DPTTOOLS_WEB_SEARCH_PROVIDER and DPTTOOLS_WEB_SEARCH_API_KEY environment variables. Supported providers: tavily, serper, brave. **Args** - `query`: The search query. - `num_results`: Number of results to return (default 5). - `search_type`: Type of search — 'general' or 'news' (default 'general'). - `search_depth`: Tavily search depth — 'basic' (default), 'advanced', 'fast' or 'ultra-fast'. 'advanced' costs twice as much. Ignored by serper and brave. **Returns** JSON string with `provider`, `query`, `results` (title, URL, snippet for each result) and `usage` (the provider's reported usage when it gives one; otherwise marked `estimated` or `provider_reported: false`). ??? note "Exact text the model receives" ```text Search the web using the configured search provider. The search provider is configured via the DPTTOOLS_WEB_SEARCH_PROVIDER and DPTTOOLS_WEB_SEARCH_API_KEY environment variables. Supported providers: tavily, serper, brave. Args: query: The search query. num_results: Number of results to return (default 5). search_type: Type of search — 'general' or 'news' (default 'general'). search_depth: Tavily search depth — 'basic' (default), 'advanced', 'fast' or 'ultra-fast'. 'advanced' costs twice as much. Ignored by serper and brave. Returns: JSON string with ``provider``, ``query``, ``results`` (title, URL, snippet for each result) and ``usage`` (the provider's reported usage when it gives one; otherwise marked ``estimated`` or ``provider_reported: false``). ``` | Parameter | Type | Required | Default | |---|---|---|---| | `query` | string | yes | — | | `num_results` | integer | no | `5` | | `search_type` | string | no | `"general"` | | `search_depth` | string | no | `"basic"` | ??? note "JSON schema" ```json { "properties": { "query": { "title": "Query", "type": "string" }, "num_results": { "default": 5, "title": "Num Results", "type": "integer" }, "search_type": { "default": "general", "title": "Search Type", "type": "string" }, "search_depth": { "default": "basic", "title": "Search Depth", "type": "string" } }, "required": [ "query" ], "title": "web_searchArguments", "type": "object" } ``` ### `fetch_webpage` Fetch a web page or other text resource and return its content. Only HTTP/HTTPS URLs are allowed. Private/internal IP ranges and localhost are rejected for SSRF protection, on every redirect. Reads text only: HTML, plain text, Markdown, JSON, XML and CSV. Any other content type (images, PDFs, archives) is refused before it is downloaded; for an image, use `fetch_image` if it is available. Returns a JSON object by default: `{url, final_url, status, content_type, title, format, extractor, text, truncated, total_chars, offset, next_offset, has_more, byte_cap_reached, likely_js_rendered}`. `format` chooses how an HTML page is returned: `"text"` (readable text, the default), `"markdown"` (headings, lists, links and tables kept) or `"html"` (the source). When the optional trafilatura library is installed, text and Markdown hold the page's main content without navigation and footers (`extractor: "trafilatura"`); otherwise a built-in extractor keeps the whole page (`extractor: "stdlib"`). Non-HTML text (JSON, plain text, CSV) is returned as-is. With `include_assets=True` the object also has `images` (up to 30: `{url, alt, width, height, source}` from `<img>`, `og:image` and icons), `links` (up to 100: `{url, text}`) and `assets_truncated`. URLs are absolute; nothing is downloaded. To look at an image, pass its URL to `fetch_image`. - `text` holds at most `max_length` characters starting at `offset`. When `has_more` is true, call again with `offset=next_offset` to read the next part. - `likely_js_rendered` is true when the page looks like it is built by JavaScript (little text, many scripts, or an empty app root). Retry with `fetch_webpage_rendered` if that tool is available. - `byte_cap_reached` is true when the response was larger than the download limit and was cut; the text ends early. **Args** - `url`: The URL to fetch (must be HTTP or HTTPS). - `extract_text`: Legacy switch: False returns the HTML source, the same as `format="html"`. - `max_length`: Maximum characters of text to return (default 10000). - `offset`: Character offset to start from, for reading a long page in parts (default 0). - `structured`: Return the JSON object (default True). False returns only the text, with a truncation notice, as older versions did, always using the built-in extractor. - `format`: `"text"` (default), `"markdown"` or `"html"`. - `include_assets`: Add the page's image and link references (default False). Ignored when `structured=False`. **Returns** The JSON object (or plain text when `structured=False`). On failure, a string starting with `"Error: "`. ??? note "Exact text the model receives" ```text Fetch a web page or other text resource and return its content. Only HTTP/HTTPS URLs are allowed. Private/internal IP ranges and localhost are rejected for SSRF protection, on every redirect. Reads text only: HTML, plain text, Markdown, JSON, XML and CSV. Any other content type (images, PDFs, archives) is refused before it is downloaded; for an image, use ``fetch_image`` if it is available. Returns a JSON object by default: ``{url, final_url, status, content_type, title, format, extractor, text, truncated, total_chars, offset, next_offset, has_more, byte_cap_reached, likely_js_rendered}``. ``format`` chooses how an HTML page is returned: ``"text"`` (readable text, the default), ``"markdown"`` (headings, lists, links and tables kept) or ``"html"`` (the source). When the optional trafilatura library is installed, text and Markdown hold the page's main content without navigation and footers (``extractor: "trafilatura"``); otherwise a built-in extractor keeps the whole page (``extractor: "stdlib"``). Non-HTML text (JSON, plain text, CSV) is returned as-is. With ``include_assets=True`` the object also has ``images`` (up to 30: ``{url, alt, width, height, source}`` from ``<img>``, ``og:image`` and icons), ``links`` (up to 100: ``{url, text}``) and ``assets_truncated``. URLs are absolute; nothing is downloaded. To look at an image, pass its URL to ``fetch_image``. - ``text`` holds at most ``max_length`` characters starting at ``offset``. When ``has_more`` is true, call again with ``offset=next_offset`` to read the next part. - ``likely_js_rendered`` is true when the page looks like it is built by JavaScript (little text, many scripts, or an empty app root). Retry with ``fetch_webpage_rendered`` if that tool is available. - ``byte_cap_reached`` is true when the response was larger than the download limit and was cut; the text ends early. Args: url: The URL to fetch (must be HTTP or HTTPS). extract_text: Legacy switch: False returns the HTML source, the same as ``format="html"``. max_length: Maximum characters of text to return (default 10000). offset: Character offset to start from, for reading a long page in parts (default 0). structured: Return the JSON object (default True). False returns only the text, with a truncation notice, as older versions did, always using the built-in extractor. format: ``"text"`` (default), ``"markdown"`` or ``"html"``. include_assets: Add the page's image and link references (default False). Ignored when ``structured=False``. Returns: The JSON object (or plain text when ``structured=False``). On failure, a string starting with ``"Error: "``. ``` | Parameter | Type | Required | Default | |---|---|---|---| | `url` | string | yes | — | | `extract_text` | boolean | no | `true` | | `max_length` | integer | no | `10000` | | `offset` | integer | no | `0` | | `structured` | boolean | no | `true` | | `format` | string | no | `"text"` | | `include_assets` | boolean | no | `false` | ??? note "JSON schema" ```json { "properties": { "url": { "title": "Url", "type": "string" }, "extract_text": { "default": true, "title": "Extract Text", "type": "boolean" }, "max_length": { "default": 10000, "title": "Max Length", "type": "integer" }, "offset": { "default": 0, "title": "Offset", "type": "integer" }, "structured": { "default": true, "title": "Structured", "type": "boolean" }, "format": { "default": "text", "enum": [ "text", "markdown", "html" ], "title": "Format", "type": "string" }, "include_assets": { "default": false, "title": "Include Assets", "type": "boolean" } }, "required": [ "url" ], "title": "fetch_webpageArguments", "type": "object" } ``` ### `fetch_image` Download an image from a URL and show it to the model. Use this to look at an image found on a web page (`fetch_webpage` with `include_assets=True` lists a page's images). Accepts PNG, JPEG, GIF and WebP. The same SSRF protection as `fetch_webpage` applies: HTTP/HTTPS only, private and internal addresses refused, every redirect re-checked. Images larger than `DPTTOOLS_FETCH_IMAGE_MAX_BYTES` (10 MB by default) are refused. Images larger than `DPTTOOLS_IMAGE_MAX_EDGE_PX` (1568 px on the long edge) or `DPTTOOLS_IMAGE_MAX_MB` (3 MB) are scaled down before they are returned. Requires `httpx` and `Pillow` (the `[web]` extra). **Args** - `url`: The image URL (must be HTTP or HTTPS). **Returns** A short caption (source URL, format, size) followed by the image as an MCP `ImageContent` block. On failure, a string starting with `"Error: "`. ??? note "Exact text the model receives" ```text Download an image from a URL and show it to the model. Use this to look at an image found on a web page (``fetch_webpage`` with ``include_assets=True`` lists a page's images). Accepts PNG, JPEG, GIF and WebP. The same SSRF protection as ``fetch_webpage`` applies: HTTP/HTTPS only, private and internal addresses refused, every redirect re-checked. Images larger than ``DPTTOOLS_FETCH_IMAGE_MAX_BYTES`` (10 MB by default) are refused. Images larger than ``DPTTOOLS_IMAGE_MAX_EDGE_PX`` (1568 px on the long edge) or ``DPTTOOLS_IMAGE_MAX_MB`` (3 MB) are scaled down before they are returned. Requires ``httpx`` and ``Pillow`` (the ``[web]`` extra). Args: url: The image URL (must be HTTP or HTTPS). Returns: A short caption (source URL, format, size) followed by the image as an MCP ``ImageContent`` block. On failure, a string starting with ``"Error: "``. ``` | Parameter | Type | Required | Default | |---|---|---|---| | `url` | string | yes | — | ??? note "JSON schema" ```json { "properties": { "url": { "title": "Url", "type": "string" } }, "required": [ "url" ], "title": "fetch_imageArguments", "type": "object" } ``` ### `format_search_results` Format raw search results with inline citation markers and a references section. Accepts raw search results (JSON with URL, title, content per result) and formats them with inline citation markers ([1], [2], etc.) and a references section at the end. The output of `web_search` can be passed in unchanged. **Args** - `results`: JSON string — either the object `web_search` returns (with a `results` array) or a plain array of results, each with url, title, and content/snippet fields. **Returns** Formatted text with inline citation markers and a references section. ??? note "Exact text the model receives" ```text Format raw search results with inline citation markers and a references section. Accepts raw search results (JSON with URL, title, content per result) and formats them with inline citation markers ([1], [2], etc.) and a references section at the end. The output of ``web_search`` can be passed in unchanged. Args: results: JSON string — either the object ``web_search`` returns (with a ``results`` array) or a plain array of results, each with url, title, and content/snippet fields. Returns: Formatted text with inline citation markers and a references section. ``` | Parameter | Type | Required | Default | |---|---|---|---| | `results` | string | yes | — | ??? note "JSON schema" ```json { "properties": { "results": { "title": "Results", "type": "string" } }, "required": [ "results" ], "title": "format_search_resultsArguments", "type": "object" } ``` ## memory server `memory-tools` — see the [server page](../servers/memory.md). ### `write_memory` Store a value in long-term memory. The key is used as the filename (with `.json` extension) inside the memory directory. Overwrites any existing value for the same key. **Args** - `key`: The memory key (used as filename). - `value`: The value to store. **Returns** Confirmation message. ??? note "Exact text the model receives" ```text Store a value in long-term memory. The key is used as the filename (with ``.json`` extension) inside the memory directory. Overwrites any existing value for the same key. Args: key: The memory key (used as filename). value: The value to store. Returns: Confirmation message. ``` | Parameter | Type | Required | Default | |---|---|---|---| | `key` | string | yes | — | | `value` | string | yes | — | ??? note "JSON schema" ```json { "properties": { "key": { "title": "Key", "type": "string" }, "value": { "title": "Value", "type": "string" } }, "required": [ "key", "value" ], "title": "write_memoryArguments", "type": "object" } ``` ### `read_memory` Retrieve a value from long-term memory by key. **Args** - `key`: The memory key to look up. **Returns** The stored value, or an error message if not found. ??? note "Exact text the model receives" ```text Retrieve a value from long-term memory by key. Args: key: The memory key to look up. Returns: The stored value, or an error message if not found. ``` | Parameter | Type | Required | Default | |---|---|---|---| | `key` | string | yes | — | ??? note "JSON schema" ```json { "properties": { "key": { "title": "Key", "type": "string" } }, "required": [ "key" ], "title": "read_memoryArguments", "type": "object" } ``` ### `search_memory` Search memory for matching keys and values (case-insensitive substring). Scans all `.json` memory files and returns entries where either the key or value contains the query string. **Args** - `query`: Search string to match against keys and values. **Returns** JSON list of matching memory entries. ??? note "Exact text the model receives" ```text Search memory for matching keys and values (case-insensitive substring). Scans all ``.json`` memory files and returns entries where either the key or value contains the query string. Args: query: Search string to match against keys and values. Returns: JSON list of matching memory entries. ``` | Parameter | Type | Required | Default | |---|---|---|---| | `query` | string | yes | — | ??? note "JSON schema" ```json { "properties": { "query": { "title": "Query", "type": "string" } }, "required": [ "query" ], "title": "search_memoryArguments", "type": "object" } ``` ### `list_memory_keys` List all stored memory keys. **Returns** JSON list of all memory keys. ??? note "Exact text the model receives" ```text List all stored memory keys. Returns: JSON list of all memory keys. ``` No parameters. ??? note "JSON schema" ```json { "properties": {}, "title": "list_memory_keysArguments", "type": "object" } ``` ### `delete_memory` Delete a memory entry. **Args** - `key`: The memory key to delete. **Returns** Confirmation message. ??? note "Exact text the model receives" ```text Delete a memory entry. Args: key: The memory key to delete. Returns: Confirmation message. ``` | Parameter | Type | Required | Default | |---|---|---|---| | `key` | string | yes | — | ??? note "JSON schema" ```json { "properties": { "key": { "title": "Key", "type": "string" } }, "required": [ "key" ], "title": "delete_memoryArguments", "type": "object" } ``` ### `memory_view` View a memory directory listing or file contents with line numbers. If the path points to a directory, lists its contents (like `ls`). If the path points to a file, shows its contents with 1-indexed line numbers. Use `view_range` to show only a specific range of lines. **Args** - `path`: Relative path within the memory directory. Use `/` or `""` for the root of the memory directory. - `view_range`: Optional `[start_line, end_line]` (1-indexed) to view only those lines. **Returns** Directory listing or file contents with line numbers. ??? note "Exact text the model receives" ```text View a memory directory listing or file contents with line numbers. If the path points to a directory, lists its contents (like ``ls``). If the path points to a file, shows its contents with 1-indexed line numbers. Use ``view_range`` to show only a specific range of lines. Args: path: Relative path within the memory directory. Use ``/`` or ``""`` for the root of the memory directory. view_range: Optional ``[start_line, end_line]`` (1-indexed) to view only those lines. Returns: Directory listing or file contents with line numbers. ``` | Parameter | Type | Required | Default | |---|---|---|---| | `path` | string | yes | — | | `view_range` | array of integer or null | no | `null` | ??? note "JSON schema" ```json { "properties": { "path": { "title": "Path", "type": "string" }, "view_range": { "anyOf": [ { "items": { "type": "integer" }, "type": "array" }, { "type": "null" } ], "default": null, "title": "View Range" } }, "required": [ "path" ], "title": "memory_viewArguments", "type": "object" } ``` ### `memory_create` Create a new memory file. Fails if the file already exists. **Args** - `path`: Relative path within the memory directory. - `file_text`: Content for the new file. **Returns** Confirmation message. ??? note "Exact text the model receives" ```text Create a new memory file. Fails if the file already exists. Args: path: Relative path within the memory directory. file_text: Content for the new file. Returns: Confirmation message. ``` | Parameter | Type | Required | Default | |---|---|---|---| | `path` | string | yes | — | | `file_text` | string | yes | — | ??? note "JSON schema" ```json { "properties": { "path": { "title": "Path", "type": "string" }, "file_text": { "title": "File Text", "type": "string" } }, "required": [ "path", "file_text" ], "title": "memory_createArguments", "type": "object" } ``` ### `memory_str_replace` Replace text in a memory file. The old_str must appear exactly once. **Args** - `path`: Relative path within the memory directory. - `old_str`: Exact text to find (must appear exactly once). - `new_str`: Replacement text. **Returns** Confirmation message. ??? note "Exact text the model receives" ```text Replace text in a memory file. The old_str must appear exactly once. Args: path: Relative path within the memory directory. old_str: Exact text to find (must appear exactly once). new_str: Replacement text. Returns: Confirmation message. ``` | Parameter | Type | Required | Default | |---|---|---|---| | `path` | string | yes | — | | `old_str` | string | yes | — | | `new_str` | string | yes | — | ??? note "JSON schema" ```json { "properties": { "path": { "title": "Path", "type": "string" }, "old_str": { "title": "Old Str", "type": "string" }, "new_str": { "title": "New Str", "type": "string" } }, "required": [ "path", "old_str", "new_str" ], "title": "memory_str_replaceArguments", "type": "object" } ``` ### `memory_insert` Insert text at a specific line in a memory file (0-based line number). Line 0 inserts before the first line. A line number equal to the file's line count appends to the end. **Args** - `path`: Relative path within the memory directory. - `insert_line`: 0-based line number to insert at. - `insert_text`: Text to insert. **Returns** Confirmation message. ??? note "Exact text the model receives" ```text Insert text at a specific line in a memory file (0-based line number). Line 0 inserts before the first line. A line number equal to the file's line count appends to the end. Args: path: Relative path within the memory directory. insert_line: 0-based line number to insert at. insert_text: Text to insert. Returns: Confirmation message. ``` | Parameter | Type | Required | Default | |---|---|---|---| | `path` | string | yes | — | | `insert_line` | integer | yes | — | | `insert_text` | string | yes | — | ??? note "JSON schema" ```json { "properties": { "path": { "title": "Path", "type": "string" }, "insert_line": { "title": "Insert Line", "type": "integer" }, "insert_text": { "title": "Insert Text", "type": "string" } }, "required": [ "path", "insert_line", "insert_text" ], "title": "memory_insertArguments", "type": "object" } ``` ### `memory_delete` Delete a memory file or directory (and all its contents). **Args** - `path`: Relative path within the memory directory. **Returns** Confirmation message. ??? note "Exact text the model receives" ```text Delete a memory file or directory (and all its contents). Args: path: Relative path within the memory directory. Returns: Confirmation message. ``` | Parameter | Type | Required | Default | |---|---|---|---| | `path` | string | yes | — | ??? note "JSON schema" ```json { "properties": { "path": { "title": "Path", "type": "string" } }, "required": [ "path" ], "title": "memory_deleteArguments", "type": "object" } ``` ### `memory_rename` Rename or move a memory file within the memory directory. **Args** - `old_path`: Current relative path within the memory directory. - `new_path`: New relative path within the memory directory. **Returns** Confirmation message. ??? note "Exact text the model receives" ```text Rename or move a memory file within the memory directory. Args: old_path: Current relative path within the memory directory. new_path: New relative path within the memory directory. Returns: Confirmation message. ``` | Parameter | Type | Required | Default | |---|---|---|---| | `old_path` | string | yes | — | | `new_path` | string | yes | — | ??? note "JSON schema" ```json { "properties": { "old_path": { "title": "Old Path", "type": "string" }, "new_path": { "title": "New Path", "type": "string" } }, "required": [ "old_path", "new_path" ], "title": "memory_renameArguments", "type": "object" } ``` ## document_processing server `document-processing-tools` — see the [server page](../servers/documents.md). ### `read_pdf` Extract text (and optionally tables) from a **text-based** PDF. For scanned / image PDFs without selectable text, use `ocr_pdf` instead (requires `LLMWHISPERER_API_KEY`). Requires the `pdfplumber` library. Install the `[docs]` extra. **Args** - `path`: Path to the PDF file. - `pages`: Which pages to read — 'all', '1-5', or '1,3,7'. - `extract_tables`: Whether to extract tables as well. **Returns** The extracted text content (and tables if requested). ??? note "Exact text the model receives" ```text Extract text (and optionally tables) from a **text-based** PDF. For scanned / image PDFs without selectable text, use ``ocr_pdf`` instead (requires ``LLMWHISPERER_API_KEY``). Requires the ``pdfplumber`` library. Install the ``[docs]`` extra. Args: path: Path to the PDF file. pages: Which pages to read — 'all', '1-5', or '1,3,7'. extract_tables: Whether to extract tables as well. Returns: The extracted text content (and tables if requested). ``` | Parameter | Type | Required | Default | |---|---|---|---| | `path` | string | yes | — | | `pages` | string | no | `"all"` | | `extract_tables` | boolean | no | `false` | ??? note "JSON schema" ```json { "properties": { "path": { "title": "Path", "type": "string" }, "pages": { "default": "all", "title": "Pages", "type": "string" }, "extract_tables": { "default": false, "title": "Extract Tables", "type": "boolean" } }, "required": [ "path" ], "title": "read_pdfArguments", "type": "object" } ``` ### `ocr_pdf` OCR a scanned / image-only **PDF** via the LLM Whisperer API. Use this for PDFs whose text is **not** selectable (scans, photos of documents). For text-based PDFs prefer `read_pdf` — it is faster and needs no API key. For a single image file (PNG/JPEG/…), use `ocr_image`. Requires the `LLMWHISPERER_API_KEY` environment variable and the `[ocr]` extra. **Args** - `path`: Path to the PDF file. - `pages`: Which pages to OCR — `'all'` (default), a range like `'1-5'`, a list like `'1,3,7'`, mixed (`'1-3,7'`), or an open-ended range like `'21-'` (page 21 to the end). Narrowing pages reduces latency and OCR cost. **Returns** Page-delimited OCR text (`--- Page N ---` headers), or an actionable `Error: ...` message. ??? note "Exact text the model receives" ```text OCR a scanned / image-only **PDF** via the LLM Whisperer API. Use this for PDFs whose text is **not** selectable (scans, photos of documents). For text-based PDFs prefer ``read_pdf`` — it is faster and needs no API key. For a single image file (PNG/JPEG/…), use ``ocr_image``. Requires the ``LLMWHISPERER_API_KEY`` environment variable and the ``[ocr]`` extra. Args: path: Path to the PDF file. pages: Which pages to OCR — ``'all'`` (default), a range like ``'1-5'``, a list like ``'1,3,7'``, mixed (``'1-3,7'``), or an open-ended range like ``'21-'`` (page 21 to the end). Narrowing pages reduces latency and OCR cost. Returns: Page-delimited OCR text (``--- Page N ---`` headers), or an actionable ``Error: ...`` message. ``` | Parameter | Type | Required | Default | |---|---|---|---| | `path` | string | yes | — | | `pages` | string | no | `"all"` | ??? note "JSON schema" ```json { "properties": { "path": { "title": "Path", "type": "string" }, "pages": { "default": "all", "title": "Pages", "type": "string" } }, "required": [ "path" ], "title": "ocr_pdfArguments", "type": "object" } ``` ### `ocr_image` OCR a single raster **image** (PNG/JPEG/WEBP/TIFF/BMP) via LLM Whisperer. Use this for a photo or screenshot of text. For multi-page documents use `ocr_pdf`. For text already selectable in a PDF prefer `read_pdf`. Requires the `LLMWHISPERER_API_KEY` environment variable and the `[ocr]` extra. **Args** - `path`: Path to the image file. A single image is a single page, so there is no `pages` parameter. **Returns** The OCR-extracted text, or an actionable `Error: ...` message. ??? note "Exact text the model receives" ```text OCR a single raster **image** (PNG/JPEG/WEBP/TIFF/BMP) via LLM Whisperer. Use this for a photo or screenshot of text. For multi-page documents use ``ocr_pdf``. For text already selectable in a PDF prefer ``read_pdf``. Requires the ``LLMWHISPERER_API_KEY`` environment variable and the ``[ocr]`` extra. Args: path: Path to the image file. A single image is a single page, so there is no ``pages`` parameter. Returns: The OCR-extracted text, or an actionable ``Error: ...`` message. ``` | Parameter | Type | Required | Default | |---|---|---|---| | `path` | string | yes | — | ??? note "JSON schema" ```json { "properties": { "path": { "title": "Path", "type": "string" } }, "required": [ "path" ], "title": "ocr_imageArguments", "type": "object" } ``` ### `read_excel` Read Excel file content: cell values and sheet metadata. Cells come back as the values saved in the file (the last calculated result of a formula, not the formula itself), as strings. The first row is returned as `headers`. The result also lists merged ranges and says whether the sheet has images or charts. Requires the `openpyxl` library. Install the `[docs]` extra. **Args** - `path`: Path to the Excel file. - `sheet`: Specific sheet name to read (default: the sheet that was active when the file was saved). - `range`: Cell range to read, e.g. 'A1:D10' (default: all data). **Returns** JSON string with sheet data. ??? note "Exact text the model receives" ```text Read Excel file content: cell values and sheet metadata. Cells come back as the values saved in the file (the last calculated result of a formula, not the formula itself), as strings. The first row is returned as ``headers``. The result also lists merged ranges and says whether the sheet has images or charts. Requires the ``openpyxl`` library. Install the ``[docs]`` extra. Args: path: Path to the Excel file. sheet: Specific sheet name to read (default: the sheet that was active when the file was saved). range: Cell range to read, e.g. 'A1:D10' (default: all data). Returns: JSON string with sheet data. ``` | Parameter | Type | Required | Default | |---|---|---|---| | `path` | string | yes | — | | `sheet` | string or null | no | `null` | | `range` | string or null | no | `null` | ??? note "JSON schema" ```json { "properties": { "path": { "title": "Path", "type": "string" }, "sheet": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Sheet" }, "range": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Range" } }, "required": [ "path" ], "title": "read_excelArguments", "type": "object" } ``` ### `read_word` Read Word document content (text, headings, tables). Requires the `python-docx` library. Install the `[docs]` extra. **Args** - `path`: Path to the Word document. **Returns** The document content as text. ??? note "Exact text the model receives" ```text Read Word document content (text, headings, tables). Requires the ``python-docx`` library. Install the ``[docs]`` extra. Args: path: Path to the Word document. Returns: The document content as text. ``` | Parameter | Type | Required | Default | |---|---|---|---| | `path` | string | yes | — | ??? note "JSON schema" ```json { "properties": { "path": { "title": "Path", "type": "string" } }, "required": [ "path" ], "title": "read_wordArguments", "type": "object" } ``` ### `read_powerpoint` Read text, slide structure, and speaker notes from a `.pptx` file. For Word documents use `read_word`; for Excel use `read_excel`; for PDFs use `read_pdf` or `ocr_pdf`. Requires the `python-pptx` library. Install the `[docs]` extra. **Args** - `path`: Path to the PowerPoint file. **Returns** JSON string with slide data. ??? note "Exact text the model receives" ```text Read text, slide structure, and speaker notes from a ``.pptx`` file. For Word documents use ``read_word``; for Excel use ``read_excel``; for PDFs use ``read_pdf`` or ``ocr_pdf``. Requires the ``python-pptx`` library. Install the ``[docs]`` extra. Args: path: Path to the PowerPoint file. Returns: JSON string with slide data. ``` | Parameter | Type | Required | Default | |---|---|---|---| | `path` | string | yes | — | ??? note "JSON schema" ```json { "properties": { "path": { "title": "Path", "type": "string" } }, "required": [ "path" ], "title": "read_powerpointArguments", "type": "object" } ``` ### `read_image` Read an image into the model's vision context. The result is delivered to the LLM as an MCP `ImageContent` block — the model SEES the image and can transcribe its text, describe its contents, or compare it against another image. Use this tool when the agent needs visual access to a PNG / JPEG / WEBP / GIF / BMP / TIFF file inside the sandbox. For dimensions / file-size / format metadata WITHOUT the image itself (i.e. text-only inspection), use `get_image_metadata`. Images larger than `DPTTOOLS_IMAGE_MAX_EDGE_PX` (1568 px on the long edge by default) or `DPTTOOLS_IMAGE_MAX_MB` (3 MB) are scaled down before they are returned, so every model provider accepts them. Requires the `Pillow` library. Install the `[docs]` extra. **Args** - `path`: Path to the image file. **Returns** An MCP ImageContent block (the LLM consumes this as vision input). On error, returns a plain string starting with `"Error: "` so the agent can surface the failure. ??? note "Exact text the model receives" ```text Read an image into the model's vision context. The result is delivered to the LLM as an MCP ``ImageContent`` block — the model SEES the image and can transcribe its text, describe its contents, or compare it against another image. Use this tool when the agent needs visual access to a PNG / JPEG / WEBP / GIF / BMP / TIFF file inside the sandbox. For dimensions / file-size / format metadata WITHOUT the image itself (i.e. text-only inspection), use ``get_image_metadata``. Images larger than ``DPTTOOLS_IMAGE_MAX_EDGE_PX`` (1568 px on the long edge by default) or ``DPTTOOLS_IMAGE_MAX_MB`` (3 MB) are scaled down before they are returned, so every model provider accepts them. Requires the ``Pillow`` library. Install the ``[docs]`` extra. Args: path: Path to the image file. Returns: An MCP ImageContent block (the LLM consumes this as vision input). On error, returns a plain string starting with ``"Error: "`` so the agent can surface the failure. ``` | Parameter | Type | Required | Default | |---|---|---|---| | `path` | string | yes | — | ??? note "JSON schema" ```json { "properties": { "path": { "title": "Path", "type": "string" } }, "required": [ "path" ], "title": "read_imageArguments", "type": "object" } ``` ### `get_image_metadata` Read image metadata (format / dimensions / file size) WITHOUT delivering the image bytes to the model. Use this when the agent only needs to inspect the file's shape (e.g. to decide whether to crop / resize before reading). Returns a JSON string. For visual inspection (the agent needs to SEE the image), use `read_image` instead. Requires the `Pillow` library. Install the `[docs]` extra. **Args** - `path`: Path to the image file. **Returns** JSON string with keys: `format`, `width`, `height`, `mode`, `file_size`, `file_size_bytes`. ??? note "Exact text the model receives" ```text Read image metadata (format / dimensions / file size) WITHOUT delivering the image bytes to the model. Use this when the agent only needs to inspect the file's shape (e.g. to decide whether to crop / resize before reading). Returns a JSON string. For visual inspection (the agent needs to SEE the image), use ``read_image`` instead. Requires the ``Pillow`` library. Install the ``[docs]`` extra. Args: path: Path to the image file. Returns: JSON string with keys: ``format``, ``width``, ``height``, ``mode``, ``file_size``, ``file_size_bytes``. ``` | Parameter | Type | Required | Default | |---|---|---|---| | `path` | string | yes | — | ??? note "JSON schema" ```json { "properties": { "path": { "title": "Path", "type": "string" } }, "required": [ "path" ], "title": "get_image_metadataArguments", "type": "object" } ``` ### `render_document_page` Render a single page of a PDF / DOCX / XLSX / PPTX into the model's vision context. PDF files are rendered directly via pymupdf; DOCX / XLSX / PPTX are first converted to PDF via LibreOffice headless, then rendered. The result is delivered to the LLM as an MCP `ImageContent` block — the model SEES the rendered page directly. **Rejects image files (.png, .jpg, .gif, .bmp, .tiff, .webp).** For raw image inspection use `read_image`. For image thumbnails use `image_resize`. Images larger than `DPTTOOLS_IMAGE_MAX_EDGE_PX` (1568 px on the long edge by default) or `DPTTOOLS_IMAGE_MAX_MB` (3 MB) are scaled down before they are returned, so every model provider accepts them. Requires the `pymupdf` library. Install the `[docs]` extra. Office files need LibreOffice in the environment (on Debian/Ubuntu: `apt-get install libreoffice-core libreoffice-writer libreoffice-calc libreoffice-impress`), or `DPTTOOLS_SOFFICE_BIN` pointing at `soffice`. **Args** - `path`: Path to the document file. - `page`: Page number to render (1-indexed, default 1). - `scale`: Render scale factor (default 2.0 for high-DPI). **Returns** An MCP ImageContent block on success; an error string starting with `"Error: "` on failure. ??? note "Exact text the model receives" ```text Render a single page of a PDF / DOCX / XLSX / PPTX into the model's vision context. PDF files are rendered directly via pymupdf; DOCX / XLSX / PPTX are first converted to PDF via LibreOffice headless, then rendered. The result is delivered to the LLM as an MCP ``ImageContent`` block — the model SEES the rendered page directly. **Rejects image files (.png, .jpg, .gif, .bmp, .tiff, .webp).** For raw image inspection use ``read_image``. For image thumbnails use ``image_resize``. Images larger than ``DPTTOOLS_IMAGE_MAX_EDGE_PX`` (1568 px on the long edge by default) or ``DPTTOOLS_IMAGE_MAX_MB`` (3 MB) are scaled down before they are returned, so every model provider accepts them. Requires the ``pymupdf`` library. Install the ``[docs]`` extra. Office files need LibreOffice in the environment (on Debian/Ubuntu: ``apt-get install libreoffice-core libreoffice-writer libreoffice-calc libreoffice-impress``), or ``DPTTOOLS_SOFFICE_BIN`` pointing at ``soffice``. Args: path: Path to the document file. page: Page number to render (1-indexed, default 1). scale: Render scale factor (default 2.0 for high-DPI). Returns: An MCP ImageContent block on success; an error string starting with ``"Error: "`` on failure. ``` | Parameter | Type | Required | Default | |---|---|---|---| | `path` | string | yes | — | | `page` | integer | no | `1` | | `scale` | number | no | `2.0` | ??? note "JSON schema" ```json { "properties": { "path": { "title": "Path", "type": "string" }, "page": { "default": 1, "title": "Page", "type": "integer" }, "scale": { "default": 2.0, "title": "Scale", "type": "number" } }, "required": [ "path" ], "title": "render_document_pageArguments", "type": "object" } ``` ### `render_document_thumbnail` Render low-resolution thumbnails for the first N pages of a PDF / DOCX / XLSX / PPTX into the model's vision context. Uses the same conversion pipeline as `render_document_page`. PDF files are rendered directly via pymupdf. DOCX / XLSX / PPTX are first converted to PDF via LibreOffice headless. Each thumbnail is delivered to the LLM as a separate MCP `ImageContent` block — the model SEES every page in sequence and can use them for at-a-glance inspection (e.g. picking the page to operate on). **Rejects image files.** For thumbnailing an image, use `image_resize` instead. Images larger than `DPTTOOLS_IMAGE_MAX_EDGE_PX` (1568 px on the long edge by default) or `DPTTOOLS_IMAGE_MAX_MB` (3 MB) are scaled down before they are returned, so every model provider accepts them. Requires the `pymupdf` library. Install the `[docs]` extra. Office files need LibreOffice in the environment (on Debian/Ubuntu: `apt-get install libreoffice-core libreoffice-writer libreoffice-calc libreoffice-impress`), or `DPTTOOLS_SOFFICE_BIN` pointing at `soffice`. **Args** - `path`: Path to the document file. - `max_pages`: Maximum number of pages to render thumbnails for (default 5). **Returns** A list of MCP ImageContent blocks (one per rendered page). On error, returns a plain string starting with `"Error: "`. ??? note "Exact text the model receives" ```text Render low-resolution thumbnails for the first N pages of a PDF / DOCX / XLSX / PPTX into the model's vision context. Uses the same conversion pipeline as ``render_document_page``. PDF files are rendered directly via pymupdf. DOCX / XLSX / PPTX are first converted to PDF via LibreOffice headless. Each thumbnail is delivered to the LLM as a separate MCP ``ImageContent`` block — the model SEES every page in sequence and can use them for at-a-glance inspection (e.g. picking the page to operate on). **Rejects image files.** For thumbnailing an image, use ``image_resize`` instead. Images larger than ``DPTTOOLS_IMAGE_MAX_EDGE_PX`` (1568 px on the long edge by default) or ``DPTTOOLS_IMAGE_MAX_MB`` (3 MB) are scaled down before they are returned, so every model provider accepts them. Requires the ``pymupdf`` library. Install the ``[docs]`` extra. Office files need LibreOffice in the environment (on Debian/Ubuntu: ``apt-get install libreoffice-core libreoffice-writer libreoffice-calc libreoffice-impress``), or ``DPTTOOLS_SOFFICE_BIN`` pointing at ``soffice``. Args: path: Path to the document file. max_pages: Maximum number of pages to render thumbnails for (default 5). Returns: A list of MCP ImageContent blocks (one per rendered page). On error, returns a plain string starting with ``"Error: "``. ``` | Parameter | Type | Required | Default | |---|---|---|---| | `path` | string | yes | — | | `max_pages` | integer | no | `5` | ??? note "JSON schema" ```json { "properties": { "path": { "title": "Path", "type": "string" }, "max_pages": { "default": 5, "title": "Max Pages", "type": "integer" } }, "required": [ "path" ], "title": "render_document_thumbnailArguments", "type": "object" } ``` ### `read_document_pages_as_images` Render multiple pages of a PDF / DOCX / XLSX / PPTX into the model's vision context as full-resolution images. PDF files are rendered directly via pymupdf; DOCX / XLSX / PPTX are first converted to PDF via LibreOffice headless, then rendered. Each requested page is delivered to the LLM as a separate MCP `ImageContent` block — the model SEES every page directly. This is the vision fallback for scanned or image/diagram-heavy documents that `read_pdf` / `read_word` / `read_excel` / `read_powerpoint` read as near-empty. **WARNING:** `pages="all"` on a large PDF will return one image per page and will very likely overload the model's context window. Prefer a bounded range (e.g. `"1-10"`). `max_pages` caps how many pages are returned; pages beyond the cap are dropped. **Rejects image files (.png, .jpg, .gif, .bmp, .tiff, .webp).** For raw image inspection use `read_image`. Images larger than `DPTTOOLS_IMAGE_MAX_EDGE_PX` (1568 px on the long edge by default) or `DPTTOOLS_IMAGE_MAX_MB` (3 MB) are scaled down before they are returned, so every model provider accepts them. Requires the `pymupdf` library. Install the `[docs]` extra. Office files need LibreOffice in the environment (on Debian/Ubuntu: `apt-get install libreoffice-core libreoffice-writer libreoffice-calc libreoffice-impress`), or `DPTTOOLS_SOFFICE_BIN` pointing at `soffice`. **Args** - `path`: Path to the document file. - `pages`: Which pages to render — 'all', '1-5', or '1,3,7' (1-indexed). - `scale`: Render scale factor (default 2.0 for high-DPI). - `max_pages`: Maximum number of page images to return (default 20). **Returns** A list of MCP ImageContent blocks (one per rendered page). On error, returns a plain string starting with `"Error: "`. ??? note "Exact text the model receives" ```text Render multiple pages of a PDF / DOCX / XLSX / PPTX into the model's vision context as full-resolution images. PDF files are rendered directly via pymupdf; DOCX / XLSX / PPTX are first converted to PDF via LibreOffice headless, then rendered. Each requested page is delivered to the LLM as a separate MCP ``ImageContent`` block — the model SEES every page directly. This is the vision fallback for scanned or image/diagram-heavy documents that ``read_pdf`` / ``read_word`` / ``read_excel`` / ``read_powerpoint`` read as near-empty. **WARNING:** ``pages="all"`` on a large PDF will return one image per page and will very likely overload the model's context window. Prefer a bounded range (e.g. ``"1-10"``). ``max_pages`` caps how many pages are returned; pages beyond the cap are dropped. **Rejects image files (.png, .jpg, .gif, .bmp, .tiff, .webp).** For raw image inspection use ``read_image``. Images larger than ``DPTTOOLS_IMAGE_MAX_EDGE_PX`` (1568 px on the long edge by default) or ``DPTTOOLS_IMAGE_MAX_MB`` (3 MB) are scaled down before they are returned, so every model provider accepts them. Requires the ``pymupdf`` library. Install the ``[docs]`` extra. Office files need LibreOffice in the environment (on Debian/Ubuntu: ``apt-get install libreoffice-core libreoffice-writer libreoffice-calc libreoffice-impress``), or ``DPTTOOLS_SOFFICE_BIN`` pointing at ``soffice``. Args: path: Path to the document file. pages: Which pages to render — 'all', '1-5', or '1,3,7' (1-indexed). scale: Render scale factor (default 2.0 for high-DPI). max_pages: Maximum number of page images to return (default 20). Returns: A list of MCP ImageContent blocks (one per rendered page). On error, returns a plain string starting with ``"Error: "``. ``` | Parameter | Type | Required | Default | |---|---|---|---| | `path` | string | yes | — | | `pages` | string | no | `"all"` | | `scale` | number | no | `2.0` | | `max_pages` | integer | no | `20` | ??? note "JSON schema" ```json { "properties": { "path": { "title": "Path", "type": "string" }, "pages": { "default": "all", "title": "Pages", "type": "string" }, "scale": { "default": 2.0, "title": "Scale", "type": "number" }, "max_pages": { "default": 20, "title": "Max Pages", "type": "integer" } }, "required": [ "path" ], "title": "read_document_pages_as_imagesArguments", "type": "object" } ``` ### `convert_document` Convert a document to another format and save the result as a file. Converts Office and OpenDocument files (DOCX, XLSX, PPTX, DOC, XLS, PPT, ODT, ODS, ODP, RTF and similar) with LibreOffice headless, and writes the output to disk. It does NOT show the result to the model: it returns the paths of the files it wrote. To look at pages, use `read_document_pages_as_images` or `render_document_page` instead. `to` is one of: pdf, docx, xlsx, pptx, odt, ods, odp, csv, html, txt, png, jpg. - **png / jpg** write one image per page, named `<name>-p<N>.png`. The document is converted to PDF first, then each page is rendered. Choose pages with `page` ('all', '1-3' or '2,5', 1-indexed). A PDF source can only be converted to png or jpg. - **csv** exports only the first sheet of a spreadsheet. - Other targets write one file, `<name>.<to>`. Output goes next to the source file unless `outdir` names an existing directory. An existing output file is never replaced unless `overwrite=True`. Requires LibreOffice in the environment (on Debian/Ubuntu: `apt-get install libreoffice-core libreoffice-writer libreoffice-calc libreoffice-impress`), or `DPTTOOLS_SOFFICE_BIN` pointing at `soffice`. png / jpg also need `pymupdf` (the `[docs]` extra). Each conversion times out after `DPTTOOLS_CONVERT_TIMEOUT_S` seconds (default 120). **Args** - `path`: Path to the source document. - `to`: Target format: pdf, docx, xlsx, pptx, odt, ods, odp, csv, html, txt, png or jpg. - `outdir`: Existing directory to write into. Defaults to the source file's directory. - `page`: Pages to render for png / jpg: 'all', '1-5' or '1,3,7' (1-indexed). Must be 'all' for other targets. - `overwrite`: Replace output files that already exist (default False). **Returns** JSON with `output_files` (absolute paths), `pages` (pages written for png / jpg, the page count for pdf when pymupdf is installed, otherwise null) and `engine`. On failure, a string starting with `"Error: "`. ??? note "Exact text the model receives" ```text Convert a document to another format and save the result as a file. Converts Office and OpenDocument files (DOCX, XLSX, PPTX, DOC, XLS, PPT, ODT, ODS, ODP, RTF and similar) with LibreOffice headless, and writes the output to disk. It does NOT show the result to the model: it returns the paths of the files it wrote. To look at pages, use ``read_document_pages_as_images`` or ``render_document_page`` instead. ``to`` is one of: pdf, docx, xlsx, pptx, odt, ods, odp, csv, html, txt, png, jpg. - **png / jpg** write one image per page, named ``<name>-p<N>.png``. The document is converted to PDF first, then each page is rendered. Choose pages with ``page`` ('all', '1-3' or '2,5', 1-indexed). A PDF source can only be converted to png or jpg. - **csv** exports only the first sheet of a spreadsheet. - Other targets write one file, ``<name>.<to>``. Output goes next to the source file unless ``outdir`` names an existing directory. An existing output file is never replaced unless ``overwrite=True``. Requires LibreOffice in the environment (on Debian/Ubuntu: ``apt-get install libreoffice-core libreoffice-writer libreoffice-calc libreoffice-impress``), or ``DPTTOOLS_SOFFICE_BIN`` pointing at ``soffice``. png / jpg also need ``pymupdf`` (the ``[docs]`` extra). Each conversion times out after ``DPTTOOLS_CONVERT_TIMEOUT_S`` seconds (default 120). Args: path: Path to the source document. to: Target format: pdf, docx, xlsx, pptx, odt, ods, odp, csv, html, txt, png or jpg. outdir: Existing directory to write into. Defaults to the source file's directory. page: Pages to render for png / jpg: 'all', '1-5' or '1,3,7' (1-indexed). Must be 'all' for other targets. overwrite: Replace output files that already exist (default False). Returns: JSON with ``output_files`` (absolute paths), ``pages`` (pages written for png / jpg, the page count for pdf when pymupdf is installed, otherwise null) and ``engine``. On failure, a string starting with ``"Error: "``. ``` | Parameter | Type | Required | Default | |---|---|---|---| | `path` | string | yes | — | | `to` | string | yes | — | | `outdir` | string or null | no | `null` | | `page` | string | no | `"all"` | | `overwrite` | boolean | no | `false` | ??? note "JSON schema" ```json { "properties": { "path": { "title": "Path", "type": "string" }, "to": { "title": "To", "type": "string" }, "outdir": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Outdir" }, "page": { "default": "all", "title": "Page", "type": "string" }, "overwrite": { "default": false, "title": "Overwrite", "type": "boolean" } }, "required": [ "path", "to" ], "title": "convert_documentArguments", "type": "object" } ``` ### `get_file_overview` Return a **lightweight, file-type-specific summary** without full content. Use for fast context-gathering; for full content extraction use `read_word`, `read_excel`, `read_pdf`, `read_powerpoint`, or `read_image`. Accepted extensions: - **PDF**: `.pdf` - **Excel**: `.xlsx`, `.xls` - **Word**: `.docx`, `.doc` - **PowerPoint**: `.pptx`, `.ppt` - **Image**: `.png`, `.jpg`, `.jpeg`, `.gif`, `.bmp`, `.tiff`, `.tif`, `.webp` - **Other**: any other extension is treated as a text/JSON file (file size, line count, first/last lines preview). The overview is customised based on the file type: - **PDF**: Page count, page dimensions, text preview per page, has_images, has_tables - **Excel**: Sheet names, row/column counts per sheet, column headers, sample rows - **Word**: Section headings, paragraph count, table count, word count - **PowerPoint**: Slide count, slide titles, speaker notes presence - **Image**: Dimensions, format, color mode, file size - **Text/JSON/Other**: File size, line count, first/last lines preview **Args** - `path`: Path to the file. **Returns** JSON string with file-type-specific metadata and summary. ??? note "Exact text the model receives" ```text Return a **lightweight, file-type-specific summary** without full content. Use for fast context-gathering; for full content extraction use ``read_word``, ``read_excel``, ``read_pdf``, ``read_powerpoint``, or ``read_image``. Accepted extensions: - **PDF**: ``.pdf`` - **Excel**: ``.xlsx``, ``.xls`` - **Word**: ``.docx``, ``.doc`` - **PowerPoint**: ``.pptx``, ``.ppt`` - **Image**: ``.png``, ``.jpg``, ``.jpeg``, ``.gif``, ``.bmp``, ``.tiff``, ``.tif``, ``.webp`` - **Other**: any other extension is treated as a text/JSON file (file size, line count, first/last lines preview). The overview is customised based on the file type: - **PDF**: Page count, page dimensions, text preview per page, has_images, has_tables - **Excel**: Sheet names, row/column counts per sheet, column headers, sample rows - **Word**: Section headings, paragraph count, table count, word count - **PowerPoint**: Slide count, slide titles, speaker notes presence - **Image**: Dimensions, format, color mode, file size - **Text/JSON/Other**: File size, line count, first/last lines preview Args: path: Path to the file. Returns: JSON string with file-type-specific metadata and summary. ``` | Parameter | Type | Required | Default | |---|---|---|---| | `path` | string | yes | — | ??? note "JSON schema" ```json { "properties": { "path": { "title": "Path", "type": "string" } }, "required": [ "path" ], "title": "get_file_overviewArguments", "type": "object" } ``` ## code_execution server `code-execution-tools` — see the [server page](../servers/code_execution.md). ### `execute_command` Execute a shell command in the persistent bash session. The session preserves the working directory across calls — if you `cd` into a directory, subsequent commands will run there until `restart_session` is called. The primary security boundary is the Docker container the agent runs in. **Args** - `command`: Shell command to execute. Supports pipes, redirects, and chained commands (`&&`, `||`, `;`). - `timeout`: Execution timeout in seconds. Defaults to the `DPTTOOLS_SANDBOX_TIMEOUT` environment variable (30 s). **Returns** JSON string with keys `stdout`, `stderr`, `exit_code` (int), and `timed_out` (bool). ??? note "Exact text the model receives" ```text Execute a shell command in the persistent bash session. The session preserves the working directory across calls — if you ``cd`` into a directory, subsequent commands will run there until ``restart_session`` is called. The primary security boundary is the Docker container the agent runs in. Args: command: Shell command to execute. Supports pipes, redirects, and chained commands (``&&``, ``||``, ``;``). timeout: Execution timeout in seconds. Defaults to the ``DPTTOOLS_SANDBOX_TIMEOUT`` environment variable (30 s). Returns: JSON string with keys ``stdout``, ``stderr``, ``exit_code`` (int), and ``timed_out`` (bool). ``` | Parameter | Type | Required | Default | |---|---|---|---| | `command` | string | yes | — | | `timeout` | integer or null | no | `null` | ??? note "JSON schema" ```json { "properties": { "command": { "title": "Command", "type": "string" }, "timeout": { "anyOf": [ { "type": "integer" }, { "type": "null" } ], "default": null, "title": "Timeout" } }, "required": [ "command" ], "title": "execute_commandArguments", "type": "object" } ``` ### `restart_session` Reset the persistent bash session. Resets the working directory back to the initial value it had when the server started. **Returns** Confirmation message including the reset working directory path. ??? note "Exact text the model receives" ```text Reset the persistent bash session. Resets the working directory back to the initial value it had when the server started. Returns: Confirmation message including the reset working directory path. ``` No parameters. ??? note "JSON schema" ```json { "properties": {}, "title": "restart_sessionArguments", "type": "object" } ``` ## screenshot server `screenshot-tools` — see the [server page](../servers/screenshot.md). ### `take_screenshot` Capture a screenshot (full screen, specific monitor, or region). When `output_path` is provided, saves the PNG to that path and returns the path string. When omitted, returns the screenshot as an MCP `ImageContent` block — the LLM SEES the captured pixels directly. Captures the real screen of the machine this server runs on (all monitors combined by default). The image block returned to the model is scaled down when it is larger than `DPTTOOLS_IMAGE_MAX_EDGE_PX` (1568 px on the long edge by default) or `DPTTOOLS_IMAGE_MAX_MB` (3 MB); a file saved to `output_path` keeps full resolution. Requires the `mss` and `Pillow` libraries. Install the `[screenshot]` extra. **Args** - `output_path`: Path to save the screenshot as PNG. If omitted, returns a vision-readable image block. - `monitor`: Monitor index to capture. 0 = all monitors combined (default), 1 = first monitor, 2 = second, etc. - `region`: Region to capture as `{"left": int, "top": int, "width": int, "height": int}`. Overrides monitor selection. **Returns** File path string (when `output_path` is set) OR an MCP ImageContent block (when `output_path` is omitted). On error, returns a plain string starting with `"Error: "`. ??? note "Exact text the model receives" ```text Capture a screenshot (full screen, specific monitor, or region). When ``output_path`` is provided, saves the PNG to that path and returns the path string. When omitted, returns the screenshot as an MCP ``ImageContent`` block — the LLM SEES the captured pixels directly. Captures the real screen of the machine this server runs on (all monitors combined by default). The image block returned to the model is scaled down when it is larger than ``DPTTOOLS_IMAGE_MAX_EDGE_PX`` (1568 px on the long edge by default) or ``DPTTOOLS_IMAGE_MAX_MB`` (3 MB); a file saved to ``output_path`` keeps full resolution. Requires the ``mss`` and ``Pillow`` libraries. Install the ``[screenshot]`` extra. Args: output_path: Path to save the screenshot as PNG. If omitted, returns a vision-readable image block. monitor: Monitor index to capture. 0 = all monitors combined (default), 1 = first monitor, 2 = second, etc. region: Region to capture as ``{"left": int, "top": int, "width": int, "height": int}``. Overrides monitor selection. Returns: File path string (when ``output_path`` is set) OR an MCP ImageContent block (when ``output_path`` is omitted). On error, returns a plain string starting with ``"Error: "``. ``` | Parameter | Type | Required | Default | |---|---|---|---| | `output_path` | string or null | no | `null` | | `monitor` | integer or null | no | `null` | | `region` | object or null | no | `null` | ??? note "JSON schema" ```json { "properties": { "output_path": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Output Path" }, "monitor": { "anyOf": [ { "type": "integer" }, { "type": "null" } ], "default": null, "title": "Monitor" }, "region": { "anyOf": [ { "additionalProperties": true, "type": "object" }, { "type": "null" } ], "default": null, "title": "Region" } }, "title": "take_screenshotArguments", "type": "object" } ``` ### `take_window_screenshot` Capture a specific window by title. Note: window-specific capture is not implemented yet, because it is not reliable across platforms. Use `take_screenshot()` with a `region` parameter for targeted captures. **Args** - `window_title`: Title of the window to capture. - `output_path`: Path to save the screenshot (optional; returns a vision-readable image block if omitted). **Returns** An error message explaining the limitation, with a suggestion to use take_screenshot with a region parameter instead. ??? note "Exact text the model receives" ```text Capture a specific window by title. Note: window-specific capture is not implemented yet, because it is not reliable across platforms. Use ``take_screenshot()`` with a ``region`` parameter for targeted captures. Args: window_title: Title of the window to capture. output_path: Path to save the screenshot (optional; returns a vision-readable image block if omitted). Returns: An error message explaining the limitation, with a suggestion to use take_screenshot with a region parameter instead. ``` | Parameter | Type | Required | Default | |---|---|---|---| | `window_title` | string | yes | — | | `output_path` | string or null | no | `null` | ??? note "JSON schema" ```json { "properties": { "window_title": { "title": "Window Title", "type": "string" }, "output_path": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Output Path" } }, "required": [ "window_title" ], "title": "take_window_screenshotArguments", "type": "object" } ``` ## image_tools server `image-tools` — see the [server page](../servers/image_tools.md). ### `image_crop` Crop an image to specified coordinates. **Args** - `path`: Path to the input image. - `left`: Left coordinate of the crop box. - `top`: Top coordinate of the crop box. - `right`: Right coordinate of the crop box. - `bottom`: Bottom coordinate of the crop box. - `output_path`: Path for the output image (default: new file next to input, same format). - `frame`: Frame index for multi-frame inputs (default 0). **Returns** Path to the cropped image. ??? note "Exact text the model receives" ```text Crop an image to specified coordinates. Args: path: Path to the input image. left: Left coordinate of the crop box. top: Top coordinate of the crop box. right: Right coordinate of the crop box. bottom: Bottom coordinate of the crop box. output_path: Path for the output image (default: new file next to input, same format). frame: Frame index for multi-frame inputs (default 0). Returns: Path to the cropped image. ``` | Parameter | Type | Required | Default | |---|---|---|---| | `path` | string | yes | — | | `left` | integer | yes | — | | `top` | integer | yes | — | | `right` | integer | yes | — | | `bottom` | integer | yes | — | | `output_path` | string or null | no | `null` | | `frame` | integer | no | `0` | ??? note "JSON schema" ```json { "properties": { "path": { "title": "Path", "type": "string" }, "left": { "title": "Left", "type": "integer" }, "top": { "title": "Top", "type": "integer" }, "right": { "title": "Right", "type": "integer" }, "bottom": { "title": "Bottom", "type": "integer" }, "output_path": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Output Path" }, "frame": { "default": 0, "title": "Frame", "type": "integer" } }, "required": [ "path", "left", "top", "right", "bottom" ], "title": "image_cropArguments", "type": "object" } ``` ### `image_resize` Resize an image. Maintains aspect ratio if only one dimension is given. **Args** - `path`: Path to the input image. - `width`: Target width in pixels (optional). - `height`: Target height in pixels (optional). - `output_path`: Path for the output image (default: new file next to input, same format). - `frame`: Frame index for multi-frame inputs (default 0). **Returns** Path to the resized image. ??? note "Exact text the model receives" ```text Resize an image. Maintains aspect ratio if only one dimension is given. Args: path: Path to the input image. width: Target width in pixels (optional). height: Target height in pixels (optional). output_path: Path for the output image (default: new file next to input, same format). frame: Frame index for multi-frame inputs (default 0). Returns: Path to the resized image. ``` | Parameter | Type | Required | Default | |---|---|---|---| | `path` | string | yes | — | | `width` | integer or null | no | `null` | | `height` | integer or null | no | `null` | | `output_path` | string or null | no | `null` | | `frame` | integer | no | `0` | ??? note "JSON schema" ```json { "properties": { "path": { "title": "Path", "type": "string" }, "width": { "anyOf": [ { "type": "integer" }, { "type": "null" } ], "default": null, "title": "Width" }, "height": { "anyOf": [ { "type": "integer" }, { "type": "null" } ], "default": null, "title": "Height" }, "output_path": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Output Path" }, "frame": { "default": 0, "title": "Frame", "type": "integer" } }, "required": [ "path" ], "title": "image_resizeArguments", "type": "object" } ``` ### `image_zoom` Zoom into a specific region of an image. Crops the specified region, then scales it up by the zoom factor. **Args** - `path`: Path to the input image. - `region`: Region to zoom as {"left": int, "top": int, "width": int, "height": int}. - `zoom_factor`: Zoom multiplier (default 2.0). - `output_path`: Path for the output image (default: new file next to input, same format). - `frame`: Frame index for multi-frame inputs (default 0). **Returns** Path to the zoomed image. ??? note "Exact text the model receives" ```text Zoom into a specific region of an image. Crops the specified region, then scales it up by the zoom factor. Args: path: Path to the input image. region: Region to zoom as {"left": int, "top": int, "width": int, "height": int}. zoom_factor: Zoom multiplier (default 2.0). output_path: Path for the output image (default: new file next to input, same format). frame: Frame index for multi-frame inputs (default 0). Returns: Path to the zoomed image. ``` | Parameter | Type | Required | Default | |---|---|---|---| | `path` | string | yes | — | | `region` | object | yes | — | | `zoom_factor` | number | no | `2.0` | | `output_path` | string or null | no | `null` | | `frame` | integer | no | `0` | ??? note "JSON schema" ```json { "properties": { "path": { "title": "Path", "type": "string" }, "region": { "additionalProperties": true, "title": "Region", "type": "object" }, "zoom_factor": { "default": 2.0, "title": "Zoom Factor", "type": "number" }, "output_path": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Output Path" }, "frame": { "default": 0, "title": "Frame", "type": "integer" } }, "required": [ "path", "region" ], "title": "image_zoomArguments", "type": "object" } ``` ### `image_rotate` Rotate an image by a given angle (counter-clockwise). The canvas is expanded to fit the entire rotated image. **Args** - `path`: Path to the input image. - `angle`: Rotation angle in degrees (counter-clockwise). - `output_path`: Path for the output image (default: new file next to input, same format). - `frame`: Frame index for multi-frame inputs (default 0). **Returns** Path to the rotated image. ??? note "Exact text the model receives" ```text Rotate an image by a given angle (counter-clockwise). The canvas is expanded to fit the entire rotated image. Args: path: Path to the input image. angle: Rotation angle in degrees (counter-clockwise). output_path: Path for the output image (default: new file next to input, same format). frame: Frame index for multi-frame inputs (default 0). Returns: Path to the rotated image. ``` | Parameter | Type | Required | Default | |---|---|---|---| | `path` | string | yes | — | | `angle` | number | yes | — | | `output_path` | string or null | no | `null` | | `frame` | integer | no | `0` | ??? note "JSON schema" ```json { "properties": { "path": { "title": "Path", "type": "string" }, "angle": { "title": "Angle", "type": "number" }, "output_path": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Output Path" }, "frame": { "default": 0, "title": "Frame", "type": "integer" } }, "required": [ "path", "angle" ], "title": "image_rotateArguments", "type": "object" } ``` ### `image_annotate` Add annotations (text, rectangles, circles, arrows) to an image. **Args** - `path`: Path to the input image. - `annotations`: JSON list of annotation objects. Supported types: - {"type": "rectangle", "x": int, "y": int, "width": int, "height": int, "color": str} - {"type": "text", "x": int, "y": int, "text": str, "font_size": int, "color": str} - {"type": "circle", "cx": int, "cy": int, "radius": int, "color": str} - {"type": "arrow", "x1": int, "y1": int, "x2": int, "y2": int, "color": str} - `output_path`: Path for the output image (default: new file next to input, same format). - `frame`: Frame index for multi-frame inputs (default 0). **Returns** Path to the annotated image. ??? note "Exact text the model receives" ```text Add annotations (text, rectangles, circles, arrows) to an image. Args: path: Path to the input image. annotations: JSON list of annotation objects. Supported types: - {"type": "rectangle", "x": int, "y": int, "width": int, "height": int, "color": str} - {"type": "text", "x": int, "y": int, "text": str, "font_size": int, "color": str} - {"type": "circle", "cx": int, "cy": int, "radius": int, "color": str} - {"type": "arrow", "x1": int, "y1": int, "x2": int, "y2": int, "color": str} output_path: Path for the output image (default: new file next to input, same format). frame: Frame index for multi-frame inputs (default 0). Returns: Path to the annotated image. ``` | Parameter | Type | Required | Default | |---|---|---|---| | `path` | string | yes | — | | `annotations` | string | yes | — | | `output_path` | string or null | no | `null` | | `frame` | integer | no | `0` | ??? note "JSON schema" ```json { "properties": { "path": { "title": "Path", "type": "string" }, "annotations": { "title": "Annotations", "type": "string" }, "output_path": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Output Path" }, "frame": { "default": 0, "title": "Frame", "type": "integer" } }, "required": [ "path", "annotations" ], "title": "image_annotateArguments", "type": "object" } ``` ### `image_convert` Convert an image between formats (PNG, JPEG, WEBP, BMP, TIFF). Non-RGB/L modes are normalized to RGB for JPEG, compositing any alpha onto a **white** background (not black). **Args** - `path`: Path to the input image. - `format`: Target format — 'png', 'jpeg', 'webp', 'bmp', 'tiff'. - `output_path`: Path for the output image (default: new file with the new extension). - `frame`: Frame index for multi-frame inputs (default 0). **Returns** Path to the converted image. ??? note "Exact text the model receives" ```text Convert an image between formats (PNG, JPEG, WEBP, BMP, TIFF). Non-RGB/L modes are normalized to RGB for JPEG, compositing any alpha onto a **white** background (not black). Args: path: Path to the input image. format: Target format — 'png', 'jpeg', 'webp', 'bmp', 'tiff'. output_path: Path for the output image (default: new file with the new extension). frame: Frame index for multi-frame inputs (default 0). Returns: Path to the converted image. ``` | Parameter | Type | Required | Default | |---|---|---|---| | `path` | string | yes | — | | `format` | string | yes | — | | `output_path` | string or null | no | `null` | | `frame` | integer | no | `0` | ??? note "JSON schema" ```json { "properties": { "path": { "title": "Path", "type": "string" }, "format": { "title": "Format", "type": "string" }, "output_path": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Output Path" }, "frame": { "default": 0, "title": "Frame", "type": "integer" } }, "required": [ "path", "format" ], "title": "image_convertArguments", "type": "object" } ``` ### `image_get_info` Get image dimensions, format, color mode, frame count, and file size. Dimensions reflect the **EXIF-corrected** orientation (what a viewer shows), so they match the coordinate space the other tools operate in. **Args** - `path`: Path to the image file. **Returns** JSON string with width, height, format, mode, n_frames, is_animated, file_size_bytes, and file_size_human. ??? note "Exact text the model receives" ```text Get image dimensions, format, color mode, frame count, and file size. Dimensions reflect the **EXIF-corrected** orientation (what a viewer shows), so they match the coordinate space the other tools operate in. Args: path: Path to the image file. Returns: JSON string with width, height, format, mode, n_frames, is_animated, file_size_bytes, and file_size_human. ``` | Parameter | Type | Required | Default | |---|---|---|---| | `path` | string | yes | — | ??? note "JSON schema" ```json { "properties": { "path": { "title": "Path", "type": "string" } }, "required": [ "path" ], "title": "image_get_infoArguments", "type": "object" } ``` ## skills server `skills-tools` — see the [server page](../servers/skills.md). ### `list_skills` Discover and list all available skills from the configured skills directory. Reads YAML frontmatter from SKILL.md files in each subdirectory of DPTTOOLS_SKILLS_DIR. Returns Level-1 metadata (name, description) plus the absolute path to each SKILL.md file and any optional spec fields (license, compatibility, metadata). **Returns** JSON object with: - `skills`: list of objects with name, description, path, directory, and any optional fields present in the frontmatter - `skills_dir`: absolute path of the configured skills directory - `count`: number of valid skills discovered - `diagnostics`: list of non-fatal parse warnings (omitted when empty) ??? note "Exact text the model receives" ```text Discover and list all available skills from the configured skills directory. Reads YAML frontmatter from SKILL.md files in each subdirectory of DPTTOOLS_SKILLS_DIR. Returns Level-1 metadata (name, description) plus the absolute path to each SKILL.md file and any optional spec fields (license, compatibility, metadata). Returns: JSON object with: - ``skills``: list of objects with name, description, path, directory, and any optional fields present in the frontmatter - ``skills_dir``: absolute path of the configured skills directory - ``count``: number of valid skills discovered - ``diagnostics``: list of non-fatal parse warnings (omitted when empty) ``` No parameters. ??? note "JSON schema" ```json { "properties": {}, "title": "list_skillsArguments", "type": "object" } ``` ### `load_skill` Load the full SKILL.md content for a specific skill (Level 2 instructions). The returned content is wrapped in `<skill_content>` tags identifying the skill name and absolute directory, with a nested `<skill_resources>` listing of bundled files (scripts, references, assets) in the skill folder and one level of subfolders, at most 50 entries. Use the directory path in the wrapper to resolve any relative references inside the skill body. **Args** - `skill_name`: Name of the skill to load (must match directory name). **Returns** The wrapped SKILL.md content (frontmatter + body + resource listing). ??? note "Exact text the model receives" ```text Load the full SKILL.md content for a specific skill (Level 2 instructions). The returned content is wrapped in ``<skill_content>`` tags identifying the skill name and absolute directory, with a nested ``<skill_resources>`` listing of bundled files (scripts, references, assets) in the skill folder and one level of subfolders, at most 50 entries. Use the directory path in the wrapper to resolve any relative references inside the skill body. Args: skill_name: Name of the skill to load (must match directory name). Returns: The wrapped SKILL.md content (frontmatter + body + resource listing). ``` | Parameter | Type | Required | Default | |---|---|---|---| | `skill_name` | string | yes | — | ??? note "JSON schema" ```json { "properties": { "skill_name": { "title": "Skill Name", "type": "string" } }, "required": [ "skill_name" ], "title": "load_skillArguments", "type": "object" } ``` ### `load_skill_resource` Read a specific resource file from within a skill directory (Level 3). **Args** - `skill_name`: Name of the skill. - `resource_path`: Relative path to the resource file within the skill directory. **Returns** The resource file content. ??? note "Exact text the model receives" ```text Read a specific resource file from within a skill directory (Level 3). Args: skill_name: Name of the skill. resource_path: Relative path to the resource file within the skill directory. Returns: The resource file content. ``` | Parameter | Type | Required | Default | |---|---|---|---| | `skill_name` | string | yes | — | | `resource_path` | string | yes | — | ??? note "JSON schema" ```json { "properties": { "skill_name": { "title": "Skill Name", "type": "string" }, "resource_path": { "title": "Resource Path", "type": "string" } }, "required": [ "skill_name", "resource_path" ], "title": "load_skill_resourceArguments", "type": "object" } ``` ### `execute_skill_script` Execute a script bundled with a skill. The script runs via subprocess with the configured timeout (DPTTOOLS_SANDBOX_TIMEOUT). The container boundary provides the primary isolation. Only `.py` scripts are supported and they run with the **same** Python interpreter as the server (`sys.executable`), not whatever `python` happens to be on `PATH`. `args` is split with POSIX `shlex` rules (note: non-ASCII / quoted args must be quoted as in a POSIX shell). **Args** - `skill_name`: Name of the skill containing the script. - `script_path`: Relative path to the `.py` script within the skill directory. - `args`: Arguments to pass to the script (POSIX `shlex` splitting). **Returns** JSON string with stdout, stderr, and exit_code from the script execution. ??? note "Exact text the model receives" ```text Execute a script bundled with a skill. The script runs via subprocess with the configured timeout (DPTTOOLS_SANDBOX_TIMEOUT). The container boundary provides the primary isolation. Only ``.py`` scripts are supported and they run with the **same** Python interpreter as the server (``sys.executable``), not whatever ``python`` happens to be on ``PATH``. ``args`` is split with POSIX ``shlex`` rules (note: non-ASCII / quoted args must be quoted as in a POSIX shell). Args: skill_name: Name of the skill containing the script. script_path: Relative path to the ``.py`` script within the skill directory. args: Arguments to pass to the script (POSIX ``shlex`` splitting). Returns: JSON string with stdout, stderr, and exit_code from the script execution. ``` | Parameter | Type | Required | Default | |---|---|---|---| | `skill_name` | string | yes | — | | `script_path` | string | yes | — | | `args` | string | no | `""` | ??? note "JSON schema" ```json { "properties": { "skill_name": { "title": "Skill Name", "type": "string" }, "script_path": { "title": "Script Path", "type": "string" }, "args": { "default": "", "title": "Args", "type": "string" } }, "required": [ "skill_name", "script_path" ], "title": "execute_skill_scriptArguments", "type": "object" } ``` ### `create_skill` Scaffold a new skill directory with a template SKILL.md. Creates a new subdirectory under DPTTOOLS_SKILLS_DIR with a SKILL.md file containing the required YAML frontmatter and a placeholder body. **Args** - `skill_name`: Name for the new skill (1-64 chars, lowercase alphanumeric + hyphens). - `description`: Description of what the skill does (1-1024 chars). **Returns** Confirmation message with the path to the created skill directory. ??? note "Exact text the model receives" ```text Scaffold a new skill directory with a template SKILL.md. Creates a new subdirectory under DPTTOOLS_SKILLS_DIR with a SKILL.md file containing the required YAML frontmatter and a placeholder body. Args: skill_name: Name for the new skill (1-64 chars, lowercase alphanumeric + hyphens). description: Description of what the skill does (1-1024 chars). Returns: Confirmation message with the path to the created skill directory. ``` | Parameter | Type | Required | Default | |---|---|---|---| | `skill_name` | string | yes | — | | `description` | string | yes | — | ??? note "JSON schema" ```json { "properties": { "skill_name": { "title": "Skill Name", "type": "string" }, "description": { "title": "Description", "type": "string" } }, "required": [ "skill_name", "description" ], "title": "create_skillArguments", "type": "object" } ``` ## browser server `browser-tools` — see the [server page](../servers/browser.md). Shown with the `full` profile and every opt-in tool enabled; each tool says when it is registered. ### `browser_navigate` *Profiles: core and full.* Open a URL in the browser, or go "back", "forward" or "reload". A URL without a scheme gets https://. Only http and https URLs can be opened. Opens a tab first if none is open. Refs from before the navigation stop working; read the page again. **Args** - `url`: The URL, or "back", "forward" or "reload". - `tab_id`: Tab to use (default: the active tab). - `intent`: Optional note on why you are doing this (logged only). ??? note "Exact text the model receives" ```text Open a URL in the browser, or go "back", "forward" or "reload". A URL without a scheme gets https://. Only http and https URLs can be opened. Opens a tab first if none is open. Refs from before the navigation stop working; read the page again. Args: url: The URL, or "back", "forward" or "reload". tab_id: Tab to use (default: the active tab). intent: Optional note on why you are doing this (logged only). ``` | Parameter | Type | Required | Default | |---|---|---|---| | `url` | string | yes | — | | `tab_id` | integer or null | no | `null` | | `intent` | string or null | no | `null` | ??? note "JSON schema" ```json { "properties": { "url": { "title": "Url", "type": "string" }, "tab_id": { "anyOf": [ { "type": "integer" }, { "type": "null" } ], "default": null, "title": "Tab Id" }, "intent": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Intent" } }, "required": [ "url" ], "title": "browser_navigateArguments", "type": "object" } ``` ### `browser_screenshot` *Profiles: core and full.* Take a screenshot of the visible part of the page. Prefer browser_read_page or browser_get_page_text to read a page; a screenshot is for checking layout or visual state. x/y arguments of the other tools are in this screenshot's pixels. **Args** - `tab_id`: Tab to capture (default: the active tab). - `intent`: Optional note on why you are doing this (logged only). ??? note "Exact text the model receives" ```text Take a screenshot of the visible part of the page. Prefer browser_read_page or browser_get_page_text to read a page; a screenshot is for checking layout or visual state. x/y arguments of the other tools are in this screenshot's pixels. Args: tab_id: Tab to capture (default: the active tab). intent: Optional note on why you are doing this (logged only). ``` | Parameter | Type | Required | Default | |---|---|---|---| | `tab_id` | integer or null | no | `null` | | `intent` | string or null | no | `null` | ??? note "JSON schema" ```json { "properties": { "tab_id": { "anyOf": [ { "type": "integer" }, { "type": "null" } ], "default": null, "title": "Tab Id" }, "intent": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Intent" } }, "title": "browser_screenshotArguments", "type": "object" } ``` ### `browser_zoom` *Profiles: core and full.* Capture one region of the page, enlarged, to read small text or details. The region is a rectangle in screenshot pixels, from (x0, y0) to (x1, y1). **Args** - `x0`: Left edge. y0: Top edge. x1: Right edge. y1: Bottom edge. - `tab_id`: Tab to capture (default: the active tab). - `intent`: Optional note on why you are doing this (logged only). ??? note "Exact text the model receives" ```text Capture one region of the page, enlarged, to read small text or details. The region is a rectangle in screenshot pixels, from (x0, y0) to (x1, y1). Args: x0: Left edge. y0: Top edge. x1: Right edge. y1: Bottom edge. tab_id: Tab to capture (default: the active tab). intent: Optional note on why you are doing this (logged only). ``` | Parameter | Type | Required | Default | |---|---|---|---| | `x0` | integer | yes | — | | `y0` | integer | yes | — | | `x1` | integer | yes | — | | `y1` | integer | yes | — | | `tab_id` | integer or null | no | `null` | | `intent` | string or null | no | `null` | ??? note "JSON schema" ```json { "properties": { "x0": { "title": "X0", "type": "integer" }, "y0": { "title": "Y0", "type": "integer" }, "x1": { "title": "X1", "type": "integer" }, "y1": { "title": "Y1", "type": "integer" }, "tab_id": { "anyOf": [ { "type": "integer" }, { "type": "null" } ], "default": null, "title": "Tab Id" }, "intent": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Intent" } }, "required": [ "x0", "y0", "x1", "y1" ], "title": "browser_zoomArguments", "type": "object" } ``` ### `browser_left_click` *Profiles: core and full.* Click an element or a point with the left mouse button. Target an element with ref (from browser_read_page or browser_find; preferred), or with x and y in screenshot pixels. The result names the element that was clicked. **Args** - `ref`: Element ref such as "ref_12". - `x`: Horizontal position in screenshot pixels (with y). - `y`: Vertical position in screenshot pixels (with x). - `modifiers`: Keys to hold, such as "ctrl" or "shift+ctrl". - `tab_id`: Tab to act in (default: the active tab). - `intent`: Optional note on why you are doing this (logged only). ??? note "Exact text the model receives" ```text Click an element or a point with the left mouse button. Target an element with ref (from browser_read_page or browser_find; preferred), or with x and y in screenshot pixels. The result names the element that was clicked. Args: ref: Element ref such as "ref_12". x: Horizontal position in screenshot pixels (with y). y: Vertical position in screenshot pixels (with x). modifiers: Keys to hold, such as "ctrl" or "shift+ctrl". tab_id: Tab to act in (default: the active tab). intent: Optional note on why you are doing this (logged only). ``` | Parameter | Type | Required | Default | |---|---|---|---| | `ref` | string or null | no | `null` | | `x` | integer or null | no | `null` | | `y` | integer or null | no | `null` | | `modifiers` | string or null | no | `null` | | `tab_id` | integer or null | no | `null` | | `intent` | string or null | no | `null` | ??? note "JSON schema" ```json { "properties": { "ref": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Ref" }, "x": { "anyOf": [ { "type": "integer" }, { "type": "null" } ], "default": null, "title": "X" }, "y": { "anyOf": [ { "type": "integer" }, { "type": "null" } ], "default": null, "title": "Y" }, "modifiers": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Modifiers" }, "tab_id": { "anyOf": [ { "type": "integer" }, { "type": "null" } ], "default": null, "title": "Tab Id" }, "intent": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Intent" } }, "title": "browser_left_clickArguments", "type": "object" } ``` ### `browser_hover` *Profiles: core and full.* Move the mouse over an element or a point, for menus and tooltips. Target an element with ref (preferred), or with x and y in screenshot pixels. **Args** - `ref`: Element ref such as "ref_12". - `x`: Horizontal position in screenshot pixels (with y). - `y`: Vertical position in screenshot pixels (with x). - `modifiers`: Keys to hold while hovering. - `tab_id`: Tab to act in (default: the active tab). - `intent`: Optional note on why you are doing this (logged only). ??? note "Exact text the model receives" ```text Move the mouse over an element or a point, for menus and tooltips. Target an element with ref (preferred), or with x and y in screenshot pixels. Args: ref: Element ref such as "ref_12". x: Horizontal position in screenshot pixels (with y). y: Vertical position in screenshot pixels (with x). modifiers: Keys to hold while hovering. tab_id: Tab to act in (default: the active tab). intent: Optional note on why you are doing this (logged only). ``` | Parameter | Type | Required | Default | |---|---|---|---| | `ref` | string or null | no | `null` | | `x` | integer or null | no | `null` | | `y` | integer or null | no | `null` | | `modifiers` | string or null | no | `null` | | `tab_id` | integer or null | no | `null` | | `intent` | string or null | no | `null` | ??? note "JSON schema" ```json { "properties": { "ref": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Ref" }, "x": { "anyOf": [ { "type": "integer" }, { "type": "null" } ], "default": null, "title": "X" }, "y": { "anyOf": [ { "type": "integer" }, { "type": "null" } ], "default": null, "title": "Y" }, "modifiers": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Modifiers" }, "tab_id": { "anyOf": [ { "type": "integer" }, { "type": "null" } ], "default": null, "title": "Tab Id" }, "intent": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Intent" } }, "title": "browser_hoverArguments", "type": "object" } ``` ### `browser_scroll` *Profiles: core and full.* Scroll the page, or the scrollable area under an element or point. Without a target, scrolls at the centre of the page. Use browser_scroll_to to bring a known element into view. **Args** - `scroll_direction`: up, down, left or right. - `scroll_amount`: 1-10 steps of about 100 pixels (default 3). - `ref`: Element ref to scroll within. - `x`: Horizontal position in screenshot pixels (with y). - `y`: Vertical position in screenshot pixels (with x). - `tab_id`: Tab to act in (default: the active tab). - `intent`: Optional note on why you are doing this (logged only). ??? note "Exact text the model receives" ```text Scroll the page, or the scrollable area under an element or point. Without a target, scrolls at the centre of the page. Use browser_scroll_to to bring a known element into view. Args: scroll_direction: up, down, left or right. scroll_amount: 1-10 steps of about 100 pixels (default 3). ref: Element ref to scroll within. x: Horizontal position in screenshot pixels (with y). y: Vertical position in screenshot pixels (with x). tab_id: Tab to act in (default: the active tab). intent: Optional note on why you are doing this (logged only). ``` | Parameter | Type | Required | Default | |---|---|---|---| | `scroll_direction` | string | yes | — | | `scroll_amount` | integer | no | `3` | | `ref` | string or null | no | `null` | | `x` | integer or null | no | `null` | | `y` | integer or null | no | `null` | | `tab_id` | integer or null | no | `null` | | `intent` | string or null | no | `null` | ??? note "JSON schema" ```json { "properties": { "scroll_direction": { "enum": [ "up", "down", "left", "right" ], "title": "Scroll Direction", "type": "string" }, "scroll_amount": { "default": 3, "title": "Scroll Amount", "type": "integer" }, "ref": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Ref" }, "x": { "anyOf": [ { "type": "integer" }, { "type": "null" } ], "default": null, "title": "X" }, "y": { "anyOf": [ { "type": "integer" }, { "type": "null" } ], "default": null, "title": "Y" }, "tab_id": { "anyOf": [ { "type": "integer" }, { "type": "null" } ], "default": null, "title": "Tab Id" }, "intent": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Intent" } }, "required": [ "scroll_direction" ], "title": "browser_scrollArguments", "type": "object" } ``` ### `browser_scroll_to` *Profiles: core and full.* Scroll an element into view. **Args** - `ref`: Element ref such as "ref_12". - `tab_id`: Tab to act in (default: the active tab). - `intent`: Optional note on why you are doing this (logged only). ??? note "Exact text the model receives" ```text Scroll an element into view. Args: ref: Element ref such as "ref_12". tab_id: Tab to act in (default: the active tab). intent: Optional note on why you are doing this (logged only). ``` | Parameter | Type | Required | Default | |---|---|---|---| | `ref` | string | yes | — | | `tab_id` | integer or null | no | `null` | | `intent` | string or null | no | `null` | ??? note "JSON schema" ```json { "properties": { "ref": { "title": "Ref", "type": "string" }, "tab_id": { "anyOf": [ { "type": "integer" }, { "type": "null" } ], "default": null, "title": "Tab Id" }, "intent": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Intent" } }, "required": [ "ref" ], "title": "browser_scroll_toArguments", "type": "object" } ``` ### `browser_type` *Profiles: core and full.* Type text into the focused element, as if on a keyboard. Click a field first to focus it, or use browser_form_input to set a field's value by ref. **Args** - `text`: The text to type. - `tab_id`: Tab to act in (default: the active tab). - `intent`: Optional note on why you are doing this (logged only). ??? note "Exact text the model receives" ```text Type text into the focused element, as if on a keyboard. Click a field first to focus it, or use browser_form_input to set a field's value by ref. Args: text: The text to type. tab_id: Tab to act in (default: the active tab). intent: Optional note on why you are doing this (logged only). ``` | Parameter | Type | Required | Default | |---|---|---|---| | `text` | string | yes | — | | `tab_id` | integer or null | no | `null` | | `intent` | string or null | no | `null` | ??? note "JSON schema" ```json { "properties": { "text": { "title": "Text", "type": "string" }, "tab_id": { "anyOf": [ { "type": "integer" }, { "type": "null" } ], "default": null, "title": "Tab Id" }, "intent": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Intent" } }, "required": [ "text" ], "title": "browser_typeArguments", "type": "object" } ``` ### `browser_key` *Profiles: core and full.* Press a key, a chord, or a space-separated sequence of them. Examples: "Enter", "ctrl+a", "shift+Tab", "Escape", "ctrl+a Delete". Key names follow xdotool (Return, BackSpace, Page_Down) or the usual names (Enter, Backspace, PageDown, ArrowLeft). **Args** - `text`: The key, chord or sequence. - `repeat`: Times to press the whole sequence, 1-100 (default 1). - `tab_id`: Tab to act in (default: the active tab). - `intent`: Optional note on why you are doing this (logged only). ??? note "Exact text the model receives" ```text Press a key, a chord, or a space-separated sequence of them. Examples: "Enter", "ctrl+a", "shift+Tab", "Escape", "ctrl+a Delete". Key names follow xdotool (Return, BackSpace, Page_Down) or the usual names (Enter, Backspace, PageDown, ArrowLeft). Args: text: The key, chord or sequence. repeat: Times to press the whole sequence, 1-100 (default 1). tab_id: Tab to act in (default: the active tab). intent: Optional note on why you are doing this (logged only). ``` | Parameter | Type | Required | Default | |---|---|---|---| | `text` | string | yes | — | | `repeat` | integer | no | `1` | | `tab_id` | integer or null | no | `null` | | `intent` | string or null | no | `null` | ??? note "JSON schema" ```json { "properties": { "text": { "title": "Text", "type": "string" }, "repeat": { "default": 1, "title": "Repeat", "type": "integer" }, "tab_id": { "anyOf": [ { "type": "integer" }, { "type": "null" } ], "default": null, "title": "Tab Id" }, "intent": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Intent" } }, "required": [ "text" ], "title": "browser_keyArguments", "type": "object" } ``` ### `browser_wait` *Profiles: core and full.* Wait a number of seconds (at most 30), for a page to finish loading or animating. **Args** - `duration`: Seconds to wait, at most 30. - `tab_id`: Tab to report on (default: the active tab). - `intent`: Optional note on why you are doing this (logged only). ??? note "Exact text the model receives" ```text Wait a number of seconds (at most 30), for a page to finish loading or animating. Args: duration: Seconds to wait, at most 30. tab_id: Tab to report on (default: the active tab). intent: Optional note on why you are doing this (logged only). ``` | Parameter | Type | Required | Default | |---|---|---|---| | `duration` | number | yes | — | | `tab_id` | integer or null | no | `null` | | `intent` | string or null | no | `null` | ??? note "JSON schema" ```json { "properties": { "duration": { "title": "Duration", "type": "number" }, "tab_id": { "anyOf": [ { "type": "integer" }, { "type": "null" } ], "default": null, "title": "Tab Id" }, "intent": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Intent" } }, "required": [ "duration" ], "title": "browser_waitArguments", "type": "object" } ``` ### `browser_read_page` *Profiles: core and full.* Read the page as an accessibility tree, with a [ref_N] tag on each element. Pass a ref to the action tools to act on that element. Refs stay valid until the tab navigates or the element is removed; a stale ref gives an error telling you to read the page again. "interactive" (default) lists links, buttons and form fields; "all" adds headings, images, text and landmarks. Covers the whole page, not just the visible part. **Args** - `filter`: "interactive" (default) or "all". - `depth`: Maximum nesting depth, 1-50 (default 15). - `ref`: Read only the subtree under this ref. - `tab_id`: Tab to read (default: the active tab). - `intent`: Optional note on why you are doing this (logged only). ??? note "Exact text the model receives" ```text Read the page as an accessibility tree, with a [ref_N] tag on each element. Pass a ref to the action tools to act on that element. Refs stay valid until the tab navigates or the element is removed; a stale ref gives an error telling you to read the page again. "interactive" (default) lists links, buttons and form fields; "all" adds headings, images, text and landmarks. Covers the whole page, not just the visible part. Args: filter: "interactive" (default) or "all". depth: Maximum nesting depth, 1-50 (default 15). ref: Read only the subtree under this ref. tab_id: Tab to read (default: the active tab). intent: Optional note on why you are doing this (logged only). ``` | Parameter | Type | Required | Default | |---|---|---|---| | `filter` | string | no | `"interactive"` | | `depth` | integer | no | `15` | | `ref` | string or null | no | `null` | | `tab_id` | integer or null | no | `null` | | `intent` | string or null | no | `null` | ??? note "JSON schema" ```json { "properties": { "filter": { "default": "interactive", "enum": [ "interactive", "all" ], "title": "Filter", "type": "string" }, "depth": { "default": 15, "title": "Depth", "type": "integer" }, "ref": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Ref" }, "tab_id": { "anyOf": [ { "type": "integer" }, { "type": "null" } ], "default": null, "title": "Tab Id" }, "intent": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Intent" } }, "title": "browser_read_pageArguments", "type": "object" } ``` ### `browser_find` *Profiles: core and full.* Find elements on the page matching a short description, with their refs. Matching is by words (the element's role, label, placeholder and nearby text), not by meaning: "search box", "Sign in button", "email". Returns up to 20 matches. If nothing matches, use browser_read_page(filter="interactive"). **Args** - `query`: Words describing the element. - `tab_id`: Tab to search (default: the active tab). - `intent`: Optional note on why you are doing this (logged only). ??? note "Exact text the model receives" ```text Find elements on the page matching a short description, with their refs. Matching is by words (the element's role, label, placeholder and nearby text), not by meaning: "search box", "Sign in button", "email". Returns up to 20 matches. If nothing matches, use browser_read_page(filter="interactive"). Args: query: Words describing the element. tab_id: Tab to search (default: the active tab). intent: Optional note on why you are doing this (logged only). ``` | Parameter | Type | Required | Default | |---|---|---|---| | `query` | string | yes | — | | `tab_id` | integer or null | no | `null` | | `intent` | string or null | no | `null` | ??? note "JSON schema" ```json { "properties": { "query": { "title": "Query", "type": "string" }, "tab_id": { "anyOf": [ { "type": "integer" }, { "type": "null" } ], "default": null, "title": "Tab Id" }, "intent": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Intent" } }, "required": [ "query" ], "title": "browser_findArguments", "type": "object" } ``` ### `browser_get_page_text` *Profiles: core and full.* Get the readable text of the page, main content first. The cheapest way to read an article, a statement or search results. Has no refs; use browser_read_page or browser_find to act on elements. **Args** - `tab_id`: Tab to read (default: the active tab). - `intent`: Optional note on why you are doing this (logged only). ??? note "Exact text the model receives" ```text Get the readable text of the page, main content first. The cheapest way to read an article, a statement or search results. Has no refs; use browser_read_page or browser_find to act on elements. Args: tab_id: Tab to read (default: the active tab). intent: Optional note on why you are doing this (logged only). ``` | Parameter | Type | Required | Default | |---|---|---|---| | `tab_id` | integer or null | no | `null` | | `intent` | string or null | no | `null` | ??? note "JSON schema" ```json { "properties": { "tab_id": { "anyOf": [ { "type": "integer" }, { "type": "null" } ], "default": null, "title": "Tab Id" }, "intent": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Intent" } }, "title": "browser_get_page_textArguments", "type": "object" } ``` ### `browser_form_input` *Profiles: core and full.* Set a form field's value directly: text field, select, checkbox or radio. Selects take an option's value or visible label; checkboxes and radios take true or false. **Args** - `ref`: Element ref of the field. - `value`: The value to set. - `tab_id`: Tab to act in (default: the active tab). - `intent`: Optional note on why you are doing this (logged only). ??? note "Exact text the model receives" ```text Set a form field's value directly: text field, select, checkbox or radio. Selects take an option's value or visible label; checkboxes and radios take true or false. Args: ref: Element ref of the field. value: The value to set. tab_id: Tab to act in (default: the active tab). intent: Optional note on why you are doing this (logged only). ``` | Parameter | Type | Required | Default | |---|---|---|---| | `ref` | string | yes | — | | `value` | string or integer or number or boolean | yes | — | | `tab_id` | integer or null | no | `null` | | `intent` | string or null | no | `null` | ??? note "JSON schema" ```json { "properties": { "ref": { "title": "Ref", "type": "string" }, "value": { "anyOf": [ { "type": "string" }, { "type": "integer" }, { "type": "number" }, { "type": "boolean" } ], "title": "Value" }, "tab_id": { "anyOf": [ { "type": "integer" }, { "type": "null" } ], "default": null, "title": "Tab Id" }, "intent": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Intent" } }, "required": [ "ref", "value" ], "title": "browser_form_inputArguments", "type": "object" } ``` ### `browser_new_tab` *Profiles: core and full.* Open a new tab, optionally at a URL, and make it the active tab. **Args** - `url`: URL to open in the new tab. - `intent`: Optional note on why you are doing this (logged only). ??? note "Exact text the model receives" ```text Open a new tab, optionally at a URL, and make it the active tab. Args: url: URL to open in the new tab. intent: Optional note on why you are doing this (logged only). ``` | Parameter | Type | Required | Default | |---|---|---|---| | `url` | string or null | no | `null` | | `intent` | string or null | no | `null` | ??? note "JSON schema" ```json { "properties": { "url": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Url" }, "intent": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Intent" } }, "title": "browser_new_tabArguments", "type": "object" } ``` ### `browser_list_tabs` *Profiles: core and full.* List the open tabs with their tabId, title and URL. **Args** - `intent`: Optional note on why you are doing this (logged only). ??? note "Exact text the model receives" ```text List the open tabs with their tabId, title and URL. Args: intent: Optional note on why you are doing this (logged only). ``` | Parameter | Type | Required | Default | |---|---|---|---| | `intent` | string or null | no | `null` | ??? note "JSON schema" ```json { "properties": { "intent": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Intent" } }, "title": "browser_list_tabsArguments", "type": "object" } ``` ### `browser_switch_tab` *Profiles: core and full.* Make another tab the active tab. **Args** - `tab_id`: The tab to switch to. - `intent`: Optional note on why you are doing this (logged only). ??? note "Exact text the model receives" ```text Make another tab the active tab. Args: tab_id: The tab to switch to. intent: Optional note on why you are doing this (logged only). ``` | Parameter | Type | Required | Default | |---|---|---|---| | `tab_id` | integer | yes | — | | `intent` | string or null | no | `null` | ??? note "JSON schema" ```json { "properties": { "tab_id": { "title": "Tab Id", "type": "integer" }, "intent": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Intent" } }, "required": [ "tab_id" ], "title": "browser_switch_tabArguments", "type": "object" } ``` ### `browser_close_tab` *Profiles: core and full.* Close a tab. **Args** - `tab_id`: The tab to close. - `intent`: Optional note on why you are doing this (logged only). ??? note "Exact text the model receives" ```text Close a tab. Args: tab_id: The tab to close. intent: Optional note on why you are doing this (logged only). ``` | Parameter | Type | Required | Default | |---|---|---|---| | `tab_id` | integer | yes | — | | `intent` | string or null | no | `null` | ??? note "JSON schema" ```json { "properties": { "tab_id": { "title": "Tab Id", "type": "integer" }, "intent": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Intent" } }, "required": [ "tab_id" ], "title": "browser_close_tabArguments", "type": "object" } ``` ### `browser_right_click` *Profile: full only.* Right-click an element or a point (opens a context menu). Target an element with ref (preferred), or with x and y in screenshot pixels. **Args** - `ref`: Element ref such as "ref_12". - `x`: Horizontal position in screenshot pixels (with y). - `y`: Vertical position in screenshot pixels (with x). - `modifiers`: Keys to hold, such as "shift". - `tab_id`: Tab to act in (default: the active tab). - `intent`: Optional note on why you are doing this (logged only). ??? note "Exact text the model receives" ```text Right-click an element or a point (opens a context menu). Target an element with ref (preferred), or with x and y in screenshot pixels. Args: ref: Element ref such as "ref_12". x: Horizontal position in screenshot pixels (with y). y: Vertical position in screenshot pixels (with x). modifiers: Keys to hold, such as "shift". tab_id: Tab to act in (default: the active tab). intent: Optional note on why you are doing this (logged only). ``` | Parameter | Type | Required | Default | |---|---|---|---| | `ref` | string or null | no | `null` | | `x` | integer or null | no | `null` | | `y` | integer or null | no | `null` | | `modifiers` | string or null | no | `null` | | `tab_id` | integer or null | no | `null` | | `intent` | string or null | no | `null` | ??? note "JSON schema" ```json { "properties": { "ref": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Ref" }, "x": { "anyOf": [ { "type": "integer" }, { "type": "null" } ], "default": null, "title": "X" }, "y": { "anyOf": [ { "type": "integer" }, { "type": "null" } ], "default": null, "title": "Y" }, "modifiers": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Modifiers" }, "tab_id": { "anyOf": [ { "type": "integer" }, { "type": "null" } ], "default": null, "title": "Tab Id" }, "intent": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Intent" } }, "title": "browser_right_clickArguments", "type": "object" } ``` ### `browser_middle_click` *Profile: full only.* Middle-click an element or a point (often opens a link in a new tab). Target an element with ref (preferred), or with x and y in screenshot pixels. **Args** - `ref`: Element ref such as "ref_12". - `x`: Horizontal position in screenshot pixels (with y). - `y`: Vertical position in screenshot pixels (with x). - `modifiers`: Keys to hold. - `tab_id`: Tab to act in (default: the active tab). - `intent`: Optional note on why you are doing this (logged only). ??? note "Exact text the model receives" ```text Middle-click an element or a point (often opens a link in a new tab). Target an element with ref (preferred), or with x and y in screenshot pixels. Args: ref: Element ref such as "ref_12". x: Horizontal position in screenshot pixels (with y). y: Vertical position in screenshot pixels (with x). modifiers: Keys to hold. tab_id: Tab to act in (default: the active tab). intent: Optional note on why you are doing this (logged only). ``` | Parameter | Type | Required | Default | |---|---|---|---| | `ref` | string or null | no | `null` | | `x` | integer or null | no | `null` | | `y` | integer or null | no | `null` | | `modifiers` | string or null | no | `null` | | `tab_id` | integer or null | no | `null` | | `intent` | string or null | no | `null` | ??? note "JSON schema" ```json { "properties": { "ref": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Ref" }, "x": { "anyOf": [ { "type": "integer" }, { "type": "null" } ], "default": null, "title": "X" }, "y": { "anyOf": [ { "type": "integer" }, { "type": "null" } ], "default": null, "title": "Y" }, "modifiers": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Modifiers" }, "tab_id": { "anyOf": [ { "type": "integer" }, { "type": "null" } ], "default": null, "title": "Tab Id" }, "intent": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Intent" } }, "title": "browser_middle_clickArguments", "type": "object" } ``` ### `browser_double_click` *Profile: full only.* Double-click an element or a point (for example to select a word). Target an element with ref (preferred), or with x and y in screenshot pixels. **Args** - `ref`: Element ref such as "ref_12". - `x`: Horizontal position in screenshot pixels (with y). - `y`: Vertical position in screenshot pixels (with x). - `modifiers`: Keys to hold. - `tab_id`: Tab to act in (default: the active tab). - `intent`: Optional note on why you are doing this (logged only). ??? note "Exact text the model receives" ```text Double-click an element or a point (for example to select a word). Target an element with ref (preferred), or with x and y in screenshot pixels. Args: ref: Element ref such as "ref_12". x: Horizontal position in screenshot pixels (with y). y: Vertical position in screenshot pixels (with x). modifiers: Keys to hold. tab_id: Tab to act in (default: the active tab). intent: Optional note on why you are doing this (logged only). ``` | Parameter | Type | Required | Default | |---|---|---|---| | `ref` | string or null | no | `null` | | `x` | integer or null | no | `null` | | `y` | integer or null | no | `null` | | `modifiers` | string or null | no | `null` | | `tab_id` | integer or null | no | `null` | | `intent` | string or null | no | `null` | ??? note "JSON schema" ```json { "properties": { "ref": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Ref" }, "x": { "anyOf": [ { "type": "integer" }, { "type": "null" } ], "default": null, "title": "X" }, "y": { "anyOf": [ { "type": "integer" }, { "type": "null" } ], "default": null, "title": "Y" }, "modifiers": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Modifiers" }, "tab_id": { "anyOf": [ { "type": "integer" }, { "type": "null" } ], "default": null, "title": "Tab Id" }, "intent": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Intent" } }, "title": "browser_double_clickArguments", "type": "object" } ``` ### `browser_triple_click` *Profile: full only.* Triple-click an element or a point (for example to select a paragraph). Target an element with ref (preferred), or with x and y in screenshot pixels. **Args** - `ref`: Element ref such as "ref_12". - `x`: Horizontal position in screenshot pixels (with y). - `y`: Vertical position in screenshot pixels (with x). - `modifiers`: Keys to hold. - `tab_id`: Tab to act in (default: the active tab). - `intent`: Optional note on why you are doing this (logged only). ??? note "Exact text the model receives" ```text Triple-click an element or a point (for example to select a paragraph). Target an element with ref (preferred), or with x and y in screenshot pixels. Args: ref: Element ref such as "ref_12". x: Horizontal position in screenshot pixels (with y). y: Vertical position in screenshot pixels (with x). modifiers: Keys to hold. tab_id: Tab to act in (default: the active tab). intent: Optional note on why you are doing this (logged only). ``` | Parameter | Type | Required | Default | |---|---|---|---| | `ref` | string or null | no | `null` | | `x` | integer or null | no | `null` | | `y` | integer or null | no | `null` | | `modifiers` | string or null | no | `null` | | `tab_id` | integer or null | no | `null` | | `intent` | string or null | no | `null` | ??? note "JSON schema" ```json { "properties": { "ref": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Ref" }, "x": { "anyOf": [ { "type": "integer" }, { "type": "null" } ], "default": null, "title": "X" }, "y": { "anyOf": [ { "type": "integer" }, { "type": "null" } ], "default": null, "title": "Y" }, "modifiers": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Modifiers" }, "tab_id": { "anyOf": [ { "type": "integer" }, { "type": "null" } ], "default": null, "title": "Tab Id" }, "intent": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Intent" } }, "title": "browser_triple_clickArguments", "type": "object" } ``` ### `browser_left_click_drag` *Profile: full only.* Drag with the left mouse button from one point to an element or point. Starts at (from_x, from_y) in screenshot pixels; ends at ref, or at x and y. **Args** - `from_x`: Start, horizontal, in screenshot pixels. - `from_y`: Start, vertical, in screenshot pixels. - `ref`: Element ref to drop on. - `x`: End, horizontal, in screenshot pixels (with y). - `y`: End, vertical, in screenshot pixels (with x). - `tab_id`: Tab to act in (default: the active tab). - `intent`: Optional note on why you are doing this (logged only). ??? note "Exact text the model receives" ```text Drag with the left mouse button from one point to an element or point. Starts at (from_x, from_y) in screenshot pixels; ends at ref, or at x and y. Args: from_x: Start, horizontal, in screenshot pixels. from_y: Start, vertical, in screenshot pixels. ref: Element ref to drop on. x: End, horizontal, in screenshot pixels (with y). y: End, vertical, in screenshot pixels (with x). tab_id: Tab to act in (default: the active tab). intent: Optional note on why you are doing this (logged only). ``` | Parameter | Type | Required | Default | |---|---|---|---| | `from_x` | integer | yes | — | | `from_y` | integer | yes | — | | `ref` | string or null | no | `null` | | `x` | integer or null | no | `null` | | `y` | integer or null | no | `null` | | `tab_id` | integer or null | no | `null` | | `intent` | string or null | no | `null` | ??? note "JSON schema" ```json { "properties": { "from_x": { "title": "From X", "type": "integer" }, "from_y": { "title": "From Y", "type": "integer" }, "ref": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Ref" }, "x": { "anyOf": [ { "type": "integer" }, { "type": "null" } ], "default": null, "title": "X" }, "y": { "anyOf": [ { "type": "integer" }, { "type": "null" } ], "default": null, "title": "Y" }, "tab_id": { "anyOf": [ { "type": "integer" }, { "type": "null" } ], "default": null, "title": "Tab Id" }, "intent": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Intent" } }, "required": [ "from_x", "from_y" ], "title": "browser_left_click_dragArguments", "type": "object" } ``` ### `browser_left_mouse_down` *Profile: full only.* Press and hold the left mouse button where the mouse is. **Args** - `tab_id`: Tab to act in (default: the active tab). - `intent`: Optional note on why you are doing this (logged only). ??? note "Exact text the model receives" ```text Press and hold the left mouse button where the mouse is. Args: tab_id: Tab to act in (default: the active tab). intent: Optional note on why you are doing this (logged only). ``` | Parameter | Type | Required | Default | |---|---|---|---| | `tab_id` | integer or null | no | `null` | | `intent` | string or null | no | `null` | ??? note "JSON schema" ```json { "properties": { "tab_id": { "anyOf": [ { "type": "integer" }, { "type": "null" } ], "default": null, "title": "Tab Id" }, "intent": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Intent" } }, "title": "browser_left_mouse_downArguments", "type": "object" } ``` ### `browser_left_mouse_up` *Profile: full only.* Release the left mouse button where the mouse is. **Args** - `tab_id`: Tab to act in (default: the active tab). - `intent`: Optional note on why you are doing this (logged only). ??? note "Exact text the model receives" ```text Release the left mouse button where the mouse is. Args: tab_id: Tab to act in (default: the active tab). intent: Optional note on why you are doing this (logged only). ``` | Parameter | Type | Required | Default | |---|---|---|---| | `tab_id` | integer or null | no | `null` | | `intent` | string or null | no | `null` | ??? note "JSON schema" ```json { "properties": { "tab_id": { "anyOf": [ { "type": "integer" }, { "type": "null" } ], "default": null, "title": "Tab Id" }, "intent": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Intent" } }, "title": "browser_left_mouse_upArguments", "type": "object" } ``` ### `browser_mouse_move` *Profile: full only.* Move the mouse to an element or a point without clicking. Use with browser_left_mouse_down and browser_left_mouse_up for drags that need intermediate steps. **Args** - `ref`: Element ref such as "ref_12". - `x`: Horizontal position in screenshot pixels (with y). - `y`: Vertical position in screenshot pixels (with x). - `tab_id`: Tab to act in (default: the active tab). - `intent`: Optional note on why you are doing this (logged only). ??? note "Exact text the model receives" ```text Move the mouse to an element or a point without clicking. Use with browser_left_mouse_down and browser_left_mouse_up for drags that need intermediate steps. Args: ref: Element ref such as "ref_12". x: Horizontal position in screenshot pixels (with y). y: Vertical position in screenshot pixels (with x). tab_id: Tab to act in (default: the active tab). intent: Optional note on why you are doing this (logged only). ``` | Parameter | Type | Required | Default | |---|---|---|---| | `ref` | string or null | no | `null` | | `x` | integer or null | no | `null` | | `y` | integer or null | no | `null` | | `tab_id` | integer or null | no | `null` | | `intent` | string or null | no | `null` | ??? note "JSON schema" ```json { "properties": { "ref": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Ref" }, "x": { "anyOf": [ { "type": "integer" }, { "type": "null" } ], "default": null, "title": "X" }, "y": { "anyOf": [ { "type": "integer" }, { "type": "null" } ], "default": null, "title": "Y" }, "tab_id": { "anyOf": [ { "type": "integer" }, { "type": "null" } ], "default": null, "title": "Tab Id" }, "intent": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Intent" } }, "title": "browser_mouse_moveArguments", "type": "object" } ``` ### `browser_hold_key` *Profile: full only.* Hold a key or chord down for a number of seconds (at most 30). **Args** - `text`: The key or chord, such as "shift" or "ArrowRight". - `duration`: Seconds to hold, at most 30. - `tab_id`: Tab to act in (default: the active tab). - `intent`: Optional note on why you are doing this (logged only). ??? note "Exact text the model receives" ```text Hold a key or chord down for a number of seconds (at most 30). Args: text: The key or chord, such as "shift" or "ArrowRight". duration: Seconds to hold, at most 30. tab_id: Tab to act in (default: the active tab). intent: Optional note on why you are doing this (logged only). ``` | Parameter | Type | Required | Default | |---|---|---|---| | `text` | string | yes | — | | `duration` | number | yes | — | | `tab_id` | integer or null | no | `null` | | `intent` | string or null | no | `null` | ??? note "JSON schema" ```json { "properties": { "text": { "title": "Text", "type": "string" }, "duration": { "title": "Duration", "type": "number" }, "tab_id": { "anyOf": [ { "type": "integer" }, { "type": "null" } ], "default": null, "title": "Tab Id" }, "intent": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Intent" } }, "required": [ "text", "duration" ], "title": "browser_hold_keyArguments", "type": "object" } ``` ### `browser_javascript_exec` *Opt-in: add it to DPTTOOLS_BROWSER_ENABLE.* Run a JavaScript expression in the page and return its value (opt-in). The value is returned as text (JSON for objects); a promise is awaited. Use only when the other tools cannot do the job. **Args** - `expression`: The JavaScript expression, or a function body like "() => document.title". - `tab_id`: Tab to run in (default: the active tab). - `intent`: Optional note on why you are doing this (logged only). ??? note "Exact text the model receives" ```text Run a JavaScript expression in the page and return its value (opt-in). The value is returned as text (JSON for objects); a promise is awaited. Use only when the other tools cannot do the job. Args: expression: The JavaScript expression, or a function body like "() => document.title". tab_id: Tab to run in (default: the active tab). intent: Optional note on why you are doing this (logged only). ``` | Parameter | Type | Required | Default | |---|---|---|---| | `expression` | string | yes | — | | `tab_id` | integer or null | no | `null` | | `intent` | string or null | no | `null` | ??? note "JSON schema" ```json { "properties": { "expression": { "title": "Expression", "type": "string" }, "tab_id": { "anyOf": [ { "type": "integer" }, { "type": "null" } ], "default": null, "title": "Tab Id" }, "intent": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Intent" } }, "required": [ "expression" ], "title": "browser_javascript_execArguments", "type": "object" } ``` ### `browser_file_upload` *Opt-in: add it to DPTTOOLS_BROWSER_ENABLE.* Attach files to a file input (opt-in). Files must be in the upload directory the operator configured; give paths relative to it (or absolute paths inside it). **Args** - `ref`: Element ref of the file input. - `paths`: Files to attach. - `tab_id`: Tab to act in (default: the active tab). - `intent`: Optional note on why you are doing this (logged only). ??? note "Exact text the model receives" ```text Attach files to a file input (opt-in). Files must be in the upload directory the operator configured; give paths relative to it (or absolute paths inside it). Args: ref: Element ref of the file input. paths: Files to attach. tab_id: Tab to act in (default: the active tab). intent: Optional note on why you are doing this (logged only). ``` | Parameter | Type | Required | Default | |---|---|---|---| | `ref` | string | yes | — | | `paths` | array of string | yes | — | | `tab_id` | integer or null | no | `null` | | `intent` | string or null | no | `null` | ??? note "JSON schema" ```json { "properties": { "ref": { "title": "Ref", "type": "string" }, "paths": { "items": { "type": "string" }, "title": "Paths", "type": "array" }, "tab_id": { "anyOf": [ { "type": "integer" }, { "type": "null" } ], "default": null, "title": "Tab Id" }, "intent": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Intent" } }, "required": [ "ref", "paths" ], "title": "browser_file_uploadArguments", "type": "object" } ``` ### `browser_read_console` *Opt-in: add it to DPTTOOLS_BROWSER_ENABLE.* Read the page's console messages since the last read (opt-in). **Args** - `tab_id`: Tab to read (default: the active tab). - `intent`: Optional note on why you are doing this (logged only). ??? note "Exact text the model receives" ```text Read the page's console messages since the last read (opt-in). Args: tab_id: Tab to read (default: the active tab). intent: Optional note on why you are doing this (logged only). ``` | Parameter | Type | Required | Default | |---|---|---|---| | `tab_id` | integer or null | no | `null` | | `intent` | string or null | no | `null` | ??? note "JSON schema" ```json { "properties": { "tab_id": { "anyOf": [ { "type": "integer" }, { "type": "null" } ], "default": null, "title": "Tab Id" }, "intent": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Intent" } }, "title": "browser_read_consoleArguments", "type": "object" } ``` ### `browser_read_network` *Opt-in: add it to DPTTOOLS_BROWSER_ENABLE.* List the page's network requests since the last read (opt-in). **Args** - `tab_id`: Tab to read (default: the active tab). - `intent`: Optional note on why you are doing this (logged only). ??? note "Exact text the model receives" ```text List the page's network requests since the last read (opt-in). Args: tab_id: Tab to read (default: the active tab). intent: Optional note on why you are doing this (logged only). ``` | Parameter | Type | Required | Default | |---|---|---|---| | `tab_id` | integer or null | no | `null` | | `intent` | string or null | no | `null` | ??? note "JSON schema" ```json { "properties": { "tab_id": { "anyOf": [ { "type": "integer" }, { "type": "null" } ], "default": null, "title": "Tab Id" }, "intent": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Intent" } }, "title": "browser_read_networkArguments", "type": "object" } ``` ### `browser_run_script` *Opt-in: add it to DPTTOOLS_BROWSER_ENABLE.* Run a Python script against the browser with Playwright (opt-in). For multi-step work the other tools make slow, such as extracting a whole table or looping over pages. The script runs in a separate process, with these names ready to use (Playwright's async API, so use await): - `page`: the tab (`await page.title()`, `await page.locator("table").inner_text()`) - `context`: the browser context - `await screenshot()`: capture the tab; the image is returned with the result - `await read_page()`: the accessibility tree with refs other tools accept `print()` output and the value of a final expression are returned. Variables persist between calls in the same session. A script that runs longer than `DPTTOOLS_BROWSER_SCRIPT_TIMEOUT_S` (60 s) is stopped and the variables are lost. The script is not network-contained: connections it opens itself are not checked. The browser's own traffic is (in cdp mode, only in the tabs this server manages). **Args** - `code`: Python code; top-level await is allowed. - `tab_id`: Tab bound to `page` (default: the active tab). - `intent`: Optional note on why you are doing this (logged only). ??? note "Exact text the model receives" ```text Run a Python script against the browser with Playwright (opt-in). For multi-step work the other tools make slow, such as extracting a whole table or looping over pages. The script runs in a separate process, with these names ready to use (Playwright's async API, so use await): - ``page``: the tab (``await page.title()``, ``await page.locator("table").inner_text()``) - ``context``: the browser context - ``await screenshot()``: capture the tab; the image is returned with the result - ``await read_page()``: the accessibility tree with refs other tools accept ``print()`` output and the value of a final expression are returned. Variables persist between calls in the same session. A script that runs longer than ``DPTTOOLS_BROWSER_SCRIPT_TIMEOUT_S`` (60 s) is stopped and the variables are lost. The script is not network-contained: connections it opens itself are not checked. The browser's own traffic is (in cdp mode, only in the tabs this server manages). Args: code: Python code; top-level await is allowed. tab_id: Tab bound to ``page`` (default: the active tab). intent: Optional note on why you are doing this (logged only). ``` | Parameter | Type | Required | Default | |---|---|---|---| | `code` | string | yes | — | | `tab_id` | integer or null | no | `null` | | `intent` | string or null | no | `null` | ??? note "JSON schema" ```json { "properties": { "code": { "title": "Code", "type": "string" }, "tab_id": { "anyOf": [ { "type": "integer" }, { "type": "null" } ], "default": null, "title": "Tab Id" }, "intent": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Intent" } }, "required": [ "code" ], "title": "browser_run_scriptArguments", "type": "object" } ``` ### `fetch_webpage_rendered` *Registered in both profiles.* Fetch a web page in a real browser, running its JavaScript, and return its content. Use this when `fetch_webpage` returns little or no text, or says `likely_js_rendered: true`: single-page apps and client-rendered pages only have content after their scripts run. Slower than `fetch_webpage` (seconds, not milliseconds), so try that first. Returns the same JSON object as `fetch_webpage` (`url`, `final_url`, `status`, `title`, `format`, `extractor`, `text`, `truncated`, `total_chars`, `offset`, `next_offset`, `has_more`, plus `"rendered": true`). Each call uses a fresh, credential-free browser context that is closed afterwards; the browser's network policy applies (private addresses refused). **Args** - `url`: The page to fetch (http or https). - `max_length`: Maximum characters of text to return (default 10000). - `offset`: Character offset to start from, for long pages (default 0). - `format`: "text" (default), "markdown" or "html". - `include_assets`: Add the page's image and link references (default False). ??? note "Exact text the model receives" ```text Fetch a web page in a real browser, running its JavaScript, and return its content. Use this when ``fetch_webpage`` returns little or no text, or says ``likely_js_rendered: true``: single-page apps and client-rendered pages only have content after their scripts run. Slower than ``fetch_webpage`` (seconds, not milliseconds), so try that first. Returns the same JSON object as ``fetch_webpage`` (``url``, ``final_url``, ``status``, ``title``, ``format``, ``extractor``, ``text``, ``truncated``, ``total_chars``, ``offset``, ``next_offset``, ``has_more``, plus ``"rendered": true``). Each call uses a fresh, credential-free browser context that is closed afterwards; the browser's network policy applies (private addresses refused). Args: url: The page to fetch (http or https). max_length: Maximum characters of text to return (default 10000). offset: Character offset to start from, for long pages (default 0). format: "text" (default), "markdown" or "html". include_assets: Add the page's image and link references (default False). ``` | Parameter | Type | Required | Default | |---|---|---|---| | `url` | string | yes | — | | `max_length` | integer | no | `10000` | | `offset` | integer | no | `0` | | `format` | string | no | `"text"` | | `include_assets` | boolean | no | `false` | ??? note "JSON schema" ```json { "properties": { "url": { "title": "Url", "type": "string" }, "max_length": { "default": 10000, "title": "Max Length", "type": "integer" }, "offset": { "default": 0, "title": "Offset", "type": "integer" }, "format": { "default": "text", "enum": [ "text", "markdown", "html" ], "title": "Format", "type": "string" }, "include_assets": { "default": false, "title": "Include Assets", "type": "boolean" } }, "required": [ "url" ], "title": "fetch_webpage_renderedArguments", "type": "object" } ``` ## computer server `computer-tools` — see the [server page](../servers/computer.md). Composed only when `DPTTOOLS_ENABLE_COMPUTER=true`. ### `computer_screenshot` Take a screenshot of the desktop. x/y arguments of the other computer tools are in this screenshot's pixels. Take one before acting, and again to check a result. **Args** - `intent`: Optional note on why you are doing this (logged only). ??? note "Exact text the model receives" ```text Take a screenshot of the desktop. x/y arguments of the other computer tools are in this screenshot's pixels. Take one before acting, and again to check a result. Args: intent: Optional note on why you are doing this (logged only). ``` | Parameter | Type | Required | Default | |---|---|---|---| | `intent` | string or null | no | `null` | ??? note "JSON schema" ```json { "properties": { "intent": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Intent" } }, "title": "computer_screenshotArguments", "type": "object" } ``` ### `computer_zoom` Capture one region of the screen, enlarged, to read small text. **Args** - `x0`: Left edge of the region, in screenshot pixels. - `y0`: Top edge of the region, in screenshot pixels. - `x1`: Right edge of the region, in screenshot pixels. - `y1`: Bottom edge of the region, in screenshot pixels. - `intent`: Optional note on why you are doing this (logged only). ??? note "Exact text the model receives" ```text Capture one region of the screen, enlarged, to read small text. Args: x0: Left edge of the region, in screenshot pixels. y0: Top edge of the region, in screenshot pixels. x1: Right edge of the region, in screenshot pixels. y1: Bottom edge of the region, in screenshot pixels. intent: Optional note on why you are doing this (logged only). ``` | Parameter | Type | Required | Default | |---|---|---|---| | `x0` | integer | yes | — | | `y0` | integer | yes | — | | `x1` | integer | yes | — | | `y1` | integer | yes | — | | `intent` | string or null | no | `null` | ??? note "JSON schema" ```json { "properties": { "x0": { "title": "X0", "type": "integer" }, "y0": { "title": "Y0", "type": "integer" }, "x1": { "title": "X1", "type": "integer" }, "y1": { "title": "Y1", "type": "integer" }, "intent": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Intent" } }, "required": [ "x0", "y0", "x1", "y1" ], "title": "computer_zoomArguments", "type": "object" } ``` ### `computer_left_click` Click the left mouse button at a point. **Args** - `x`: Horizontal position in screenshot pixels. - `y`: Vertical position in screenshot pixels. - `modifiers`: Keys to hold while clicking, such as "ctrl" or "shift+ctrl": key names joined by "+". - `intent`: Optional note on why you are doing this (logged only). ??? note "Exact text the model receives" ```text Click the left mouse button at a point. Args: x: Horizontal position in screenshot pixels. y: Vertical position in screenshot pixels. modifiers: Keys to hold while clicking, such as "ctrl" or "shift+ctrl": key names joined by "+". intent: Optional note on why you are doing this (logged only). ``` | Parameter | Type | Required | Default | |---|---|---|---| | `x` | integer | yes | — | | `y` | integer | yes | — | | `modifiers` | string or null | no | `null` | | `intent` | string or null | no | `null` | ??? note "JSON schema" ```json { "properties": { "x": { "title": "X", "type": "integer" }, "y": { "title": "Y", "type": "integer" }, "modifiers": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Modifiers" }, "intent": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Intent" } }, "required": [ "x", "y" ], "title": "computer_left_clickArguments", "type": "object" } ``` ### `computer_right_click` Click the right mouse button at a point (opens a context menu). **Args** - `x`: Horizontal position in screenshot pixels. - `y`: Vertical position in screenshot pixels. - `modifiers`: Keys to hold while clicking, such as "ctrl" or "shift+ctrl": key names joined by "+". - `intent`: Optional note on why you are doing this (logged only). ??? note "Exact text the model receives" ```text Click the right mouse button at a point (opens a context menu). Args: x: Horizontal position in screenshot pixels. y: Vertical position in screenshot pixels. modifiers: Keys to hold while clicking, such as "ctrl" or "shift+ctrl": key names joined by "+". intent: Optional note on why you are doing this (logged only). ``` | Parameter | Type | Required | Default | |---|---|---|---| | `x` | integer | yes | — | | `y` | integer | yes | — | | `modifiers` | string or null | no | `null` | | `intent` | string or null | no | `null` | ??? note "JSON schema" ```json { "properties": { "x": { "title": "X", "type": "integer" }, "y": { "title": "Y", "type": "integer" }, "modifiers": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Modifiers" }, "intent": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Intent" } }, "required": [ "x", "y" ], "title": "computer_right_clickArguments", "type": "object" } ``` ### `computer_middle_click` Click the middle mouse button at a point. **Args** - `x`: Horizontal position in screenshot pixels. - `y`: Vertical position in screenshot pixels. - `modifiers`: Keys to hold while clicking, such as "ctrl" or "shift+ctrl": key names joined by "+". - `intent`: Optional note on why you are doing this (logged only). ??? note "Exact text the model receives" ```text Click the middle mouse button at a point. Args: x: Horizontal position in screenshot pixels. y: Vertical position in screenshot pixels. modifiers: Keys to hold while clicking, such as "ctrl" or "shift+ctrl": key names joined by "+". intent: Optional note on why you are doing this (logged only). ``` | Parameter | Type | Required | Default | |---|---|---|---| | `x` | integer | yes | — | | `y` | integer | yes | — | | `modifiers` | string or null | no | `null` | | `intent` | string or null | no | `null` | ??? note "JSON schema" ```json { "properties": { "x": { "title": "X", "type": "integer" }, "y": { "title": "Y", "type": "integer" }, "modifiers": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Modifiers" }, "intent": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Intent" } }, "required": [ "x", "y" ], "title": "computer_middle_clickArguments", "type": "object" } ``` ### `computer_double_click` Double-click at a point (for example to open a file or select a word). **Args** - `x`: Horizontal position in screenshot pixels. - `y`: Vertical position in screenshot pixels. - `modifiers`: Keys to hold while clicking, such as "ctrl" or "shift+ctrl": key names joined by "+". - `intent`: Optional note on why you are doing this (logged only). ??? note "Exact text the model receives" ```text Double-click at a point (for example to open a file or select a word). Args: x: Horizontal position in screenshot pixels. y: Vertical position in screenshot pixels. modifiers: Keys to hold while clicking, such as "ctrl" or "shift+ctrl": key names joined by "+". intent: Optional note on why you are doing this (logged only). ``` | Parameter | Type | Required | Default | |---|---|---|---| | `x` | integer | yes | — | | `y` | integer | yes | — | | `modifiers` | string or null | no | `null` | | `intent` | string or null | no | `null` | ??? note "JSON schema" ```json { "properties": { "x": { "title": "X", "type": "integer" }, "y": { "title": "Y", "type": "integer" }, "modifiers": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Modifiers" }, "intent": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Intent" } }, "required": [ "x", "y" ], "title": "computer_double_clickArguments", "type": "object" } ``` ### `computer_triple_click` Triple-click at a point (for example to select a line). **Args** - `x`: Horizontal position in screenshot pixels. - `y`: Vertical position in screenshot pixels. - `modifiers`: Keys to hold while clicking, such as "ctrl" or "shift+ctrl": key names joined by "+". - `intent`: Optional note on why you are doing this (logged only). ??? note "Exact text the model receives" ```text Triple-click at a point (for example to select a line). Args: x: Horizontal position in screenshot pixels. y: Vertical position in screenshot pixels. modifiers: Keys to hold while clicking, such as "ctrl" or "shift+ctrl": key names joined by "+". intent: Optional note on why you are doing this (logged only). ``` | Parameter | Type | Required | Default | |---|---|---|---| | `x` | integer | yes | — | | `y` | integer | yes | — | | `modifiers` | string or null | no | `null` | | `intent` | string or null | no | `null` | ??? note "JSON schema" ```json { "properties": { "x": { "title": "X", "type": "integer" }, "y": { "title": "Y", "type": "integer" }, "modifiers": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Modifiers" }, "intent": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Intent" } }, "required": [ "x", "y" ], "title": "computer_triple_clickArguments", "type": "object" } ``` ### `computer_left_click_drag` Drag with the left mouse button from one point to another. **Args** - `from_x`: Start, horizontal, in screenshot pixels. - `from_y`: Start, vertical, in screenshot pixels. - `x`: End, horizontal, in screenshot pixels. - `y`: End, vertical, in screenshot pixels. - `intent`: Optional note on why you are doing this (logged only). ??? note "Exact text the model receives" ```text Drag with the left mouse button from one point to another. Args: from_x: Start, horizontal, in screenshot pixels. from_y: Start, vertical, in screenshot pixels. x: End, horizontal, in screenshot pixels. y: End, vertical, in screenshot pixels. intent: Optional note on why you are doing this (logged only). ``` | Parameter | Type | Required | Default | |---|---|---|---| | `from_x` | integer | yes | — | | `from_y` | integer | yes | — | | `x` | integer | yes | — | | `y` | integer | yes | — | | `intent` | string or null | no | `null` | ??? note "JSON schema" ```json { "properties": { "from_x": { "title": "From X", "type": "integer" }, "from_y": { "title": "From Y", "type": "integer" }, "x": { "title": "X", "type": "integer" }, "y": { "title": "Y", "type": "integer" }, "intent": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Intent" } }, "required": [ "from_x", "from_y", "x", "y" ], "title": "computer_left_click_dragArguments", "type": "object" } ``` ### `computer_mouse_move` Move the mouse to a point without clicking. **Args** - `x`: Horizontal position in screenshot pixels. - `y`: Vertical position in screenshot pixels. - `intent`: Optional note on why you are doing this (logged only). ??? note "Exact text the model receives" ```text Move the mouse to a point without clicking. Args: x: Horizontal position in screenshot pixels. y: Vertical position in screenshot pixels. intent: Optional note on why you are doing this (logged only). ``` | Parameter | Type | Required | Default | |---|---|---|---| | `x` | integer | yes | — | | `y` | integer | yes | — | | `intent` | string or null | no | `null` | ??? note "JSON schema" ```json { "properties": { "x": { "title": "X", "type": "integer" }, "y": { "title": "Y", "type": "integer" }, "intent": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Intent" } }, "required": [ "x", "y" ], "title": "computer_mouse_moveArguments", "type": "object" } ``` ### `computer_left_mouse_down` Press and hold the left mouse button where the mouse is. **Args** - `intent`: Optional note on why you are doing this (logged only). ??? note "Exact text the model receives" ```text Press and hold the left mouse button where the mouse is. Args: intent: Optional note on why you are doing this (logged only). ``` | Parameter | Type | Required | Default | |---|---|---|---| | `intent` | string or null | no | `null` | ??? note "JSON schema" ```json { "properties": { "intent": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Intent" } }, "title": "computer_left_mouse_downArguments", "type": "object" } ``` ### `computer_left_mouse_up` Release the left mouse button where the mouse is. **Args** - `intent`: Optional note on why you are doing this (logged only). ??? note "Exact text the model receives" ```text Release the left mouse button where the mouse is. Args: intent: Optional note on why you are doing this (logged only). ``` | Parameter | Type | Required | Default | |---|---|---|---| | `intent` | string or null | no | `null` | ??? note "JSON schema" ```json { "properties": { "intent": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Intent" } }, "title": "computer_left_mouse_upArguments", "type": "object" } ``` ### `computer_cursor_position` Report where the mouse cursor is, in screenshot pixels. **Args** - `intent`: Optional note on why you are doing this (logged only). ??? note "Exact text the model receives" ```text Report where the mouse cursor is, in screenshot pixels. Args: intent: Optional note on why you are doing this (logged only). ``` | Parameter | Type | Required | Default | |---|---|---|---| | `intent` | string or null | no | `null` | ??? note "JSON schema" ```json { "properties": { "intent": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Intent" } }, "title": "computer_cursor_positionArguments", "type": "object" } ``` ### `computer_scroll` Scroll at a point (default: the centre of the screen). **Args** - `scroll_direction`: up, down, left or right. - `scroll_amount`: 1-10 wheel steps (default 3). - `x`: Horizontal position in screenshot pixels (with y). - `y`: Vertical position in screenshot pixels (with x). - `intent`: Optional note on why you are doing this (logged only). ??? note "Exact text the model receives" ```text Scroll at a point (default: the centre of the screen). Args: scroll_direction: up, down, left or right. scroll_amount: 1-10 wheel steps (default 3). x: Horizontal position in screenshot pixels (with y). y: Vertical position in screenshot pixels (with x). intent: Optional note on why you are doing this (logged only). ``` | Parameter | Type | Required | Default | |---|---|---|---| | `scroll_direction` | string | yes | — | | `scroll_amount` | integer | no | `3` | | `x` | integer or null | no | `null` | | `y` | integer or null | no | `null` | | `intent` | string or null | no | `null` | ??? note "JSON schema" ```json { "properties": { "scroll_direction": { "enum": [ "up", "down", "left", "right" ], "title": "Scroll Direction", "type": "string" }, "scroll_amount": { "default": 3, "title": "Scroll Amount", "type": "integer" }, "x": { "anyOf": [ { "type": "integer" }, { "type": "null" } ], "default": null, "title": "X" }, "y": { "anyOf": [ { "type": "integer" }, { "type": "null" } ], "default": null, "title": "Y" }, "intent": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Intent" } }, "required": [ "scroll_direction" ], "title": "computer_scrollArguments", "type": "object" } ``` ### `computer_type` Type text, as if on the keyboard, into whatever has focus. Text only: tabs and newlines are typed, other control characters are refused. Press keys such as Delete, Escape or BackSpace with computer_key. **Args** - `text`: The text to type. - `intent`: Optional note on why you are doing this (logged only). ??? note "Exact text the model receives" ```text Type text, as if on the keyboard, into whatever has focus. Text only: tabs and newlines are typed, other control characters are refused. Press keys such as Delete, Escape or BackSpace with computer_key. Args: text: The text to type. intent: Optional note on why you are doing this (logged only). ``` | Parameter | Type | Required | Default | |---|---|---|---| | `text` | string | yes | — | | `intent` | string or null | no | `null` | ??? note "JSON schema" ```json { "properties": { "text": { "title": "Text", "type": "string" }, "intent": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Intent" } }, "required": [ "text" ], "title": "computer_typeArguments", "type": "object" } ``` ### `computer_key` Press a key, a chord, or a space-separated sequence of them. Each key is a single character or an xdotool key name made of letters, digits and "_", spelled as xdotool spells it: "Return", "Tab", "Page_Down", "super", "XF86AudioMute"; never a key code such as "119". Join keys into a chord with "+" ("ctrl+a", "alt+Tab") and separate a sequence with spaces ("ctrl+a BackSpace"). The plus key itself is "+" ("ctrl++" zooms in). Anything else is refused. **Args** - `text`: The key, chord or sequence. - `repeat`: Times to press the whole sequence, 1-100 (default 1). - `intent`: Optional note on why you are doing this (logged only). ??? note "Exact text the model receives" ```text Press a key, a chord, or a space-separated sequence of them. Each key is a single character or an xdotool key name made of letters, digits and "_", spelled as xdotool spells it: "Return", "Tab", "Page_Down", "super", "XF86AudioMute"; never a key code such as "119". Join keys into a chord with "+" ("ctrl+a", "alt+Tab") and separate a sequence with spaces ("ctrl+a BackSpace"). The plus key itself is "+" ("ctrl++" zooms in). Anything else is refused. Args: text: The key, chord or sequence. repeat: Times to press the whole sequence, 1-100 (default 1). intent: Optional note on why you are doing this (logged only). ``` | Parameter | Type | Required | Default | |---|---|---|---| | `text` | string | yes | — | | `repeat` | integer | no | `1` | | `intent` | string or null | no | `null` | ??? note "JSON schema" ```json { "properties": { "text": { "title": "Text", "type": "string" }, "repeat": { "default": 1, "title": "Repeat", "type": "integer" }, "intent": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Intent" } }, "required": [ "text" ], "title": "computer_keyArguments", "type": "object" } ``` ### `computer_hold_key` Hold a key or chord down for a number of seconds (at most 30). **Args** - `text`: The key or chord, such as "shift" or "Right": single characters or key names joined by "+". - `duration`: Seconds to hold, at most 30. - `intent`: Optional note on why you are doing this (logged only). ??? note "Exact text the model receives" ```text Hold a key or chord down for a number of seconds (at most 30). Args: text: The key or chord, such as "shift" or "Right": single characters or key names joined by "+". duration: Seconds to hold, at most 30. intent: Optional note on why you are doing this (logged only). ``` | Parameter | Type | Required | Default | |---|---|---|---| | `text` | string | yes | — | | `duration` | number | yes | — | | `intent` | string or null | no | `null` | ??? note "JSON schema" ```json { "properties": { "text": { "title": "Text", "type": "string" }, "duration": { "title": "Duration", "type": "number" }, "intent": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Intent" } }, "required": [ "text", "duration" ], "title": "computer_hold_keyArguments", "type": "object" } ``` ### `computer_wait` Wait a number of seconds (at most 30), for an application to respond. **Args** - `duration`: Seconds to wait, at most 30. - `intent`: Optional note on why you are doing this (logged only). ??? note "Exact text the model receives" ```text Wait a number of seconds (at most 30), for an application to respond. Args: duration: Seconds to wait, at most 30. intent: Optional note on why you are doing this (logged only). ``` | Parameter | Type | Required | Default | |---|---|---|---| | `duration` | number | yes | — | | `intent` | string or null | no | `null` | ??? note "JSON schema" ```json { "properties": { "duration": { "title": "Duration", "type": "number" }, "intent": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Intent" } }, "required": [ "duration" ], "title": "computer_waitArguments", "type": "object" } ``` <!-- source: docs/reference/api/index.md --> # API Reference These pages are generated from the source docstrings and type hints with `mkdocstrings`. They are the Python view of each module. A tool function's docstring is also the description the model sees for that tool, so the text here is what an agent reads when it chooses a tool. The [Tool Reference](../tools.md) shows each tool's exact description and parameter schema as the MCP server sends them. ## Coverage - [File System](file_system.md) — `doc_processing_std_tools.file_system` - [Web Search](web_search.md) — `doc_processing_std_tools.web_search` - [Memory](memory.md) — `doc_processing_std_tools.memory` - [Document Processing](document_processing.md) — `doc_processing_std_tools.document_processing` - [Code Execution](code_execution.md) — `doc_processing_std_tools.code_execution` - [Screenshot](screenshot.md) — `doc_processing_std_tools.screenshot` - [Image Tools](image_tools.md) — `doc_processing_std_tools.image_tools` - [Skills](skills.md) — `doc_processing_std_tools.skills` - [Browser](browser.md) — `doc_processing_std_tools.browser` (optional server) - [Computer](computer.md) — `doc_processing_std_tools.computer` and its backends, `doc_processing_std_tools._computer_backends` (optional server) - [Launcher](launcher.md) — `doc_processing_std_tools` and `doc_processing_std_tools.__main__` - [Shared Internals](config_and_safety.md) — `doc_processing_std_tools._config`, `doc_processing_std_tools._safety`, `doc_processing_std_tools._images`, `doc_processing_std_tools._display` and `doc_processing_std_tools._sessions`, the private modules the servers share ## Notes - The pages reflect the source under `src/` in this version of the documentation. - For how to use each server, start with its page under **Servers**; these pages are the reference. <!-- source: docs/reference/api/file_system.md --> # File System API ## File System Server ::: doc_processing_std_tools.file_system <!-- source: docs/reference/api/web_search.md --> # Web Search API ## Web Search Server ::: doc_processing_std_tools.web_search <!-- source: docs/reference/api/memory.md --> # Memory API ## Memory Server ::: doc_processing_std_tools.memory <!-- source: docs/reference/api/document_processing.md --> # Document Processing API ## Document Processing Server ::: doc_processing_std_tools.document_processing <!-- source: docs/reference/api/code_execution.md --> # Code Execution API ## Code Execution Server ::: doc_processing_std_tools.code_execution <!-- source: docs/reference/api/screenshot.md --> # Screenshot API ## Screenshot Server ::: doc_processing_std_tools.screenshot <!-- source: docs/reference/api/image_tools.md --> # Image Tools API ## Image Tools Server ::: doc_processing_std_tools.image_tools <!-- source: docs/reference/api/skills.md --> # Skills API !!! note "The five skills tools are not listed as functions here" `list_skills`, `load_skill`, `load_skill_resource`, `execute_skill_script` and `create_skill` are registered inside `_register_skills_tools()`, which both `skills_server` and `make_skills_server()` call, so they are not module-level functions that mkdocstrings can document. Their parameters and behaviour are on the [Skills server](../../servers/skills.md) page, and the [Tool Reference](../tools.md) will carry their exact descriptions. ## Skills Server ::: doc_processing_std_tools.skills <!-- source: docs/reference/api/browser.md --> # Browser API !!! note "Tools, not just functions" The functions below are registered as MCP tools by `make_browser_server()`, named `browser_<member>`; which ones are registered depends on the profile and the opt-in list. Their exact model-visible descriptions are in the [Tool Reference](../tools.md#browser-server), and how to use them is on the [Browser server](../../servers/browser.md) page. ## Browser Server ::: doc_processing_std_tools.browser <!-- source: docs/reference/api/computer.md --> # Computer API !!! note "Tools, not just functions" The `computer_*` functions below are registered as MCP tools by `make_computer_server()`. Their exact model-visible descriptions are in the [Tool Reference](../tools.md#computer-server), and how to use them is on the [Computer (Desktop) server](../../servers/computer.md) page. ## Computer Server ::: doc_processing_std_tools.computer ## Backends and Key Chords ::: doc_processing_std_tools._computer_backends <!-- source: docs/reference/api/launcher.md --> # Launcher API ## Package ::: doc_processing_std_tools ## Composed Launcher ::: doc_processing_std_tools.__main__ <!-- source: docs/reference/api/config_and_safety.md --> # Shared Internals API The modules the servers share. They are private (their names start with `_`): documented here for contributors and for reading the source, not as a stable API. ## Configuration ::: doc_processing_std_tools._config ## Safety Utilities ::: doc_processing_std_tools._safety ## Image Sizing ::: doc_processing_std_tools._images ## Screen Capture Used by `take_screenshot` and the computer server's `host` backend. ::: doc_processing_std_tools._display ## Per-session State Maps an MCP session to a stable key, for the browser and computer servers. ::: doc_processing_std_tools._sessions <!-- source: docs/changelog.md --> # Changelog All notable changes to `doc-processing-std-tools` are recorded here. Versions follow `vMAJOR.MINOR.PATCH`, equal to `pyproject.toml`'s `version`, and every release is tagged. ## Unreleased Nothing yet. ## 0.6.1 — 2026-10-09 A security and documentation release: the desktop server's key input can no longer run commands or slip past the confirmation gate, the private-address check covers every non-public address, and the docs site is ready to be linked publicly. Upgrade by pinning `@v0.6.1`. ### Migration notes - **The fetch tools and the browser refuse every address that is not public.** Carrier-grade NAT addresses (`100.64.0.0/10`, which Tailscale uses), `0.0.0.0`, the IETF protocol block `192.0.0.0/24`, the benchmarking, documentation and reserved ranges, multicast, and IPv4 addresses inside IPv6 forms were reachable before and are now refused. If `fetch_webpage`, `fetch_image` or the browser reach an internal host at such an address, add its hostname to `DPTTOOLS_WEB_FETCH_ALLOW_HOSTS` and `DPTTOOLS_BROWSER_ALLOW_HOSTS`. - **The desktop server checks key names.** Each key in `computer_key`, `computer_hold_key` and the click tools' `modifiers` must be a single character or a key name (letters, digits and `_`, such as `Page_Down` or `XF86AudioMute`). A name that starts with a digit must be a hex keysym (`0x1008ff13`) or one of the `3270_*` names: `xdotool` reads any other, such as `119`, as a raw key code. Anything else is refused with an `Error:` naming it. Every real X key name still works; a client that sent other text as a key gets an error instead of a silent no-op. Upgrade: this closes a security issue (below). - **The desktop confirmation gate covers more.** With `DPTTOOLS_COMPUTER_CONFIRM=deny` or `elicit`, a chord that contains a gated chord is gated too (`ctrl+shift+w`, `alt+shift+F4`, `ctrl+Delete`, `Alt_L+F4`, `alt+0xffc1`, `ctrl+U0077`), and so are the click tools' `modifiers` (`modifiers="alt+F4"`). A pipeline on `deny` that pressed such a chord now gets an `Error:`; with `off`, the default, nothing changes. Separately, whatever the gate's setting, `computer_type` refuses control characters other than tab, newline and carriage return; press keys such as Delete, Escape or BackSpace with `computer_key`. - **`ocr_pdf` output can differ.** Pages are labelled in document order, and the note on a result cut on a page boundary names exactly the pages left out (`Re-call ocr_pdf with pages='3,7' to read them.`). Code that parses the labels or the note should be checked against the new form. - **Install hints in errors changed wording.** A missing optional library now gives "Install the `[docs]` extra." rather than a pip command; code that matches the old text must match the new one. ### Added - The docs site's eight recipes are written: OCR a scanned PDF, Office files to page images, search then summarise, JavaScript-rendered pages, portal download to text, desktop actions with hooks, skills setup, and the launcher with other MCP clients. Each has its goal, steps, what to validate and the tests behind it, and each example was run against real services or software (LLMWhisperer, Tavily, LibreOffice, Chromium, a virtual desktop and a toolkit agent). The home page, the server pages and the README link them. - The docs site has a link-preview card. Every page carries Open Graph and Twitter tags (title, description, the card image), from a theme override in `overrides/`, so a shared link shows a preview. The site also publishes a `robots.txt` that names the sitemap, and the Architecture page opens with a diagram of the launcher, the servers and the safety controls. ### Changed - The README links the documentation site and gains a Documentation section, as the toolkit's README has: how to serve the site locally, an index of its key pages, the API reference, and the `llms.txt` / `llms-full.txt` files for coding agents. Its Overview now mentions the two optional servers. - Missing-dependency errors and tool descriptions name the extra to install ("Install the `[docs]` extra.") instead of giving a pip command for the package. std-tools is not on PyPI, so that command could install someone else's package of the same name. 47 messages and docstrings change; code that matches the old error text must match the new wording. - The changelog and the module docstrings shown on the API pages no longer carry internal planning IDs. - The security documentation matches the code. `browser_run_script` is described as what it is: model-written Python running as the server's user, which can open connections that bypass the egress proxy and read the process environment, so enable it only where `execute_command` would be acceptable. Its loopback DevTools port, the egress proxy over the browser's own traffic and the fresh browser per session are described for the modes that have them: in `cdp` mode the script attaches to the configured endpoint and no proxy applies. "No `eval`/`exec` of model input" now says "in the server process". The output-truncation row names what is not cut (directory listings, memory views and memory search). The browser page says downloads have no size or count limit of their own, so the container's disk quota is the control. The README's old "Security Model" section is replaced by a short summary that links the Security guide. The computer page says what the desktop confirmation gate does not know: an application's own shortcuts, keys such as `XF86Close`, typed text and clicks. - The Tool Reference page formats each description for reading (RST ``` ``x`` ``` becomes code, the Args section a list), and keeps the exact text the model receives in a collapsed block under each tool. Code in table cells stays on one line, and wide tables scroll inside their box on a phone. The examples use one model ID, `claude-sonnet-5`, and the home page has a byline. - The home page and the quickstart say up front, in a "Source code" note, that the code is in a private repository, that the pages document every tool, setting and API, and how to ask to review the code. The footer of every page reads the same sentence. - Browser tool counts read the same everywhere: 18 `browser_*` tools (`core`) or 27 (`full`), plus `fetch_webpage_rendered`. The `llms.txt` introduction names the browser and desktop servers. ### Fixed - **Security: `computer_key` could run commands on the virtual desktop.** Its text was split on spaces and every word went to `xdotool` as a separate argument, and `xdotool` reads a word that names one of its commands as the start of a chained command. So `computer_key("a exec touch /tmp/x")` ran `touch`, with the server's environment, API keys included: command execution for a deployment that exposes only the desktop tools, and a way past hooks that watch `execute_command`. Click `modifiers` and `computer_hold_key` also passed model text to the same parser. Now every key is checked (a single character or a key name, never an `xdotool` command word), each chord goes to its own `xdotool` call after `--`, and `keydown`/`keyup` get `--` too. Host mode (`pynput`) was not affected. Reproduced in the Linux test image before the fix, and refused after it, with real keys still pressed. - **Desktop key names other than the built-in ones reach `xdotool` as written.** Every name the server did not know (it knows `ctrl`, `Return`, `Tab`, `F1`–`F12` and a few more) was lowercased, and X key names are case-sensitive: `XF86AudioMute`, `ISO_Left_Tab`, `U20AC` or `F13` were silently ignored while the tool reported them pressed, and `3270_Enter` pressed a different key. The descriptions of `computer_key`, `computer_hold_key` and the click tools' `modifiers` now state which keys are accepted. - **Security: the desktop confirmation gate could be bypassed.** The click tools never showed their `modifiers` to the gate, and the modifiers are pressed before the click, so `computer_left_click(x, y, modifiers="alt+F4")` closed the focused window under `DPTTOOLS_COMPUTER_CONFIRM=deny`. The gate also matched chords exactly, so an extra key (`alt+shift+F4`, `ctrl+shift+w`, `ctrl+Delete`) or, on the virtual desktop, a side-specific X name (`Alt_L+F4`, `Control_L+w`, `KP_Delete`) got past it. Numeric spellings got past it too: a hex keysym (`0xffff` is Delete, `alt+0xffc1` is alt+F4), a Unicode form (`ctrl+U0077` is ctrl+w), or a raw key code, which `xdotool` reads from any token that starts with a digit and is not a key name (`119` is Delete, `64+70` alt+F4, on a standard keymap); each was checked live with `xev`. Now the click tools check their modifiers before anything else and show them to the gate as one chord; a chord is gated when it contains a gated chord, with side-specific names and hex and Unicode spellings read as the key they name; and a digit-led token that is not a hex keysym or a `3270_*` name is refused. A third route went through `computer_type`: `xdotool` types a character with the key that produces it, and the Delete key produces U+007F, so on the virtual desktop `computer_type("\x7f")` pressed Delete. `computer_type` now refuses control characters other than tab, newline and carriage return (host mode on Windows sends U+007F as a character, not the Delete key). Present since 0.6.0; only deployments that set `DPTTOOLS_COMPUTER_CONFIRM` relied on the gate. - **The desktop's plus key.** `+` separates the keys of a chord, so the plus key itself could not be pressed: `computer_key("+")` was refused and `"ctrl++"` pressed `ctrl` alone. A `+` where a key is expected is now the plus key (`"+"`, `"ctrl++"`), and a chord with a dangling or doubled separator (`"ctrl+"`, `"ctrl++a"`) is refused instead of being read as a shorter chord. - **The private-address check missed ranges.** It refused seven networks (`127.0.0.0/8`, `10.0.0.0/8`, `172.16.0.0/12`, `192.168.0.0/16`, `169.254.0.0/16`, `::1`, `fc00::/7`), so `0.0.0.0` (which reaches localhost on Linux), `::ffff:127.0.0.1`, IPv6 link-local, carrier-grade NAT, NAT64 and 6to4 forms of private addresses got through, contrary to the documented policy. It now refuses every block the IANA special-purpose address registries list as not globally reachable, multicast, reserved addresses and any IPv6 address outside `2000::/3`, and judges an IPv6 address that carries an IPv4 one (IPv4-mapped, 6to4, Teredo, NAT64) by that IPv4 address. The blocks are written out in the code rather than read from `ipaddress.is_global`, whose tables depend on the Python release (on 3.10.11 they call `192.0.0.8` and `3fff::1` global), so the result is the same on every supported Python. `fetch_webpage`, `fetch_image`, the browser's egress proxy and its `cdp`-mode checks all use it. - `ocr_pdf` page labels. OCR returns the requested pages in document order, and the `--- Page N ---` labels now follow it. An open-ended range (`pages="21-"`) was numbered from 1 instead of from its start, and a list out of order (`"3,1"`) was labelled in the order given, so page 1's text came back as `--- Page 3 ---`. - `ocr_pdf`'s note on a result cut on a page boundary now names exactly the pages left out (`Re-call ocr_pdf with pages='3,7'`). It gave one range from the first page left out to the last, which could take in pages never requested (`'2-7'` for `pages="1-3,7"`, so following it paid for pages 4 to 6) or be invalid (`'5-2'`). ## 0.6.0 — 2026-10-03 ### Migration notes - **`fetch_webpage` no longer uses `DPTTOOLS_SANDBOX_TIMEOUT`**. Its timeout is now `DPTTOOLS_WEB_FETCH_TIMEOUT`, default 30 s. If you raised the sandbox timeout to allow slow pages, set the new variable when you upgrade. - **`web_search` output has a new `usage` key**. Strict-schema parsers must allow it. `search_type="news"` no longer forces Tavily's 2-credit `advanced` depth; pass `search_depth="advanced"` to keep it. - **`DPTTOOLS_OCR_TIMEOUT` is validated at start-up**. A non-integer or non-positive value now raises a `ConfigError` naming the variable, instead of silently falling back to 180 s. The value is read once per process, so restart after changing it. Values outside 30–600 are still clamped. - **`fetch_webpage` returns a JSON object by default**: `url`, `final_url`, `status`, `content_type`, `title`, `text`, `truncated`, `total_chars`, `offset`, `next_offset`, `has_more`, `byte_cap_reached`, `likely_js_rendered`. Code that used the return value as page text should read `text`, or pass `structured=False` for the old plain string. Non-text responses are now refused, and plain text and JSON are no longer run through the HTML-to-text step. ### Added - `DPTTOOLS_WEB_FETCH_TIMEOUT` (default 30 s) for `fetch_webpage`, with connects capped at 10 s. A slow or hostile URL can no longer hold a fetch for the code-execution budget (often several minutes). - `web_search` reports per-call `usage`, and takes an optional `search_depth` parameter (`basic` | `advanced` | `fast` | `ultra-fast`, Tavily only). Tavily's own usage is passed through (`include_usage`), with a fallback estimate clearly marked `estimated`; Serper and Brave report `provider_reported: false` rather than an invented cost. - **Images are scaled to model-safe limits**. `read_image`, `render_document_page`, `render_document_thumbnail`, `read_document_pages_as_images` and `take_screenshot` scale any image above `DPTTOOLS_IMAGE_MAX_EDGE_PX` (default 1568 px long edge) or `DPTTOOLS_IMAGE_MAX_MB` (default 3 MB). Images within the limits are returned byte-identical. The shared helper, `_images.fit_image`, also reports the scale it applied. - `tests/integration/test_toolkit_compat.py` checks std-tools against an installed `doc-processing-toolkit` without a model call: the skills catalog reaches the system prompt, and an in-process `read_image` reaches the toolkit as an image. It is skipped when the toolkit is not installed. - README section "Compatibility with doc-processing-toolkit": tested versions, the in-process image requirement (toolkit 0.5.1+), the skills naming rule, passing `env` to stdio servers, and a working pre-built-session example for a toolkit issue (0.5.2 and earlier) that leaves the skills catalog empty in-process and with `mcp_command` configs. - `.env.example` listing every environment variable with its default, and a Configuration section rebuilt as one table per area (default, accepted values, when it is read), plus how configuration is loaded. Both are generated from one table in `_config.py` by `scripts/gen_reference.py`; unit tests fail if a variable or a tool goes undocumented. - `doc_processing_std_tools.__version__`, read from the installed package metadata, so a running deployment can report its version. - `convert_document(path, to, outdir=None, page="all", overwrite=False)`: converts Office and OpenDocument files with LibreOffice to pdf, docx, xlsx, pptx, odt, ods, odp, csv, html, txt, or png/jpg page images (through PDF and pymupdf), writes next to the source or into an existing `outdir`, never replaces a file unless `overwrite=True`, and returns the output paths as JSON. New settings `DPTTOOLS_SOFFICE_BIN` and `DPTTOOLS_CONVERT_TIMEOUT_S` (default 120, the previous fixed value). The composed server now has 55 tools. - `fetch_webpage` paging (`offset`, `next_offset`, `has_more`), a `likely_js_rendered` hint, and the settings `DPTTOOLS_WEB_FETCH_MAX_BYTES`, `DPTTOOLS_WEB_FETCH_MAX_REDIRECTS` and `DPTTOOLS_WEB_FETCH_ALLOW_HOSTS` (an operator allow-list of internal hostnames, still resolved once and pinned). - `fetch_webpage(format="text" | "markdown" | "html")` and the optional `[extract]` extra (trafilatura): with it, text and Markdown hold the page's main content without navigation and footers; without it, a built-in extractor (now also producing Markdown) is used. The result reports `format` and `extractor`. `[all]` includes `[extract]`. - `fetch_image(url)`: downloads a PNG, JPEG, GIF or WebP image under `fetch_webpage`'s SSRF policy, refuses images over `DPTTOOLS_FETCH_IMAGE_MAX_BYTES` (default 10,000,000) before downloading, checks the real format and pixel count, and returns a caption plus the image scaled to the model limits. `fetch_webpage(include_assets=True)` adds bounded `images` (30) and `links` (100) reference lists. The `[web]` extra now includes Pillow. The composed server now has 56 tools. - `docker/integration.Dockerfile` and `tests/integration/run_in_docker.py`: a Linux test image (Python 3.12, LibreOffice, Xvfb, xdotool, CJK fonts) that runs the integration suite, so LibreOffice and virtual-display tests run on any machine with Docker. A `.dockerignore` keeps `.env` and local environments out of images. - Optional browser server (`browser_server`, `[browser]` extra): drives Chromium through element refs, modelled on Anthropic's browser use toolset, with `browser_<member>` tools (`core` profile: 18, `full`: 27). It has three modes (`launch`, `persistent`, `cdp`), one browser context per MCP session, an idle reaper, and a Tab Context footer in every result. Playwright runs on its own thread, so the browser survives callers that create a new event loop per call. It is off by default: the launcher composes it only with `DPTTOOLS_ENABLE_BROWSER=true`. New `DPTTOOLS_BROWSER_*` settings are validated only when it is enabled. - Browser network policy and confirmation gate. Every connection a launched Chromium makes goes through an egress proxy inside std-tools that applies the domain lists, refuses private and reserved addresses, and connects to the address it validated, closing DNS rebinding. That covers sub-resources, fetch and WebSockets. Navigation refuses non-web schemes and re-checks redirects, and service workers are blocked. `DPTTOOLS_BROWSER_CONFIRM` = `off` (default) | `deny` | `elicit` gates consequential clicks, password entry, uploads and scripts. With `off`, approval belongs to the client app. Includes the `BROWSER_SAFETY_PROMPT` template. - Browser downloads are saved per session and reported once, with a path the document tools open. Opt-in tools (`DPTTOOLS_BROWSER_ENABLE`): `browser_javascript_exec`, `browser_read_console`, `browser_read_network` (both drained per read, per tab) and `browser_file_upload` (only files inside `DPTTOOLS_BROWSER_UPLOAD_DIR`). - `browser_run_script` (opt-in): runs model-written Python with Playwright against the session's tab. It runs in a per-session child process, started without the environment variables whose names look like secrets, attached over a loopback DevTools port that is opened only while the tool is enabled (in `cdp` mode, to the configured endpoint). Variables persist between calls; in `launch` and `persistent` modes the browser's own traffic still goes through the egress proxy; a timeout (`DPTTOOLS_BROWSER_SCRIPT_TIMEOUT_S`, 60) kills the process tree and resets the namespace. The script itself is not sandboxed: it can open its own connections, bypassing the proxy, and read the server's environment (stated in the docs from 0.6.1). - `fetch_webpage_rendered` (browser server): `fetch_webpage`'s JSON envelope (plus `"rendered": true`) from a page rendered by Chromium, for single-page apps and other pages `fetch_webpage` sees as an empty shell. Each call uses a fresh, credential-free context behind the egress proxy, closed afterwards; it exists only where the browser server is enabled. - Optional computer (desktop) server (`computer_server`, `[computer]` extra): 17 `computer_<member>` tools modelled on Anthropic's computer use toolset. `virtual` backend (default): an Xvfb display per MCP session driven through `xdotool`, with an optional VNC view; Linux containers only. `host` backend: the real screen through `pynput` and `mss`, one session at a time, on a chosen monitor, with per-monitor DPI awareness on Windows, a kill switch (top-left corner or `DPTTOOLS_COMPUTER_KILL_HOTKEY`) and a pause on human activity. Large screens are scaled for the model and coordinates mapped back. `DPTTOOLS_COMPUTER_CONFIRM` = `off` (default) | `deny` | `elicit` gates host sessions and key chords that close, quit or delete. Off by default: the launcher composes it only with `DPTTOOLS_ENABLE_COMPUTER=true`. On Linux the extra leaves out `pynput`, whose `evdev` dependency compiles from source. ### Changed - `DPTTOOLS_OCR_TIMEOUT` is read through the shared `ToolsConfig`, like every other variable. It and `LLMWHISPERER_API_KEY` are now documented in the README. - The `mcp` floor is now `>=1.25`, the range the toolkit already requires and the one std-tools is tested on. - Contributor guides for coding agents (`CLAUDE.md`, `AGENTS.md`). - `take_screenshot` and the computer server's host backend share their capture code (`_display.py`); the browser and computer servers share their MCP-session keys (`_sessions.py`). The Screenshot Server README entry now says plainly that it captures the real screen of the machine the server runs on. - The README installs from the repository at the release tag (`pip install "doc-processing-std-tools[...] @ git+https://github.com/rzoeb/doc-processing-std-tools.git@v0.6.0"`), as the docs site does: std-tools is not on PyPI, and a bare `pip install doc-processing-std-tools` would fetch whatever package holds that name there. Unit tests now also check the docs site: its generated tool reference and llms files are current, its configuration page lists every variable, no page names an unknown variable, and no page carries an email address, a link to the repository or a Vercel preview URL. ### Fixed - **`apply_patch` holds the read-modify-write lock** like `edit_file`. In the composed server two overlapping patches, or a patch and an `edit_file` call, could both report success while the last write silently discarded the other change. - **Images are turned upright before they are resized.** A phone photo stored sideways with an EXIF orientation tag came back sideways once `fit_image` had to shrink it, because re-encoding dropped the tag. Images that need no resizing are still returned byte-identical. - **Colour-key transparency survives resizing.** An RGB or greyscale PNG made transparent with a `tRNS` chunk (no alpha channel) was treated as opaque and could be re-encoded as JPEG. - **`browser_run_script` responses are bounded in the child process.** Output is cut to the browser's 50,000-character limit, a call returns at most five screenshots, each fitted to the image limits within an overall budget, so a response can no longer exceed the 64 MiB pipe limit; if a response is still too large or unreadable, the worker is stopped and the namespace reset. The child keeps the protocol on private file descriptors, so a script that writes to the real stdout, reads input or starts a subprocess cannot corrupt it. Browser truncation notices no longer end with a stray number (the limit was passed as the notice's hint). - **`browser_run_script` no longer lets one client's script reach another client's pages.** A script attaches to the whole browser, so while `run_script` is enabled only one browser session is open at a time (a second client gets an error until the first is idle), in `launch` mode each session starts with a fresh browser, and `cdp` mode requires `DPTTOOLS_BROWSER_CDP_SHARE_TABS=true` (a `ConfigError` otherwise). To serve several clients with scripts enabled, run one server per client. - **Exclusive creates under concurrency.** `create_file` and `memory_create` create the file exclusively, and `memory_create` and `memory_rename` hold the memory lock, so overlapping calls for the same path can no longer both succeed with one silently overwriting the other. - **Every file change in the File System and Memory servers holds the lock**, not only read-modify-write edits. `write_file`, `create_file`, `delete_file`, `move_file`, `copy_file`, `write_memory`, `delete_memory` and `memory_delete` now take it too. Before, two overlapping overwrites could interleave (the shorter text followed by the longer one's tail, invalid JSON for a memory entry), and an edit overlapping a delete or move wrote the old file back. A unit test fails if a tool changes a file outside the lock. - **A `browser_run_script` screenshot sets the click scaling.** Coordinates are mapped back through the scale of the latest screenshot, but a script's `await screenshot()` did not update it, so with a viewport larger than `DPTTOOLS_IMAGE_MAX_EDGE_PX` a click read off a script's screenshot missed (by about 20% at 1920x1080). The worker now reports each screenshot's scale; a viewport screenshot updates the mapping, a full-page one does not. - `tests/integration/run_in_docker.py` makes the integration folder pytest's default (`testpaths`) instead of a fixed argument, so `-- tests/integration/<file>.py` runs that file only. - **Host mode picks the real primary monitor on Linux.** With `DPTTOOLS_COMPUTER_MONITOR=primary` (the default) the computer server used the monitor at the desktop's origin, which on X11 need not be the primary output. It now uses the monitor mss marks primary (the XRandR primary output on X11, the primary display on Windows) and falls back to the origin rule. The `[computer]` extra now requires `mss>=10.2`, the first release that reports it. - **The Dockerfile examples install `git`** (absent from `python:3.12-slim`, so the `git+https` install failed) and show how to pass a GitHub token as a BuildKit build secret without storing it in the image. - The browser integration tests restore the environment when Chromium cannot start. Before, a skip during start-up left loopback allow-lists set for every later test. - **Child processes no longer inherit the server's stdin.** `execute_command`, `execute_skill_script`, the LibreOffice conversions and the computer server's helpers start their children with stdin closed. Under the stdio launcher the server's stdin is the MCP message stream, so a command that read input could consume protocol messages or hang the session. - Tool descriptions the model reads: `read_excel` says it returns saved cell values (not formulas) from the sheet active when the file was saved; `load_skill` states its listing limits instead of a private constant name; `take_window_screenshot` no longer cites a version number. The `DPTTOOLS_COMPUTER_CONFIRM` description lists all seven chords the gate checks, and the README explains how to reach the VNC view safely. - **Tavily search authenticates with a Bearer header**. The key used to travel in the JSON body, which Tavily no longer documents and which development-tier keys reject with a 401. Errors returned by `web_search` no longer include request URLs or the API key. - **`format_search_results` accepts `web_search`'s output unchanged**. It used to accept only a bare JSON array, so the two tools could not be chained. Plain arrays still work as before. - The live OCR integration tests skip when the `[ocr]` extra is missing. Before, a key in `.env` without the client installed made them fail instead of skip. - **The composed server no longer freezes on a slow tool**. FastMCP calls synchronous tools on its event loop, so one OCR job, LibreOffice conversion or long command stalled every other request and the transport. The launcher now registers every synchronous tool as an async wrapper that runs it on a worker thread. Names, descriptions and schemas are unchanged, and a test checks this. `edit_file`, `memory_str_replace` and `memory_insert` lock their read-modify-write, so overlapping calls cannot lose updates. - Concurrent LibreOffice conversions in `render_document_page`, `render_document_thumbnail` and `read_document_pages_as_images` no longer collide: each run gets its own profile (`-env:UserInstallation`), so a second conversion no longer exits without output. Failed conversions no longer leave temporary directories behind. - `fetch_webpage` no longer decodes images, PDFs or other binary responses into garbage: they are refused, with their size, before the body is downloaded. The page charset now comes from a byte-order mark, the header or `<meta charset>`, so pages that declare their encoding only in HTML decode correctly. - `take_screenshot` on Linux with mss 10.2: captures now use `mss.MSS` with the `xgetimage` backend. mss's `default` backend, called from a worker thread (where the composed server runs every capture), left the X server unable to map new windows. - The browser and computer servers tolerate the placeholders that strict function calling (OpenAI) sends for unused optional arguments: `tab_id` 0 means the active tab and `browser_navigate` opens a tab when none is open whatever `tab_id` says; `ref=""` means no ref; `x=0, y=0` next to a ref is ignored; `browser_scroll` and `computer_scroll` at (0, 0) scroll at the centre. Found in end-to-end tests with a model that uses strict function calling. ## 0.5.0 — baseline (tag `v0.5.0`, commit `565c3e4`) The first tagged release. It contains 8 FastMCP tool servers with 54 tools: file system, web search, memory, document processing, code execution, screenshot, image tools and skills, plus the composed launcher `python -m doc_processing_std_tools`. Changes before this release are recorded in the git history only. ## Before the first tag The 0.5.0 baseline above is the first tagged release; until then, changes were recorded in the git history only. In summary: ### 0.5.0 — 2026-06-09 - New `read_document_pages_as_images`: renders several PDF or Office pages as images, the vision fallback for scanned and visual documents. - Richer Office extraction, all additive: - `read_word` reads headers, footers and text boxes, and reports embedded images. - `read_excel` reports merged ranges and whether the sheet has images or charts. - `read_powerpoint` reads tables and grouped shapes, and counts images and charts per slide. ### 0.4.0 — 2026-05-18 - **Breaking:** `extract_pdf_via_ocr` is replaced by `ocr_pdf(path, pages)` and `ocr_image(path)`, which call LLMWhisperer directly. OCR gains a deadline (`DPTTOOLS_OCR_TIMEOUT`), page-aware truncation and sanitised errors. - `sanitize_path` no longer claims to contain paths: it rejects only NUL bytes, and the container is the boundary. `take_screenshot` no longer creates folders. - Image tools keep orientation, colour profiles and quality, add a `frame` parameter for multi-frame images, and refuse decompression bombs. ### 0.3.1 — 2026-05-09 - `read_image` reports the image format detected from the file's bytes, not its extension. ### 0.3.0 — 2026-05-07 - `read_image`, the page renderers and `take_screenshot` return images the model can see, instead of base64 text. - New `get_image_metadata`. ### 0.2.2 — 2026-05-06 - Tool descriptions rewritten so each states its accepted formats and points to the right alternative. ### 0.2.1 — 2026-05-01 - `fetch_webpage` re-validates every redirect against the SSRF policy. - `apply_patch` refuses a patch whose context no longer matches the file. - `search_in_files` skips files over the size limit. ### 0.2.0 — 2026-05-01 - Skills server rebuilt: PyYAML frontmatter parsing, `path` and `diagnostics` in `list_skills`, the `<skill_content>` wrapper in `load_skill`, and `make_skills_server()`. - New composed launcher, `python -m doc_processing_std_tools`. ### 0.1.0 - First version: the file system, web search, memory, document processing, code execution, screenshot, image tools and skills servers.