Web Search API¶
Web Search Server¶
doc_processing_std_tools.web_search
¶
Web Search tools MCP server.
Provides web search capabilities through configurable search providers, with citation formatting support.
fetch_image(url)
¶
Download an image from a URL and show it to the model.
Use this to look at an image found on a web page (fetch_webpage
with include_assets=True lists a page's images). Accepts PNG, JPEG,
GIF and WebP. The same SSRF protection as fetch_webpage applies:
HTTP/HTTPS only, private and internal addresses refused, every redirect
re-checked.
Images larger than DPTTOOLS_FETCH_IMAGE_MAX_BYTES (10 MB by default)
are refused. Images larger than DPTTOOLS_IMAGE_MAX_EDGE_PX (1568 px
on the long edge) or DPTTOOLS_IMAGE_MAX_MB (3 MB) are scaled down
before they are returned.
Requires httpx and Pillow
(the [web] extra).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
url
|
str
|
The image URL (must be HTTP or HTTPS). |
required |
Returns:
| Type | Description |
|---|---|
list[str | Image] | str
|
A short caption (source URL, format, size) followed by the image as |
list[str | Image] | str
|
an MCP |
list[str | Image] | str
|
|
fetch_webpage(url, extract_text=True, max_length=10000, offset=0, structured=True, format='text', include_assets=False)
¶
Fetch a web page or other text resource and return its content.
Only HTTP/HTTPS URLs are allowed. Private/internal IP ranges and localhost are rejected for SSRF protection, on every redirect.
Reads text only: HTML, plain text, Markdown, JSON, XML and CSV. Any
other content type (images, PDFs, archives) is refused before it is
downloaded; for an image, use fetch_image if it is available.
Returns a JSON object by default:
{url, final_url, status, content_type, title, format, extractor, text,
truncated, total_chars, offset, next_offset, has_more, byte_cap_reached,
likely_js_rendered}.
format chooses how an HTML page is returned: "text" (readable
text, the default), "markdown" (headings, lists, links and tables
kept) or "html" (the source). When the optional trafilatura library
is installed, text and Markdown hold the page's main content without
navigation and footers (extractor: "trafilatura"); otherwise a
built-in extractor keeps the whole page (extractor: "stdlib").
Non-HTML text (JSON, plain text, CSV) is returned as-is.
With include_assets=True the object also has images (up to 30:
{url, alt, width, height, source} from <img>, og:image and
icons), links (up to 100: {url, text}) and assets_truncated.
URLs are absolute; nothing is downloaded. To look at an image, pass its
URL to fetch_image.
textholds at mostmax_lengthcharacters starting atoffset. Whenhas_moreis true, call again withoffset=next_offsetto read the next part.likely_js_renderedis true when the page looks like it is built by JavaScript (little text, many scripts, or an empty app root). Retry withfetch_webpage_renderedif that tool is available.byte_cap_reachedis true when the response was larger than the download limit and was cut; the text ends early.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
url
|
str
|
The URL to fetch (must be HTTP or HTTPS). |
required |
extract_text
|
bool
|
Legacy switch: False returns the HTML source, the
same as |
True
|
max_length
|
int
|
Maximum characters of text to return (default 10000). |
10000
|
offset
|
int
|
Character offset to start from, for reading a long page in parts (default 0). |
0
|
structured
|
bool
|
Return the JSON object (default True). False returns only the text, with a truncation notice, as older versions did, always using the built-in extractor. |
True
|
format
|
Literal['text', 'markdown', 'html']
|
|
'text'
|
include_assets
|
bool
|
Add the page's image and link references (default
False). Ignored when |
False
|
Returns:
| Type | Description |
|---|---|
str
|
The JSON object (or plain text when |
str
|
a string starting with |
format_search_results(results)
¶
Format raw search results with inline citation markers and a references section.
Accepts raw search results (JSON with URL, title, content per result)
and formats them with inline citation markers ([1], [2], etc.) and a
references section at the end. The output of web_search can be
passed in unchanged.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
results
|
str
|
JSON string — either the object |
required |
Returns:
| Type | Description |
|---|---|
str
|
Formatted text with inline citation markers and a references section. |
web_search(query, num_results=5, search_type='general', search_depth='basic')
¶
Search the web using the configured search provider.
The search provider is configured via the DPTTOOLS_WEB_SEARCH_PROVIDER and DPTTOOLS_WEB_SEARCH_API_KEY environment variables. Supported providers: tavily, serper, brave.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
query
|
str
|
The search query. |
required |
num_results
|
int
|
Number of results to return (default 5). |
5
|
search_type
|
str
|
Type of search — 'general' or 'news' (default 'general'). |
'general'
|
search_depth
|
str
|
Tavily search depth — 'basic' (default), 'advanced', 'fast' or 'ultra-fast'. 'advanced' costs twice as much. Ignored by serper and brave. |
'basic'
|
Returns:
| Type | Description |
|---|---|
str
|
JSON string with |
str
|
snippet for each result) and |
str
|
usage when it gives one; otherwise marked |
str
|
|