[direct][research] add a first-class read-only research tool pack for Auto-Dig-style workers #324

New issue

Open

opened 2026-09-02 17:25:38 +00:00 by nsaspy · 0 comments

nsaspy commented

2026-09-02 17:25:38 +00:00

Owner

Goal

Give native direct-mode research agents a serious read-only tool surface without every consumer rebuilding MCP process policy, capability projection, and prompt-compiler wiring by hand.

Auto-Dig currently exposes the complete trusted inventory from its pinned Brave Search 2.1.0 + Fetch 1.1.2 servers:

Brave

brave_web_search
brave_local_search
brave_video_search
brave_image_search
brave_news_search
brave_summarizer
brave_llm_context
brave_place_search

Fetch

fetch_html
fetch_markdown
fetch_readable
fetch_txt
fetch_json
fetch_youtube_transcript

That is enough to get the worker unstuck, but it still leaves obvious research gaps.

Tool families to add next

Keep these read-only and capability/authority gated. Prefer adapters/MCP profiles over baking service logic into the scheduler.

JS/browser retrieval
- bounded Playwright/Chromium page render
- DOM snapshot / readable extraction after JS
- network request capture for public endpoints
- screenshot only when text/DOM is insufficient
PDF/document retrieval
- download PDF
- extract text + metadata by page
- page-range retrieval
- preserve source URL/page numbers for citations
Web archives
- Wayback CDX lookup
- archived-page fetch
- timestamped snapshot enumeration
Git/code research
- repository/file search
- commits/tags/releases
- issue/PR read/search
- raw file fetch
- read-only only for research workers
Feeds / structured endpoints
- RSS/Atom enumerate + fetch entry
- generic bounded HTTP GET with headers/query params
- CSV/JSON/XML parsing into bounded structured observations
Internet infrastructure lookup
- DNS records
- RDAP/WHOIS-style registration data where public
- TLS certificate metadata / CT search
Primary-record adapters worth packaging as optional profiles
- SEC/EDGAR
- FEC
- USASpending / SAM.gov
- court/regulator document search where public APIs exist

Runtime requirements

All tools enter direct mode through the existing registry -> capability -> authority -> confinement path.
Read-only effect classification is host-owned and explicit.
Large results become retained contexts and are exposed through context_peek/search/slice, not dumped into the model prompt.
Every adapter has strict time, byte, and result-count bounds.
Tool schemas remain stable across continuation turns for prompt-cache friendliness.
No generic shell/browser automation capability should silently become a write-capable escape hatch.

Consumer feedback loop

Auto-Dig direct mode is being instructed to include a missing tool/datasource capability section when a research pass is blocked or materially weakened by tool coverage. Those observations should be triaged into this issue (or child issues) rather than patched ad hoc into each consumer.

Related: #296 for model-limit metadata and trusted imported-tool capability projection.

## Goal Give native direct-mode research agents a serious read-only tool surface without every consumer rebuilding MCP process policy, capability projection, and prompt-compiler wiring by hand. Auto-Dig currently exposes the complete trusted inventory from its pinned Brave Search 2.1.0 + Fetch 1.1.2 servers: ### Brave - `brave_web_search` - `brave_local_search` - `brave_video_search` - `brave_image_search` - `brave_news_search` - `brave_summarizer` - `brave_llm_context` - `brave_place_search` ### Fetch - `fetch_html` - `fetch_markdown` - `fetch_readable` - `fetch_txt` - `fetch_json` - `fetch_youtube_transcript` That is enough to get the worker unstuck, but it still leaves obvious research gaps. ## Tool families to add next Keep these read-only and capability/authority gated. Prefer adapters/MCP profiles over baking service logic into the scheduler. 1. **JS/browser retrieval** - bounded Playwright/Chromium page render - DOM snapshot / readable extraction after JS - network request capture for public endpoints - screenshot only when text/DOM is insufficient 2. **PDF/document retrieval** - download PDF - extract text + metadata by page - page-range retrieval - preserve source URL/page numbers for citations 3. **Web archives** - Wayback CDX lookup - archived-page fetch - timestamped snapshot enumeration 4. **Git/code research** - repository/file search - commits/tags/releases - issue/PR read/search - raw file fetch - read-only only for research workers 5. **Feeds / structured endpoints** - RSS/Atom enumerate + fetch entry - generic bounded HTTP GET with headers/query params - CSV/JSON/XML parsing into bounded structured observations 6. **Internet infrastructure lookup** - DNS records - RDAP/WHOIS-style registration data where public - TLS certificate metadata / CT search 7. **Primary-record adapters worth packaging as optional profiles** - SEC/EDGAR - FEC - USASpending / SAM.gov - court/regulator document search where public APIs exist ## Runtime requirements - All tools enter direct mode through the existing registry -> capability -> authority -> confinement path. - Read-only effect classification is host-owned and explicit. - Large results become retained contexts and are exposed through `context_peek/search/slice`, not dumped into the model prompt. - Every adapter has strict time, byte, and result-count bounds. - Tool schemas remain stable across continuation turns for prompt-cache friendliness. - No generic shell/browser automation capability should silently become a write-capable escape hatch. ## Consumer feedback loop Auto-Dig direct mode is being instructed to include a `missing tool/datasource capability` section when a research pass is blocked or materially weakened by tool coverage. Those observations should be triaged into this issue (or child issues) rather than patched ad hoc into each consumer. Related: #296 for model-limit metadata and trusted imported-tool capability projection.