Agent Mag
SUBSCRIBE
Documentation
Web Intelligence

Search And Map

Find, rank, and map public URLs before scraping or crawling them.

Status: Planned.

Search and map workflows help agents find relevant public URLs before spending credits on capture. This is useful for competitive research, documentation lookup, vendor scans, and targeted monitoring.

Planned Search Request

code
{
  "query": "best practices for agent memory storage",
  "limit": 10,
  "scrapeTop": 3,
  "formats": ["markdown", "metadata"],
  "freshness": "30d"
}

Output

FieldPurpose
results[]Ranked URLs with title, snippet, source, and score.
selected[]URLs chosen for scrape/crawl enrichment.
map[]Site graph rows with depth, parent URL, and status.
creditsEstimateExpected credits before enrichment runs.

Crawl Endpoint

Plan and inspect multi-page crawl jobs for docs sites, blogs, and product surfaces.

Extract Endpoint

Convert one or more pages into schema-shaped JSON for agent workflows.

On this page

Planned Search RequestOutput