--- title: "Search Tools Studio" version: 3.8.7 lastUpdated: 2026-05-30 --- # Search Tools Studio > **Feature:** Search Tools Studio — unified web tools workspace for `/dashboard/search-tools`. > **Plans:** `18-search-tools-studio-redesign.plan.md` + `_orchestration/master-plan-group-C.md` > **Status:** Released in v3.8.6 --- ## Overview Search Tools Studio transforms `/dashboard/search-tools` from a basic search playground into a three-tab Studio unifying web search, web scraping, and side-by-side provider comparison. ``` ┌ Search Tools ──────────────────────────────────────────────────────────┐ │ [🔍 Search] [📄 Scrape] [⚖ Compare] 142ms · $0.001 │ │ ⓘ [Modalities guide] │ ├──────────────────────────────────────────┬─────────────────────────────┤ │ {active tab content} │ ─ Config │ │ │ Provider [auto ∨] │ │ │ 🟢 Serper $0.001 │ │ │ 🟢 Tavily $0.008 │ │ │ 🔥 Firecrawl (fetch) │ │ │ Type [web | news] │ │ │ Full page [ ] (scrape) │ │ │ Format [md|text|html] │ │ │ Rerank model [∨] │ └──────────────────────────────────────────┴─────────────────────────────┘ ``` --- ## Tabs ### Search Tab Evolves the existing `SearchForm` + `ResultsPanel` + `RerankPanel` into a tab: - Query → results (title, URL, snippet, relevance score). - Provider metadata in Config pane (cost, quota, status). - Rerank section: pick a rerank model, reorder results, show `positionDelta`. - Empty state with CTA when no search providers are configured. - Search history via `SearchHistory.tsx`. - Calls `POST /v1/search` (existing endpoint, no changes). ### Scrape Tab New tab for extracting content from a URL via `POST /v1/web/fetch` (created in plan 05): - Input: URL + full-page toggle + format selector (markdown / text / HTML). - Submit → fetch → render `ScrapeResult.tsx`. - `ScrapeResult` renders markdown preview + raw toggle. - Cap: if response body > **256 KB**, UI shows `(truncated, view raw)` and opens raw in a Monaco modal (D21). - Metadata panel: provider (firecrawl/jina-reader/tavily-search), latency, cost, response size, links count. - Uses `useScrapeFetch.ts` hook. ### Compare Tab Runs the same query/URL across up to **4 providers in parallel** (D22): - Side-by-side columns per provider. - Metrics: latency, cost, result count, response size. - URL overlap calculation for search (number of shared URLs vs initial result). - Calls `POST /v1/search` (search) or `POST /v1/web/fetch` (scrape) per provider. --- ## Config Pane (Shared) `SearchToolsConfigPane.tsx` — always visible, collapsible. | Field | Notes | |-------|-------| | Provider | Dropdown with status badge (configured / missing / rate limited) | | Type | `web` or `news` (search only) | | Full page | Toggle for scrape — fetches entire page vs first visible content | | Format | `markdown`, `text`, or `html` (scrape only) | | Rerank model | Optional model for post-search reranking | | History | Collapsible search history section | --- ## SearchConceptCard `SearchConceptCard.tsx` — always visible, collapsible accordion. Explains: | Concept | One-liner | |---------|-----------| | **Search** | Fetches a list of web results (title, URL, snippet, relevance score) | | **Scrape** | Extracts the full content of a URL (markdown, text or HTML) | | **Compare** | Runs the same query in N providers side-by-side | | **Rerank** | Reorders results via LLM to improve query relevance | | **Auto (cheapest)** | Picks the cheapest available provider automatically | --- ## Provider Catalog `ProviderCatalog.tsx` exposes the full provider list from `GET /api/search/providers` (extended in F4 to include fetch providers): | Field | Source | |-------|--------| | `id`, `name` | `searchRegistry.ts` | | `kind` | `"search"` (12 providers) or `"fetch"` (firecrawl, jina-reader, tavily-search) | | `costPerQuery` | Registry data | | `freeMonthlyQuota` | Registry data | | `searchTypes` / `fetchFormats` | Registry data | | `status` | `"configured"` / `"missing"` / `"rate_limited"` — derived at runtime from credential store | | `configureHref` | `/dashboard/providers` | The status is **derived at request time** by checking whether credentials exist and whether all keys are currently in cooldown. --- ## Export Code `ExportCodeModal` (imported from Playground Studio) + `codeExport.ts` generate curl / Python / TypeScript snippets for both `/v1/search` and `/v1/web/fetch` calls. API key placeholder is always `$OMNIROUTE_API_KEY` (D11, shared with Playground Studio). --- ## Backend Changes Only one backend change was needed for this feature: ### Extended `GET /api/search/providers` `src/app/api/search/providers/route.ts` was extended to: - Include all 3 fetch providers (`firecrawl`, `jina-reader`, `tavily-search`) in the array. - Add `kind: "search" | "fetch"` to every item. - Add `status: "configured" | "missing" | "rate_limited"` derived from live credential state. - Maintain backward compatibility — existing fields (`id`, `name`, etc.) unchanged. --- ## Key Files | Path | Purpose | |------|---------| | `src/app/(dashboard)/dashboard/search-tools/SearchToolsClient.tsx` | Studio shell, tab orchestrator | | `src/app/(dashboard)/dashboard/search-tools/components/SearchToolsTopBar.tsx` | Tabs + metrics + export button | | `src/app/(dashboard)/dashboard/search-tools/components/SearchToolsConfigPane.tsx` | Shared config panel | | `src/app/(dashboard)/dashboard/search-tools/components/SearchConceptCard.tsx` | Explainer cards (always visible) | | `src/app/(dashboard)/dashboard/search-tools/components/ProviderCatalog.tsx` | Provider list with metadata | | `src/app/(dashboard)/dashboard/search-tools/components/ScrapeResult.tsx` | Markdown preview + raw toggle | | `src/app/(dashboard)/dashboard/search-tools/components/tabs/SearchTab.tsx` | Search + rerank tab | | `src/app/(dashboard)/dashboard/search-tools/components/tabs/ScrapeTab.tsx` | Scrape tab | | `src/app/(dashboard)/dashboard/search-tools/components/tabs/CompareTab.tsx` | Multi-provider compare tab | | `src/app/(dashboard)/dashboard/search-tools/hooks/useScrapeFetch.ts` | Scrape fetch hook | | `src/app/api/search/providers/route.ts` | Extended with `kind` + `status` + fetch providers | | `open-sse/config/searchRegistry.ts` | Source of truth for search provider metadata | --- ## Troubleshooting | Symptom | Cause | Fix | |---------|-------|-----| | Scrape tab shows "endpoint not available" | `/v1/web/fetch` not wired | Verify plan 05 is merged; check `src/app/api/v1/web/fetch/route.ts` exists | | Provider catalog shows all as "missing" | Credentials not configured | Add credentials in `/dashboard/providers` | | Scrape content is truncated | Response > 256 KB cap | Expected behavior (D21). Use "view raw" button for full content | | Compare tab only shows 2 providers | Rate limit active | Two or more providers may be in cooldown — check provider status in Config pane | | "Size" shown as raw key in table | Missing i18n key | Verify `search.size` exists in the locale file; rebuild i18n | --- ## References - Master plan: `_tasks/features-v3.8.6/refactorpages/_orchestration/master-plan-group-C.md` - Feature plan: `_tasks/features-v3.8.6/refactorpages/18-search-tools-studio-redesign.plan.md` - Search provider registry: `open-sse/config/searchRegistry.ts` - Playground Studio (shared `ExportCodeModal` + `codeExport.ts`): `docs/frameworks/PLAYGROUND_STUDIO.md` - Web fetch backend: `src/app/api/v1/web/fetch/route.ts`