Algolia DocSearch Integration for API Portals

Algolia DocSearch adds fast, typo-tolerant, keyboard-driven search to a developer portal by crawling rendered docs into an Algolia index and rendering a search modal with a drop-in widget. This guide is part of Developer Portal Frameworks & UI Setup and covers the crawler configuration, the @docsearch/react and @docsearch/js widgets, the appId / apiKey / indexName triad, and wiring reindexing into CI so search stays current.

Key objectives:

• Configure the DocSearch crawler to index the right content and ignore navigation chrome
• Mount the search widget with the correct credentials
• Keep the Search-Only key in the browser and the Write key in CI secrets
• Trigger a reindex automatically after each docs deploy

For a step-by-step install on a single portal, see Adding Algolia DocSearch to a docs portal.

Quick reference: which DocSearch setup fits

Three delivery models exist, and the choice is made for you by whether your docs are public. Use this to pick before configuring anything:

The middle column is the default for most commercial API portals: the hosted program cannot see a private staging site, and its weekly cadence means a spec change published on Monday is not searchable until the following crawl. Running the same open-source scraper yourself costs one CI job and gives you deploy-synchronised search.

Prerequisites & Environment Setup

DocSearch has two halves: a crawler that reads your deployed docs and writes records to an Algolia index, and a widget that queries that index from the browser. You need an Algolia application and two of its keys.

Requirements:

• An Algolia application with an appId. Create one in the Algolia dashboard, or apply to the hosted DocSearch program if your docs are public and open-source.
• A Search-Only API key — safe to ship in client-side JavaScript. It can only run queries.
• A Write (Admin) API key — used by the crawler to push records. Keep it in CI secrets only; never expose it to the browser.
• A publicly reachable, fully rendered docs site. The crawler reads server-rendered or pre-rendered HTML. If your portal renders content only after client-side hydration, ensure the crawler can still see the text (static export, or DocSearch’s JS rendering mode).

Install the widget for a React-based portal:

npm install @docsearch/[email protected] @docsearch/[email protected]

Or for a non-React site:

npm install @docsearch/[email protected] @docsearch/[email protected]

Pin the version. The widget’s CSS class names and modal markup are stable within a major version but can shift across minors, which matters if you override the styling.

If you self-host the crawler, install the open-source scraper image. It runs as a one-shot container that reads a config file and exits:

docker pull algolia/docsearch-scraper:latest

The two keys are not interchangeable, and the distinction is the single most important security property of this integration. The Write key can create, replace, and delete every record in the application; the Search-Only key can do nothing but run queries against indexes you have marked readable. Because the widget ships in client-side JavaScript, whatever key you give it is public — treat the widget’s apiKey as if it were printed in the page source, because it is:

If the Write key ever reaches a built page, rotate it immediately in the Algolia dashboard rather than editing the page — the old key remains valid in every CDN copy and browser cache until it is revoked at the source.

Core Configuration

The crawler is driven by a JSON config that tells it which URLs to start from, which DOM selectors map to record fields, and which index to write. The selectors block is the part that determines search quality: it splits each page into hierarchical records (lvl0–lvl4 plus text) so results group by section.

{
  "index_name": "api-portal",
  "start_urls": ["https://docs.example.com/"],
  "sitemap_urls": ["https://docs.example.com/sitemap.xml"],
  "stop_urls": ["https://docs.example.com/changelog/"],
  "selectors": {
    "lvl0": {
      "selector": ".sidebar .menu__link--active",
      "global": true,
      "default_value": "Documentation"
    },
    "lvl1": "article h1",
    "lvl2": "article h2",
    "lvl3": "article h3",
    "lvl4": "article h4",
    "text": "article p, article li, article td"
  },
  "custom_settings": {
    "attributesForFaceting": ["lang", "version"]
  }
}

What each key controls:

• index_name is the Algolia index the crawler writes to. This exact string must match the widget’s indexName — a mismatch is the most common reason search returns nothing.
• start_urls are the entry points the crawler follows links from. sitemap_urls speeds up discovery by giving it the full URL list directly.
• stop_urls excludes paths from indexing — a regex list. Use it to skip a high-churn changelog or generated reference that you do not want polluting results.
• selectors map page structure to record levels. lvl0 is usually the active navigation category (set global: true so it applies to the whole page), and text captures the body. Scope every selector to the content region (article …) so navigation, footer, and sidebar text do not enter the index.
• custom_settings.attributesForFaceting declares attributes you can later filter on, such as version for a multi-version portal.

The lvl0–lvl4 hierarchy is not decoration — it is what makes a result readable. Algolia stores one record per text block, each carrying the full heading path above it, so the modal can show “Authentication › OAuth2 › Refresh tokens” instead of an anonymous paragraph. Get the selectors wrong and every record collapses to the same lvl1, which is why results from a misconfigured crawl all look identical:

Run the crawler with the application credentials passed as environment variables, never in the config file:

docker run -it --env-file=.env \
  -e "CONFIG=$(cat docsearch.json | jq -r tostring)" \
  algolia/docsearch-scraper

The .env file holds APPLICATION_ID and API_KEY (the Write key). Keeping credentials out of docsearch.json lets you commit the config to version control safely.

Integration Pattern

In CI, run the crawler after the docs deploy so the index reflects the live site. The widget then needs no rebuild — it always queries the current index. The workflow below assumes a preceding deploy job has published the portal.

# .github/workflows/reindex-docsearch.yml
name: Reindex DocSearch
on:
  workflow_run:
    workflows: ["Deploy Docs Portal"]
    types: [completed]
jobs:
  reindex:
    # Only reindex if the deploy succeeded
    if: ${{ github.event.workflow_run.conclusion == 'success' }}
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - name: Wait for CDN to serve new content
        run: sleep 30
      - name: Run DocSearch crawler
        run: |
          docker run \
            -e "APPLICATION_ID=${{ secrets.ALGOLIA_APP_ID }}" \
            -e "API_KEY=${{ secrets.ALGOLIA_WRITE_KEY }}" \
            -e "CONFIG=$(cat docsearch.json | jq -r tostring)" \
            algolia/docsearch-scraper:latest

The workflow_run trigger chains this job onto the deploy workflow, and the conclusion == 'success' guard prevents reindexing a failed deploy. The ALGOLIA_WRITE_KEY lives only in repository secrets — it is the one credential that must never reach the browser. The short sleep gives the CDN time to serve the new pages before the crawler reads them; tune it to your cache propagation time.

For portals built on a framework with a native search slot, wire the same index into that slot rather than mounting a second widget. Docusaurus for API Portals ships a DocSearch theme that reads appId, apiKey, and indexName from config; Mintlify Setup & Migration provides its own search that you can swap for DocSearch when you need cross-domain indexing.

Advanced Options

Mounting @docsearch/react. Render the DocSearch component anywhere in the tree, typically in the portal header. Import the CSS once at the app root:

import { DocSearch } from '@docsearch/react';
import '@docsearch/css';

export function SearchButton() {
  return (
    <DocSearch
      appId="YOUR_APP_ID"
      apiKey="YOUR_SEARCH_ONLY_KEY"
      indexName="api-portal"
      placeholder="Search the API docs"
    />
  );
}

The component renders a button that opens the search modal; Ctrl/Cmd+K opens it from anywhere. All three credentials are public-safe here because apiKey is the Search-Only key.

Mounting @docsearch/js. For a non-React portal, call docsearch() against a container element:

<div id="docsearch"></div>
<link rel="stylesheet" href="https://cdn.jsdelivr.net/npm/@docsearch/[email protected]" />
<script src="https://cdn.jsdelivr.net/npm/@docsearch/[email protected]"></script>
<script>
  docsearch({
    container: '#docsearch',
    appId: 'YOUR_APP_ID',
    apiKey: 'YOUR_SEARCH_ONLY_KEY',
    indexName: 'api-portal',
  });
</script>

Filtering and ranking with searchParameters. Restrict results to the active version or language by passing Algolia query parameters — this is where the attributesForFaceting from the crawler config pays off:

<DocSearch
  appId="YOUR_APP_ID"
  apiKey="YOUR_SEARCH_ONLY_KEY"
  indexName="api-portal"
  searchParameters={{ facetFilters: ['version:v2', 'lang:en'] }}
/>

This keeps a user reading the v2 docs from getting v1 results, without maintaining separate indexes. It is also cheaper than the alternative: a separate index per version multiplies your record count and forces the widget to know which index to query, whereas a facet filter is a query-time parameter you can derive from the current URL. For the crawler side of the same problem — keeping deprecated versions out of the index entirely rather than merely filtering them — see Scoping DocSearch crawls to versioned docs.

Weighting the fields that matter. By default every record level contributes equally to relevance, which means a passing mention of pagination in body text can outrank the page whose H1 is “Pagination”. Set the searchable-attribute order explicitly so headings win:

{
  "custom_settings": {
    "searchableAttributes": [
      "unordered(hierarchy.lvl0)",
      "unordered(hierarchy.lvl1)",
      "unordered(hierarchy.lvl2)",
      "unordered(hierarchy.lvl3)",
      "content"
    ],
    "attributesForFaceting": ["lang", "version"],
    "customRanking": ["desc(weight.pageRank)", "desc(weight.level)"]
  }
}

Attributes earlier in searchableAttributes rank higher, so a match on lvl1 beats a match in content. The customRanking tiebreaker then prefers pages you have marked important and deeper, more specific headings over generic ones.

Reindex timing. The crawler reads whatever the CDN is currently serving, which is not necessarily what you just deployed. If your host serves stale HTML for a few seconds after a deploy, the crawl captures the previous build and search silently lags one release behind — a bug that reproduces only in production and looks like a caching problem. Either wait for cache propagation before crawling (the sleep in the workflow above) or purge the CDN as the final step of the deploy job and crawl after the purge confirms.

Verification & Testing

After the crawler runs, confirm records actually landed in the index before trusting the widget. Query the index directly with the Search-Only key:

curl -s "https://YOUR_APP_ID-dsn.algolia.net/1/indexes/api-portal/query" \
  -H "X-Algolia-API-Key: YOUR_SEARCH_ONLY_KEY" \
  -H "X-Algolia-Application-Id: YOUR_APP_ID" \
  -d '{"query":"authentication"}' | jq '.nbHits'

A non-zero nbHits confirms the crawler populated the index and the Search-Only key can read it. A zero result for a term you know exists in the docs means either the crawler did not run, wrote to a different index_name, or your selectors excluded the content region.

Then verify the widget mounts and opens in the browser:

npx [email protected] install --with-deps chromium
node -e "
const { chromium } = require('playwright');
(async () => {
  const b = await chromium.launch();
  const p = await b.newPage();
  await p.goto('https://docs.example.com/');
  await p.click('.DocSearch-Button');
  await p.fill('.DocSearch-Input', 'authentication');
  await p.waitForSelector('.DocSearch-Hit', { timeout: 10000 });
  await b.close();
  console.log('DocSearch returned hits in the modal');
})();
"

The waitForSelector('.DocSearch-Hit') fails the check if the modal opens but returns nothing — catching the credential or index-name mismatch that a static page load would not reveal.

Troubleshooting

• Index does not exist or empty modal. The widget’s indexName does not match the crawler’s index_name, or the crawler has never run successfully. Confirm both strings are identical and check the crawler logs for a write confirmation.
• Method not allowed / 403 when the widget queries. You shipped the Write or Admin key instead of the Search-Only key. Replace apiKey with the Search-Only key from the Algolia dashboard; the write key is rejected for client-side search and is a security risk if exposed.
• Results include navigation, footer, or sidebar text. The selectors are not scoped to the content region. Prefix every selector with the article container (e.g. article h2, article p) so the crawler ignores chrome.
• Crawler indexes nothing on a client-rendered SPA. The scraper read empty HTML before hydration. Pre-render or statically export the docs, or enable the crawler’s JavaScript rendering option so it waits for content to appear.
• Record count collapses after a redesign. A theme upgrade renamed the class the lvl0 selector depends on, so every page now falls back to default_value and the hierarchy flattens. The symptom is search that still “works” but whose results all show the same breadcrumb. Assert a minimum record count in CI (nbHits for a known term, as in the verification step) so a selector regression fails the pipeline instead of quietly degrading relevance.
• Search returns results for pages that no longer exist. The crawler adds and updates records but does not always remove URLs that disappeared, so a deleted endpoint page can linger in the index and 404 when clicked. Run the crawl against a sitemap rather than link-following alone, and clear the index on major restructures rather than crawling on top of the old records.
• Every query is slow from one region. The widget is hitting the primary cluster rather than a distributed search network endpoint. Confirm the request host is the -dsn.algolia.net domain, which routes to the nearest replica, rather than the write endpoint.

Beyond the widget itself, treat search as an observable feature rather than a set-and-forget install. Algolia records every query, including the ones that return nothing, and the zero-result list is the single most useful backlog a documentation team can have: each entry is a developer who expected your API to do something and could not find out whether it does. Review it monthly. The pattern is usually one of three things — vocabulary drift where the docs say “principal” and developers search “user”, a genuinely missing page, or an operation whose summary in the OpenAPI document is too terse to be indexed usefully. The first two are content fixes; the third is a spec fix, which is why search analytics belong in the same review as spec quality rather than in a separate marketing dashboard.

FAQ

Do I have to run the DocSearch crawler myself?

No. Open-source and qualifying docs sites can apply for Algolia’s free hosted DocSearch program, where Algolia runs the crawler on a schedule. Self-hosted teams run the open-source crawler container themselves against their own Algolia application.

Which API key does the widget use?

The widget uses the Search-Only API key, which is safe to expose in client-side code because it can only run queries. The Admin or Write API key, used by the crawler to push records, must stay in CI secrets and never appear in the browser.

Should I use @docsearch/react or @docsearch/js?

Use @docsearch/react for React, Next.js, and Docusaurus portals so the modal integrates with the component tree. Use @docsearch/js for plain HTML, server-rendered pages, or any non-React stack, mounting it onto a container element.

Why does the search modal return zero results after deploy?

The most common cause is a mismatch between the indexName in the widget and the index the crawler wrote to, or a crawler that has not run since the content changed. Confirm both reference the same index and trigger a reindex in CI after each deploy.

Related

• Developer Portal Frameworks & UI Setup — the parent overview of portal frameworks and tooling.
• Adding Algolia DocSearch to a docs portal — the step-by-step single-portal install.
• Docusaurus for API Portals — a framework with a native DocSearch theme.
• Mintlify Setup & Migration — a portal with built-in search you can swap for DocSearch.