Algolia 101: Key Search Technology Driving Business Success

1. What you’ll build

You’ll (1) use Algolia Crawler to pull content from several domains into one Algolia application, (2) send that content into one or more indices, and (3) expose a unified search-as-you-type UI that can be embedded in WordPress, Shopify, Webflow or any other front-end.

---

2. Prerequisites

---

3. Create a single Algolia application

1. Log in → New application → choose nearest region.

2. In Settings › Team & Access invite stakeholders if you need separate API keys per environment.
   Why single app? Algolia places no restriction on mapping sites to indices inside one application, so analytics and quota stay centralised. (support.algolia.com)

---

4. Plan your indexing strategy

---

5. Configure Algolia Crawler

5.1 Add a crawler

1. Dashboard → Crawler › Create.

2. Give the crawler a name (e.g. multi-site-search).

5.2 Define start URLs

Add one start URL per domain, e.g.

[
  "https://docs.myproduct.com/",
  "https://blog.myproduct.com/",
  "https://help.anotherbrand.io/"
]

5.3 Create actions (one per domain or content-type)

Each action sets:

{
  "indexName": "global_content",   // or "blog_index"
  "pathsToMatch": ["https://blog.myproduct.com/**"],
  "selectors": {
    "record": {
      "title": "h1",
      "content": "main",
      "url": "url",
      "site": "static:blog"        // custom attribute for faceting
    }
  }
}

You can edit all of this in the visual Crawler editor or paste JSON directly. (algolia.com)

5.4 Respect crawl rules

• Make sure robots.txt allows Algolia’s user-agent (Algolia Crawler), or whitelist its IPs on private sites.

• For password-protected staging domains, set Basic auth credentials in Crawler › Advanced settings.

5.5 Schedule & monitor

• Schedule tab → choose frequency (daily, hourly…).

• Use Crawler logs and Algolia CLI/API for CI-based monitoring. (algolia.com)

---

6. Fine-tune the indices

1. In Indices › Configuration set Searchable attributes (title → content).

2. Enable Attributes for faceting: site, language, type.

3. Create Synonyms for branded terms (Settings › Synonyms).

4. Add Rules (e.g. pin the docs homepage for query “documentation”).

---

7. Build a shared search UI

7.1 Choose a flavour of InstantSearch

7.2 Example React component with federated search

import algoliasearch from 'algoliasearch/lite';
import { InstantSearch, Index, SearchBox, Hits } from 'react-instantsearch-hooks-web';

const searchClient = algoliasearch('APP_ID', 'SEARCH_ONLY_KEY');

export default function GlobalSearch() {
  return (
    <InstantSearch searchClient={searchClient} indexName="global_content">
      <SearchBox placeholder="Search across all sites…" />
      {/* If you kept separate indices, repeat <Index> blocks */}
      <Index indexName="global_content">
        <h2>All results</h2>
        <Hits hitComponent={Hit} />
      </Index>
      <Index indexName="blog_index">
        <h2>Blog</h2>
        <Hits hitComponent={Hit} />
      </Index>
    </InstantSearch>
  );
}

function Hit({ hit }) {
  return (
    <a href={hit.url} className="block p-4 border-b">
      <p className="text-sm text-gray-500">{hit.site}</p>
      <h3 className="font-medium">{hit.title}</h3>
      <p className="line-clamp-2">{hit.content}</p>
    </a>
  );
}

Styling is Tailwind-ready; adjust to match each CMS theme.

---

8. Embed the widget in each CMS

8.1 WordPress

1. Install WP Search with Algolia. (algolia.com)

2. Paste your Application ID & API keys.

3. In the plugin settings, disable its built-in crawler (you now control indexing centrally).

4. Insert the React bundle (or use the plugin’s shortcode) in your theme’s header or via Gutenberg block.

8.2 Shopify

Install the “Algolia AI Search & Discovery” app → Customize › Search → choose Custom layout → paste React bundle or tweak Liquid snippets provided by the app. (algolia.com)

8.3 Webflow

1. Host the compiled JS bundle on a CDN.

2. In Page settings › Before add the <script src="...bundle.js"></script> tag.

3. Optionally use No-Code Supply example for filters & pagination. (nocodesupply.co)

---

9. Security & performance tips

• Server keys stay server-side. Always expose only a search-only key in your JS bundles.

• Turn on HTTP referrer restrictions for Search-only keys.

• Add Algolia Insights to track click & conversion events for relevance tuning.

• Use Replica indices if you need per-country ranking variations.

---

10. Maintenance checklist

---

Next steps

• Explore Algolia Recommend for “related content” widgets once the indices stabilise.

• If some sites expose large JSON APIs, consider bypassing the crawler and pushing data via the Batch API for faster indexing.

With this workflow you end up with one search service, one UX, many sites—maintainable, analytics-friendly and blazing-fast for users.