Rockbox Zig's search just got a significant overhaul. I replaced the embedded Tantivy full-text search engine with Typesense, an open-source, self-hosted search server. This post covers what changed, why we made the switch, and how the new system works end-to-end.

The Old Approach: Tantivy

The previous search was powered by Tantivy, a Rust full-text search library that runs embedded inside the Rockbox process. The implementation lived in a dedicated /crates/search/ crate and maintained separate on-disk indexes for each entity type:

  • Tracks

  • Albums

  • Artists

  • Liked tracks / liked albums

  • Files

Indexes were stored on the filesystem under ~/.config/rockbox.org/indexes/ using Tantivy's MmapDirectory. Each entity type had its own schema definition, and search queries ran against these local indexes synchronously inside the Rockbox server process.

The problems:

  • Searches took long enough that the macOS client needed a loading spinner and an isLoading state variable just to indicate work was happening.

  • Tantivy indexes lived inside the process — scaling or offloading that work required non-trivial changes.

  • Running the full-text indexing and query logic inside the server added overhead during library scans.

The New Approach: Typesense

Typesense is a standalone search server written in C++ that exposes a clean REST API. Instead of embedding search logic in the Rockbox process, we now run Typesense as a sidecar process and talk to it over HTTP.

How Typesense is provisioned

The new /crates/typesense/ crate handles setup. On startup, it:

  1. 1.

    Checks whether typesense-server is already on $PATH.

  2. 2.

    If not, downloads the correct binary for the current OS/arch from https://dl.typesense.org/releases/{version}/typesense-server-{os}-{arch}.tar.gz (default version: 30.1).

  3. 3.

    Stores data in ~/.config/rockbox.org/typesense/.

  4. 4.

    Generates (or reuses) a UUID-based API key stored at ~/.config/rockbox.org/typesense/api-key.

  5. 5.

    Starts the server on port 8109 (configurable via RB_TYPESENSE_PORT).

Both macOS (darwin) and Linux are supported. The whole provisioning step is transparent — users don't need to install or configure anything separately.

Collection schemas

Three Typesense collections mirror the main entity types:

Tracks

path : string, unique document ID

title, artist, album, album_artist: string, searchable

genre, composer: string

bitrate, frequency, filesize, length: int

track_number, disc_number, year: int

md5: string, content hash

album_art, artist_id, album_id, genre_id: string, optional

Default sort: title. Query fields: title, artist, album, path.

Albums

title, artist: string, searchable

year: int

md5, artist_id: string

album_art, label: string, optional

Artists

| Field | Type | Notes |

|---|---|---|

| name | string | searchable |

| bio, image | string | optional |

Data Flow

Library scan and indexing

When Rockbox scans your music library it:

  1. 1.

    Walks the music directory recursively looking for supported audio formats (mp3, flac, ogg, m4a, aac, wav, wv, mpc, aiff, opus, ape, wma, and more).

  2. 2.

    Extracts metadata and album art using rockbox_sys FFI calls, running up to 8 files concurrently.

  3. 3.

    Persists everything to a local SQLite database.

  4. 4.

    Fetches all tracks, albums, and artists from SQLite, converts them to the Typesense types, serialises to JSONL (JSON Lines), and batch-imports via:


POST /collections/{collection}/documents/import?action=upsert

X-TYPESENSE-API-KEY: <key>

The upsert action means re-scanning the library is idempotent — existing documents are updated in place rather than duplicated.

Search queries

The HTTP search handler sits at:


GET /search?q={query}

It fans out to three parallel Typesense requests, one per collection, using query_by=title,artist,album,path for tracks, query_by=title,artist,label for albums, and query_by=name for artists. The responses are merged and returned as a single JSON payload:


{
  "tracks": [...],
  "albums": [...],
  "artists": [...]
}

The same data is also available through the GraphQL API as a SearchResults type with tracks, albums, artists, liked_tracks, and liked_albums fields.

What Changed in the Client

The most visible sign of the improvement is what we were able to remove from the macOS app. With Tantivy, searches were slow enough to require explicit loading UI:


// Before

@Published var isLoading: Bool = false

Typesense returns results fast enough that the loading state and spinner are simply gone. The macOS SearchManager now fires a 300 ms debounced query and the results appear — no spinner, no progress view.

Configuration Reference

RB_TYPESENSE_API_KEY: auto-generated UUID, Authentication key RB_TYPESENSE_PORT: 8109, Port for the Typesense server RB_TYPESENSE_VERSION: 30.1, Version to download if not in PATH ROCKBOX_UPDATE_LIBRARY: Set to 1 to force a library rescan on startup

You can also trigger a rescan with index rebuild at any time via the HTTP API:


POST /system/scan_library?rebuild_index=true

Summary

| | Tantivy (before) | Typesense (after) |

|---------------|----------------|-------------------------|

| Deployment | Embedded in process | Sidecar server (auto-provisioned) |

| Storage | ~/.config/rockbox.org/indexes/ | ~/.config/rockbox.org/typesense/ |

| Index format | Tantivy segments (mmap) | Typesense native |

| Import format | Tantivy documents | JSONL batch via REST |

| Search API | In-process function call | HTTP REST |

| Client loading UI | Required | Removed |

| Platform | Rust only | OS binary (macOS + Linux) |

The migration removes an entire crate from the project, replaces it with a thin HTTP client, and delivers noticeably faster results. Typesense stays self-hosted and open-source, so there are no external dependencies or data leaving your machine.

The full implementation is available at github.com/tsirysndr/rockbox-zig.